第5章 使用pandas进行数据预处理 实训

第5章实训

实训1 插补用户用电量缺失值

# 拉格朗日插值
from scipy.interpolate import lagrange
import pandas as pd
import numpy as np
data = pd.read_csv('../data/missing_data.csv')
# 查看一下数据
data.info()

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 21 entries, 0 to 20
Data columns (total 3 columns):
 #   Column  Non-Null Count  Dtype  
---  ------  --------------  -----  
 0   A       19 non-null     float64
 1   B       17 non-null     float64
 2   C       17 non-null     float64
dtypes: float64(3)
memory usage: 632.0 bytes

#拉格朗日插值
#dropna().index用于记录非缺失值的下标
#dropna().values用于记录非缺失值的实际值
for i in['A','B','C']:
    # 训练模型
    la = lagrange(data.loc[:,i].dropna().index,data.loc[:,i].dropna().values)
    # 用list_d用于记录当前列缺失值所在的行(记录缺失值下标)
    list_d = list(set(np.arange(0,21)).difference(set(data.loc[:,i].dropna().index)))
    data.loc[list_d,i] = la(list_d)

# 查看一下数据
data.info()

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 21 entries, 0 to 20
Data columns (total 3 columns):
 #   Column  Non-Null Count  Dtype  
---  ------  --------------  -----  
 0   A       21 non-null     float64
 1   B       21 non-null     float64
 2   C       21 non-null     float64
dtypes: float64(3)
memory usage: 632.0 bytes

实训 2 合并线损,用电量趋势与线路告警数据

data_ele = pd.read_csv('../data/ele_loss.csv')
data_ala = pd.read_csv('../data/alarm.csv',encoding="gbk")

(49, 4)

print('查看ele_loss.csv数据形状:')
data_ele.shape

查看ele_loss.csv数据形状:





(49, 4)

print('查看alarm.csv数据形状:')
data_ala.shape

查看alarm.csv数据形状:





(25, 3)

data = pd.merge(data_ele,data_ala,left_on = ['ID','date'],right_on = ['ID','date'],how='inner')
print('合并后的形状:',data.shape)
data

合并后的形状: (6, 5)

	ID	date	ele	loss	alarm
0	21261001	2010/9/3	858.0	0.151048	电流不平衡
1	21261001	2010/9/6	883.5	0.162778	C相电流过负荷
2	21261001	2010/9/16	915.5	0.162028	C相电流过负荷
3	21261001	2010/9/16	915.5	0.162028	C相电流过负荷
4	21261001	2010/9/17	961.0	0.166650	A相电流过负荷
5	21261001	2010/9/17	961.0	0.166650	电流不平衡

实训3 标准化建模专家样本数据

data = pd.read_csv('../data/model.csv',encoding = 'gbk')
def StandardScaler(data):
    data = (data - data.mean())/data.std()
    return data
S = StandardScaler(data)
S.head()

	电量趋势下降指标	线损指标	告警类指标	是否窃漏电
0	0.612777	0.974549	-0.161216	2.500687
1	0.612777	-1.022590	2.653616	2.500687
2	-0.343352	0.974549	-0.161216	2.500687
3	3.003099	-1.022590	-1.099494	-0.398516
4	0.134712	0.974549	-1.099494	-0.398516