python data Preprocessing a method for converting category data into numerical values
- 2020-06-07 04:49:48
- OfStack
When analyzing python data, data preprocessing should be carried out first.
Sometimes you have to deal with a non-numeric category of data. Well, today's lesson is what to do with that data.
There are about three ways to do this:
1. Fast conversion through LabelEncoder;
2. Map the category to the value through mapping. But the scope of this approach is limited;
3, using the get_dummies method to convert.
import pandas as pd
from io import StringIO
csv_data = '''A,B,C,D
1,2,3,4
5,6,,8
0,11,12,'''
df = pd.read_csv(StringIO(csv_data))
print(df)
# Count the number that is empty
print(df.isnull().sum())
print(df.values)
# Discard the empty
print(df.dropna())
print('after', df)
from sklearn.preprocessing import Imputer
# axis=0 column axis = 1 line
imr = Imputer(missing_values='NaN', strategy='mean', axis=0)
imr.fit(df) # fit Build data
imputed_data = imr.transform(df.values) #transform Populate the data
print(imputed_data)
df = pd.DataFrame([['green', 'M', 10.1, 'class1'],
['red', 'L', 13.5, 'class2'],
['blue', 'XL', 15.3, 'class1']])
df.columns =['color', 'size', 'price', 'classlabel']
print(df)
size_mapping = {'XL':3, 'L':2, 'M':1}
df['size'] = df['size'].map(size_mapping)
print(df)
## traverse Series
for idx, label in enumerate(df['classlabel']):
print(idx, label)
#1, using LabelEncoder Class fast coding , But at the moment of color Not suitable for ,
# look , There seems to be some size
from sklearn.preprocessing import LabelEncoder
class_le = LabelEncoder()
color_le = LabelEncoder()
df['classlabel'] = class_le.fit_transform(df['classlabel'].values)
#df['color'] = color_le.fit_transform(df['color'].values)
print(df)
#2, A mapping dictionary converts a class label to an integer
import numpy as np
class_mapping = {label: idx for idx, label in enumerate(np.unique(df['classlabel']))}
df['classlabel'] = df['classlabel'].map(class_mapping)
print('2,', df)
#3, To deal with 1 Does not apply to the
# Used to create 1 A new virtual feature
from sklearn.preprocessing import OneHotEncoder
pf = pd.get_dummies(df[['color']])
df = pd.concat([df, pf], axis=1)
df.drop(['color'], axis=1, inplace=True)
print(df)