Detailed Explanation of pandas Function for Python Data Analysis
- 2021-11-01 03:47:52
- OfStack
1. apply and applymap
1. Functions that can be used directly with NumPy
Sample code:
# Numpy ufunc Function
df = pd.DataFrame(np.random.randn(5,4) - 1)
print(df)
print(np.abs(df))Run results:
0 1 2 3
0 -0.062413 0.844813 -1.853721 -1.980717
1 -0.539628 -1.975173 -0.856597 -2.612406
2 -1.277081 -1.088457 -0.152189 0.530325
3 -1.356578 -1.996441 0.368822 -2.211478
4 -0.562777 0.518648 -2.007223 0.059411
0 1 2 3
0 0.062413 0.844813 1.853721 1.980717
1 0.539628 1.975173 0.856597 2.612406
2 1.277081 1.088457 0.152189 0.530325
3 1.356578 1.996441 0.368822 2.211478
4 0.562777 0.518648 2.007223 0.059411
2. Apply a function to a column or row via apply
Sample code:
# Use apply Apply row or column data
#f = lambda x : x.max()
print(df.apply(lambda x : x.max()))Run results:
0 -0.062413
1 0.844813
2 0.368822
3 0.530325
dtype: float64
3. Note that the direction of the specified axis is axis=0 by default, and the direction is column
Sample code:
# Specifies the axis direction, axis=1 The direction is line
print(df.apply(lambda x : x.max(), axis=1))Run results:
0 0.844813
1 -0.539628
2 0.530325
3 0.368822
4 0.518648
dtype: float64
4. Apply the function to each data through applymap
Sample code:
# Use applymap Apply to each data
f2 = lambda x : '%.2f' % x
print(df.applymap(f2))Run results:
0 1 2 3
0 -0.06 0.84 -1.85 -1.98
1 -0.54 -1.98 -0.86 -2.61
2 -1.28 -1.09 -0.15 0.53
3 -1.36 -2.00 0.37 -2.21
4 -0.56 0.52 -2.01 0.06
STEP 2 Sort
1. Index sorting
sort_index()
By default, ascending sort is used, and ascending=False is descending sort
Sample code:
# Series
s4 = pd.Series(range(10, 15), index = np.random.randint(5, size=5))
print(s4)
# Index sorting
s4.sort_index() # 0 0 1 3 3Run results:
0 10
3 11
1 12
3 13
0 14
dtype: int64
0 10
0 14
1 12
3 11
3 13
dtype: int64
2. Pay attention to the axis direction when operating DataFrame
Sample code:
# DataFrame
df4 = pd.DataFrame(np.random.randn(3, 5),
index=np.random.randint(3, size=3),
columns=np.random.randint(5, size=5))
print(df4)
df4_isort = df4.sort_index(axis=1, ascending=False)
print(df4_isort) # 4 2 1 1 0
Run results:
1 4 0 1 2
2 -0.416686 -0.161256 0.088802 -0.004294 1.164138
1 -0.671914 0.531256 0.303222 -0.509493 -0.342573
1 1.988321 -0.466987 2.787891 -1.105912 0.889082
4 2 1 1 0
2 -0.161256 1.164138 -0.416686 -0.004294 0.088802
1 0.531256 -0.342573 -0.671914 -0.509493 0.303222
1 -0.466987 0.889082 1.988321 -1.105912 2.787891
3. Sort by Value
sort_values(by='column name')
Sort according to a column name with only 1, and report an error if there are other identical column names.
Sample code:
# Sort by Value
df4_vsort = df4.sort_values(by=0, ascending=False)
print(df4_vsort)Run results:
1 4 0 1 2
1 1.988321 -0.466987 2.787891 -1.105912 0.889082
1 -0.671914 0.531256 0.303222 -0.509493 -0.342573
2 -0.416686 -0.161256 0.088802 -0.004294 1.164138
STEP 3 Deal with missing data
Sample code:
df_data = pd.DataFrame([np.random.randn(3), [1., 2., np.nan],
[np.nan, 4., np.nan], [1., 2., 3.]])
print(df_data.head())
Run results:
0 1 2
0 -0.281885 -0.786572 0.487126
1 1.000000 2.000000 NaN
2 NaN 4.000000 NaN
3 1.000000 2.000000 3.000000
1. Determine if there is a missing value: isnull ()
Sample code:
# isnull
print(df_data.isnull())Run results:
0 1 2
0 False False False
1 False False True
2 True False True
3 False False False
2. Discard missing data: dropna ()
According to the axis axis direction, the row or column containing NaN is discarded. Sample code:
# dropna
print(df_data.dropna())
print(df_data.dropna(axis=1))Run results:
0 1 2
0 -0.281885 -0.786572 0.487126
3 1.000000 2.000000 3.000000
1
0 -0.786572
1 2.000000
2 4.000000
3 2.000000
3. Fill in missing data: fillna ()
Sample code:
# Use apply Apply row or column data
#f = lambda x : x.max()
print(df.apply(lambda x : x.max()))Run results:
0 1 2
0 -0.281885 -0.786572 0.487126
1 1.000000 2.000000 -100.000000
2 -100.000000 4.000000 -100.000000
3 1.000000 2.000000 3.000000