Detailed explanation of 16 Pandas functions to improve your "data cleaning" ability by 100 times!-Python Tutorial-php.cn

As a data analyst, Data cleaning is an essential link. Sometimes because the data is too messy, it often takes us a lot of time to process it. Therefore, mastering more data cleaning methods will increase your ability by 100 times.

This article is based on this, tells about the super easy-to-use str vectorized string function in Pandas. After learning it, I instantly felt that my data cleaning ability has improved.

1 data set, 16 Pandas functions

The data set is carefully compiled by Huang Huang for everyone, just to help everyone learn knowledge. The data set is as follows:

import pandas as pd

df ={&#39;姓名&#39;:[&#39; 黄同学&#39;,&#39;黄至尊&#39;,&#39;黄老邪 &#39;,&#39;陈大美&#39;,&#39;孙尚香&#39;],
     &#39;英文名&#39;:[&#39;Huang tong_xue&#39;,&#39;huang zhi_zun&#39;,&#39;Huang Lao_xie&#39;,&#39;Chen Da_mei&#39;,&#39;sun shang_xiang&#39;],
     &#39;性别&#39;:[&#39;男&#39;,&#39;women&#39;,&#39;men&#39;,&#39;女&#39;,&#39;男&#39;],
     &#39;身份证&#39;:[&#39;463895200003128433&#39;,&#39;429475199912122345&#39;,&#39;420934199110102311&#39;,&#39;431085200005230122&#39;,&#39;420953199509082345&#39;],
     &#39;身高&#39;:[&#39;mid:175_good&#39;,&#39;low:165_bad&#39;,&#39;low:159_bad&#39;,&#39;high:180_verygood&#39;,&#39;low:172_bad&#39;],
     &#39;家庭住址&#39;:[&#39;湖北广水&#39;,&#39;河南信阳&#39;,&#39;广西桂林&#39;,&#39;湖北孝感&#39;,&#39;广东广州&#39;],
     &#39;电话号码&#39;:[&#39;13434813546&#39;,&#39;19748672895&#39;,&#39;16728613064&#39;,&#39;14561586431&#39;,&#39;19384683910&#39;],
     &#39;收入&#39;:[&#39;1.1万&#39;,&#39;8.5千&#39;,&#39;0.9万&#39;,&#39;6.5千&#39;,&#39;2.0万&#39;]}
df = pd.DataFrame(df)
df

The results are as follows:

Observe the above data , the data set is messy. Next, we will use 16 Pandas to clean the above data.

① cat函数：用于字符串的拼接

df["姓名"].str.cat(df["家庭住址"],sep=&#39;-&#39;*3)

结果如下：

② contains：判断某个字符串是否包含给定字符

df["家庭住址"].str.contains("广")

结果如下：

③ startswith/endswith：判断某个字符串是否以…开头/结尾

# 第一个行的“ 黄伟”是以空格开头的
df["姓名"].str.startswith("黄") 
df["英文名"].str.endswith("e")

结果如下：

④ count：计算给定字符在字符串中出现的次数

df["电话号码"].str.count("3")

结果如下：

⑤ get：获取指定位置的字符串

df["姓名"].str.get(-1)
df["身高"].str.split(":")
df["身高"].str.split(":").str.get(0)

结果如下：

⑥ len：计算字符串长度

df["性别"].str.len()

结果如下：

⑦ upper/lower：英文大小写转换

df["英文名"].str.upper()
df["英文名"].str.lower()

结果如下：

⑧ pad+side参数/center：在字符串的左边、右边或左右两边添加给定字符

df["家庭住址"].str.pad(10,fillchar="*")      # 相当于ljust()
df["家庭住址"].str.pad(10,side="right",fillchar="*")    # 相当于rjust()
df["家庭住址"].str.center(10,fillchar="*")

结果如下：

⑨ repeat：重复字符串几次

df["性别"].str.repeat(3)

结果如下：

⑩ slice_replace：使用给定的字符串，替换指定的位置的字符

df["电话号码"].str.slice_replace(4,8,"*"*4)

结果如下：

⑪ replace：将指定位置的字符，替换为给定的字符串

df["身高"].str.replace(":","-")

结果如下：

⑫ replace：将指定位置的字符，替换为给定的字符串(接受正则表达式)

replace中传入正则表达式，才叫好用；
先不要管下面这个案例有没有用，你只需要知道，使用正则做数据清洗多好用；

df["收入"].str.replace("\d+\.\d+","正则")

结果如下：

⑬ split方法+expand参数：搭配join方法功能很强大

# 普通用法
df["身高"].str.split(":")
# split方法，搭配expand参数
df[["身高描述","final身高"]] = df["身高"].str.split(":",expand=True)
df
# split方法搭配join方法
df["身高"].str.split(":").str.join("?"*5)

结果如下：

⑭ strip/rstrip/lstrip：去除空白符、换行符

df["姓名"].str.len()
df["姓名"] = df["姓名"].str.strip()
df["姓名"].str.len()

结果如下：

⑮ findall：利用正则表达式，去字符串中匹配，返回查找结果的列表

findall使用正则表达式，做数据清洗，真的很香！

df["身高"]
df["身高"].str.findall("[a-zA-Z]+")

结果如下：

⑯ extract/extractall：接受正则表达式，抽取匹配的字符串(一定要加上括号)

df["身高"].str.extract("([a-zA-Z]+)")
# extractall提取得到复合索引
df["身高"].str.extractall("([a-zA-Z]+)")
# extract搭配expand参数
df["身高"].str.extract("([a-zA-Z]+).*?([a-zA-Z]+)",expand=True)

结果如下：

The above is the detailed content of Detailed explanation of 16 Pandas functions to improve your 'data cleaning' ability by 100 times!. For more information, please follow other related articles on the PHP Chinese website!

Statement of this Website

The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn

Hot AI Tools

Undress AI Tool

Undress images for free

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

Online AI tool for removing clothes from photos.

Clothoff.io

AI clothes remover

Video Face Swap

Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Article

PHP Variable Scope Explained

1 months ago By 百草

Tips for Writing PHP Comments

4 weeks ago By 百草

Commenting Out Code in PHP

4 weeks ago By 百草

Roblox: Grow A Garden - Complete Guide To Travelling Merchants

3 weeks ago By Jack chen

Destiny 2: The Edge Of Fate - How Difficulty Modifiers Work

4 weeks ago By DDD

Hot Tools

Notepad++7.3.1

Easy-to-use and free code editor

SublimeText3 Chinese version

Chinese version, very easy to use

Zend Studio 13.0.1

Powerful PHP integrated development environment

Dreamweaver CS6

Visual web development tools

SublimeText3 Mac version

God-level code editing software (SublimeText3)

Hot Topics

Laravel Tutorial

1604

PHP Tutorial

1510

276

Related knowledge

How to create a virtual environment in Python Aug 05, 2025 pm 01:05 PM

To create a Python virtual environment, you can use the venv module. The steps are: 1. Enter the project directory to execute the python-mvenvenv environment to create the environment; 2. Use sourceenv/bin/activate to Mac/Linux and env\Scripts\activate to Windows; 3. Use the pipinstall installation package, pipfreeze>requirements.txt to export dependencies; 4. Be careful to avoid submitting the virtual environment to Git, and confirm that it is in the correct environment during installation. Virtual environments can isolate project dependencies to prevent conflicts, especially suitable for multi-project development, and editors such as PyCharm or VSCode are also

What are common strategies for debugging a memory leak in Python? Aug 06, 2025 pm 01:43 PM

Usetracemalloctotrackmemoryallocationsandidentifyhigh-memorylines;2.Monitorobjectcountswithgcandobjgraphtodetectgrowingobjecttypes;3.Inspectreferencecyclesandlong-livedreferencesusingobjgraph.show_backrefsandcheckforuncollectedcycles;4.Usememory_prof

How to work with timezones in Python? Aug 05, 2025 pm 04:53 PM

UsezoneinfoforPython3.9 tocreatetimezone-awaredatetimesandconvertbetweentimezoneswithastimezone();2.ForPython3.6–3.8,usepytzwithlocalize()toavoidDSTerrors;3.AlwaysworkinUTCinternallyandconverttolocaltimeonlyfordisplay;4.Parsetimezone-awarestringsusin

How to automate data entry from Excel to a web form with Python? Aug 12, 2025 am 02:39 AM

The method of filling Excel data into web forms using Python is: first use pandas to read Excel data, and then use Selenium to control the browser to automatically fill and submit the form; the specific steps include installing pandas, openpyxl and Selenium libraries, downloading the corresponding browser driver, using pandas to read Name, Email, Phone and other fields in the data.xlsx file, launching the browser through Selenium to open the target web page, locate the form elements and fill in the data line by line, using WebDriverWait to process dynamic loading content, add exception processing and delay to ensure stability, and finally submit the form and process all data lines in a loop.

How to implement a custom iterator within a Python class? Aug 06, 2025 pm 01:17 PM

Define__iter__()toreturntheiteratorobject,typicallyselforaseparateiteratorinstance.2.Define__next__()toreturnthenextvalueandraiseStopIterationwhenexhausted.Tocreateareusablecustomiterator,managestatewithin__iter__()oruseaseparateiteratorclass,ensurin

How to pretty print a JSON file in Python? Aug 07, 2025 pm 12:10 PM

To beautify and print JSON files, you need to use the indent parameters of the json module. The specific steps are: 1. Use json.load() to read the JSON file data; 2. Use json.dump() and set indent to 4 or 2 to write to a new file, and then the formatted JSON file can be generated and the beautified printing can be completed.

How to set up Python virtual environment in VSCode Aug 06, 2025 am 02:30 AM

Create a virtual environment: Run python-mvenvvenv in the project folder. 2. Activate the virtual environment: Windows uses venv\Scripts\activate, macOS/Linux uses sourcevenv/bin/activate. 3. Open the project in VSCode and press Ctrl Shift P to select the Python interpreter, specify the interpreter in the virtual environment. 4. Verify whether it is effective: run importsys;print(sys.executable), and the output path should point to the venv folder. 5. Optional configuration: enable python.terminal.a in settings

How to connect to a Redis datastore from a Python application? Aug 06, 2025 am 05:51 AM

Installtheredispackageusingpipinstallredis.2.ConnecttoRedisusingredis.Redis(host,port,db,decode_responses=True)forlocalorremoteservers,providingcredentialsifneeded.3.Alternatively,useredis.from_url()withaRedisURL,includingrediss://forSSLconnections.4

See all articles