String manipulations in Pandas DataFrame

String manipulation is the process of changing, parsing, splicing, pasting or analyzing strings. As we know that sometimes data in the string is not suitable for manipulating the analysis or get a description of the data. But Python is known for its ability to manipulate strings. In this article we will understand how Pandas provides us the ways to manipulate to modify and process string data-frame using some builtin functions.

Create a String Dataframe using Pandas

First of all we will know ways to create a string dataframe using Pandas.

import pandas as pd
import numpy as np

data = {'Names': ['Gulshan', 'Shashank', 'Bablu', 'Abhishek', 'Anand', np.nan, 'Pratap'],
        'City': ['Delhi', 'Mumbai', 'Kolkata', 'Delhi', 'Chennai', 'Bangalore', 'Hyderabad']}

df = pd.DataFrame(data)
print(df)

Output:

Change Column Datatype in Pandas

To change the type of the created dataframe to string type. we can do this with the help of .astype() . Let's have a look at them in the below example

print(df.astype('string'))

Output:

String Manipulations in Pandas

Now we see the string manipulations inside a Pandas Dataframe, so first create a Dataframe and manipulate all string operations on this single data frame below so that everyone can get to know about it easily.

Example:

import pandas as pd
import numpy as np

data = {'Names': ['Gulshan', 'Shashank', 'Bablu', 'Abhishek', 'Anand', np.nan, 'Pratap'],
        'City': ['Delhi', 'Mumbai', 'Kolkata', 'Delhi', 'Chennai', 'Bangalore', 'Hyderabad']}

df = pd.DataFrame(data)
print(df)

Output:

Let's have a look at various methods provided by this library for string manipulations.

• lower(): Converts all uppercase characters in strings in the DataFrame to lower case and returns the lowercase strings in the result.

print(df['Names'].str.lower())

Output:

• upper(): Converts all lowercase characters in strings in the DataFrame to upper case and returns the uppercase strings in result.

print(df['Names'].str.upper())

Output:

• strip(): If there are spaces at the beginning or end of a string, we should trim the strings to eliminate spaces using strip() or remove the extra spaces contained by a string in DataFrame.

print(df['Names'].str.strip())

Output:

• split(' '): Splits each string with the given pattern. Strings are split and the new elements after the performed split operation, are stored in a list.

df['Split_Names'] = df['Names'].str.split('a')
print(df[['Names', 'Split_Names']])

Output:

• len(): With the help of len() we can compute the length of each string in DataFrame & if there is empty data in DataFrame, it returns NaN.

print(df['Names'].str.len())

Output:

• cat(sep=' '): It concatenates the data-frame index elements or each string in DataFrame with given separator.

print(df)

print("\nafter using cat:")
print(df['Names'].str.cat(sep=', ')) 

Output:

• get_dummies(): It returns the DataFrame with One-Hot Encoded values like we can see that it returns boolean value 1 if it exists in relative index or 0 if not exists.

print(df['City'].str.get_dummies())

Output:

• startswith(pattern): It returns true if the element or string in the DataFrame Index starts with the pattern.

print(df['Names'].str.startswith('G'))

Output:

• endswith(pattern): It returns true if the element or string in the DataFrame Index ends with the pattern.

print(df['Names'].str.endswith('h'))

Output:

• Python replace(a,b): It replaces the value a with the value b like below in example 'Gulshan' is being replaced by 'Gaurav'.

print(df['Names'].str.replace('Gulshan', 'Gaurav'))

Output:

• Python repeat(value): It repeats each element with a given number of times like below in example, there are two appearances of each string in DataFrame.

print(df['Names'].str.repeat(2))

Output:

• Python count(pattern): It returns the count of the appearance of pattern in each element in Data-Frame like below in example it counts 'n' in each string of DataFrame and returns the total counts of 'a' in each string.

print(df['Names'].str.count('a'))

Output:

• Python find(pattern): It returns the first position of the first occurrence of the pattern. We can see in the example below that it returns the index value of appearance of character 'a' in each string throughout the DataFrame.

print(df['Names'].str.find('a'))

Output:

• findall(pattern): It returns a list of all occurrences of the pattern. As we can see in below, there is a returned list consisting n as it appears only once in the string.

print(df['Names'].str.findall('a'))

Output:

• islower(): It checks whether all characters in each string in the Index of the Data-Frame in lower case or not, and returns a Boolean value.

print(df['Names'].str.islower())

Output:

• isupper(): It checks whether all characters in each string in the Index of the Data-Frame in upper case or not, and returns a Boolean value.

print(df['Names'].str.isupper())

Output:

• isnumeric(): It checks whether all characters in each string in the Index of the Data-Frame are numeric or not, and returns a Boolean value.

print(df['Names'].str.isnumeric())

Output:

• swapcase(): It swaps the case lower to upper and vice-versa. Like in the example below, it converts all uppercase characters in each string into lowercase and vice-versa (lowercase -> uppercase).

print(df['Names'].str.swapcase())

Output: