Interactive Session

A cartoon panda in a frame shop. MidJourney 5

Getting Started

Before we begin our interactive session, please follow these steps to set up your Jupyter Notebook:

Open JupyterLab and create a new notebook:
- Click on the + button in the top left corner
- Select Python 3.10.0 from the Notebook options
Rename your notebook:
- Right-click on the Untitled.ipynb tab
- Select “Rename”
- Name your notebook with the format: Session_XY_Topic.ipynb (Replace X with the day number and Y with the session number)
Add a title cell:
- In the first cell of your notebook, change the cell type to “Markdown”
- Add the following content (replace the placeholders with the actual information):

# Day X: Session Y - [Session Topic]

[Link to session webpage]

Date: [Current Date]

Add a code cell:
- Below the title cell, add a new cell
- Ensure it’s set as a “Code” cell
- This will be where you start writing your Python code for the session
Throughout the session:
- Take notes in Markdown cells
- Copy or write code in Code cells
- Run cells to test your code
- Ask questions if you need clarification

Caution

Remember to save your work frequently by clicking the save icon or using the keyboard shortcut (Ctrl+S or Cmd+S).

Let’s begin our interactive session!

Introduction

In this interactive session, we’ll explore the basics of working with pandas DataFrames using a dataset of world cities. We’ll cover importing data, basic DataFrame operations, and essential methods for data exploration and manipulation. This session will prepare you for more advanced data analysis tasks and upcoming collaborative coding exercises.

Learning Objectives

By the end of this session, you will be able to:

Import data into a pandas DataFrame
Explore basic DataFrame properties and methods
Perform simple data filtering and selection operations
Use basic aggregation and grouping functions

Setting Up

Let’s start by importing the pandas library and loading our dataset.

Code

import pandas as pd
import numpy as np

1. Basic Data Importing

Code

url = "https://raw.githubusercontent.com/datasets/world-cities/master/data/world-cities.csv"
cities_df = pd.read_csv(url)

2. Basic DataFrame Exploration

Viewing the Data

Let’s take a look at the first few rows of our DataFrame:

Code

print(cities_df.head())

                  name               country             subcountry  geonameid
0         les Escaldes               Andorra     Escaldes-Engordany    3040051
1     Andorra la Vella               Andorra       Andorra la Vella    3041563
2   Umm Al Quwain City  United Arab Emirates  Imārat Umm al Qaywayn     290594
3  Ras Al Khaimah City  United Arab Emirates        Raʼs al Khaymah     291074
4           Zayed City  United Arab Emirates              Abu Dhabi     291580

To see the last few rows, we can use:

Code

print(cities_df.tail())

              name   country           subcountry  geonameid
26462     Bulawayo  Zimbabwe             Bulawayo     894701
26463      Bindura  Zimbabwe  Mashonaland Central     895061
26464   Beitbridge  Zimbabwe   Matabeleland South     895269
26465      Epworth  Zimbabwe               Harare    1085510
26466  Chitungwiza  Zimbabwe               Harare    1106542

DataFrame Properties

Now, let’s explore some basic properties of our DataFrame:

Code

# Number of rows and columns
print("Shape:", cities_df.shape)

# Column names
print("\nColumns:", cities_df.columns)

# Data types of each column
print("\nData types:\n", cities_df.dtypes)

# Summary statistics of numeric columns (if any)
print("\nSummary statistics:\n", cities_df.describe())

Shape: (26467, 4)

Columns: Index(['name', 'country', 'subcountry', 'geonameid'], dtype='object')

Data types:
 name          object
country       object
subcountry    object
geonameid      int64
dtype: object

Summary statistics:
           geonameid
count  2.646700e+04
mean   2.858410e+06
std    2.167506e+06
min    1.057000e+04
25%    1.274182e+06
50%    2.524907e+06
75%    3.589464e+06
max    1.254173e+07

Checking for Missing Values

It’s important to identify any missing data in your DataFrame:

Code

print(cities_df.isnull().sum())

name           0
country        0
subcountry    28
geonameid      0
dtype: int64

3. Baisc Cleaning

Remove rows with missing data in subcountry using dropna() and the subset argument.

Code

cities_df = cities_df.dropna(subset=['subcountry'])

4. Basic Data Selection and Filtering

Selecting Columns

To select specific columns:

Code

# Select a single column
print(cities_df['name'].head())

# Select multiple columns
print(cities_df[['name', 'country', 'subcountry']].head())

0           les Escaldes
1       Andorra la Vella
2     Umm Al Quwain City
3    Ras Al Khaimah City
4             Zayed City
Name: name, dtype: object
                  name               country             subcountry
0         les Escaldes               Andorra     Escaldes-Engordany
1     Andorra la Vella               Andorra       Andorra la Vella
2   Umm Al Quwain City  United Arab Emirates  Imārat Umm al Qaywayn
3  Ras Al Khaimah City  United Arab Emirates        Raʼs al Khaymah
4           Zayed City  United Arab Emirates              Abu Dhabi

Filtering Rows

We can filter rows based on conditions:

Code

# Cities in the United States
us_cities = cities_df[cities_df['country'] == 'United States']
print(us_cities[['name', 'country']].head())

# Cities in California
california_cities = cities_df[(cities_df['country'] == 'United States') & (cities_df['subcountry'] == 'California')]
print(california_cities[['name', 'country', 'subcountry']].head())

             name        country
22451   Fort Hunt  United States
22452    Bessemer  United States
22453     Paducah  United States
22454  Birmingham  United States
22455     Cordova  United States
               name        country  subcountry
24818      Fillmore  United States  California
24867      Adelanto  United States  California
24868        Agoura  United States  California
24869  Agoura Hills  United States  California
24870       Alameda  United States  California

Combining Conditions

We can use logical operators to combine multiple conditions:

Code

# Cities in Canada that start with the letter 'T'
canadian_t_cities = cities_df[(cities_df['country'] == 'Canada') & (cities_df['name'].str.startswith('T'))]
print(canadian_t_cities[['name', 'country', 'subcountry']])

                        name country        subcountry
2813  Tam O'Shanter-Sullivan  Canada           Ontario
2814                Tecumseh  Canada           Ontario
2815                 Terrace  Canada  British Columbia
2816              Terrebonne  Canada            Quebec
2817             The Beaches  Canada           Ontario
2818                 Thorold  Canada           Ontario
2819             Thunder Bay  Canada           Ontario
2820                 Timmins  Canada           Ontario
2821                 Toronto  Canada           Ontario
2822          Trois-Rivières  Canada            Quebec
2823              Tsawwassen  Canada  British Columbia
2860          Thetford-Mines  Canada            Quebec
2872       Trinity-Bellwoods  Canada           Ontario
2901           Taylor-Massey  Canada           Ontario
2915        Thorncliffe Park  Canada           Ontario

5. Basic Sorting and Ranking

To sort the DataFrame based on one or more columns:

Code

# Sort cities alphabetically
sorted_cities = cities_df.sort_values('name')
print(sorted_cities[['name', 'country']].head())

# Sort cities by country, then by name
sorted_cities_by_country = cities_df.sort_values(['country', 'name'])
print(sorted_cities_by_country[['name', 'country']].head())

                   name      country
7018        'Ali Sabieh     Djibouti
17467    's-Gravenzande  Netherlands
17466  's-Hertogenbosch  Netherlands
8037           A Coruña        Spain
8036          A Estrada        Spain
        name      country
67   Andkhōy  Afghanistan
66  Asadābād  Afghanistan
29     Aībak  Afghanistan
63   Baghlān  Afghanistan
62     Balkh  Afghanistan

6. Basic Transformations

Creating New Columns

We can create new columns based on existing data:

Code

# Create a column for city name length
cities_df['name_length'] = cities_df['name'].str.len()

# Display the top 5 cities with the longest names
long_named_cities = cities_df.nlargest(5, 'name_length')
print(long_named_cities[['name', 'country', 'name_length']])

                                        name     country  name_length
5134   Sandaoling Lutiankuang Wuqi Nongchang       China           37
8032   Sant Pere, Santa Caterina i La Ribera       Spain           37
8392    Palikir - National Government Center  Micronesia           36
16421   Nanchital de Lázaro Cárdenas del Río      Mexico           36
2896     Dovercourt-Wallace Emerson-Junction      Canada           35

7-8: Basic Grouping and Aggregation

Grouping allows us to perform operations on subsets of the data:

Code

# Number of cities by country
cities_per_country = cities_df.groupby('country')['name'].count().sort_values(ascending=False)
print(cities_per_country.head())

# Number of subcountries (e.g., states, provinces) by country
subcountries_per_country = cities_df.groupby('country')['subcountry'].nunique().sort_values(ascending=False)
print(subcountries_per_country.head())

country
United States    3273
India            2480
China            1955
Brazil           1217
Germany          1117
Name: name, dtype: int64
country
Russia      83
Turkey      81
Thailand    75
Vietnam     62
Algeria     53
Name: subcountry, dtype: int64

Conclusion

In this session, we’ve covered the basics of working with pandas DataFrames using a world cities dataset, including:

Importing data
Exploring DataFrame properties
Selecting and filtering data
Sorting and ranking
Grouping and aggregation
Creating new columns

These skills form the foundation of data analysis with pandas and will be essential for upcoming exercises and projects. Remember, pandas has many more functions and methods that we haven’t covered here. Don’t hesitate to explore the pandas documentation for more advanced features!

Getting Started

Introduction

Learning Objectives

Setting Up

1. Basic Data Importing

2. Basic DataFrame Exploration

Viewing the Data

DataFrame Properties

Checking for Missing Values

3. Baisc Cleaning

4. Basic Data Selection and Filtering

Selecting Columns

Filtering Rows

Combining Conditions

5. Basic Sorting and Ranking

6. Basic Transformations

Creating New Columns

7-8: Basic Grouping and Aggregation

Conclusion

Resources