Interactive Session

A panda, investigating. MidJourney 5

You have a DataFrame. Before you compute anything with it, you need to know what you’re holding: how big it is, what the columns are, what type each one is, what’s missing, and what the values look like. That’s this session.

This is the step people skip, and skipping it is how you end up confidently reporting an average of two different units. We’re going to do it properly, and by the end of the hour you’ll have found two real problems in a real dataset.

By the end of this session you will be able to:

size up a DataFrame with .shape, .columns, and .dtypes
run a health check with .info() and .isnull().sum()
summarize numbers with .describe(), and know when not to trust it
count categories with .value_counts()
rename columns using a Python dictionary

Getting Started

Create a new notebook from the Command Palette (Create: New Jupyter Notebook), and confirm its kernel reads eds217_2026.
Save your notebook (Ctrl + S, or Cmd + S on macOS) as: Session_2B_Exploring_Data.ipynb
Add a title cell (Markdown), updating the date to today:

# Day 2: Session 2B - Exploring a DataFrame

[Session Webpage](https://eds-217-essential-python.github.io/course-materials/interactive-sessions/2b_exploring_data.html)

Date: 09/01/2026

Read in the same Goleta air quality data from the last session:

Code

import pandas as pd

url = 'https://eds-217-essential-python.github.io/data/openaq_goleta_measurments.csv'
goleta = pd.read_csv(url)

Look at it

Always start by looking. .head() shows the first five rows, .tail() the last five:

Code

goleta.head()

	location_id	location_name	parameter	value	unit	datetimeUtc	datetimeLocal	timezone	latitude	longitude	country_iso	isMobile	isMonitor	owner_name	provider
0	1186	Goleta	o3	0.025	ppm	2024-07-12T01:00:00+00:00	2024-07-11T18:00:00-07:00	America/Los_Angeles	34.445301	-119.827797	NaN	NaN	NaN	Unknown Governmental Organization	AirNow
1	1186	Goleta	o3	0.028	ppm	2024-07-12T02:00:00+00:00	2024-07-11T19:00:00-07:00	America/Los_Angeles	34.445301	-119.827797	NaN	NaN	NaN	Unknown Governmental Organization	AirNow
2	1186	Goleta	o3	0.029	ppm	2024-07-12T03:00:00+00:00	2024-07-11T20:00:00-07:00	America/Los_Angeles	34.445301	-119.827797	NaN	NaN	NaN	Unknown Governmental Organization	AirNow
3	1186	Goleta	o3	0.027	ppm	2024-07-12T04:00:00+00:00	2024-07-11T21:00:00-07:00	America/Los_Angeles	34.445301	-119.827797	NaN	NaN	NaN	Unknown Governmental Organization	AirNow
4	1186	Goleta	o3	0.026	ppm	2024-07-12T05:00:00+00:00	2024-07-11T22:00:00-07:00	America/Los_Angeles	34.445301	-119.827797	NaN	NaN	NaN	Unknown Governmental Organization	AirNow

Code

goleta.tail()

	location_id	location_name	parameter	value	unit	datetimeUtc	datetimeLocal	timezone	latitude	longitude	country_iso	isMobile	isMonitor	owner_name	provider
1957	1186	Goleta	pm25	10.0	µg/m³	2024-08-11T20:00:00+00:00	2024-08-11T13:00:00-07:00	America/Los_Angeles	34.445301	-119.827797	NaN	NaN	NaN	Unknown Governmental Organization	AirNow
1958	1186	Goleta	pm25	8.0	µg/m³	2024-08-11T21:00:00+00:00	2024-08-11T14:00:00-07:00	America/Los_Angeles	34.445301	-119.827797	NaN	NaN	NaN	Unknown Governmental Organization	AirNow
1959	1186	Goleta	pm25	8.0	µg/m³	2024-08-11T22:00:00+00:00	2024-08-11T15:00:00-07:00	America/Los_Angeles	34.445301	-119.827797	NaN	NaN	NaN	Unknown Governmental Organization	AirNow
1960	1186	Goleta	pm25	9.0	µg/m³	2024-08-11T23:00:00+00:00	2024-08-11T16:00:00-07:00	America/Los_Angeles	34.445301	-119.827797	NaN	NaN	NaN	Unknown Governmental Organization	AirNow
1961	1186	Goleta	pm25	5.0	µg/m³	2024-08-12T00:00:00+00:00	2024-08-11T17:00:00-07:00	America/Los_Angeles	34.445301	-119.827797	NaN	NaN	NaN	Unknown Governmental Organization	AirNow

Both take a number if you want more or fewer:

Code

goleta.head(3)

	location_id	location_name	parameter	value	unit	datetimeUtc	datetimeLocal	timezone	latitude	longitude	country_iso	isMobile	isMonitor	owner_name	provider
0	1186	Goleta	o3	0.025	ppm	2024-07-12T01:00:00+00:00	2024-07-11T18:00:00-07:00	America/Los_Angeles	34.445301	-119.827797	NaN	NaN	NaN	Unknown Governmental Organization	AirNow
1	1186	Goleta	o3	0.028	ppm	2024-07-12T02:00:00+00:00	2024-07-11T19:00:00-07:00	America/Los_Angeles	34.445301	-119.827797	NaN	NaN	NaN	Unknown Governmental Organization	AirNow
2	1186	Goleta	o3	0.029	ppm	2024-07-12T03:00:00+00:00	2024-07-11T20:00:00-07:00	America/Los_Angeles	34.445301	-119.827797	NaN	NaN	NaN	Unknown Governmental Organization	AirNow

How big is it?

.shape gives you (rows, columns):

Code

goleta.shape

(1962, 15)

It’s a pair of numbers, so you can pull either one out by position, exactly like the lists you met in the last session:

Code

print("rows:", goleta.shape[0])
print("columns:", goleta.shape[1])

rows: 1962
columns: 15

What are the columns, and what type is each?

Code

goleta.columns.tolist()

['location_id',
 'location_name',
 'parameter',
 'value',
 'unit',
 'datetimeUtc',
 'datetimeLocal',
 'timezone',
 'latitude',
 'longitude',
 'country_iso',
 'isMobile',
 'isMonitor',
 'owner_name',
 'provider']

Code

goleta.dtypes

location_id        int64
location_name     object
parameter         object
value            float64
unit              object
datetimeUtc       object
datetimeLocal     object
timezone          object
latitude         float64
longitude        float64
country_iso      float64
isMobile         float64
isMonitor        float64
owner_name        object
provider          object
dtype: object

object almost always means text. float64 and int64 are numbers. Types matter because they decide what you’re allowed to do: you can average a float64, and you cannot average an object.

✏️ Test your knowledge

How many columns in goleta hold text? Use .dtypes to answer, and say how you counted.

The health check: `.info()`

.info() is the single most useful line in this session. It combines the shape, the column names, the types, and, critically, how many non-null values each column has:

Code

goleta.info()

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 1962 entries, 0 to 1961
Data columns (total 15 columns):
 #   Column         Non-Null Count  Dtype  
---  ------         --------------  -----  
 0   location_id    1962 non-null   int64  
 1   location_name  1962 non-null   object 
 2   parameter      1962 non-null   object 
 3   value          1962 non-null   float64
 4   unit           1962 non-null   object 
 5   datetimeUtc    1962 non-null   object 
 6   datetimeLocal  1962 non-null   object 
 7   timezone       1962 non-null   object 
 8   latitude       1962 non-null   float64
 9   longitude      1962 non-null   float64
 10  country_iso    0 non-null      float64
 11  isMobile       0 non-null      float64
 12  isMonitor      0 non-null      float64
 13  owner_name     1962 non-null   object 
 14  provider       1962 non-null   object 
dtypes: float64(6), int64(1), object(8)
memory usage: 230.1+ KB

Read the Non-Null Count column carefully. Most columns have 1962 non-null values, one for every row. Three of them have 0.

country_iso, isMobile, and isMonitor are completely empty. They exist as column headers in the file, and there is not one single value in any of them.

Important

This is why we explore. Those three columns look perfectly real in head(), which shows NaN, and you might not think twice. .info() tells you they are empty from top to bottom.

You can ask the same question directly. .isnull() marks every missing value, and .sum() counts them per column:

Code

goleta.isnull().sum()

location_id         0
location_name       0
parameter           0
value               0
unit                0
datetimeUtc         0
datetimeLocal       0
timezone            0
latitude            0
longitude           0
country_iso      1962
isMobile         1962
isMonitor        1962
owner_name          0
provider            0
dtype: int64

We won’t fix this today. Deciding what to do about missing data is Day 4’s job. Today we only need to know it’s there.

✏️ Test your knowledge

Read the Santa Barbara file into santa_barbara and run .info() on it. Does it have the same three empty columns?

Summarizing numbers with `.describe()`

.describe() gives you count, mean, standard deviation, min, max, and quartiles for every numeric column:

Code

goleta.describe()

	location_id	value	latitude	longitude	country_iso	isMobile	isMonitor
count	1962.0	1962.000000	1.962000e+03	1.962000e+03	0.0	0.0	0.0
mean	1186.0	6.378173	3.444530e+01	-1.198278e+02	NaN	NaN	NaN
std	0.0	7.313763	2.132172e-14	2.842896e-14	NaN	NaN	NaN
min	1186.0	-4.000000	3.444530e+01	-1.198278e+02	NaN	NaN	NaN
25%	1186.0	0.027000	3.444530e+01	-1.198278e+02	NaN	NaN	NaN
50%	1186.0	5.000000	3.444530e+01	-1.198278e+02	NaN	NaN	NaN
75%	1186.0	10.000000	3.444530e+01	-1.198278e+02	NaN	NaN	NaN
max	1186.0	40.000000	3.444530e+01	-1.198278e+02	NaN	NaN	NaN

Look at the value column. The mean is about 6.4 and the minimum is −4.

Two things should bother you.

First, a negative concentration. You cannot have less than zero of a pollutant. That’s either an instrument artifact or a sentinel value, and it’s the kind of thing you flag now and decide about later.

Second, and worse, that mean is meaningless. Look at what’s in the unit column:

Code

goleta['unit'].value_counts()

unit
µg/m³    1251
ppm       711
Name: count, dtype: int64

Some rows are in micrograms per cubic metre and some are in parts per million. The mean of 6.4 was computed by averaging those together. It is a number, it is not a fact.

.describe() will happily lie to you

.describe() averages whatever is in the column. It has no idea that this table stacks different pollutants, in different units, in a single value column. Data shaped like this, with one row per measurement and a column saying which kind, is extremely common in environmental data. Getting a real answer out of it means grouping first, which is Day 5.

Counting categories with `.value_counts()`

.describe() is for numbers. For text columns, the question is usually “what values appear, and how often?” That’s .value_counts():

Code

goleta['parameter'].value_counts()

parameter
pm25    734
o3      711
pm10    517
Name: count, dtype: int64

Three pollutants: pm25, o3, and pm10. Now the unit result above makes sense, since ozone is reported in ppm and particulates in µg/m³.

.value_counts() is also how you spot a column that carries no information:

Code

goleta['location_name'].value_counts()

location_name
Goleta    1962
Name: count, dtype: int64

Every row says Goleta. That column is a constant. It’s not wrong, and it will matter later when we stack several stations together, but within this file it tells you nothing.

Add .head(n) when a column has many distinct values:

Code

goleta['datetimeLocal'].value_counts().head()

datetimeLocal
2024-07-27T21:00:00-07:00    3
2024-07-28T13:00:00-07:00    3
2024-07-27T01:00:00-07:00    3
2024-07-27T00:00:00-07:00    3
2024-07-26T23:00:00-07:00    3
Name: count, dtype: int64

✏️ Test your knowledge

Use .value_counts() on the provider and owner_name columns. Are either of them constants, like location_name?

Renaming columns with a dictionary

datetimeUtc and datetimeLocal are in camelCase, while everything else uses lowercase with underscores. Mixed naming styles cause typos. Let’s fix them.

To do that you need Python’s other essential container: the dictionary.

A dictionary stores pairs. Each pair has a key and a value, written key: value, and the whole thing goes in curly braces:

Code

new_names = {
    'datetimeUtc': 'datetime_utc',
    'datetimeLocal': 'datetime_local',
}
new_names

{'datetimeUtc': 'datetime_utc', 'datetimeLocal': 'datetime_local'}

You look up a value by its key, using square brackets:

Code

new_names['datetimeUtc']

'datetime_utc'

Where a list is ordered and you index it by position, a dictionary is looked up by name. That makes it exactly the right shape for “rename this to that”:

Code

goleta_renamed = goleta.rename(columns=new_names)
goleta_renamed.columns.tolist()

['location_id',
 'location_name',
 'parameter',
 'value',
 'unit',
 'datetime_utc',
 'datetime_local',
 'timezone',
 'latitude',
 'longitude',
 'country_iso',
 'isMobile',
 'isMonitor',
 'owner_name',
 'provider']

You can write the dictionary inline if it’s short:

Code

goleta.rename(columns={'value': 'concentration'}).columns.tolist()

['location_id',
 'location_name',
 'parameter',
 'concentration',
 'unit',
 'datetimeUtc',
 'datetimeLocal',
 'timezone',
 'latitude',
 'longitude',
 'country_iso',
 'isMobile',
 'isMonitor',
 'owner_name',
 'provider']

Note

🐍 .rename() returns a new DataFrame and leaves the original alone. If you want to keep the change, assign it to a name, as we did with goleta_renamed. Almost every pandas method behaves this way, and forgetting it is the single most common source of “why didn’t my change stick?”

✏️ Test your knowledge

Build a dictionary that renames value to concentration and parameter to pollutant, apply it, and confirm with .columns.tolist() that both changes took effect.

The exploration ritual

Five lines, every time you open a new dataset:

df.shape          # how big
df.info()         # types and what's missing
df.head()         # what does a row look like
df.describe()     # what do the numbers look like
df['col'].value_counts()   # what's in the text columns

Run them before you compute anything. You will catch problems in ninety seconds that would otherwise surface in your final figure.

Key points

.shape is (rows, columns). .columns and .dtypes say what’s in the table.
.info() is the health check. Read the non-null counts; that’s where empty columns hide.
.isnull().sum() counts missing values per column.
.describe() summarizes numeric columns, and will average across units without complaint. Check what a column actually contains before you trust its mean.
.value_counts() counts categories, and reveals constants and unexpected values.
A dictionary maps keys to values, written {key: value}, looked up by name.
.rename(columns={...}) returns a new DataFrame. Assign it to keep it.

Getting Started

Look at it

How big is it?

What are the columns, and what type is each?

The health check: .info()

Summarizing numbers with .describe()

Counting categories with .value_counts()

Renaming columns with a dictionary

The exploration ritual

Key points

Resources

The health check: `.info()`

Summarizing numbers with `.describe()`

Counting categories with `.value_counts()`