Tasks & Activities

🌊 Microplastics: From Raw File to Real Numbers

A panda, taking a moment before the hard part.

NOAA’s National Centers for Environmental Information maintain an archive of every marine microplastics measurement they can find: 16,245 water samples, pooled from thirty-seven separate studies, taken between 1972 and 2022 by research vessels, citizen scientists, and undergraduates with nets.

Nobody designed this dataset. It was assembled. That is true of most environmental data you will ever be handed, and it is why today mattered.

You have already met this file twice today. This evening you take it the whole way: from the raw download to a defensible answer about whether the Atlantic and the Pacific are different.

Reference

NOAA National Centers for Environmental Information, Marine Microplastics database. https://www.ncei.noaa.gov/products/microplastics

Today’s three sentences

Everything below is built from these, plus the two you learned yesterday. You should not need to look them up more than once.

df['new'] = expression                     # the derived-column sentence
df.dropna(subset=['col'])                  # the missing-data sentence, treat half
df['col'] = df['col'].fillna(value)
df['col'] = df['col'].str.strip()          # the string-cleaning sentence

And the payoff from this morning:

df['new'] = df['col'].apply(my_function)   # your own verb, run down a column

Setup

  1. Create a new notebook named EOD_Day4_Microplastics.ipynb.

  2. Add a title cell:

# Day 4 EOD: Microplastics, From Raw File to Real Numbers

Date: 09/03/2026
  1. Import what you need and read the data. The Date column is text, and pandas can convert it while it reads if you tell it exactly what the text looks like.
🧭 Field Note: reading dates, and asking one for its year

read_csv() takes a parse_dates= argument naming the columns that hold dates, and a date_format= argument describing how they are written. Here is the whole line. Run it as written:

df = pd.read_csv(url, parse_dates=['Date'], date_format='%m/%d/%Y %I:%M:%S %p')

The format string is a small language of its own. %m/%d/%Y is month, day, four-digit year; %I:%M:%S is a twelve-hour clock; and %p is the AM or PM that goes with it. You are not expected to be able to write one of these yet.

Once a column is a real date, you reach the pieces of it through .dt. You will need exactly one of these tonight, in task 13, and it also comes supplied:

samples['year'] = samples['Date'].dt.year

Dates get a full session on Day 6. By the middle of next week you will be writing both of these yourself. Tonight, copy them.

πŸ“– Cheatsheet: reading CSV files Β· Cheatsheet: time series

Part 1: Get your bearings

Answer each of these in a markdown cell underneath the code that produced it.

  1. How many rows and columns? Display the first few rows.
  2. Run .info(). Which columns are object, and which of those should be something else?
  3. Run .isnull().sum(). Five columns have gaps. Name them and their counts.
  4. Oceans is missing on 271 rows and Measurement on 5,792. Are those the same rows? Find out, and say how you know.

Part 2: Decide what to keep

This is the part that takes judgement rather than syntax. Every task here removes rows, so print .shape after every one of them and record the number in a markdown cell.

  1. Use .dropna() with subset= to keep only the rows that carry a Measurement. How many rows are left, and what fraction of the file is that?

  2. Compare df['Unit'].value_counts() now with what it was before task 5. What did that one line cost you, and which sampling programme did it belong to? (Check Sampling Method and Organization on the rows you removed, if you want to be sure.)

  3. Fill the gaps in Regions and SubRegions with 'Unspecified', one column at a time, assigning each result back. Confirm with .isnull().sum() that only Keywords is still missing anything.

  4. In a markdown cell: task 5 removed rows and task 7 kept them. Both were the right call. Explain in two or three sentences what made them different.

  5. Check for duplicated rows. Do it twice: once bare, and once with subset= naming the columns you think define a single observation. Report both numbers and say which one you believe.

Part 3: Make it comparable

  1. Filter to the samples reported in pieces/m3, end the line with .copy(), and call the result samples. How many rows?

  2. Accession Number is a label, not a quantity. Convert it to text with .astype(str) and assign it back. Confirm with .dtype.

  3. The Oceans values all end in the word β€œOcean”. Use .str.replace() to make a new column called ocean holding the short name, and check it with .value_counts().

  4. Add a column called year holding the year each sample was taken, using the supplied line from the setup Field Note. Then run .value_counts() on year. Which single year contributed the most samples? In a markdown cell, say whether this archive is spread evenly across its fifty years, and what that means for anyone who wants to use it to describe a trend.

Part 4: The derived columns

  1. Run .describe() on Measurement. Write down the median and the maximum. How many orders of magnitude separate them?

  2. Draw a histogram of Measurement.

🧭 Field Note: a histogram in one line

On Day 1 you drew a line with plt.plot(series). A histogram is the same shape of call:

plt.hist(samples['Measurement'])
plt.xlabel('pieces per cubic metre')
plt.show()

plt.hist() sorts the values into bins and draws a bar for each. Plotting gets two full days next week, where you will learn to control bins, axes, and everything else. Tonight you need it only to see a shape.

πŸ“– Cheatsheet: matplotlib

  1. That plot is useless, and it is useless for a reason you can state in one sentence. State it in a markdown cell.

  2. Filter samples to the rows where Measurement is greater than zero, end the line with .copy(), and call it positive. How many samples reported exactly zero pieces? Is zero a measurement or a missing value? Defend your answer in one sentence.

  3. Add a column to positive called log10_measurement holding np.log10() of the measurement. Run .describe() on it, then draw the histogram again.

  4. In a markdown cell, describe the shape you can now see and could not see before. Name one thing about the data that this plot tells you and .describe() did not.

Part 5: A classifier of your own

The archive ships a column called Density Class with values like Low and Very High. Somebody decided those. Tonight you decide your own, and then find out whether you agree.

  1. Write a function called density_class that takes a measurement in pieces per cubic metre and returns:

    • 'low' below 0.01
    • 'medium' from 0.01 up to 1
    • 'high' at 1 and above

    Test it on 0.001, 0.5, and 50 before you go near the DataFrame.

  2. Use .apply() to run your function down positive['Measurement'], store the result in a column called my_class, and report the counts.

  3. Now compare with NOAA’s. Filter positive to the rows NOAA labelled 'Medium', and run .value_counts() on my_class for just those rows. Do the same for NOAA’s 'Very Low'.

  4. Something is wrong, and it is not your function. In a markdown cell, explain what NOAA’s Density Class must actually mean, given that samples they called Very Low fall in your high bin. Then say whether you would use their column in an analysis, and why.

A hint, if you want one

Look at Density Range alongside Density Class. Run positive[['Density Class', 'Density Range']].drop_duplicates() and read the result carefully.

Part 6: The question

  1. Build two tables with the filter sentence: the Atlantic samples and the Pacific samples, from positive. Report the row count of each.

  2. Run .describe() on Measurement for each. Compare the two medians (the 50% row), and then compare the two means. They tell different stories. Which one would you put in a report, and why?

  3. Compare the mean of log10_measurement for each instead. In a markdown cell, say what that comparison is doing that the comparison of raw means was not.

  4. Before you conclude anything: how many Pacific samples are there, out of how many total? Look back at task 12. In one sentence, say what you would need to know about how the samples were collected before you would publish a comparison between these two oceans.

Part 7: Write it up

In a single markdown cell of 200 to 300 words, answer this:

A journalist emails you: β€œIs there more plastic in the Pacific than the Atlantic? I heard about the garbage patch.” You have this dataset and one evening. What do you tell them?

Your answer must cite at least three specific numbers you computed tonight, must name at least one cleaning decision you made and what it cost, and must include at least one thing this dataset cannot tell them. Complete sentences, no bullet fragments.

The grammar minute

Look back through your notebook. Every code cell you wrote tonight was one of a handful of sentences:

df.dropna(subset=['col'])                  # keep rows that have what you need
df['col'] = df['col'].fillna(value)        # or keep the row and fill the hole
df['col'] = df['col'].astype(str)          # tell pandas what a column is
df['col'] = df['col'].str.replace(a, b)    # tidy text
df['new'] = expression                     # add a column that was not in the file
df['new'] = df['col'].apply(my_function)   # add a column only you know how to compute

Six sentences, and two of them you also used yesterday and the day before.

Notice which one did the most work tonight. It was not any of the cleaning sentences. It was df['new'] = expression, four times: ocean, year, log10_measurement, my_class. The answer to the journalist’s question was in none of the twenty-two columns you downloaded. You made the columns that held it.

Wrap-up

Before you close your notebook, check that:

  • every step that removed rows has the new row count printed under it
  • every cleaning decision has a markdown sentence saying why, not just what
  • your Part 7 answer names a number that came from a column you derived
  • you can say out loud what subset= does, and why .duplicated() without it returned zero
  • your notebook reads top to bottom as a document, not as a pile of cells

Resources