Survey of data scientists, CrowdFlower, 2016.
Yesterday we interrogated a table that behaved itself. Today we will meet one that does not, and will spend the whole day learning how to wrangle it into something useful.
Day 4 has three new code patterns. First weβll learn a missing-data pattern that decides what to do about gaps. Then weβll tackle the string-cleaning pattern that tidies up text heterogeneities so that comparisons can work. Finally, we will use the derived-column pattern to add a new column that holds your answer (because what you actually need is almost never what is contained in the raw data!). We also get formally introduced to functions this morning after seeing their utility during yesterdayβs EOD activity.
Class materials
| Session | Session 1 | Session 2 |
|---|---|---|
| day 4 / morning | π§Ό Missing, Duplicated, Miscast | βοΈ Write It Once, Name It, Use It |
| day 4 / afternoon | β The Derived-Column Pattern | π From Field Sheet to DataFrame |
End-of-day practice
This afternoon we will analyze NOAAβs marine microplastics archive, going from the raw download to a defensible answer about whether the patterns of microplastics in the Atlantic Ocean and the Pacific Ocean are different.