Answer Key

How to use this key

Work the exercise first, then come here. The code below is one correct answer, not the only one. If your code looks different but produces the same numbers, you were right.

The written answers matter more than the code. You can already tell whether your code ran. What you cannot check on your own is whether you read the result correctly, and that is what the green Answer boxes are for. Compare your markdown cells against them.

⬅️ Back to the exercise

Setup

import pandas as pd

url = 'https://eds-217-essential-python.github.io/data/eurovision_contestants.csv'
eurovision = pd.read_csv(url)

Part 1: What is in this file, and what is missing from it

1. How many rows and columns? What are the column names, and what is the range of year?

print(eurovision.shape)
print(list(eurovision.columns))
print(eurovision['year'].min(), eurovision['year'].max())

(1603, 21)
['year', 'to_country_id', 'to_country', 'performer', 'song', 'place_contest', 'sf_num', 'running_final', 'running_sf', 'place_final', 'points_final', 'place_sf', 'points_sf', 'points_tele_final', 'points_jury_final', 'points_tele_sf', 'points_jury_sf', 'composers', 'lyricists', 'lyrics', 'youtube_url']
1956 2020

✅ Answer

1,603 rows and 21 columns, covering 1956 to 2020. The unit of a row is one country’s entry in one contest, not one contest, which is why there are 1,603 rows for 65 years.

That span of 65 years contains 65 distinct values of year, so every year in the range is represented in the file, including the one that did not happen. Of the 21 columns, most describe a result rather than a song: place and points in the final, place and points in the semi-final, and since 2016 the jury and televote components of each.

2. Run .isnull().sum(). Several columns are null on well over a thousand of the 1,603 rows. Pick two of them and, in a markdown cell, propose an explanation for each that has nothing to do with anybody making a mistake.

eurovision.isnull().sum()

year                    0
to_country_id           0
to_country              0
performer               0
song                    3
place_contest          41
sf_num               1046
running_final         282
running_sf           1081
place_final           283
points_final          295
place_sf             1081
points_sf            1081
points_tele_final    1499
points_jury_final    1499
points_tele_sf       1459
points_jury_sf       1459
composers              42
lyricists             673
lyrics                  0
youtube_url             0
dtype: int64

✅ Answer

Both of the worst columns are null because the thing they measure did not exist yet.

sf_num is null on 1,046 rows. The 557 rows where it is present all fall in 2004 or later, which is when the contest introduced a semi-final. Before 2004 there was no semi-final to number, so place_sf, points_sf and running_sf are null on 1,081 rows for the same reason.

points_tele_final and points_jury_final are null on 1,499 rows. The 104 rows where they are present are all from 2016 to 2019, which is when the contest began publishing the televote and the jury vote as two separate scores instead of one combined number. The earlier scores were recorded as one combined number and never split into two.

This is the distinction worth carrying forward: a null can mean “not recorded”, but here it means “not applicable”, and neither one is a mistake. Dropping every row with a null in this file would leave you with almost nothing.

3. points_final is the column the rest of this exercise depends on, and it is null on 295 rows. Group by year and count the non-null points_final values in each. Look at the last few years.

eurovision.groupby('year')['points_final'].count().tail(6)

year
2015    27
2016    26
2017    26
2018    26
2019    26
2020     0
Name: points_final, dtype: int64

✅ Answer

The recent years sit at 26 or 27 finalists each, and then 2020 reports 0. Note that .count() counts non-null values, which is exactly why it is the right verb here: 26 is the number of countries that reached the final and were scored, not the number that entered.

4. One year has a count of zero. Which one, and what happened? (You may look this up. It is the only fact in this exercise that is not in the file.)

✅ Answer

2020. The contest was cancelled in March 2020 because of the COVID-19 pandemic, the first cancellation in its history. The 41 entries are in the file with their performers and songs because they had been selected before the cancellation, but no country ever performed and no points were ever awarded, so points_final and place_contest are null on all 41 rows.

There is a second year at zero, and the .tail(6) hides it. Print the head of the same series:

eurovision.groupby('year')['points_final'].count().head(6)

year
1956     0
1957    10
1958    10
1959    11
1960    13
1961    16
Name: points_final, dtype: int64

1956 also has a count of zero, on 12 entries, and for a completely different reason: the scores of the first contest were never published. Only the winner was announced. The two zeros look identical and have different causes: in 2020 no contest took place, and in 1956 the contest took place and the numbers were not released. A count is a useful diagnostic, but it tells you the size of a hole and not its cause.

5. Build a table called contests that excludes that year, using the filter sentence and ending the line with .copy(). How many rows does it have?

contests = eurovision[eurovision['year'] != 2020].copy()
contests.shape

(1562, 21)

✅ Answer

1,562 rows, exactly 41 fewer than the 1,603 you started with, which confirms you removed the 2020 entries and nothing else. Checking the row count against a number you can predict is the cheapest way to know a filter did what you meant.

Note that 1956 is still in contests. Its 12 rows have null points_final, so every mean and count in the rest of the evening quietly ignores them, and that is the correct behaviour: pandas skips nulls in .mean() and .count() rather than treating them as zero.

Part 2: Decades

6. Add a decade column. Divide year by ten, convert the result to an integer, and multiply by ten. .astype(int) truncates toward zero, which is exactly what you want here. Then count the entries in each decade.

contests['decade'] = (contests['year'] / 10).astype(int) * 10

contests['decade'].value_counts().sort_index()

decade
1950     43
1960    163
1970    176
1980    200
1990    236
2000    336
2010    408
Name: count, dtype: int64

✅ Answer

The count rises from 43 entries in the 1950s to 408 in the 2010s, roughly a tenfold increase. Some of that is real growth in the number of competing countries, but part of it is an artifact of the bin: the 1950s decade holds only four contests, 1956 to 1959, while every later decade holds ten. A per-decade total is not comparable across decades unless you also know how many contests each decade contains.

7. Count the entries in each year from 2000 to 2007. Something changes abruptly. What, and in which year? In a markdown cell, say what a sudden jump in entries per contest would do to a comparison of average scores between the 1990s and the 2010s.

contests.groupby('year')['to_country'].count().loc[2000:2007]

year
2000    24
2001    23
2002    24
2003    26
2004    36
2005    39
2006    37
2007    42
Name: to_country, dtype: int64

✅ Answer

The field jumps from 26 entries in 2003 to 36 in 2004, and keeps growing to 42 by 2007. That is the year the semi-final was introduced, which is the same 2004 that first fills in sf_num. Once countries no longer all had to fit into one evening, many more could enter.

For a comparison of average scores this is a serious problem. Points are awarded by the competing countries, so a larger field means more voters, which means more points in circulation, which raises every score whether or not any song improved. A 2010s average and a 1990s average are measured on scales that differ by roughly a factor of two in the number of voters, so comparing them directly measures the size of the contest rather than the quality of the entries.

Part 3: The best country of each decade

8. Build a table with one row per decade, holding the country with the highest average points_final in that decade, how many entries that average came from, and the average itself.

decade_winners_list = []

for decade, decade_data in contests.groupby('decade'):
    country_avg = decade_data.groupby('to_country')['points_final'].agg(['count', 'mean'])
    country_avg = country_avg.reset_index()
    top_country = country_avg.sort_values('mean', ascending=False).head(1).copy()
    top_country['decade'] = decade
    decade_winners_list.append(top_country)

decade_winners = pd.concat(decade_winners_list, ignore_index=True)

decade_winners[['decade', 'to_country', 'count', 'mean']]

	decade	to_country	count	mean
0	1950	France	3	19.666667
1	1960	United Kingdom	10	23.100000
2	1970	United Kingdom	10	93.200000
3	1980	Ireland	9	99.222222
4	1990	Ireland	10	119.200000
5	2000	Serbia	2	214.000000
6	2010	Bulgaria	3	362.666667

9. Read the count column before you read the mean column. Three of the seven winners won on fewer than four entries. Name them, and say in one sentence why you would not put any of them in a headline.

✅ Answer

France, which tops the 1950s at 19.7 points from 3 entries; Serbia, which tops the 2000s at 214.0 from 2 finals; and Bulgaria, which tops the 2010s at 362.7 from 3. A mean of two or three numbers moves by tens of points if a single entry is different, so none of the three is stable enough to call a country the best of its decade.

The other four winners are the ones with enough behind them to report: the United Kingdom in the 1960s and 1970s, on 10 entries each, and Ireland in the 1980s and 1990s, on 9 and 10. Three of seven decade winners resting on three entries or fewer is itself a result about the method rather than about the countries, since taking the highest mean in a group is exactly the operation that selects for small counts.

10. The mean column rises from about 20 in the 1950s to about 360 in the 2010s. In a markdown cell, two or three sentences: is Europe getting better at writing songs? What else changed between 1956 and 2019 that would produce exactly this pattern, and what would you have to divide by to remove it?

✅ Answer

No. What changed is the scale on which points are awarded. Points come from the competing countries, and the field grew from about ten entrants in the 1950s to 42 in 2007, so the total number of points available grew with it. On top of that, the contest split the jury vote and the televote into two separate awards in 2016, which roughly doubled the points available again: the winning score was 152 in 1975 and 148 in 1995, but 365 in 2015 and 498 in 2019.

To remove the effect you would divide each score by the total points available in that year, or by the year’s winning score, turning an absolute total into a share. A mean of shares is comparable across decades. A mean of raw points measures how many countries were competing.

Part 4: The second table

11. Read the population file at https://eds-217-essential-python.github.io/data/eurovision_country_populations.csv. How many rows, and what are its columns?

pop_url = 'https://eds-217-essential-python.github.io/data/eurovision_country_populations.csv'
populations = pd.read_csv(pop_url)

print(populations.shape)
print(list(populations.columns))

(52, 2)
['country_name', 'population']

✅ Answer

52 rows and two columns, country_name and population. 52 is the number of countries that have ever entered, so this file has one row per country over the whole history of the contest, while contests has one row per country per year. The two tables are at different grains, and that is exactly why a merge is needed rather than a concatenation.

12. Build a table called modern holding only contests from 1990 onwards, then count the entries per country and turn that count into a two-column table using the Series-to-table move. Rename the columns to country and entries.

modern = contests[contests['year'] >= 1990].copy()

counts = modern['to_country'].value_counts().reset_index()
counts = counts.rename(columns={'to_country': 'country', 'count': 'entries'})

counts.head()

	country	entries
0	Spain	30
1	United Kingdom	30
2	France	30
3	Sweden	30
4	Ireland	29

✅ Answer

51 countries, and four of them are at the ceiling of 30: France, the United Kingdom, Spain and Sweden. The window 1990 to 2019 holds exactly 30 contests, and no country can enter a contest twice, so 30 means “entered every single year” and no number in this column can ever exceed it. Knowing the maximum a count is allowed to reach is worth as much as knowing the count.

13. Merge counts with populations. The key columns have different names, so you will need left_on= and right_on=. Do it twice, once with the default how= and once with how='right', and report both shapes.

entries = pd.merge(counts, populations, left_on='country', right_on='country_name')
check = pd.merge(counts, populations, left_on='country', right_on='country_name', how='right')

print(entries.shape)
print(check.shape)

(51, 4)
(52, 4)

✅ Answer

51 rows from the default inner merge and 52 from the right merge, a difference of exactly one row. The default how='inner' keeps only keys present in both tables, so it returns the 51 countries that have both a modern entry count and a population. how='right' keeps every row of the population file whether or not the left table matched, so the 52nd country arrives with nulls in its country and entries columns.

Running the merge twice is the diagnostic. The inner merge alone would have told you 51 and given you no way to know that anything was dropped.

14. One country is in the population file and not in your counts. Find it, name it, and then find every row for it in the full eurovision table. In a markdown cell, say what you learned and why the inner merge was right to drop it.

print(check[check['entries'].isnull()][['country_name', 'population']])
print(eurovision[eurovision['to_country'] == 'Morocco'][['year', 'performer', 'song', 'place_contest']])

   country_name  population
22      Morocco    24807462
     year       performer         song  place_contest
399  1980  Samira Bensaïd  Bitakat Hob           18.0

✅ Answer

Morocco, population 24,807,462, with exactly one row in the whole file: Samira Bensaïd singing “Bitakat Hob” in 1980, finishing 18th with 7 points. Morocco has never returned.

The inner merge was right because counts is a table of entries since 1990, and Morocco has none. It is absent from value_counts() output for the plain reason that value_counts() can only count values that appear, and a country with no rows in modern has no value to count. Dropping it keeps the table honest about what it contains: countries that compete in the modern contest.

The subtle part is what the null in the right merge means. It does not mean Morocco’s entry count is unknown, and it does not mean it is zero in general, since the 1980 entry is real. It means “this key had no match on the left”, and only you can decide which of those three readings applies. Had you filled it with zero and kept it, the entries-per-million table in the next question would carry a country that stopped competing ten years before the window opens.

15. Add an entries_per_million column to entries: the entry count divided by the population in millions. (1_000_000 is a perfectly good way to write a million in Python; the underscores are ignored.) Rank it and show the top eight and the bottom five.

entries['entries_per_million'] = entries['entries'] / (entries['population'] / 1_000_000)

ranked = entries.sort_values('entries_per_million', ascending=False)

print(ranked[['country', 'entries', 'population', 'entries_per_million']].head(8))
print(ranked[['country', 'entries', 'population', 'entries_per_million']].tail(5))

       country  entries  population  entries_per_million
42  San Marino       10       23186           431.294747
45     Andorra        6       54507           110.077605
8      Iceland       28      255866           109.432281
48      Monaco        3       29972           100.093421
5        Malta       29      352430            82.285844
10      Cyprus       28      766615            36.524201
41  Montenegro       11      615035            17.885161
19     Estonia       25     1570599            15.917494
                country  entries  population  entries_per_million
33              Ukraine       15    51944000             0.288773
35                Italy       14    56719240             0.246830
50  Serbia & Montenegro        2     9791475             0.204259
24               Russia       22   148005704             0.148643
49           Yugoslavia        3    23818000             0.125955

16. The top of that ranking is San Marino, Andorra, Iceland, Monaco and Malta, and the bottom is Russia, Italy and Ukraine. In a markdown cell, three or four sentences: is entries per million people a meaningful quantity? What is it actually measuring, and what would you have to know about how the file’s population numbers were collected before you would publish this table?

✅ Answer

It is arithmetically correct and close to meaningless, because it is almost entirely a ranking of small populations. The numerator cannot exceed 30, so it varies by a factor of about ten across the table, while the denominator varies by a factor of more than six thousand, from San Marino at 23,186 to Russia at 148,005,704. Whichever country is smallest will top this ranking almost regardless of how often it competes: San Marino reaches 431.3 entries per million on only 10 entries, while Russia’s 22 entries give 0.149.

What it actually measures is roughly one divided by population, presented as a rate. It says nothing about enthusiasm per person, because a country sends one entry no matter how many people live in it. The quantity would be meaningful only if the numerator could scale with population, and here by rule it cannot.

Before publishing it you would need to know what year each population figure is from, and the file does not say. Iceland is listed at 255,866 and the United Kingdom at 57,247,586, both of which are roughly late-1980s figures, and the file still carries rows for Yugoslavia (23,818,000) and Serbia & Montenegro (9,791,475), states that ceased to exist in 1992 and 2006. So the denominators come from mixed and unstated years, and some of them describe countries that did not exist for most of the 1990 to 2019 window the numerator covers.

Part 5: The whole contest in one grid

17. Build a wide table with to_country down the rows, decade across the columns, and the mean points_final in the cells. What shape is it?

grid = contests.pivot_table(index='to_country', columns='decade', values='points_final')

grid.shape

(51, 7)

✅ Answer

51 rows by 7 columns, one row per country and one column per decade. The seven columns are the seven decades, which you already know from question 6.

The 51 is worth a second look. There are 52 distinct countries in contests, so one is absent from this grid entirely. It is Andorra, whose six entries between 2004 and 2009 never reached a final, so points_final is null on every one of them. pivot_table drops an all-null row by default, and it does so silently. A pivot table with fewer rows than your data has groups is reporting that some group had nothing to aggregate.

18. Build the same table again with aggfunc='count', and look at both for four countries that have competed throughout: Ireland, Sweden, the United Kingdom and Norway.

grid_n = contests.pivot_table(index='to_country', columns='decade',
                              values='points_final', aggfunc='count')

print(grid.loc[['Ireland', 'Sweden', 'United Kingdom', 'Norway']].round(1))
print(grid_n.loc[['Ireland', 'Sweden', 'United Kingdom', 'Norway']])

decade          1950  1960  1970  1980   1990   2000   2010
to_country                                                 
Ireland          NaN  15.0  68.1  99.2  119.2   42.7   66.2
Sweden           7.0   6.9  48.2  82.8   76.8   86.8  268.3
United Kingdom  11.0  23.1  93.2  94.5  108.4   44.5   42.2
Norway           NaN   5.0  35.9  48.7   61.7  114.5  132.0
decade          1950  1960  1970  1980  1990  2000  2010
to_country                                              
Ireland          NaN   5.0  10.0   9.0  10.0   6.0   5.0
Sweden           2.0   9.0   8.0  10.0  10.0  10.0   9.0
United Kingdom   2.0  10.0  10.0  10.0  10.0  10.0  10.0
Norway           NaN  10.0   9.0  10.0  10.0   8.0   8.0

✅ Answer

The counts grid counts finals reached, not entries submitted, and the difference shows up after 2004. Ireland’s counts run 5, 10, 9, 10 through the 1990s and then fall to 6 and 5, but Ireland entered nine times in the 2000s and ten times in the 2010s. The missing entries were eliminated in the semi-final and so have a null points_final. The United Kingdom shows 10 in every full decade, because the largest financial contributors are seeded straight into the final, a rule that is not in the file.

That changes how you read the means beside them. Ireland’s 2000s mean of 42.7 is an average over the six years it qualified, so it excludes its three worst years by construction, and it is still the country’s weakest decade. Sweden’s 2010s mean of 268.3 comes from 9 finals out of 10 entries. An average computed only over the occasions when a country did well enough to be scored is biased upward, and the counts grid is how you see that it is happening.

19. In a markdown cell: the empty cells in the means grid are not all the same kind of empty. Name two different reasons a country might have no number in a given decade, and say how you would tell them apart using the counts grid.

✅ Answer

Reason one: the country did not compete in that decade at all. Ireland in the 1950s and Norway in the 1950s are examples. There are no rows to aggregate, so both grids are empty in that cell.

Reason two: the country competed and never reached a final. Slovakia in the 2000s and again in the 2010s, and the Czech Republic, Monaco, Montenegro and San Marino in the 2000s, all entered and were all eliminated in every semi-final of the decade. Every points_final is null, so the mean is empty, but there were rows to count.

The counts grid separates the two: an empty cell there means no entries, while a 0.0 means entries but no finals. Six cells in the counts grid hold 0.0 while the means grid is empty, and they are the second kind of empty. Andorra is the third and most easily missed case, since it has 0.0 in the counts grid for the 2000s but no row at all in the means grid.

The reading matters because the two look identical in a plot or a report. One says a country was not there. The other says it was there and never got far enough to be scored, which is a result, not an absence.

Part 6: Write it up

There is no single right answer. A strong response cites at least three numbers you computed, names at least two distinct reasons the question is harder than it sounds, and ends with a specific table or number you would actually send. Here is one that would earn full marks.

✅ A model answer

Partly, and only for a period that ended twenty years ago. Ireland has the highest average points_final of any country in both the 1980s and the 1990s, at 99.2 points from 9 finals and 119.2 from 10, and those are the two most trustworthy rows in the decade-winners table because every other winner rests on ten entries or fewer than four. On that evidence Ireland was the strongest country of two consecutive decades, which no other country in the file achieved.

The claim stops being supportable as soon as it is extended to “in history”, for two reasons. The first is that points are not comparable across eras. The field grew from 26 entries in 2003 to 36 in 2004 when the semi-final was introduced, and the winning score rose from 148 in 1995 to 498 in 2019, so a raw mean rewards recency rather than quality. The second is that the counts grid shows Ireland reaching only 6 finals in the 2000s and 5 in the 2010s, and its mean in those decades, 42.7 and 66.2, is an average over its better years alone, since the years it failed to qualify contribute nothing. Sweden’s 268.3 in the 2010s is on the same inflated scale and cannot be set against Ireland’s 119.2 from the 1990s.

What the data cannot tell the editor is who is best overall, because no common scale exists. I would send the two pivot tables together, means beside counts, for the countries that competed throughout, and the sentence that goes with them: Ireland dominated the 1980s and 1990s, and the question of the best country in history is not answerable from points.

Where the marks are

If you compare your notebook against this key, look for these four things before anything else.

1. You ran both merges and compared 51 to 52. The single dropped row was the whole point of question 13, and an inner merge on its own gives you no signal that anything went missing.
2. You read the count column before the mean column, every time. Three of the seven decade winners rest on fewer than four entries, and in Part 5 the counts grid is what tells you that Ireland’s later means were computed over qualifying years only.
3. You can distinguish the two kinds of empty cell. A null in the means grid with 0.0 beside it in the counts grid is a country that competed and never made a final. A null in both is a country that was not there.
4. You never called .fillna(0). Doing it to 2020 would have given 41 real scores of zero, and doing it to the merge would have put a country that last competed in 1980 into a table about the modern contest.

⬅️ Back to the exercise