Day 6: Tasks & Activities

Eurovision Data Analysis Exercise

In this exercise, you’ll analyze Eurovision Song Contest data using pandas. You’ll practice grouping, joining, and date manipulation techniques to explore trends in the contest’s history while building your data analysis skills.

Setup

First, import the necessary libraries and load the dataset:

Code 
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

# Load the dataset
url = "https://github.com/Spijkervet/eurovision-dataset/releases/download/2020.0/contestants.csv"
eurovision_df = pd.read_csv(url)

Task 1: Data Exploration and Cleaning

  1. Display the first few rows of the dataset.

  2. Check the data types of each column.

  3. Identify and handle any missing values.

  4. Convert the β€˜year’ column to datetime type.

Task 2: Filtering and Transformation

  1. Create a new dataframe containing only data from 1990 onwards
Important

Use .copy() to make sure you create a new dataframe and not just a view.

  1. Calculate the difference between final points and semi-final points for each entry and make a histogram of these values using the builtin dataframe .hist() command.

Task 3: Sorting and Aggregation

  1. Find the top 10 countries with the most Eurovision appearances (use the entire dataset for this calculation)

  2. Calculate the average final points for each country across all years. Make a simple bar plot of these data.

Note

Use value_counts() for counting appearances and groupby() for calculating averages.

Task 4: Grouping and Analysis

  1. Determine the country with the highest average final points for each decade.
Hint: Grouping Years in Pandas

When working with time series data, it’s often useful to group years into larger intervals like decades, 5-year periods, etc. Here’s a general approach using pandas:

  1. For decades (10-year intervals):

    df['decade'] = df['year'].dt.year // 10 * 10

  2. For any N-year interval:

    N = 5  # Change this to your desired interval (e.g., 2, 5, 10, 20)
df['year_group'] = df['year'].dt.year // N * N

  3. For more specific date ranges:

    df['custom_group'] = pd.cut(df['year'], 
                            bins=[1990, 1995, 2000, 2005, 2010], 
                            labels=['1990-1994', '1995-1999', '2000-2004', '2005-2009'])

Remember: - // is integer division (rounds down) - Multiplying by the interval after division ensures the start year of each group

These methods create a new column that you can use with groupby() for aggregations across your chosen time intervals.

Task 5: Joining Data

  1. Read in a new dataframe that contains population data stored at this url:
Code 
population_url = 'https://bit.ly/euro_pop'
  1. Join this data with the Eurovision dataframe.
Warning

Ensure that country names match exactly between the two dataframes before joining.

  1. Calculate total entries per capita by country.

    Substeps:

    3a. Create a new dataframe containing the counts of entries for each county (use value_counts)

    3b. Merge the dataframe of counts of entries for each country with the population dataframe.

    3c. Calculate entries per million population (using entries per million to make the numbers easier to work with)

    3d. Sort the results by entries per capita

    3e. Print the top 10 values

Task 6: Time Series Analysis

  1. Plot the trend of maximum final points awarded over the years.

  2. Identify any significant changes in the scoring system based on this trend.

(This step simply requires visual interpretation of the plot, but perhaps you could explore if there are actual rules changes underlying observed patterns using google)

Task 7: Choose your own analysis!

Come up with your own analysis of the Eurovision data that reveals some pattern across the data or through time. Feel free to discuss your ideas with others; often this leads to new ideas or refinement of ones you are already working on.

Reflection

Now that you’ve completed the Eurovision data analysis exercise, it’s time to reflect on your experience. Add a new markdown cell to your notebook and answer the following questions:

  1. Grouping Operations: Which grouping tasks felt most natural to you? How did breaking complex operations into individual steps help your understanding?

  2. Data Joining: What challenges did you encounter when merging the Eurovision and population datasets? How did the step-by-step approach help?

  3. Date Manipulation: How comfortable did you feel working with the datetime operations (like creating decades)?

  4. Visualization with matplotlib: How did creating plots help you understand the patterns in the data?

  5. Real-world Applications: How could you apply these grouping, joining, and date manipulation skills to environmental data science problems?

  6. Most Interesting Discovery: What was the most surprising pattern you found in the Eurovision data?

Looking Ahead to Day 7

Tomorrow we’ll learn about advanced visualization with seaborn, which will allow you to create even more sophisticated plots to explore patterns in your data!

Note

Remember, breaking complex operations into clear steps makes your code more readable and helps you debug problems more easily - both crucial skills for data scientists!

Remember to document your code, explain your reasoning, and interpret the results of your analysis throughout the exercise.