CI7320 Databases & Data Management Assignment 2 | KU

Part A [75 marks] 

The data set provided for this assignment contains punctuality statistics for selected UK airports. It includes 24 files, each representing a month's data, covering the years 2023 and 2024. Airport IATA codes can be found at iata.org. 

For this assignment, you will be reporting on how the data could be used for research using a data warehouse and Tableau. 

The report should include the following: 

Design a data warehouse using a star schema. You must justify your design decisions.

Write the CREATE table statements for the tables in your star schema (include all primary and foreign keys). 

Discuss the steps you took in creating and populating the database. This should include the steps you took in preparing the data and the transformation tasks performed. Include screenshots of your populated tables. 

Create 4 visualisations using Tableau. For each visualisation, you should include the following: 

• Aim of the visualisation
• Bullet points covering the data preparation and steps you followed in Tableau to produce the graph
• The effectiveness and presentation of the graph
• Key findings from the visualisation 

Part B [25 marks] 

There are two options to select from; 

Option 1

Compose a brief reflective report (500-1000 words) that demonstrates your understanding of how AWS Glue facilitates automated batch data ingestion. Specifically, address the following:

1. Core Glue Components: 

• Briefly describe the key components of AWS Glue (Crawlers, Data Catalogue, ETL Jobs, Triggers). 
• Explain the role each component plays in a typical batch ingestion workflow.

2. Automating Batch Ingestion: 

• Illustrate how these components work together to automate the process of extracting, transforming, and loading batch data.
• Focus on a simple, illustrative scenario: for example, ingesting daily CSV files from an S3 bucket.

3. Reflective Insights:

• Provide a short reflection on the benefits and potential limitations of using AWS Glue for batch ingestion.
• Consider aspects such as ease of use, scalability, and cost-effectiveness.
• Include a simple diagram or flowchart to visualise the glue workflow.

Option 2:

Complete the AWS Academy Data Engineering [AWS-KU course ref' 114897] course's Module 7 lab activity - "Lab: Performing ETL on a Dataset by Using AWS Glue". As you perform each step of the lab, capture a screenshot. Submit a Word document containing all screenshots as evidence of your completion.

Please note: Completion will be verified using lab activity statistics from the AWS educator admin console. Do not share your screenshots with others.