CC5067NI Smart Data Discovery Coursework 01 Question Paper Spring 2025 | LMU

Submission Instructions

Submit the following to Islington College's MST Portal before 01:00 on the due date: 

• A report (document) in .pdf format in the MST Portal or through any medium that the Module Leader specifies.
• Associate the Python program with a ZIP file

Plagiarism

You are reminded that there exist regulations concerning plagiarism. Extracts from these regulations are printed overleaf. Please sign below to say that you have read and understood these extracts: 

Extracts from University Regulations on Cheating, Plagiarism, and Collusion 

Section 2.3: "The following broad types of offence can be identified and are provided as indicative examples .... 

• Cheating: including taking unauthorised material into an examination; consulting unauthorised material outside the examination hall during the examination; obtaining an unseen examination paper in advance of the examination; copying from another examinee; using an unauthorised calculator during the examination or storing unauthorised material in the memory of a programmable calculator which is taken into the examination; copying coursework.
• Falsifying data in experimental results.
• Personation, where a substitute takes an examination or test on behalf of the candidate. Both the candidate and substitute may be guilty of an offence under these Regulations. 
• Bribery or attempted bribery of a person is thought to have some influence on the candidate's assessment.
• Collusion to present joint work as the work solely of one individual.

Contract Cheating

Contract cheating (also known as assessment outsourcing, commissioning or ghost writing) is when someone seeks out another party, or an AI generator service, to produce work or buy an essay or assignment, either already written or specifically written for them or the assignment to submit as their piece of work. 

Contract cheating undermines the integrity of the academic process and devalues the qualifications awarded by the university. Students are reminded that academic integrity is a fundamental principle of our institution. Engaging in contract cheating not only impacts the individual's academic record but also the reputation of the university. 

Students are encouraged to seek support if they are struggling with their coursework. The university offers a range of resources, including academic counselling, tutoring services, and workshops on study skills and time management. Utilising these resources can help students achieve their academic goals without resorting to dishonest practices.

Penalty: 

Failure in the Module: The student must re-register for the same module, and the re-registered module will be capped at a bare pass. 

Ineligibility to Continue on the Course: Where re-registration of the same module, or a suitable alternative, is not permissible, the student will not be able to continue on the course. Additionally, the following penalty will be applied to the student's final award: 

• Undergraduate Honours: The student's final classification will be reduced by one level. 
• Unclassified Bachelors: Downgraded to Diploma in Higher Education.
• Foundation Degree: Distinction downgraded to Merit; Merit downgraded to Pass; Pass downgraded to Certificate in Higher Education.
• Masters: Distinction downgraded to Merit; Merit downgraded to Pass; Pass downgraded to Postgraduate Diploma. 

Reporting and Consequences: 

Instances of contract cheating will be thoroughly investigated, and students found guilty will face the penalties outlined above. It is the responsibility of every student to ensure that their work is their own and to avoid situations that could lead to accusations of academic misconduct. By adhering to these standards, students contribute to a fair and equitable academic environment, ensuring the value and recognition of their qualifications are maintained. 

• Write a Python program to remove the NaN missing values from the updated dataframe.[5 Marks]
• Write a Python program to see the unique values from all the columns in the dataframe.[5 Marks]

Data Analysis 

• Write a Python program to show summary statistics of sum, mean, standard deviation, skewness, and kurtosis of the data frame. [5 Marks]
• Write a Python program to calculate and show the correlation of all variables. [5 Marks] 

Data Exploration 

• Provide four major insights through visualisation that you come up with after data mining. [10 Marks] 
• Arrange the complaint types according to their average 'Request_Closing_Time', categorised by various locations. Illustrate it through a graph as well. [10 Marks] 

Milestone 1 (Week 7)

1. Data Understanding 

• To understand what your data resources are and the characteristics of those resources. Write down your findings.[10 Marks] 

2. Data Preparation 

• Import the dataset[5 marks]
• .Provide your insight on the information and details that the provided dataset carries.[5 marks]
• Convert the columns "Created Date" and "Closed Date" to the datetime datatype and create a new column "Request_Closing_Time" as the time elapsed between request creation and request closing [10 Marks]
• Write a Python program to drop the irrelevant Columns which are listed below. 

['Agency Name',' Incident Address','Street Name',' Cross Street 1',' Cross Street 2',' Intersection Street 1', 'Intersection Street 2',' Address Type','Park Facility Name',' Park Borough','School Name', 'School Number','School Region','School Code','School Phone Number','School Address','School City', 'School State','School Zip','School Not Found','School or Citywide Complaint',' Vehicle Type', 'Taxi Company Borough',' Taxi Pick Up location','Bridge Highway Name','Bridge Highway Direction', 'Road Ramp',' Bridge Highway Segment',' Garage Lot Name',' Ferry Direction',' Ferry Terminal Name','Landmark', 'X Coordinate (State Plane)',' Y Coordinate (State Plane)',' Due Date',' Resolution Action Updated Date',' Community Board',' Facility Type', 'Location'] [5 Marks] 

• Write a Python program to remove the NaN missing values from the updated dataframe.[5 Marks] 
• Write a Python program to see the unique values from all the columns in the dataframe.[5 Marks]

Data Analysis 

• Write a Python program to show summary statistics of sum, mean, standard deviation, skewness, and kurtosis of the data frame. [5 Marks]
• Write a Python program to calculate and show the correlation of all variables. [5 Marks] 

Milestone 2 (Week 10) 

Data Exploration 

• Provide four major insights through visualisation that you came up with after data mining. [10 Marks]
• Arrange the complaint types according to their average 'Request_Closing_Time', categorised by various locations. Illustrate it through a graph as well. [10 Marks]

Statistical Testing 

Test 1: Whether the Average Response Time Across Complaint Types is Similar or Not. 

• State the Null Hypothesis (H0) and Alternate Hypothesis (H1).
• Perform the statistical test and provide the p-value.
• Interpret the results to accept or reject the Null Hypothesis. [10 Marks] 

Test 2: Whether the Type of Complaint or Service Requested and the Location are Related.