CTEC3754D Malware Analysis Coursework Brief 2024-25 | DMU

The Learning Outcomes that are Assessed by this Coursework Are:

• Critically analyse the goals of malware investigation as part of incident response and best practice approaches to analysing malware.
• Critically assess assembly language code for reversing and its application to advanced static and dynamic analysis.
• Evaluate malware evasive techniques, e.g., packing, obfuscation, and anti-disassembly. antis and pro-boxing, etc. and how to circumvent them.
• Investigate, select, and apply real malware through static and dynamic analyses using standard tools and techniques.
• Discuss and evaluate APT strategies and advanced malware functionality, e.g. rootkits, botnets, remote access trojans, etc.
• Apply industry-standard malware analysis tools to conduct an in-depth analysis.

Coursework Tasks:

In this coursework, you are expected to analyse several suspicious files and answer questions about the insights gained, detailing your approach with relevant evidence, e.g., screenshots, excerpts of logs, etc. Ensure that your answer includes detailed steps on how your evidence has been identified and extracted. Please refer to your lab sheets to identify virtual machines and forensic investigation tools appropriate for analysing the suspect files and the correct method of presenting your findings by following detailed steps. This will demonstrate your efforts in conducting practical analysis, thereby assisting you in achieving high marks. Any references should be cited within the body of the report and then listed at the end. Note: Malware files can be accessed from the learning zone.

Part 1: Basic Analysis (40 Marks) (Word limit: 1500 as maximum)

• Retrieve the sus-file1 zipped within the sus-file1.7z archive file. Perform a comprehensive analysis of this file and present your findings, concluding whether the file is a malicious PDF document. [12 marks]
• Retrieve sus_file2 from the archive zipped file sus-file2.7z. (a) How would you confirm the type of file it is, and how would you make it execute for analysis? (b) Is the sample packed? What observable indicators of the file suggest that it may/may not be packed? Document all your observations with any applicable tools of your choice. [12 marks]
• Assuming that your friend receives the sus_file3 (zipped in sus-file3.7z) as an email attachment on a Windows 10 machine and accidentally double-clicks the file. Is their system infected? If yes, why/how? If not, why not? Explain and support your answer with evidence. [6 marks]
• Analyse the sus_file3 dynamically and monitor its activities on the system. Outline the steps taken to execute the sample for analysis. What changes do you observe on the host? For example, is anything dropped, executed or deleted? Any other changes to the host observed? (Hint: if you use Regshot in any phase of your analysis, be careful to set the right scan directory, i.e. C:\). Support your claims with documentary evidence. [10 marks]

Part 2: Advanced Analysis (54 Marks) (Word limit: 2500 as maximum)

You have been provided with a SUS-file4, a malware file potentially employing encoding techniques. Your task is to analyse the file for:

• Identifying the encoding methods utilised.
• Identifying the encoding key for each technique employed.
• Explaining the functionality of this malware.

Note: It is always advisable to commence the analysis with static methods. In case static analysis does not produce sufficient evidence, dynamic analysis can be initiated.[15 marks]

Investigate the file sus-file5 and provide an analysis supported by evidence to determine:

• Whether the sus-file5 malware was compiled for either the x64 or x86 environment.
• The malicious actions the malware can perform.
• The alterations in behaviour exhibited by this malware when operating in an environment that differs from its current one. (For instance, if the current environment is x86, identify the malicious actions the malware may do in an x64 environment compared to an x86 environment.)[15 marks]

For this question, you are asked to analyse the sus-file6, which is a code containing a vulnerability that leads to a stack buffer overflow, using an appropriate debugger. Your task is to use a debugger to analyse this vulnerability and provide sufficient details on how to:

• Identify the vulnerable part of the code where user input is read into a buffer without proper bounds checking.
• Set breakpoints at critical points to monitor the flow of execution and analyse the memory layout.
• Investigate the mechanism through which an attacker can potentially exploit a buffer overflow to manipulate code execution. To achieve this, implement the vulnerable code across various scenarios and debug its steps. Focus on identifying the specific assembly instructions that demonstrate how user-entered data is stored in memory addresses, consequently leading to a stack buffer overflow attack.

Note:

Ensure that you conduct the analysis in a controlled environment and adhere to ethical guidelines during the lab task. Document your observations, insights, and any potential countermeasures to address stack buffer overflow vulnerabilities effectively. [24 marks]

Writing and organising the report that includes your answers to be submitted is an important component of your assignment. The report should be well-organised and written, adhering to a word limit of a maximum of 4000 words (5000 words including references and appendices). It should be structured well and coherent, making it easy to follow and assess. References should use the Harvard format with proper citation details inside the report body. [6 marks]

Deliverables to be Submitted for Assessment:

Written document consisting of two parts, with answers to each question provided as a separate item. Detailed answers and documentary evidence should be given under each question. Posting the final result/findings without providing appropriate evidence and analysis will not be marked. Where appropriate, references should be listed at the end of the report and cited within the body of the report (max. 5000 words including figures, references, appendices, etc.).

How the Work will be Marked:

Each of the questions is scored from 0 to a maximum point indicated alongside each question, following a specific marking grid that considers the substance of each written response. There is a total of 6 marks for presentation quality.

Please refer to the criteria marking grid for details of the assessment of the work.