IN3034-INM446 Cybercrime and Socio-technical Risk Resit CWK Assignment 2024-25

IN3034-INM446 General instructions

This resit coursework assignment is an assignment for individual work to be submitted by the standard deadline for this year, Friday 8 Aug 2025 at 5 p.m.

It gives a maximum overall mark of 100 marks, counting for 30% of the module mark.

The marking scheme is different between IN3034 and INM446 (see below). Answer ALL questions specified for your module.

File format: (1) Submit one file containing all your answers, as a single MS Word “.doc” or “.docx” file, or a PDF file, by the deadline. (2) You need some diagrams or sketches to illustrate your ideas: you may just sketch them by hand, neatly, and scan or photograph them; or prepare these figures using a computer program (say which program you used). In either case, incorporate them into the single Word file mentioned. (3) Please use font Times 12 or a larger font. Please ensure that everything is clearly readable if printed. (4) For your calculations, Excel spreadsheets will help to perform all the calculations quickly and consistently. Please show the spreadsheets as tables in your submission, but do NOT assume that these tables alone suffice to explain how you arrived at your answers: do clarify in your text what calculations the spreadsheet performs (not for every cell separately). Submit the spreadsheet files as well.

General guidelines about marking: for each question, I will give 4/10 of its maximum mark (bare pass for IN3034, fail for INM446) if the answer shows understanding of the question but is still incomplete or wrong. Answers that are fully correct (including the explanation of why your answer is the correct answer) and unambiguous get 7/10 of the marks. Answers that show special levels of thoroughness, clarity, readings beyond the bare minimum required, and/or ingenious spotting and solving of problems, get extra marks. Please do check that your calculation are correct; I will especially penalise in marking those calculation errors that could have been caught by common-sense checking (e.g., I will deduct many marks for probabilities that are negative or above 1).

Language: please use the terminology introduced in this module.

Be clear. If a statement that you write can be interpreted in two ways, and only one of the two is correct, you gain fewer marks than if it had only one possible meaning that is correct. Referencing: apply the usual rules about referencing. make sure to indicate by bibliographic references any sources (work by others) you have used. Always reference the sources of ideas or facts that you use, even if not quoting them verbatim. Presenting others' ideas as your own is academic misconduct. You can use any standard style of referencing. Make sure that each bibliographic entry in your reference section contains enough information for a reader to find each source cited, and there are appropriate pointers in the body of your text. Quotations: Copying and pasting text from a source is fine IF AND ONLY IF you identify what text is copied by putting it between quotation marks, and cite the source as a bibliographic reference. Otherwise it is misconduct.

Quoted text count towards your total word count.

Use of AI, e.g. ChatGPT: Using these aids without acknowledging it is academic misconduct.

Please keep in mind that I set questions in such a way that these AI aids tend to give wrong answers: I have seen such wrong answers in the March coursework. Nonetheless, in this module,

1. you are free to use such AI aids, PROVIDED THAT you add an appendix (there is no limit on its length) that shows exactly how you used them: a complete log of prompts used and of outputs received, and your description of how you used and corrected these outputs;
2. but in any case you are responsible for your answers being correct and ethical, including that they must contain proper acknowledgment of the sources used (by you or by the AI).

Length: I set word limits for some answers. You are not required to write that much. A short answer sometimes has better content than a verbose one.

The marks are distributed as in the table below. Q7 is for INM446 students; for IN3034 students it is optional (if they answer, I'll give feedback, and may give bonus marks if the answer is very good)

IN3034-INM446 The problem:

Phishing risks (Note: In this exercise, assume, unless instructed otherwise, that all the decisions are taken on the basis of expected utility alone.) 

Your company. where you are the chief information security officer, is seriously concerned about the risk of social engineering and phishing. The company is very large (hundreds of thousands of employees); many of the employees have among their duties that of keeping contacts with a numerous and evolving set of suppliers and customers, with varying levels of security awareness and maturity, so that a defence consisting only of filtering incoming Email based on provenance will be neither effective enough (dangerous messages may come from the addresses of actual, non-criminal business partners) nor acceptable (one may reject messages from new business partners, for instance, damaging the business).

You have run a mock phishing attack campaign, in which your own team sent made-up phishing messages to staff, so that you could estimate the staff's ability to deal with phishing. You were able to monitor closely what staff did with these |"mock phishing" messages: whether they opened URLs or attachments from the messages, tried to reply, etc. You obtained the results below (NB in this story, we count individual messages: if a message is broadcast to 50 people, it is counted as 50 messages):

1. out of the 478,912 mock phishing messages that were sent to them, staff responded correctly to 451,073. "Correctly" here means that they deleted the message without replying or opening links or attachments that came with the message; 
2. even when they did not respond correctly, still, for each case in which a member of staff executed all the actions that the attacker desired, there were six cases in which, before completing these actions, they noticed that something was wrong and stopped. In your reporting you call these two kinds of cases "complete successes" and "partial successes" (of the attacker), respectively 
3. note that over the same period, staff received 3,608,752,080 messages that were not part of the mock phishing exercise; 
4. staff in your company are instructed to report all messages that they think suspicious, and their mail client user interface offers an easy way of doing this; yet, during the period in question, out of all the "mock phishing" messages, staff reported only 174,076. Of these reports you received, 835 concerned "complete successes"; the reports concerning "partial successes" were 13 times more numerous 
5. during the same period, staff reported as suspected phishing a total of 693,824 messages that were not part of you mock phishing campaign. Of these, you ascertained that 45,987 were partial successes, 7,956 were total successes, and 304,002 were false alarms.

Q1. draw an event tree representing the relationship between these 
various events: messages from the mock phishing campaign and the rest of the traffic; phishing and non-phishing messages; degree of success of phishing messages; reporting or non-reporting; etc. For those nodes or edges for which you do not have numbers yet, do not write numbers (marginal or conditional probabilities). Complete these numbers (only as far as you can) after answering all the other questions. 

Q2. assume that when a staff member detects a possible phishing phishing
message, the probability of their reporting it is the same for real phishing messages as for mock ones that achieved the same degree of success (none, partial or total). How many fully successful phasing events can you assume your company suffered in the observation period in question? How many partially successful ones? From past history and industry intelligence reports, you estimate the cost to the company of all these types of events as follows: 

1. a fully successful phishing event costs on average 15,000 pounds (a combination of harm from those successful intrusions that are made possible by the phishing, cost of intervention that is needed irrespective of whether the criminals exploit their success or not, business losses from reputational damage, etc.). Note that this cost may not be known to the company for months or year after thee event 
2. a partially successful phishing event costs on average 200 pounds (mostly the cost of investigating it and taking remedial action) 
3. a phishing report that is a false alarm costs on average 100 pounds, the staff time for investigating the report. d. the cost of the user time used in rejecting phishing attempts and/or reporting them is negligible compared to all these other costs

Q3. given these numbers, what is the estimated loss to your company 
due to the events, during the observation period, whose numbers you estimated in Q2? 

Q4. if a new "copilot"-style product were advertised that would monitor
and advise staff while they deal with their Email, and this product were documented to be capable of transforming 5% of the current "total successes" into "partial successes", at the cost of doubling the false alarms, should you, reasoning in terms of expected loss for the company, adopt it or not? In deciding this, assume that the total cost of use of this product (license fee, computer time, training costs etc) would be 30 millions pounds, for the reference period considered above. 

Q5. Consider now the viewpoint of an attacker who uses phishing, let's 
call him Philip. For Philip, the cost pf the phishing activity, based on his established practice (how much he spends to perfect a phishing message, how many copies of each message he would send, etc.), is 0.0025 pounds per phishing message. About 1 in 100 fully successful phishing messages leads to a gain (through stealing and reselling data, obtaining ransom, or whatever other means), of 7000 pounds on average. During the observation period, how much of a profit (or a loss) did Philip make, assuming that he was the author of all phishing activity against your company? If you were to adopt the copilot product above, how would this profit or loss change? 

Q6. Supposing that instead of the copilot mentioned in Q4, you had the 
possibility of adopting a different, "filter" product, which automatically deletes phishing messages before they are even seen by the recipient, instead of advising the recipient about how to deal with them. For a same number of complete successes avoided, what would be the pros and cons of using this "filter" vs using the "copilot"? Consider what we studied in the lectures on event detection and on decisions under uncertainty. 

Q7. (for INM446 only. For IN3034 students, optional question that can 
yield bonus marks) Considering again Philip's viewpoint, how effective would the copilot need to be (in terms of complete successes turned into partial ones) in order to halve Philip's net expected utility? Could Philip compensate for this loss by just sending more phishing messages?