Digital Crime Analyzer

This repository contains the source code of the assignment for python, in partial fulfilment, of the module ICT1002: Programming Fundamentals assignment

Objective

Python has become one of the most popular programming languages in a lot of application scenarios such as the data crawling, scripting, data analytics and Artificial Intelligence. Thus, the objective of this project is to allow you to develop a Python program such that you can:

1. Apply and familiarize the knowledge learnt from the class
2. Learn more new Python programming skills through self-learning to deepen in your skills
3. Learn team collaboration using Python tools and libraries
4. Gain more hands-on experience in solving a real-world problem using Python

The Detail Requirements

Digital crime becomes more and more popular. It refers to the criminal activity done against computers and networks or using the computer as a tool to do that activity. It can be in the form of offences against computer data or systems, unauthorized access, modification or impairment of a computer or digital system. In this project, you are tasked to develop one tool to help the crime investigators to better investigate the digital crime. In digital crime investigations, the investigators and incident responders often need to examine and analyze log records and disk / memory images or other possible resources. For example, they may utilize a history of actions to reconstruct a chain of past events and decide whether or not a crime has been committed, the circumstances surrounding the crime and the perpetrator. Normally, the online behaviors can be identified from the various log files together with digital trails or artifacts extracted from disk or memory images. So, you need to develop a good data-driven solution such that the crime investigators can better understand the digital behaviors with various features. Overall, your team shall perform below tasks in your project:

1. Identify proper data sources to use in your project that could be potentially used for crime detection. We have also provided you some data logs files in a separate file. You are free and encouraged to use your own data resources as well in your project.
2. Develop the proper data parser and cleaner to extract the proper information from the data you have.
3. Build your own story about how to make sense of the datasets and perform some meaningful analytics with different visualization or user interfaces.
4. Provide some evaluations and comparisons if needed. In particular, your system shall allow users to perform some basic features shown as below:
   • Read data from an external data sources, e.g. txt, Jason, excel or other format.
   • Allow users to view the datasets.
   • Allow users to search the data based on some criterion.
   • Allow users to see some statistics information based on the data.
   • Allow users to export some of the results.
   • As some advanced features, you can identify your only way to see how to detect a crime, such as the machine learning, pattern detection and so on. You need to design one user-friendly UI such that your program is easy to use. As one example, the program should be able to allow users to choose which function to perform and display the result nicely. You can build any innovative UI. Note that it is not compulsory to build a GUI. However, building one GUI will be considered as one advanced features of your program.

References

1. DDoS monitoring: how to know you’re under attack
2. How to Analyze Distributed Denial-of-Service (DDos) Attack
3. ddosflowgen

Sample logs Sources

1. SecRepo.com - Samples of Security Related Data
2. The UNSW-NB15 Dataset Description
3. Apache2 access logs
4. Web Server access logs
5. Web Server access logs
6. ios-example.com Sample Access Logs
7. satsig.net Sample log
8. IIS 10.0 Sample Access Logs

Project Reference Used

1. CICFlowMeter
2. IDS 2018 | Datasets | Research | Canadian Institute for Cybersecurity | UNB

License and Copyright

Repository content (Excluding third-party resources, see below) shall be released under the MIT License.

This project is an assignment submission in partial fulfillment of the module ICT1002 Programming Fundementals.

As such, copyright and any rights to this project shall belong to the project contributors as well as to Singapore Institute of Technology (SIT)

Plagiarism is a serious offence, and SIT's policy explicitly forbids such acts. Any submission caught with plagiarised work shall receive zero marks for their submission.

Any third-party resources used for this project may be reused in accordance to their license and/or terms and conditions.