High Performance Computational Infrastructures

Project Development

OBJECTIVE OF THE ASSESSMENT
In this assessment, you are required to demonstrate the appropriate practical skills and abilities to implement solutions using modern large-scale data storage and processing infrastructures and to reflect critically on the concepts, theory, and use of high-performance computational infrastructures.

DESCRIPTION OF THE ASSESSMENT
You are required to identify and analyse a real-world problem, design and implement a solution to the problem using Hadoop, and evaluate your implementation. The problem can be a simplified version of its original scale, extent or level of difficulties etc.

LEARNING OUTCOME

LO1: Demonstrate the appropriate practical skills/abilities required to implement solutions using modern large-scale data storage and processing infrastructures.

LO2: Reflect critically on the concepts, theory and appropriate use of large-scale data storage and processing infrastructures (commonly used in modern organisational environments).

Identifying a real data analytics problem with strong motivation for using distributed processing methods
Designing, Implementing and Evaluating a working solution using distributed analytical techniques (MapReduce Algorithms)

Grade Band E and F

The candidate fails to meet the minimum requirements as outlined in the learning outcomes.

Grade Band D (D+, D, D-)
The work demonstrates significant weaknesses, but all of the learning outcomes have been met at the minimum requirement level. The work provides evidence of some critical understanding of large-scale data storage and processing infrastructures and demonstrates some abilities and skills to implement solutions using these technologies. The characteristics of a coursework in this grade band are:
The research question, which is based on a chosen dataset, is not very well set (vague and the
purpose is not clear)
The proposed MapReduce solution is the same as the analyses and programs used in the lab sheets
but on different dataset/s
The proposed MapReduce solution is partially working (this means that the MapReduce program is working and producing results but not the results that answer the research question)
Basic/No reflection on the output (results)
Grade Band C (C+, C, C-)
In addition to the requirements for a grade in D-band, it demonstrates the ability to develop an independent, systematic, logical and effective solution to the problems identified. It also demonstrates a significant degree of competence in the appropriate use of the relevant methodologies, practices, and tools, etc., to analyse the problems and evaluate the solutions. The characteristics of coursework in this grade band are:
Clear definition of the research question based on the chosen dataset
The proposed MapReduce solution is similar to the analyses and programs used in the lab sheets but on different dataset/s
The proposed MapReduce solution is working, and the MapReduce program is producing results that help
answer the research question
Basic reflection on the output (results) is provided
The code authored by the student meets basic standards for quality and clarity.
No meaningful insights are discovered from the results
Grade Band B (B+, B, B-)
In addition to the requirements for a grade in C-band, it clearly demonstrates the ability to develop an independent, systematic, logical and effective solution to the problems identified. It also demonstrates a high degree of competence in the appropriate use of the methodologies, practices, and tools, etc., to analyse the problems and evaluate the solutions. The characteristics of a coursework in this grade band are:
Clear and important research question, which is based on the chosen dataset, is set at the beginning (clear
justification of using distributed computation techniques)
The proposed MapReduce solution is slightly different compared to the analyses and programs in the lab
sheets and on different dataset/s
The proposed MapReduce solution is working, and the MapReduce program is producing results that help
answer the research question
Good reflection on the output (results) is provided
The code authored by the student is of high quality, demonstrating clarity and efficiency.
Some insights are discovered from the results
Grade Band A (A*, A+, A, A-)
In addition to the requirements for a grade in B-band, the work clearly demonstrates a sophisticated, critical and thorough understanding and ability to process large-scale data using High-Performance Computational Infrastructures. It provides evidence of originality of thought and clearly demonstrates the ability to develop an independent, systematic, logical and effective solution to the problems identified. It also demonstrates excellence in the appropriate use of methodologies, practices, and tools, etc., to analyse the problems and evaluate the solutions. The characteristics of coursework in this grade band are:
Clear and important research question, which is based on the chosen dataset, is set at the beginning (clear
justification of using distributed computation techniques)
The proposed MapReduce solution is fundamentally different compared to the analyses and programs in
the lab sheets and on different dataset/s.
The proposed MapReduce solution is working, and the MapReduce program is producing results that help
answer the research question
Excellent reflection on the output (results) is provided
The code authored by the student demonstrates exceptional quality, clarity, and efficiency.
Meaningful and actionable insights are discovered from the results

FORMAT OF THE ASSESSMENT

The maximum length of the report is 3500 words, not including table of contents, references and any appendices. The report should be submitted as a single PDF file. You should include sections on (percentage of overall mark):
Introduction, Problem description & associated dataset
Design & Implementation
Results & Evaluation & Discussion