Learning outcomes and key skills

This assignment addresses the following learning outcomes of the module.

– Demonstrate fluency in contemporary programming languages, development tools and environments.
– Evaluate and demonstrate the theory and concepts of contemporary/industry standard programming and design in the software development life cycle.
– Demonstrate awareness of industry standards of professional and ethical software development, software carpentry and codemanship.

This assignment also addresses the following key skills
– Written communication
– Use of IT
– Requirement analysis, technical interpretations of problem statements

2. Description of task and guidance notes

Part 1: Design and Develop application to recognise handwritten digits

The assignment requires you to develop a java-based application to recognise handwritten digits. The inputs to the program will be a set of training images and appropriate labels (see below for details on MNIST database), along with an image of a handwritten digit, which needs to be recognised. The image of the handwritten digit to be recognised can be provided to the program either as an external image upload (a set of test images are given), or allowing the user to draw a digit using mouse, within a canvas area available in the application's user interface.

Use case 1: User uploads an image of a handwritten digit for recognition

In this case, the program should allow the user to upload an image that contain a handwritten digit. The application should display the uploaded image, its predicted result.

Use case 2: User draws a digit using mouse inside the canvas area

In this case, the program should allow the user to draw a desired digit (0 - 9) within a canvas area using mouse motions. As a programmer, you need to think about the line thickness, background colour of the canvas based on the sample images found in the MNIST database.

Once the drawing is complete, the digit should be extracted as an image which is resized accordingly (to match the size of the training set image samples), and centred with sufficient amount of padding before being used in the recognition.

Recognition of digits using k-Nearest Neighbour (k-NN) Algorithm

Once the image with handwritten digits, and the images and labels from the training set are available, the recognition process should be initiated. There are numerous algorithms available in the literature which can be utilised for the purpose of recognition (I.e., SVMs, DNNs etc.). However, in this assignment, it is recommended to use k-NN algorithm with Euclidean distance over raw pixels as the decision criteria. The pseudo code for a potential implementation of the k-NN algorithm is given below.

train_dataset := [train_images, train_lables] # 2D array that holds training images and labels
d_img # Variable that holds handwritten digit image
distance_lbl_array # 2D array to hold distances and labels

# iterate through each element in the train_dataset 2D array
for each element e in train_dataset
t_img := train_dataset[e][0] # eth training image
t_lbl := train_dataset[e][1] # eth label for the corresponding image

# compute the Euclidian distance between training image and current image

E_d=√(∑_j=0^N∑_i=0^N(t_img(i,j) -d_img(i,j))² )

# store theEuclidian distance and the label of the training image
distance_lbl_array[e] := [distance, t_lbl]
end

# Sort distance_lbl_array according to the distance
sorted_distance_lbl_array := sort (distance_lbl_array)

# extract elements for k closest labels. e.g., if k=10, get the first 10 elements in the sorted_distance_lbl_array.
closest_k_label_array := sorted_distance_lbl_array[:k]

# Find the label class with maximum number of label counts
Estimated_label := max_label_count(closest_k_label_array)

# Find the confidence as a ratio of selected label count to total number of closest elements
Confidence := max_label_count(closest_k_label_array)/size(closest_k_label_array)

Presentation of the results

The digit recognised should be displayed with the confidence of the result.

Hint: The MNIST database contain training and testing datasets. You are encouraged to develop your code to test on the images provided in test data set, before attempting to perform the recognition for the user drawn digit.

MNIST dataset

MNIST database of handwritten digits is a popular dataset for studying machine learning and pattern recognition methods. The details of the images and their formats are given in the MNIST official web site . As specified in the web page, the images are not in any standard image format. Therefore, you may have to write your own program to read and convert them. Some helpful links are provided in the Moodle page.