Objective:

Student projects aim to provide hands-on experience with implementing, training, and testing a neural network for an image processing task. A student may choose from four topics, all of which are based on the same original dataset, but which differ in the input and output, and the formats thereof.

Project Topics:

1. Single-class Detection (project code: DET): Focuses on identifying and localizing instances of a single class within images.

2. Multi-class Classification (project code: CLA): Involves categorizing images into multiple predefined classes.

3. Multi-class Semantic Segmentation (project code: SEG): Aims to assign a semantic label to every pixel in an image, effectively dividing the image into meaningful segments.

4. Colorization (project code: COL): Entails colorizing images that were converted to grayscale from their original format.

We provide a separate dataset for every project topic and a general project template that can be adjusted for a specific topic:

Datasets

Students may only use the datasets that are provided for the projects. Using any additional datasets is strictly forbidden, but data augmentation is allowed. The provided datasets were created based on the Cityscapes dataset by applying a color transformation to the original images to prevent students from using existing solutions. While every type of preprocessing is allowed, students must not attempt to reverse this transformation.

Project requirements

1. Only the PyTorch library may be used, i.e., using Tensorflow, PyTorch Lightning, or any other DL libraries is prohibited. Of course, libraries such as numpy or albumentations can be freely used.

2. All files must be runnable on the computers in classroom B311. If you use a faculty server or your own computer, it is recommended to run at least the inference script on a classroom computer to make sure that it is runnable.

3. Pre-trained models and transfer learning are not allowed, i.e., the model must be trained completely from scratch.

4. Students are free to choose any suitable architecture and loss function, including ones described in online guides or scientific papers.

5. Only the provided datasets may be used. Preprocessing and augmentation are allowed, but recoloring the images is not (i.e., what is green in the provided images should stay green, etc.).

6. Students are allowed to add additional classes and functions in the project template. However, parts of the template marked “not to be changed” must not be changed (e.g., the get_arguments() function). 

7. Only files specified in the project template should be submitted. Any additional files for classes/functions should not be created.

Project evaluation procedure 

To evaluate the projects, we will do the following:

1. We will check the training.py file

• The file should be runnable and generate model.pt, learning_curves.png, and architecture.png

• The student should divide the provided dataset correctly (into train, validation, and test parts)

2. The files generated by us via the training.py script (model.pt, learning_curves.png, and architecture.png) and those generated by a student (final_model.pt, final_learning_curves.png and final_architecture.png) may be compared.

3. We will run the inference.py script for final_model.pt on a secret-test dataset to generate predictions. Then, we will run evaluation.py to generate the final score of the model's performance.

• The final performance score on the secret-test dataset will not influence the decision of whether the project is passed or failed (as long as the model produces something reasonable). However, students will be anonymously ranked based on their model’s performance.

During the evaluation, we will look at the code, the model's predictions, and all generated files. If the task was solved reasonably well and all of the project requirements are satisfied, the project will be evaluated as passed.

We will contact the author for help if some files are not runnable. If the files cannot be made runnable even with the student’s assistance, the project will be failed.

During the project evaluation period, we may contact students with additional questions (e.g., if model.pt and final_model.pt were to produce drastically different results).

Project checkpoint

Around week 9 of the semester, a special seminar will take place where students will have the opportunity to consult with the seminar tutors and get feedback on their projects. It is not necessary to attend this seminar, though it is recommended to present a partially completed solution. This way, the tutors can identify potential problems and provide feedback regarding best practices.

The last lecture (May 20th) and feedback 

On May 20th, 8:00, we will present a summary of all projects aggregated by topic. 

We will rank all submitted projects by the final score on the secret-test dataset and present the summary of everyone's work. We may use the submitted learning_curves.png and architecture.png files (without stating the author's name).

Each student will be assigned a random code name for their model (code names for a model will be known only to the author of the model). 

By using this code name, students will be able to identify their model's performance in the ranking table (for the secret-test dataset) that is going to be shared at the last lecture. 

The ranking is only meant to provide additional objective feedback (e.g., to demonstrate which approach worked the best).