UofT-DSI | deep_learning - Assignment 2

[Vortable]

Title: UofT-DSI | Deep Learning - Assignment 2

What changes are you trying to make?

Adding completed Assignment 2 notebook implementing zero-shot image classification with CLIP on Fashion-MNIST, and completed Labs 4, 5, and 6.

What did you learn from the changes you have made?

I learned how vision-language models like CLIP can classify images without any task-specific training using text prompts. I also learned about cosine similarity between embeddings, prompt engineering, and UMAP dimensionality reduction for visualizing embeddings.

Was there another approach you were thinking about making? If so, what approach(es) were you thinking of?

Were there any challenges? If so, what issue(s) did you face? How did you overcome it?

I ran into library version incompatibilities between the course code and the version of transformers installed on Kaggle/locally. I also could not run the full 10,000 image test set on CPU, so I used a 1,000 image subset. The Pascal VOC dataset for Lab 5 was unavailable as the URL had been removed prior to class.

How were these changes tested?

The notebook was run locally and on Kaggle, producing accuracy scores, confusion matrices, a UMAP plot, and Top-K accuracy results.

A reference to a related issue in your repository (if applicable)

Checklist

• I can confirm that my changes are working as intended

[github-actions]

Hello, thank you for your contribution. If you are a participant, please close this pull request and open it in your own forked repository instead of here. Please read the instructions on your onboarding Assignment Submission Guide more carefully. If you are not a participant, please give us up to 72 hours to review your PR. Alternatively, you can reach out to us directly to expedite the review process.

[tianyi21]

Hi @Vortable . Thank you for submitting your DL assignments. Please follow the submission guidelines and create the PR from your assignment branch to your main branch. Please let us know if you need any help with creating your PR. This PR will be closed shortly.