This is the repo containing the backend that was used by team FIRST/V-FIRST at LSC'22 and VBS 2022 respectively.
At VBS 2022, we placed 7/16, just a bit behind Best Newcomer VideoFall. We scored 67 on KIS-T, 83 on KIS-V, and 41 on AVS. This marks the return of our team since VBS 2018, and we did pretty well!
There were some difficult KIS-T tasks that require some advanced English, and/or creativity in parsing the queries and making use of different search terms.
We anticipated difficulties in KIS-V tasks, however text queries combined with visually similar image search work out quite well.
At LSC 2022, we placed 4/9, winning Best KIS (after overall winner MyScéal). This is a very proud result of our team!
As our system is designed with KIS in mind, overall our system performs best within that category. QA is a new category that requires ancillary tools (e.g., a map). As with VBS, our inferior performance in AVS tasks is due to the lack of batch-submit related features, which we have improved for LSC'22 but not quite enough.
Install milvus docker image (CPU or GPU), then run it with the script (once).
Install services.
Install Python requirements using pip + venv (requirements.txt) AND conda (requirements-conda.txt).
For the front-end, see its own repo.
Start the back-end and migrate (once)
python manage.py migrate
Insert CLIP features into milvus: see notebook
Insert milvus ID - image name mapping into django (please read the command's help/source before doing this!)
python manage.py insert <collection_name> milvus_ids.txt keyframes_list.txt
The application needs 6 things to run: milvus, elastic, CLIP, image server, front-end, and back-end. They should all be put in tmux sessions or docker containers.
Start milvus docker container:
sudo docker start milvus_cpu_1.1.1
Run the image server: script.
Run the front-end server: script.
Run CLIP & elastic services: see service repo.
Run the django back-end server: see script.
see api readme.
Check each component separately to see if they are still up and running. Mind the ports & IPs!
Test the back-end API using Postman.
Read the back-end and front-end logs.
For front-end debugging, F12 is very helpful.
For more detailed information, see api readme.