Performance problem - Pages are too slow to load

[Didayolo]

Codabench is too slow to load.

• Home page of competitions are loading everything at once. We need a lazy loading.

• When a loading is long, it is decreasing the performance for everybody else (this is normal I guess).

• Competitions list is slow to load too, despite the small logos (Add thumbnail logo to competitions to increase loading speed of the list #1290). Code of this page: https://github.com/codalab/codabench/blob/f83f94c1e492da524fef5b2f6ccf26846fa88b69/src/static/riot/competitions/public-list.tag

TODO

• Increase the CPU and RAM of the main VM
• Increase the CPU and RAM of the main VM (again)
• Competition page
  • Trying to remove "entries" column from leaderboard, which may be the one that is slow to load (Removed num_entries becasue the count was slowing down the platform #1673)
  • Put back "entries" in an efficient way
  • Pagination for "Submissions" tab (and lazy loading)
  • "Show more" on leaderboard results tab. By default show 30 rows, and add more when clicking on show more. If the participants is not on the first 30, it could still be shown at the bottom of the table after some "[...]" row
• Competition list
  • Optimize the code for loading the competitions list (@bbearce) (CompetitionSerializer Simple vs Detail #1376) (Public competition filtering #1381)
  • Public competition filtering #1381 improves the query by directly filter public competitions. We need to improve the filter to take into account the dates and page of the public list we are displaying.
• General performance and home page
  • Remove temporarily participants and submissions counters to accelerate the competition loading URGENT - removed count from competition query to fix the codabench #1645
  • Participants and submissions counters in an efficient way (Optimization PR#2 - Submissions and Participants Count #1669)
  • Featured competitions manually set (Optimization PR#1 - Featured competitions #1678)
  • Home page counters (Optimization PR#3 - Home page counters #1699)
    • Update the counts somewhere in a file or in the db with a script that is run once a day
    • Update home page to fetch the counts from the file/db to make them dynamic instead of hardcoded

[ihsaan-ullah]

• Optimize the code for loading a competition page

This needs proper planning as code cannot be optimized with current structure of the web page. It is a big update

[Didayolo]

Still very slow after merging #1343

[bbearce]

Tested and think I have a fix:
Docs for automation to create large amounts of data

Essentially we used the CompetitionDetailSerializer to load competitions and I switched to CompetitionSerializerSimple. The detail one serializes phases and tasks and I believe that is time consuming. The Simple one has everything we need and it loads things faster on my local when I create lots of data. using selenium.

[Didayolo]

@bbearce #1376 is deployed on production, let's see if it helps!

[Didayolo]

We need to make the loading of competition page lazy.

codabench/src/apps/api/views/competitions.py

Line 160 in c2cd638

qs = qs.prefetch_related(

codabench/src/apps/api/views/submissions.py

Line 31 in c2cd638

class SubmissionViewSet(ModelViewSet):

[ihsaan-ullah]

What do you think if we separate the admin procedures from the competition detail page? Similar to Edit page we can separate the submissions, participants etc in separate pages