Submissions randomly go missing

[j-mao]

For some reason, source code for submissions 316, 318, 325, 328 don't exist, but they are still in the database and cause trouble for the servers. Should investigate why these submissions disappeared.

[n8kim1]

is this not the race condition? (i'll keep digging)

[n8kim1]

thinking abt it, it might be good to separate the bucket part and pub/sub part of compilation code into two try/catch blocks

[j-mao]

I think our race condition had been "fixed" infrastructure-side when this happened. Even so I think the backend should be agnostic to the race condition and it affects the compile server more?

I hypothesise that this could be something to do with submitting twice in a row; potentially this is another race condition inside the backend/db. e.g. 484 is missing and submitted by the same team as 482-483.

[j-mao]

This is causing many problems with the compilation queue and needs to be fixed as soon as possible.

[j-mao]

Cause appears to be HTTP 500 on /api/0/submission; the row is added to the database, but no signed upload url is given to the browser and so the code never reaches the bucket.
Further testing indicates that the job also never reaches the compile server

[j-mao]

A sketchy but effective fix:

• While making the first submit, disable the submit button to prevent a second submit from starting before the first one finishes.
• If it 500s, send the request again. Can maybe even ask the backend to delete/reuse the previous request but this is probably less important.
• If it 201s, alert success, or do something that makes it physically impossible to submit again immediately. Also re-enable the submit button. This would also resolve Disable the submit button right after it's clicked #181

[n8kim1]

justify / display reason of "can't submit while prev submission processing" (as opposed to compiling, etc)

[n8kim1]

@zoemarschner @arvid220u let's make the sketchy fix for now; a more robust solution is better in long term and probably isn't difficult to carry out