Running into API limits downloading code

[FerranAlet]

Hi,

When running dl.py, I'm able to fetch standings but not submissions. fetch_submission fails because the result of fetch is FAIL, more concretely:

http://codeforces.com/data/submitSource b'submissionId=46209699&csrf_token=' {'_full_url': 'http://codeforces.com/data/submitSource', 'fragment': None, 'type': 'http', 'host': 'codeforces.com', 'selector': '/data/submitSource', 'headers': {'User-agent': '', 'X-csrf-token': '', 'Cookie': ''}, 'unredirected_hdrs': {'Content-type': 'application/x-www-form-urlencoded', 'Content-length': '33', 'Host': 'codeforces.com'}, '_data': b'submissionId=46209699&csrf_token=', '_tunnel_host': None, 'origin_req_host': 'codeforces.com', 'unverifiable': False, 'timeout': <object object at 0x7f5169d121f0>} FAIL

I then changed the code to use:

req = requests.get('http://codeforces.com/contest/'+str(contest)+
                '/submission/'+str(submissionId))
        tree2 = lxml.html.fromstring(req.text)
        code = tree2.xpath('//*[@id="pageContent"]/div[3]/pre/text()')[0]

but I'm running into limits of how many codes I can download per time (seems to be about 45 every 5 or 10 minutes)

• Does your dl.py code still work for you?
• Did you run into these code download limits?

Thanks for this nice codebase!