fix: MidSuffolkDistrictCouncil - replace sleep(5) with WebDriverWait, fix date parsing#1985
fix: MidSuffolkDistrictCouncil - replace sleep(5) with WebDriverWait, fix date parsing#1985InertiaUK wants to merge 1 commit into
Conversation
…x date parsing, improve address matching time.sleep(5) after clicking Find Address held Chrome idle at peak memory while the parallel Grid+local Chrome paths overlapped — causing the robbrad#17 __clone OOM crash in the Selenium Grid container. The dropdown populates in ~0.4s in practice, so the 5s sleep was pure idle contention. Changes: - Replace sleep(5) with WebDriverWait polling for a populated <select> (>1 option), cutting ~4.5s off each run and eliminating the overlap window that triggered the Grid crash - Add a second wait for at least one .card h3 after selecting the address, since JS builds collection cards asynchronously after the select change - Fix date parsing: use _DATE_RE regex to extract every well-formed date from each <p> — a single tag can contain multiple comma-separated dates in the 'Following Collections:' case; old split(':') approach raised 'unconverted data remains' errors in those cases - Improve address matching priority: exact prefix > comma-bounded > A-suffix > substring, so '91 THE COMMON' beats 'ANNEXE 91 THE COMMON' when searching for house number 91 - Remove unused import time
|
Warning Rate limit exceeded
To keep reviews running without waiting, you can enable usage-based add-on for your organization. This allows additional reviews beyond the hourly cap. Account admins can enable it under billing. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Included in May 2026 Release PR #1992. Closing. |
…e fix (supersedes earlier fix)
2 participants
What
Fixes a Selenium Grid OOM crash (
#17 0x701e18afaa64 __clone) and a latent date-parsing bug.Root cause of Grid crash
time.sleep(5)after clicking Find Address held two concurrent Chrome processes (Grid + local UC path run in parallel byrun_bin_lookup_parallel) at memory peak simultaneously. The address<select>actually populates in ~0.4s, so the 5s sleep was 4.6s of idle Chrome contention per run. On a loaded Selenium Grid that overlap window triggers an OOM/thread-clone crash inside the container.Fixes
1. Replace sleep(5) with WebDriverWait
Poll for a populated
<select>(>1 option) instead of sleeping. Exits as soon as the dropdown is ready (~0.4s). Also adds a second wait for at least one.card h3after address selection — the JS observer builds collection cards asynchronously.2. Fix date parsing for multi-date
<p>tagsWhen the council has 3 upcoming collections, the JS renders
"Following Collections: Wed 27 May 2026, Thu 18 Jun 2026"in one<p>. The oldsplit(":")+strptimeraisedunconverted data remainserrors. New approach: use_DATE_REregex to extract every well-formed date from each<p>, emitting one bin entry per date regardless of how many dates appear.3. Improve address matching priority
Ranked best-match (exact prefix → comma-bounded → A-suffix → substring) instead of first-match, so
"91 THE COMMON"beats"ANNEXE 91 THE COMMON"when searching for house number91.4. Remove unused
import timeTest
Tested against postcode
IP23 8EF, paon91. Returns 7 bin entries (Refuse, Recycling, Garden). Grid path tested concurrently with local path — both return correct results, no crash.