fix: ChelmsfordCityCouncil - replace Selenium with requests (emarsys popup blocks clicks)

[InertiaUK]

What

Replaces the Selenium-based scraper with a pure requests + BeautifulSoup implementation. Wall time drops from 90s+ (timeout) to ~2.5s.

Root cause

The council page now displays an emarsys popup modal (z-index: 2147483647) on first load that covers the entire viewport. Selenium's element_to_be_clickable reports the cookie ACCEPT button as clickable (it's in the DOM), but .click() raises ElementClickInterceptedException which the broad except swallows silently. The popup stays up, the postcode input is then uninteractable, and both subsequent WebDriverWait(30s) calls time out — blowing past the 90s test limit.

Fix

The search form submits via GET querystring (?<block_id>_keyword=<postcode>) and the address table, collection round text, and calendar page all render in static HTML. No browser interaction needed:

1. GET the search page → extract the dynamic block ID from the <input id="NNNN_keyword"> element
2. GET ?NNNN_keyword=<postcode> → parse the results table row matching paon → extract collection round (e.g. Tuesday B)
3. GET the round's calendar page → find the .ics link
4. Parse the ICS file for events in the next 60 days

A fallback <details> card path handles any future page structure changes.

Test

Tested against postcode CM3 7AE, paon 1 Celeborn Street, South Woodham Ferrers, Chelmsford, CM3 7AE. Returns 36 bin entries (food waste, black bin, green box, card sack / brown bin, plastics and cartons, paper sack) spanning ~60 days.

Input.json

house_number (used as paon) and postcode fields are unchanged. web_driver key is now unused but harmless.

[coderabbitai]

Warning

Rate limit exceeded

@InertiaUK has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 53 minutes and 38 seconds before requesting another review.

To keep reviews running without waiting, you can enable usage-based add-on for your organization. This allows additional reviews beyond the hourly cap. Account admins can enable it under billing.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 122d71a9-252c-4f4e-8a24-f10ce2b108bf

📥 Commits

Reviewing files that changed from the base of the PR and between 60bd3cc and f7e50b8.

📒 Files selected for processing (1)

• uk_bin_collection/uk_bin_collection/councils/ChelmsfordCityCouncil.py

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[robbrad]

Included in May 2026 Release PR #1992. Closing.