Dec release

[robbrad]

Summary by CodeRabbit

Release Notes

• Bug Fixes

  • Improved bin collection date parsing to correctly extract multiple dates per entry.
  • Enhanced error handling with better logging and cleanup.

• Documentation

  • Updated documentation across council implementations to clarify input parameters and return structures.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[coderabbitai]

Warning

Rate limit exceeded

@robbrad has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 11 minutes and 18 seconds before requesting another review.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

📥 Commits

Reviewing files that changed from the base of the PR and between e64538a and 604a49f.

📒 Files selected for processing (3)

• .github/workflows/release.yml (1 hunks)
• uk_bin_collection/uk_bin_collection/councils/HarlowCouncil.py (2 hunks)
• uk_bin_collection/uk_bin_collection/councils/NorthumberlandCouncil.py (3 hunks)

Walkthrough

This pull request adds comprehensive docstrings to parse_data methods across 26 council parser modules, documenting input parameters, return structures, and error conditions. One file (RushcliffeBoroughCouncil) includes functional changes: updated regex to extract multiple collection dates per result, sorting by date, and improved error handling with driver cleanup. Most other modules receive input validation or expanded error documentation.

Changes

Cohort / File(s)	Summary
Docstring additions to parse_data ArgyllandButeCouncil.py, ArmaghBanbridgeCraigavonCouncil.py, BlackpoolCouncil.py, EdinburghCityCouncil.py, FarehamBoroughCouncil.py, HaltonBoroughCouncil.py, HarlowCouncil.py, KingsLynnandWestNorfolkBC.py, LondonBoroughLambeth.py, MidSussexDistrictCouncil.py, NorthumberlandCouncil.py, NorwichCityCouncil.py, RushmoorCouncil.py, SouthGloucestershireCouncil.py, SouthHollandDistrictCouncil.py, SouthLanarkshireCouncil.py, SouthamptonCityCouncil.py, ThurrockCouncil.py, WestOxfordshireDistrictCouncil.py, WiltshireCouncil.py, WinchesterCityCouncil.py	Added or updated docstrings documenting purpose, input parameters, return structure with bins containing type and collectionDate, and error conditions.
Docstring additions with input validation BirminghamCityCouncil.py	Added docstring and introduced input validation raising ValueError when uprn or postcode is missing.
Docstring updates with expanded error documentation ChelmsfordCityCouncil.py	Refined docstring description and expanded error documentation to include ValueError for missing ICS calendar link.
Multiple method docstrings IsleOfAngleseyCouncil.py	Expanded docstrings across _initialise_session, _run_lookup, _get_uprn_from_postcode_and_paon, parse_data, and _extract_bin_data describing behavior, parameters, and error conditions.
Multiple method docstrings FifeCouncil.py	Added expanded docstrings to parse_data and internal helper _best_option describing behavior, inputs, outputs, and selection logic.
Input validation and error handling enhancements NorthHertfordshireDistrictCouncil.py	Enhanced input validation in lookup_uprn with granular error messages; expanded error documentation in fetch_mobile_api; improved parse_data docstring.
Functional logic changes RushcliffeBoroughCouncil.py	Removed unused import; updated regex to extract two collection dates per result; append two bin entries per date; sort by collectionDate; enhanced error handling and added finally block for WebDriver cleanup.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~22 minutes

Areas requiring extra attention:

• RushcliffeBoroughCouncil.py — Functional changes to regex parsing logic and new sorting behavior; verify extracted dates match expected format and order
• NorthHertfordshireDistrictCouncil.py — Input validation and normalization logic; confirm error messages and validation conditions are correct
• KingsLynnandWestNorfolkBC.py — High complexity rating in original estimate; review docstring accuracy against implementation
• NorwichCityCouncil.py — High complexity rating; verify docstring parameter descriptions match actual kwargs usage

Possibly related PRs

• fix:Rushcliffe BC broken scraping #1756 — Contains identical functional changes to RushcliffeBoroughCouncil regex parsing and date extraction logic
• Dec release #1754 — Overlapping docstring and functional changes across multiple council modules (ArgyllandBute, Birmingham, Blackpool, Fife, IsleOfAnglesey, Harlow, Northumberland, Rushmoor, SouthLanarkshire, Wiltshire, etc.)
• feat: add direct mobile API support to North Hertfordshire District Council scraper #1715 — Related modifications to NorthHertfordshireDistrictCouncil input validation and mobile API handling

Poem

🐰 A hop through docstrings, a leap through code,
Bin dates clarified along the parser road,
With Rushmoor's regex now catching two,
And error messages honest and true,
The council scrapers smile, refactored anew!

Pre-merge checks and finishing touches

❌ Failed checks (1 inconclusive)

Check name	Status	Explanation	Resolution
Title check	❓ Inconclusive	The title 'Dec release' is too generic and does not meaningfully describe the actual changes in the pull request, which involve adding docstrings to 25+ council parser files.	Provide a more descriptive title that captures the main changes, such as 'Add docstrings to council parser methods' or 'Document parse_data methods across council modules'.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Docstring Coverage	✅ Passed	Docstring coverage is 97.37% which is sufficient. The required threshold is 80.00%.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[codecov]

❌ 6 Tests Failed:

Tests completed	Failed	Passed	Skipped
27	6	21	0

View the full list of 6 ❄️ flaky test(s)

uk_bin_collection.tests.step_defs.test_validate_council::test_scenario_outline[HaltonBoroughCouncil]

Flake rate in main: 100.00% (Passed 0 times, Failed 227 times)

Stack Traces | 45.3s run time

fixturefunc = <function scrape_step at 0x7f840c7999e0>
request = <FixtureRequest for <Function test_scenario_outline[HaltonBoroughCouncil]>>
kwargs = {'context': <test_validate_council.Context object at 0x7f840d8587a0>, 'headless_mode': 'True', 'local_browser': 'False', 'selenium_url': 'http://localhost:4444'}

    def call_fixture_func(
        fixturefunc: _FixtureFunc[FixtureValue], request: FixtureRequest, kwargs
    ) -> FixtureValue:
        if is_generator(fixturefunc):
            fixturefunc = cast(
                Callable[..., Generator[FixtureValue, None, None]], fixturefunc
            )
            generator = fixturefunc(**kwargs)
            try:
                fixture_result = next(generator)
            except StopIteration:
                raise ValueError(f"{request.fixturename} did not yield a value") from None
            finalizer = functools.partial(_teardown_yield_fixture, fixturefunc, generator)
            request.addfinalizer(finalizer)
        else:
            fixturefunc = cast(Callable[..., FixtureValue], fixturefunc)
>           fixture_result = fixturefunc(**kwargs)

../../../..../pypoetry/virtualenvs/uk-bin-collection-EwS6Gn8s-py3.12/lib/python3.12.../site-packages/_pytest/fixtures.py:898: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.../tests/step_defs/test_validate_council.py:101: in scrape_step
    context.parse_result = CollectData.run()
uk_bin_collection/uk_bin_collection/collect_data.py:101: in run
    return self.client_code(
uk_bin_collection/uk_bin_collection/collect_data.py:121: in client_code
    return get_bin_data_class.template_method(address_url, **kwargs)
uk_bin_collection/uk_bin_collection/get_bin_data.py:61: in template_method
    bin_data_dict = self.get_and_parse_data(this_url, **kwargs)
uk_bin_collection/uk_bin_collection/get_bin_data.py:84: in get_and_parse_data
    bin_data_dict = self.parse_data("", url=address_url, **kwargs)
.../uk_bin_collection/councils/HaltonBoroughCouncil.py:101: in parse_data
    WebDriverWait(driver, 10).until(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <selenium.webdriver.support.wait.WebDriverWait (session="3c9f7912b657bbcf34d8a06a38efb457")>
method = <function presence_of_element_located.<locals>._predicate at 0x7f840c536160>
message = ''

    def until(self, method: Callable[[D], Union[Literal[False], T]], message: str = "") -> T:
        """Wait until the method returns a value that is not False.
    
        Calls the method provided with the driver as an argument until the
        return value does not evaluate to ``False``.
    
        Parameters:
        -----------
        method: callable(WebDriver)
            - A callable object that takes a WebDriver instance as an argument.
    
        message: str
            - Optional message for :exc:`TimeoutException`
    
        Return:
        -------
        object: T
            - The result of the last call to `method`
    
        Raises:
        -------
        TimeoutException
            - If 'method' does not return a truthy value within the WebDriverWait
            object's timeout
    
        Example:
        --------
        >>> from selenium.webdriver.common.by import By
        >>> from selenium.webdriver.support.ui import WebDriverWait
        >>> from selenium.webdriver.support import expected_conditions as EC
    
        # Wait until an element is visible on the page
        >>> wait = WebDriverWait(driver, 10)
        >>> element = wait.until(EC.visibility_of_element_located((By.ID, "exampleId")))
        >>> print(element.text)
        """
        screen = None
        stacktrace = None
    
        end_time = time.monotonic() + self._timeout
        while True:
            try:
                value = method(self._driver)
                if value:
                    return value
            except self._ignored_exceptions as exc:
                screen = getattr(exc, "screen", None)
                stacktrace = getattr(exc, "stacktrace", None)
            if time.monotonic() > end_time:
                break
            time.sleep(self._poll)
>       raise TimeoutException(message, screen, stacktrace)
E       selenium.common.exceptions.TimeoutException: Message: 
E       Stacktrace:
E       #0 0x55c36f0a2aea <unknown>
E       #1 0x55c36eaeecdb <unknown>
E       #2 0x55c36eb416c4 <unknown>
E       #3 0x55c36eb41901 <unknown>
E       #4 0x55c36eb908b4 <unknown>
E       #5 0x55c36eb8dc87 <unknown>
E       #6 0x55c36eb33aca <unknown>
E       #7 0x55c36eb347d1 <unknown>
E       #8 0x55c36f069ab9 <unknown>
E       #9 0x55c36f06ca8c <unknown>
E       #10 0x55c36f052d49 <unknown>
E       #11 0x55c36f06d685 <unknown>
E       #12 0x55c36f03a6c3 <unknown>
E       #13 0x55c36f08f7d8 <unknown>
E       #14 0x55c36f08f9b3 <unknown>
E       #15 0x55c36f0a1a83 <unknown>
E       #16 0x7f35a70a6aa4 <unknown>
E       #17 0x7f35a7133a64 __clone

../../../..../pypoetry/virtualenvs/uk-bin-collection-EwS6Gn8s-py3.12/lib/python3.12.../webdriver/support/wait.py:146: TimeoutException

uk_bin_collection.tests.step_defs.test_validate_council::test_scenario_outline[NorthHertfordshireDistrictCouncil]

Flake rate in main: 56.36% (Passed 96 times, Failed 124 times)

Stack Traces | 0.493s run time

fixturefunc = <function scrape_step at 0x7f6d2859f6a0>
request = <FixtureRequest for <Function test_scenario_outline[NorthHertfordshireDistrictCouncil]>>
kwargs = {'context': <test_validate_council.Context object at 0x7f6d3cc196d0>, 'headless_mode': 'True', 'local_browser': 'False', 'selenium_url': 'http://localhost:4444'}

    def call_fixture_func(
        fixturefunc: _FixtureFunc[FixtureValue], request: FixtureRequest, kwargs
    ) -> FixtureValue:
        if is_generator(fixturefunc):
            fixturefunc = cast(
                Callable[..., Generator[FixtureValue, None, None]], fixturefunc
            )
            generator = fixturefunc(**kwargs)
            try:
                fixture_result = next(generator)
            except StopIteration:
                raise ValueError(f"{request.fixturename} did not yield a value") from None
            finalizer = functools.partial(_teardown_yield_fixture, fixturefunc, generator)
            request.addfinalizer(finalizer)
        else:
            fixturefunc = cast(Callable[..., FixtureValue], fixturefunc)
>           fixture_result = fixturefunc(**kwargs)

../../../..../pypoetry/virtualenvs/uk-bin-collection-EwS6Gn8s-py3.12/lib/python3.12.../site-packages/_pytest/fixtures.py:898: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.../tests/step_defs/test_validate_council.py:101: in scrape_step
    context.parse_result = CollectData.run()
uk_bin_collection/uk_bin_collection/collect_data.py:101: in run
    return self.client_code(
uk_bin_collection/uk_bin_collection/collect_data.py:121: in client_code
    return get_bin_data_class.template_method(address_url, **kwargs)
uk_bin_collection/uk_bin_collection/get_bin_data.py:61: in template_method
    bin_data_dict = self.get_and_parse_data(this_url, **kwargs)
uk_bin_collection/uk_bin_collection/get_bin_data.py:84: in get_and_parse_data
    bin_data_dict = self.parse_data("", url=address_url, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <NorthHertfordshireDistrictCouncil.CouncilClass object at 0x7f6d27b29ee0>
page = ''
kwargs = {'council_module_str': 'NorthHertfordshireDistrictCouncil', 'dev_mode': False, 'headless': True, 'local_browser': False, ...}
bins_with_sort_date = [], uprn = '100081258147'
api_response = {'wasteCollectionDates': {'cacheKey': '1.0.959', 'container1CollectionDetails': None, 'container2CollectionDetails': None, 'container3CollectionDetails': None, ...}}
waste_collection_dates = {'cacheKey': '1.0.959', 'container1CollectionDetails': None, 'container2CollectionDetails': None, 'container3CollectionDetails': None, ...}
container_num = 8

    def parse_data(self, page: str, **kwargs) -> dict:
        """
        Parse and return sorted bin collection entries for an address using the Cloud9 mobile API.
    
        Parameters:
            page (str): Unused; present for interface compatibility.
            **kwargs: Must include either:
                - uprn (str): Validated UPRN to query the mobile API.
                OR
                - postcode (str) and paon (str): Postcode and property identifier used to resolve a UPRN via lookup_uprn.
    
        Returns:
            dict: {"bins": [entry, ...]} where each entry is a dict with:
                - "type" (str): Container description (e.g., "Refuse", "Recycling", or a default "Container N").
                - "collectionDate" (str): Date formatted according to the module's date_format.
    
        Raises:
            ValueError: If inputs are missing/invalid, UPRN lookup fails, the API response lacks wasteCollectionDates,
                        no valid collection dates can be extracted, or the API request/response is malformed.
        """
        bins_with_sort_date = []
    
        # Get UPRN either directly or via lookup
    
        uprn = kwargs.get("uprn")
    
        if uprn:
            check_uprn(uprn)
        else:
            # Try to lookup UPRN from postcode and house number
            # This is provided to maintain backward compatibility with the existing postcode/paon input method
            postcode = kwargs.get("postcode")
            paon = kwargs.get("paon")
            check_postcode(postcode)
            check_paon(paon)
    
            # Attempt UPRN lookup using postcode and paon
            uprn = lookup_uprn(postcode=postcode, paon=paon)
    
    
        # Fetch data from mobile API
        api_response = fetch_mobile_api(uprn)
    
        # Parse the API response - Cloud9 API returns WasteCollectionDates with 8 containers
        # Response structure: {"wasteCollectionDates": {"container1CollectionDetails": {...}, ...}}
        waste_collection_dates = api_response.get("wasteCollectionDates", {})
    
        if not waste_collection_dates:
            raise ValueError(
                f"No wasteCollectionDates found in API response. API response: {json.dumps(api_response)[:200]}"
            )
    
        # Process all 8 possible containers
        for container_num in range(1, MOBILE_API_NUM_CONTAINERS + 1):
            container_key = f"container{container_num}CollectionDetails"
            container = waste_collection_dates.get(container_key)
    
            if not container or not isinstance(container, dict):
                continue
    
            # Extract collection date
            collection_date_str = container.get("collectionDate", "")
    
            # Skip empty collection dates
            if not collection_date_str:
                continue
    
            # Extract container description (bin type)
            bin_type = container.get("containerDescription", f"Container {container_num}")
            try:
                collection_datetime = datetime.fromisoformat(collection_date_str)
            except ValueError:
                # skip bins with invalid date format and continue processing
                continue
    
            # Parse the date - API returns ISO format like "2025-11-25T00:00:00"
            bin_entry = {
                "type": bin_type,
                "collectionDate": collection_datetime.strftime(date_format),
                "_sort_date": collection_datetime
            }
    
            bins_with_sort_date.append(bin_entry)
    
        if not bins_with_sort_date:
>           raise ValueError(
                "No valid bin collection data could be extracted from the API response"
            )
E           ValueError: No valid bin collection data could be extracted from the API response

.../uk_bin_collection/councils/NorthHertfordshireDistrictCouncil.py:245: ValueError

uk_bin_collection.tests.step_defs.test_validate_council::test_scenario_outline[NorthumberlandCouncil]

Flake rate in main: 75.22% (Passed 56 times, Failed 170 times)

Stack Traces | 30.7s run time

fixturefunc = <function scrape_step at 0x7f0e19206d40>
request = <FixtureRequest for <Function test_scenario_outline[NorthumberlandCouncil]>>
kwargs = {'context': <test_validate_council.Context object at 0x7f0e30ffcd10>, 'headless_mode': 'True', 'local_browser': 'False', 'selenium_url': 'http://localhost:4444'}

    def call_fixture_func(
        fixturefunc: _FixtureFunc[FixtureValue], request: FixtureRequest, kwargs
    ) -> FixtureValue:
        if is_generator(fixturefunc):
            fixturefunc = cast(
                Callable[..., Generator[FixtureValue, None, None]], fixturefunc
            )
            generator = fixturefunc(**kwargs)
            try:
                fixture_result = next(generator)
            except StopIteration:
                raise ValueError(f"{request.fixturename} did not yield a value") from None
            finalizer = functools.partial(_teardown_yield_fixture, fixturefunc, generator)
            request.addfinalizer(finalizer)
        else:
            fixturefunc = cast(Callable[..., FixtureValue], fixturefunc)
>           fixture_result = fixturefunc(**kwargs)

../../../..../pypoetry/virtualenvs/uk-bin-collection-EwS6Gn8s-py3.12/lib/python3.12.../site-packages/_pytest/fixtures.py:898: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.../tests/step_defs/test_validate_council.py:101: in scrape_step
    context.parse_result = CollectData.run()
uk_bin_collection/uk_bin_collection/collect_data.py:101: in run
    return self.client_code(
uk_bin_collection/uk_bin_collection/collect_data.py:121: in client_code
    return get_bin_data_class.template_method(address_url, **kwargs)
uk_bin_collection/uk_bin_collection/get_bin_data.py:61: in template_method
    bin_data_dict = self.get_and_parse_data(this_url, **kwargs)
uk_bin_collection/uk_bin_collection/get_bin_data.py:84: in get_and_parse_data
    bin_data_dict = self.parse_data("", url=address_url, **kwargs)
.../uk_bin_collection/councils/NorthumberlandCouncil.py:81: in parse_data
    cookie_button = wait.until(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <selenium.webdriver.support.wait.WebDriverWait (session="8b2c0fe2763eec160e01144ecac3ee37")>
method = <function element_to_be_clickable.<locals>._predicate at 0x7f0e187d7380>
message = ''

    def until(self, method: Callable[[D], Union[Literal[False], T]], message: str = "") -> T:
        """Wait until the method returns a value that is not False.
    
        Calls the method provided with the driver as an argument until the
        return value does not evaluate to ``False``.
    
        Parameters:
        -----------
        method: callable(WebDriver)
            - A callable object that takes a WebDriver instance as an argument.
    
        message: str
            - Optional message for :exc:`TimeoutException`
    
        Return:
        -------
        object: T
            - The result of the last call to `method`
    
        Raises:
        -------
        TimeoutException
            - If 'method' does not return a truthy value within the WebDriverWait
            object's timeout
    
        Example:
        --------
        >>> from selenium.webdriver.common.by import By
        >>> from selenium.webdriver.support.ui import WebDriverWait
        >>> from selenium.webdriver.support import expected_conditions as EC
    
        # Wait until an element is visible on the page
        >>> wait = WebDriverWait(driver, 10)
        >>> element = wait.until(EC.visibility_of_element_located((By.ID, "exampleId")))
        >>> print(element.text)
        """
        screen = None
        stacktrace = None
    
        end_time = time.monotonic() + self._timeout
        while True:
            try:
                value = method(self._driver)
                if value:
                    return value
            except self._ignored_exceptions as exc:
                screen = getattr(exc, "screen", None)
                stacktrace = getattr(exc, "stacktrace", None)
            if time.monotonic() > end_time:
                break
            time.sleep(self._poll)
>       raise TimeoutException(message, screen, stacktrace)
E       selenium.common.exceptions.TimeoutException: Message: 
E       Stacktrace:
E       #0 0x561877d28aea <unknown>
E       #1 0x561877774cdb <unknown>
E       #2 0x5618777c76c4 <unknown>
E       #3 0x5618777c7901 <unknown>
E       #4 0x5618778168b4 <unknown>
E       #5 0x561877813c87 <unknown>
E       #6 0x5618777b9aca <unknown>
E       #7 0x5618777ba7d1 <unknown>
E       #8 0x561877cefab9 <unknown>
E       #9 0x561877cf2a8c <unknown>
E       #10 0x561877cd8d49 <unknown>
E       #11 0x561877cf3685 <unknown>
E       #12 0x561877cc06c3 <unknown>
E       #13 0x561877d157d8 <unknown>
E       #14 0x561877d159b3 <unknown>
E       #15 0x561877d27a83 <unknown>
E       #16 0x7f96169bdaa4 <unknown>
E       #17 0x7f9616a4aa64 __clone

../../../..../pypoetry/virtualenvs/uk-bin-collection-EwS6Gn8s-py3.12/lib/python3.12.../webdriver/support/wait.py:146: TimeoutException

uk_bin_collection.tests.step_defs.test_validate_council::test_scenario_outline[SouthHollandDistrictCouncil]

Flake rate in main: 78.71% (Passed 43 times, Failed 159 times)

Stack Traces | 0.39s run time

self = <Response [403]>, kwargs = {}

    def json(self, **kwargs):
        r"""Returns the json-encoded content of a response, if any.
    
        :param \*\*kwargs: Optional arguments that ``json.loads`` takes.
        :raises requests.exceptions.JSONDecodeError: If the response body does not
            contain valid json.
        """
    
        if not self.encoding and self.content and len(self.content) > 3:
            # No encoding set. JSON RFC 4627 section 3 states we should expect
            # UTF-8, -16 or -32. Detect which one to use; If the detection or
            # decoding fails, fall back to `self.text` (using charset_normalizer to make
            # a best guess).
            encoding = guess_json_utf(self.content)
            if encoding is not None:
                try:
                    return complexjson.loads(self.content.decode(encoding), **kwargs)
                except UnicodeDecodeError:
                    # Wrong UTF codec detected; usually because it's not UTF-8
                    # but some other 8-bit codec.  This is an RFC violation,
                    # and the server didn't bother to tell us what codec *was*
                    # used.
                    pass
                except JSONDecodeError as e:
                    raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
    
        try:
>           return complexjson.loads(self.text, **kwargs)

../../../..../pypoetry/virtualenvs/uk-bin-collection-EwS6Gn8s-py3.12/lib/python3.12....../site-packages/requests/models.py:971: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
....../usr/lib/python3.12/json/__init__.py:346: in loads
    return _default_decoder.decode(s)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <json.decoder.JSONDecoder object at 0x7f87ecaa2c00>, s = '403 Forbidden'
_w = <built-in method match of re.Pattern object at 0x7f87ecab8ba0>

    def decode(self, s, _w=WHITESPACE.match):
        """Return the Python representation of ``s`` (a ``str`` instance
        containing a JSON document).
    
        """
        obj, end = self.raw_decode(s, idx=_w(s, 0).end())
        end = _w(s, end).end()
        if end != len(s):
>           raise JSONDecodeError("Extra data", s, end)
E           json.decoder.JSONDecodeError: Extra data: line 1 column 5 (char 4)

....../usr/lib/python3.12/json/decoder.py:340: JSONDecodeError

During handling of the above exception, another exception occurred:

fixturefunc = <function scrape_step at 0x7f87d6b8b560>
request = <FixtureRequest for <Function test_scenario_outline[SouthHollandDistrictCouncil]>>
kwargs = {'context': <test_validate_council.Context object at 0x7f87d856d8e0>, 'headless_mode': 'True', 'local_browser': 'False', 'selenium_url': 'http://localhost:4444'}

    def call_fixture_func(
        fixturefunc: _FixtureFunc[FixtureValue], request: FixtureRequest, kwargs
    ) -> FixtureValue:
        if is_generator(fixturefunc):
            fixturefunc = cast(
                Callable[..., Generator[FixtureValue, None, None]], fixturefunc
            )
            generator = fixturefunc(**kwargs)
            try:
                fixture_result = next(generator)
            except StopIteration:
                raise ValueError(f"{request.fixturename} did not yield a value") from None
            finalizer = functools.partial(_teardown_yield_fixture, fixturefunc, generator)
            request.addfinalizer(finalizer)
        else:
            fixturefunc = cast(Callable[..., FixtureValue], fixturefunc)
>           fixture_result = fixturefunc(**kwargs)

../../../..../pypoetry/virtualenvs/uk-bin-collection-EwS6Gn8s-py3.12/lib/python3.12.../site-packages/_pytest/fixtures.py:898: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.../tests/step_defs/test_validate_council.py:101: in scrape_step
    context.parse_result = CollectData.run()
uk_bin_collection/uk_bin_collection/collect_data.py:101: in run
    return self.client_code(
uk_bin_collection/uk_bin_collection/collect_data.py:121: in client_code
    return get_bin_data_class.template_method(address_url, **kwargs)
uk_bin_collection/uk_bin_collection/get_bin_data.py:61: in template_method
    bin_data_dict = self.get_and_parse_data(this_url, **kwargs)
uk_bin_collection/uk_bin_collection/get_bin_data.py:84: in get_and_parse_data
    bin_data_dict = self.parse_data("", url=address_url, **kwargs)
.../uk_bin_collection/councils/SouthHollandDistrictCouncil.py:50: in parse_data
    bin_collection = response.json()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <Response [403]>, kwargs = {}

    def json(self, **kwargs):
        r"""Returns the json-encoded content of a response, if any.
    
        :param \*\*kwargs: Optional arguments that ``json.loads`` takes.
        :raises requests.exceptions.JSONDecodeError: If the response body does not
            contain valid json.
        """
    
        if not self.encoding and self.content and len(self.content) > 3:
            # No encoding set. JSON RFC 4627 section 3 states we should expect
            # UTF-8, -16 or -32. Detect which one to use; If the detection or
            # decoding fails, fall back to `self.text` (using charset_normalizer to make
            # a best guess).
            encoding = guess_json_utf(self.content)
            if encoding is not None:
                try:
                    return complexjson.loads(self.content.decode(encoding), **kwargs)
                except UnicodeDecodeError:
                    # Wrong UTF codec detected; usually because it's not UTF-8
                    # but some other 8-bit codec.  This is an RFC violation,
                    # and the server didn't bother to tell us what codec *was*
                    # used.
                    pass
                except JSONDecodeError as e:
                    raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
    
        try:
            return complexjson.loads(self.text, **kwargs)
        except JSONDecodeError as e:
            # Catch JSON-related errors and raise as requests.JSONDecodeError
            # This aliases json.JSONDecodeError and simplejson.JSONDecodeError
>           raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
E           requests.exceptions.JSONDecodeError: Extra data: line 1 column 5 (char 4)

../../../..../pypoetry/virtualenvs/uk-bin-collection-EwS6Gn8s-py3.12/lib/python3.12....../site-packages/requests/models.py:975: JSONDecodeError

uk_bin_collection.tests.step_defs.test_validate_council::test_scenario_outline[SouthLanarkshireCouncil]

Flake rate in main: 40.10% (Passed 115 times, Failed 77 times)

Stack Traces | 0.856s run time

fixturefunc = <function scrape_step at 0x7f87d6b8b560>
request = <FixtureRequest for <Function test_scenario_outline[SouthLanarkshireCouncil]>>
kwargs = {'context': <test_validate_council.Context object at 0x7f87d856d8e0>, 'headless_mode': 'True', 'local_browser': 'False', 'selenium_url': 'http://localhost:4444'}

    def call_fixture_func(
        fixturefunc: _FixtureFunc[FixtureValue], request: FixtureRequest, kwargs
    ) -> FixtureValue:
        if is_generator(fixturefunc):
            fixturefunc = cast(
                Callable[..., Generator[FixtureValue, None, None]], fixturefunc
            )
            generator = fixturefunc(**kwargs)
            try:
                fixture_result = next(generator)
            except StopIteration:
                raise ValueError(f"{request.fixturename} did not yield a value") from None
            finalizer = functools.partial(_teardown_yield_fixture, fixturefunc, generator)
            request.addfinalizer(finalizer)
        else:
            fixturefunc = cast(Callable[..., FixtureValue], fixturefunc)
>           fixture_result = fixturefunc(**kwargs)

../../../..../pypoetry/virtualenvs/uk-bin-collection-EwS6Gn8s-py3.12/lib/python3.12.../site-packages/_pytest/fixtures.py:898: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.../tests/step_defs/test_validate_council.py:101: in scrape_step
    context.parse_result = CollectData.run()
uk_bin_collection/uk_bin_collection/collect_data.py:101: in run
    return self.client_code(
uk_bin_collection/uk_bin_collection/collect_data.py:121: in client_code
    return get_bin_data_class.template_method(address_url, **kwargs)
uk_bin_collection/uk_bin_collection/get_bin_data.py:61: in template_method
    bin_data_dict = self.get_and_parse_data(this_url, **kwargs)
uk_bin_collection/uk_bin_collection/get_bin_data.py:82: in get_and_parse_data
    bin_data_dict = self.parse_data(page, url=address_url, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <SouthLanarkshireCouncil.CouncilClass object at 0x7f87d616f230>
page = <Response [200]>
kwargs = {'council_module_str': 'SouthLanarkshireCouncil', 'dev_mode': False, 'headless': True, 'local_browser': False, ...}
data = {'bins': []}
collection_types = ['non recyclable waste', 'food and garden', 'paper and card', 'glass, cans and plastics']
soup = <!DOCTYPE html>

<html lang="en">
<!--[if lt IE 7]> <html class="no-js ie6 oldie" lang="en"> <![endif]-->
<!--[if IE 7...4933156"></script>
<script src="//www.southlanarkshire.gov..../site/javascript/site.js"></script>
</div></body>
</html>

week_details = <div class="bin-dir-snip"><div class="clearfix">This week's collection<p>Monday 8 December 2025 to Friday 12 December ...plastic bags.</p></h4><img alt="red bin" src="https://www.southlanarkshire.gov.uk/images/bin-red.png"/></li></ul></div>
week_dates = <p>Monday 8 December 2025 to Friday 12 December 2025 </p>
week_collections = [<h4><a href="https://www.southlanarkshire.gov..../bins_and_recycling/1841/bins_-_what_goes_in_them/" title...dy bin - food and garden waste">Burgundy bin - food and garden waste</a>
<p>No liquids, oils or plastic bags.</p></h4>]
results = <re.Match object; span=(0, 49), match='Monday 8 December 2025 to Friday 12 December 2025>

    def parse_data(self, page: str, **kwargs) -> dict:
        """
        Parse an HTML page to extract scheduled bin collection types and their collection dates.
    
        Parameters:
            page: An object with a `text` attribute containing the HTML of the council's bin collection page (e.g., an HTTP response).
    
        Returns:
            dict: A dictionary with a "bins" key mapping to a list of collections, where each collection is a dict with:
                - "type": the collection description string (e.g., "Garden waste")
                - "collectionDate": the collection date formatted according to the module's `date_format`
        """
        data = {"bins": []}
        collection_types = [
            "non recyclable waste",
            "food and garden",
            "paper and card",
            "glass, cans and plastics",
        ]
    
        # Make a BS4 object
        soup = BeautifulSoup(page.text, features="html.parser")
        soup.prettify()
    
        week_details = soup.find("div", {"class": "bin-dir-snip"})
        week_dates = week_details.find("div", {"class": "clearfix"}).find("p")
        week_collections = week_details.find_all_next("h4")
    
        results = re.search(
            "([A-Za-z0-9 ]+) to ([A-Za-z0-9 ]+)", week_dates.get_text().strip()
        )
        if results:
            week_start = datetime.strptime(results.groups()[0], "%A %d %B %Y")
            week_end = datetime.strptime(results.groups()[1], "%A %d %B %Y")
            week_days = (
                week_start + timedelta(days=i)
                for i in range((week_end - week_start).days + 1)
            )
    
            week_collection_types = []
            for week_collection in week_collections:
                week_collection = (
                    week_collection.get_text().strip().lower().replace("-", " ")
                )
                for collection_type in collection_types:
                    if collection_type in week_collection:
                        week_collection_types.append(collection_type)
    
            collection_schedule = (
                soup.find("div", {"class": "serviceDetails"})
                .find("table")
                .find_all_next("tr")
            )
            for day in week_days:
                for row in collection_schedule:
                    schedule_type = row.find("th").get_text().strip()
                    results2 = re.search("([^(]+)", row.find("td").get_text().strip())
>                   schedule_cadence = row.find("td").get_text().strip().split(" ")[1]
E                   IndexError: list index out of range

.../uk_bin_collection/councils/SouthLanarkshireCouncil.py:75: IndexError

uk_bin_collection.tests.step_defs.test_validate_council::test_scenario_outline[WestOxfordshireDistrictCouncil]

Flake rate in main: 67.57% (Passed 72 times, Failed 150 times)

Stack Traces | 103s run time

fixturefunc = <function scrape_step at 0x7f87d6b8b560>
request = <FixtureRequest for <Function test_scenario_outline[WestOxfordshireDistrictCouncil]>>
kwargs = {'context': <test_validate_council.Context object at 0x7f87d856d8e0>, 'headless_mode': 'True', 'local_browser': 'False', 'selenium_url': 'http://localhost:4444'}

    def call_fixture_func(
        fixturefunc: _FixtureFunc[FixtureValue], request: FixtureRequest, kwargs
    ) -> FixtureValue:
        if is_generator(fixturefunc):
            fixturefunc = cast(
                Callable[..., Generator[FixtureValue, None, None]], fixturefunc
            )
            generator = fixturefunc(**kwargs)
            try:
                fixture_result = next(generator)
            except StopIteration:
                raise ValueError(f"{request.fixturename} did not yield a value") from None
            finalizer = functools.partial(_teardown_yield_fixture, fixturefunc, generator)
            request.addfinalizer(finalizer)
        else:
            fixturefunc = cast(Callable[..., FixtureValue], fixturefunc)
>           fixture_result = fixturefunc(**kwargs)

../../../..../pypoetry/virtualenvs/uk-bin-collection-EwS6Gn8s-py3.12/lib/python3.12.../site-packages/_pytest/fixtures.py:898: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.../tests/step_defs/test_validate_council.py:101: in scrape_step
    context.parse_result = CollectData.run()
uk_bin_collection/uk_bin_collection/collect_data.py:101: in run
    return self.client_code(
uk_bin_collection/uk_bin_collection/collect_data.py:121: in client_code
    return get_bin_data_class.template_method(address_url, **kwargs)
uk_bin_collection/uk_bin_collection/get_bin_data.py:61: in template_method
    bin_data_dict = self.get_and_parse_data(this_url, **kwargs)
uk_bin_collection/uk_bin_collection/get_bin_data.py:84: in get_and_parse_data
    bin_data_dict = self.parse_data("", url=address_url, **kwargs)
.../uk_bin_collection/councils/WestOxfordshireDistrictCouncil.py:72: in parse_data
    first_found_address = wait.until(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <selenium.webdriver.support.wait.WebDriverWait (session="930b273933109c3cb33c13c56c7e34fe")>
method = <function element_to_be_clickable.<locals>._predicate at 0x7f87e7113ec0>
message = ''

    def until(self, method: Callable[[D], Union[Literal[False], T]], message: str = "") -> T:
        """Wait until the method returns a value that is not False.
    
        Calls the method provided with the driver as an argument until the
        return value does not evaluate to ``False``.
    
        Parameters:
        -----------
        method: callable(WebDriver)
            - A callable object that takes a WebDriver instance as an argument.
    
        message: str
            - Optional message for :exc:`TimeoutException`
    
        Return:
        -------
        object: T
            - The result of the last call to `method`
    
        Raises:
        -------
        TimeoutException
            - If 'method' does not return a truthy value within the WebDriverWait
            object's timeout
    
        Example:
        --------
        >>> from selenium.webdriver.common.by import By
        >>> from selenium.webdriver.support.ui import WebDriverWait
        >>> from selenium.webdriver.support import expected_conditions as EC
    
        # Wait until an element is visible on the page
        >>> wait = WebDriverWait(driver, 10)
        >>> element = wait.until(EC.visibility_of_element_located((By.ID, "exampleId")))
        >>> print(element.text)
        """
        screen = None
        stacktrace = None
    
        end_time = time.monotonic() + self._timeout
        while True:
            try:
                value = method(self._driver)
                if value:
                    return value
            except self._ignored_exceptions as exc:
                screen = getattr(exc, "screen", None)
                stacktrace = getattr(exc, "stacktrace", None)
            if time.monotonic() > end_time:
                break
            time.sleep(self._poll)
>       raise TimeoutException(message, screen, stacktrace)
E       selenium.common.exceptions.TimeoutException: Message: 
E       Stacktrace:
E       #0 0x557d7128caea <unknown>
E       #1 0x557d70cd8cdb <unknown>
E       #2 0x557d70d2b6c4 <unknown>
E       #3 0x557d70d2b901 <unknown>
E       #4 0x557d70d7a8b4 <unknown>
E       #5 0x557d70d77c87 <unknown>
E       #6 0x557d70d1daca <unknown>
E       #7 0x557d70d1e7d1 <unknown>
E       #8 0x557d71253ab9 <unknown>
E       #9 0x557d71256a8c <unknown>
E       #10 0x557d7123cd49 <unknown>
E       #11 0x557d71257685 <unknown>
E       #12 0x557d712246c3 <unknown>
E       #13 0x557d712797d8 <unknown>
E       #14 0x557d712799b3 <unknown>
E       #15 0x557d7128ba83 <unknown>
E       #16 0x7fcbb03f7aa4 <unknown>
E       #17 0x7fcbb0484a64 __clone

../../../..../pypoetry/virtualenvs/uk-bin-collection-EwS6Gn8s-py3.12/lib/python3.12.../webdriver/support/wait.py:146: TimeoutException

To view more test analytics, go to the Test Analytics Dashboard
_{📋 Got 3 mins? Take this short survey to help us improve Test Analytics.}

[coderabbitai]

Actionable comments posted: 3

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

uk_bin_collection/uk_bin_collection/councils/HarlowCouncil.py (1)

1-3: Remove duplicate import.

requests is imported twice (lines 1 and 3).

-import requests
-
 import requests
 from bs4 import BeautifulSoup

🧹 Nitpick comments (12)

uk_bin_collection/uk_bin_collection/councils/ThurrockCouncil.py (1)

45-46: Missing input validation could cause confusing errors.

If collection_day is not in days_of_week or round is not in round_week, list.index() will raise a ValueError with a generic message. Consider validating inputs early with descriptive errors, consistent with the pattern used in other councils (e.g., check_postcode, check_uprn).

+        if collection_day not in days_of_week:
+            raise ValueError(f"Invalid collection day: {collection_day}. Expected one of {days_of_week}")
+        if round not in round_week:
+            raise ValueError(f"Invalid round: {round}. Expected 'Round A' or 'Round B'")
         offset_days = days_of_week.index(collection_day)
         round_collection = round_week.index(round)

uk_bin_collection/uk_bin_collection/councils/HarlowCouncil.py (1)

52-53: Potential AttributeError if summary is not found.

If the page structure changes and no div.summary is found, summary will be None and summary.find_all() will raise an AttributeError. Consider adding a guard or raising a descriptive error.

         summary = soup.find("div", {"class": "summary"})
+        if not summary:
+            raise ValueError("Could not find collection summary on page")
         collectionrows = summary.find_all("div", {"class": "collectionsrow"})

uk_bin_collection/uk_bin_collection/councils/KingsLynnandWestNorfolkBC.py (1)

47-47: soup.prettify() result is unused.

prettify() returns a formatted string but doesn't modify soup in place. This line has no effect and can be removed.

         soup = BeautifulSoup(response.content, features="html.parser")
-        soup.prettify()

uk_bin_collection/uk_bin_collection/councils/NorthumberlandCouncil.py (1)

1-3: datetime module is shadowed by the class import.

Line 1 imports the datetime module, but line 3 imports datetime class which shadows the module. The module import on line 1 becomes unreachable. If only the class is needed, remove line 1.

-import datetime
 import time
 from datetime import datetime

uk_bin_collection/uk_bin_collection/councils/ArmaghBanbridgeCraigavonCouncil.py (1)

18-31: Docstring accurately documents the happy path.

The docstring clearly describes parameters and return structure. However, consider updating the "Raises" section or adding a note that the function returns an empty bins list (rather than raising) when the HTTP request fails (line 96 only prints an error message).

         Returns:
             dict: Dictionary with a "bins" key mapping to a list of collections. Each collection is a dict with:
                 - "collectionDate" (str): Date in "DD/MM/YYYY" format.
                 - "type" (str): One of "Domestic", "Recycling", or "Garden".
             The list is sorted in ascending order by the parsed collection date (format "%d/%m/%Y").
+        
+        Note:
+            Returns an empty bins list if the HTTP request fails (non-200 status code).
         """

uk_bin_collection/uk_bin_collection/councils/SouthGloucestershireCouncil.py (1)

7-20: Consider removing debug print statements.

Line 10 contains print(servicename) which appears to be debug output. Similarly, line 63 has print(collection). Consider removing these or converting to proper logging to avoid noise in production.

 def format_bin_data(key: str, date: datetime):
     formatted_date = date.strftime(date_format)
     servicename = key.get("hso_servicename")
-    print(servicename)
     if re.match(r"^Recycl", servicename) is not None:

And in parse_data:

         for collection in collection_data:
-            print(collection)
             item = collection.get('hso_nextcollection')

uk_bin_collection/uk_bin_collection/councils/EdinburghCityCouncil.py (1)

58-65: Consider externalizing hardcoded start dates.

The collection start dates are hardcoded to November 2025. While functional now, these will need periodic updates to maintain accuracy as collection schedules evolve. Consider storing these dates in a configuration file or database to simplify future maintenance.

uk_bin_collection/uk_bin_collection/councils/RushcliffeBoroughCouncil.py (1)

7-7: Consider explicit imports for clarity and static analysis compatibility.

The static analysis tool flags datetime and date_format as potentially undefined due to the star import. While these are provided by the common module and work at runtime, explicit imports (as done in NorthHertfordshireDistrictCouncil.py) improve code clarity and IDE support.

-from uk_bin_collection.uk_bin_collection.common import *
+from datetime import datetime
+from uk_bin_collection.uk_bin_collection.common import (
+    check_postcode,
+    check_uprn,
+    create_webdriver,
+    date_format,
+)

uk_bin_collection/uk_bin_collection/councils/RushmoorCouncil.py (1)

18-34: Docstring accurately describes flow; consider expanding the Raises section

The implementation can raise a ValueError both when check_uprn(user_uprn) fails and when no collections are found. To keep the docstring aligned with behavior, consider mentioning that invalid UPRNs will also result in a ValueError, not just the “no collections found” case.

uk_bin_collection/uk_bin_collection/councils/FarehamBoroughCouncil.py (1)

16-31: Docstring looks good; minor clarity improvements for errors and date format

The function can also raise a ValueError from check_postcode(user_postcode) when the postcode is invalid, not only when it is “not found on the website”. If you want the docstring to be fully explicit, you could mention this validation error separately. Likewise, since collectionDate is formatted using date_format, you might phrase the return description as “formatted according to date_format (currently DD/MM/YYYY)" so it stays correct if the global format ever changes.

uk_bin_collection/uk_bin_collection/councils/ArgyllandButeCouncil.py (1)

21-36: Clear docstring; consider documenting validation and failure modes

The parameter and return descriptions match the implementation well, including UPRN normalization and date_format usage. If you want parity with other councils in this PR, you could add a brief Raises section noting that invalid uprn/postcode will cause check_uprn/check_postcode to raise ValueError, and that any Selenium/navigation issues will propagate as exceptions.

uk_bin_collection/uk_bin_collection/councils/BirminghamCityCouncil.py (1)

67-84: Tighten parameter typing/validation to match real usage and messages

Two small inconsistencies worth addressing:

• page is annotated and documented as str, but it’s passed directly to get_token(page), which accesses page.text. That implies page is a response-like object, not a plain string. Consider updating the type hint and docstring (e.g., “response object with a .text attribute”) to match actual usage.
• The error messages and docstring say uprn/postcode “must be a non-empty string”, but the current checks only guard against None. Empty strings (or whitespace-only values) will fall through to check_uprn/check_postcode and fail there instead. Either strengthen the guards (for example, checking if not uprn / if not postcode) or relax the wording in the docstring/messages so it only claims they are required, not necessarily already validated for non-emptiness.

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 66e2b7e and e64538a.

📒 Files selected for processing (27)

• uk_bin_collection/uk_bin_collection/councils/ArgyllandButeCouncil.py (2 hunks)
• uk_bin_collection/uk_bin_collection/councils/ArmaghBanbridgeCraigavonCouncil.py (3 hunks)
• uk_bin_collection/uk_bin_collection/councils/BirminghamCityCouncil.py (2 hunks)
• uk_bin_collection/uk_bin_collection/councils/BlackpoolCouncil.py (2 hunks)
• uk_bin_collection/uk_bin_collection/councils/ChelmsfordCityCouncil.py (2 hunks)
• uk_bin_collection/uk_bin_collection/councils/EdinburghCityCouncil.py (2 hunks)
• uk_bin_collection/uk_bin_collection/councils/FarehamBoroughCouncil.py (2 hunks)
• uk_bin_collection/uk_bin_collection/councils/FifeCouncil.py (3 hunks)
• uk_bin_collection/uk_bin_collection/councils/HaltonBoroughCouncil.py (2 hunks)
• uk_bin_collection/uk_bin_collection/councils/HarlowCouncil.py (2 hunks)
• uk_bin_collection/uk_bin_collection/councils/IsleOfAngleseyCouncil.py (6 hunks)
• uk_bin_collection/uk_bin_collection/councils/KingsLynnandWestNorfolkBC.py (2 hunks)
• uk_bin_collection/uk_bin_collection/councils/LondonBoroughLambeth.py (2 hunks)
• uk_bin_collection/uk_bin_collection/councils/MidSussexDistrictCouncil.py (2 hunks)
• uk_bin_collection/uk_bin_collection/councils/NorthHertfordshireDistrictCouncil.py (4 hunks)
• uk_bin_collection/uk_bin_collection/councils/NorthumberlandCouncil.py (3 hunks)
• uk_bin_collection/uk_bin_collection/councils/NorwichCityCouncil.py (2 hunks)
• uk_bin_collection/uk_bin_collection/councils/RushcliffeBoroughCouncil.py (1 hunks)
• uk_bin_collection/uk_bin_collection/councils/RushmoorCouncil.py (2 hunks)
• uk_bin_collection/uk_bin_collection/councils/SouthGloucestershireCouncil.py (2 hunks)
• uk_bin_collection/uk_bin_collection/councils/SouthHollandDistrictCouncil.py (2 hunks)
• uk_bin_collection/uk_bin_collection/councils/SouthLanarkshireCouncil.py (2 hunks)
• uk_bin_collection/uk_bin_collection/councils/SouthamptonCityCouncil.py (2 hunks)
• uk_bin_collection/uk_bin_collection/councils/ThurrockCouncil.py (2 hunks)
• uk_bin_collection/uk_bin_collection/councils/WestOxfordshireDistrictCouncil.py (2 hunks)
• uk_bin_collection/uk_bin_collection/councils/WiltshireCouncil.py (2 hunks)
• uk_bin_collection/uk_bin_collection/councils/WinchesterCityCouncil.py (2 hunks)

🧰 Additional context used

🪛 Ruff (0.14.7)

uk_bin_collection/uk_bin_collection/councils/NorthumberlandCouncil.py

25-25: Docstring contains ambiguous – (EN DASH). Did you mean - (HYPHEN-MINUS)?

(RUF002)

uk_bin_collection/uk_bin_collection/councils/RushcliffeBoroughCouncil.py

90-90: datetime may be undefined, or defined from star imports

(F405)

---

94-94: date_format may be undefined, or defined from star imports

(F405)

---

97-97: datetime may be undefined, or defined from star imports

(F405)

---

101-101: date_format may be undefined, or defined from star imports

(F405)

---

106-106: datetime may be undefined, or defined from star imports

(F405)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)

• GitHub Check: Run Unit Tests (3.12, 1.8.4)
• GitHub Check: Run Integration Tests (3.12, 1.8.4)

🔇 Additional comments (26)

uk_bin_collection/uk_bin_collection/councils/ThurrockCouncil.py (1)

15-27: Docstring accurately documents the API.

The docstring clearly describes the unconventional parameter usage (paon for weekday, postcode for round) which is helpful for maintainers.

uk_bin_collection/uk_bin_collection/councils/FifeCouncil.py (2)

22-39: Comprehensive docstring with accurate parameter and return documentation.

The docstring properly documents the parameters including optional ones, return structure, and potential exceptions. This aligns well with the implementation.

---

93-100: Helper function docstring is clear and useful.

Documents the matching logic and return behavior appropriately.

uk_bin_collection/uk_bin_collection/councils/NorthumberlandCouncil.py (1)

41-56: Docstring accurately documents the method behavior.

Parameters and return structure are well-documented, including the UPRN padding behavior.

uk_bin_collection/uk_bin_collection/councils/BlackpoolCouncil.py (1)

19-35: Well-documented docstring addition.

The docstring accurately describes the parameters (uprn, postcode), return structure, and the HTTPError that can be raised by the raise_for_status() calls. This improves code maintainability without changing behavior.

uk_bin_collection/uk_bin_collection/councils/ArmaghBanbridgeCraigavonCouncil.py (1)

42-53: Good documentation of internal helper.

The docstring for extract_bin_schedule clearly explains its purpose and parameters, making the code more maintainable.

uk_bin_collection/uk_bin_collection/councils/WestOxfordshireDistrictCouncil.py (1)

25-39: Comprehensive docstring added.

The docstring clearly documents all relevant kwargs including paon, postcode, web_driver, and headless, along with the return structure. This improves discoverability and maintainability.

uk_bin_collection/uk_bin_collection/councils/NorwichCityCouncil.py (1)

18-34: Thorough docstring with accurate exception documentation.

The docstring correctly documents all three Exception raise points (initial page load failure, address not found, no scheduled services), which aligns with the actual implementation at lines 55, 75, and 87. Good attention to detail.

uk_bin_collection/uk_bin_collection/councils/SouthGloucestershireCouncil.py (1)

25-39: Accurate docstring documenting the ValueError.

The docstring correctly documents the ValueError raised when no collection data is found (line 54). The parameter and return structure documentation is clear and helpful.

uk_bin_collection/uk_bin_collection/councils/MidSussexDistrictCouncil.py (1)

20-36: Excellent documentation addition.

The docstring comprehensively describes the method's purpose, parameters, return structure, and error conditions. The documentation accurately reflects the implementation.

uk_bin_collection/uk_bin_collection/councils/WiltshireCouncil.py (1)

17-34: Clear and comprehensive docstring.

The documentation accurately describes the multi-month query logic, input parameters, return structure, and error handling. Well done.

uk_bin_collection/uk_bin_collection/councils/LondonBoroughLambeth.py (1)

18-34: Well-documented API interaction.

The docstring clearly describes the Lambeth API endpoint, parameter requirements, commercial container normalization logic, and return structure.

uk_bin_collection/uk_bin_collection/councils/SouthHollandDistrictCouncil.py (1)

16-30: Thorough documentation of API integration.

The docstring accurately describes the JSON-RPC endpoint interaction, parameter requirements, return structure, and sorting behavior.

uk_bin_collection/uk_bin_collection/councils/HaltonBoroughCouncil.py (1)

25-40: Comprehensive documentation of browser-based scraping.

The docstring clearly describes the Selenium-based approach, input parameters including webdriver configuration, and the returned data structure.

uk_bin_collection/uk_bin_collection/councils/SouthLanarkshireCouncil.py (1)

19-29: Clear documentation of HTML parsing logic.

The docstring accurately describes the page parsing approach, input requirements, and return structure.

uk_bin_collection/uk_bin_collection/councils/EdinburghCityCouncil.py (1)

25-38: Well-documented scheduling algorithm.

The docstring clearly explains the fortnightly rota logic, input parameter semantics, and return structure.

uk_bin_collection/uk_bin_collection/councils/WinchesterCityCouncil.py (1)

20-31: Improved docstring clarity.

The wording refinements make the documentation more concise while maintaining accuracy.

uk_bin_collection/uk_bin_collection/councils/SouthamptonCityCouncil.py (1)

19-34: LGTM! Docstring accurately documents the function behavior.

The docstring correctly describes the return structure, date format, and error conditions (ValueError for missing calendar view, HTTPError for request failures) which match the implementation at lines 67 and 74-76.

uk_bin_collection/uk_bin_collection/councils/ChelmsfordCityCouncil.py (1)

18-33: LGTM! Docstring correctly documents error conditions.

The expanded documentation accurately reflects the two ValueError scenarios: when no collection round is found (line 111) and when the ICS calendar link cannot be located (line 106).

uk_bin_collection/uk_bin_collection/councils/RushcliffeBoroughCouncil.py (2)

84-109: Good implementation for capturing multiple collection dates.

The updated regex correctly captures both collection dates, and the logic properly creates separate bin entries for each. The sorting ensures consistent output ordering.

One consideration: the regex expects exactly two dates with "and" between them. If the council page ever shows a single date or different format, no results will be captured. This is acceptable given the scraper's nature, but worth noting for future maintenance.

---

110-118: Good resource cleanup pattern.

The try/except/finally structure ensures the WebDriver is properly closed even when exceptions occur, preventing resource leaks. This aligns with the pattern used in other council implementations.

uk_bin_collection/uk_bin_collection/councils/NorthHertfordshireDistrictCouncil.py (2)

34-58: Well-documented with thorough input validation.

The enhanced docstring clearly describes the function's purpose, parameters, return value, and comprehensive error conditions. The input normalization (strip/lowercase) before matching is a good defensive practice.

---

160-179: Clear documentation of the dual-input pattern.

The docstring effectively communicates that users can provide either a UPRN directly or a postcode+paon combination, with clear error conditions documented. This matches the implementation logic at lines 184-197.

uk_bin_collection/uk_bin_collection/councils/IsleOfAngleseyCouncil.py (3)

27-35: LGTM! Clear initialization documentation.

The docstring accurately describes the session setup with self._session and the self._have_session flag pattern for lazy authentication.

---

135-152: Well-documented dual-input interface.

The docstring clearly explains that users can provide either uprn alone or postcode with paon/number, matching the implementation at lines 155-170.

---

219-226: Year inference logic handles edge cases well.

The approach of assuming current year and rolling over to next year if the date is more than 30 days in the past is a reasonable heuristic for handling date strings without year information. The 30-day buffer prevents false rollovers for recently-passed dates.

[coderabbitai]

⚠️ Potential issue | 🟡 Minor

Docstring mentions parameters not used by this method.

The docstring references web_driver and headless parameters, but this council uses requests (not Selenium) and doesn't accept or use these kwargs. Remove these from the docstring to avoid confusion.

         Parameters:
             uprn (str): Unique Property Reference Number provided via kwargs key "uprn". The value will be validated and left-padded with zeros to 12 characters before use.
-        
-            web_driver (str, optional, in kwargs): WebDriver backend identifier passed to the webdriver factory.
-            headless (bool, optional, in kwargs): Whether to run the browser in headless mode.

Committable suggestion skipped: line range outside the PR's diff.

🤖 Prompt for AI Agents

In uk_bin_collection/uk_bin_collection/councils/KingsLynnandWestNorfolkBC.py
around lines 23-24, the docstring incorrectly lists web_driver and headless
parameters that are not accepted or used (this module uses requests, not
Selenium); remove those parameters from the Parameters section so it only
documents the actual kwargs used (e.g., uprn) and update wording to match the
real behavior (validate/pad uprn to 12 chars) to avoid confusion.