Feb26 release2

[robbrad]

closes #1686 #1593 #1618 #1794

Summary by CodeRabbit

Release Notes

• New Features

  • Added bin collection lookup support for Gosport Borough Council
  • Expanded calendar integration to support additional councils: Causeway Coast and Glens, and Rossendale Borough

• Improvements

  • Enhanced data retrieval and parsing with improved error handling and validation for bin collection schedules

[coderabbitai]

Caution

Review failed

The pull request is closed.

📝 Walkthrough

Walkthrough

This PR adds support for Gosport Borough Council bin collection data retrieval via the Supatrak API, updates test configuration to include the new council and two additional councils to GooglePublicCalendarCouncil, and enhances error handling and data parsing in HarboroughDistrictCouncil.

Changes

Cohort / File(s)	Summary
Test Configuration uk_bin_collection/tests/input.json	Added GosportBoroughCouncil entry with postcode, url, and LAD24CD. Expanded GooglePublicCalendarCouncil's supported councils list with Causeway Coast and Glens and Rossendale Borough, adding corresponding LAD24CD entries.
New GosportBoroughCouncil Implementation uk_bin_collection/uk_bin_collection/councils/GosportBoroughCouncil.py	New council class extending AbstractGetBinDataClass. Implements parse_data method to fetch bin collection schedules from Supatrak API using postcode, with error handling for missing input, request failures, and empty results. Parses API responses extracting WasteType and NextCollection, deduplicating by (type, date).
HarboroughDistrictCouncil Enhancement uk_bin_collection/uk_bin_collection/councils/HarboroughDistrictCouncil.py	Suppresses SSL warnings and switches POST request to form data with SSL verification disabled. Adds explicit 502 Bad Gateway error handling and response validation. Introduces enhanced parsing logic with a preferred path for span-based date extraction, falling back to regex-based parsing. Improves error handling granularity and response content validation.

Sequence Diagram(s)

sequenceDiagram
    participant Client as Client
    participant Council as GosportBoroughCouncil
    participant API as Supatrak API
    participant Parser as Data Parser
    
    Client->>Council: parse_data(postcode="PO12 4RU")
    
    Council->>Council: Validate postcode exists
    
    Council->>API: GET request with postcode header
    API-->>Council: JSON response with bins data
    
    Council->>Parser: Extract WasteType & NextCollection
    Parser->>Parser: Parse dates to dd/mm/YYYY
    Parser->>Parser: Deduplicate by (type, date)
    Parser-->>Council: Processed bin entries
    
    Council->>Council: Build response dict with bins list
    Council-->>Client: Return {"bins": [...]}

Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Suggested reviewers

• dp247

Poem

🐰 Hops of joy! A new council joins our fold,
Gosport's bins now tracked with data bold,
Harborough's parsing grows more wise,
With better guards and sharper eyes,
The collection network multiplies! 📦✨

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch feb26_release2

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[codecov]

❌ 1 Tests Failed:

Tests completed	Failed	Passed	Skipped
2	1	1	0

View the top 1 failed test(s) by shortest run time

uk_bin_collection.tests.step_defs.test_validate_council::test_scenario_outline[HarboroughDistrictCouncil]

Stack Traces | 1.39s run time

fixturefunc = <function scrape_step at 0x7f976f19b240>
request = <FixtureRequest for <Function test_scenario_outline[HarboroughDistrictCouncil]>>
kwargs = {'context': <test_validate_council.Context object at 0x7f977045db80>, 'headless_mode': 'True', 'local_browser': 'False', 'selenium_url': 'http://localhost:4444'}

    def call_fixture_func(
        fixturefunc: _FixtureFunc[FixtureValue], request: FixtureRequest, kwargs
    ) -> FixtureValue:
        if is_generator(fixturefunc):
            fixturefunc = cast(
                Callable[..., Generator[FixtureValue, None, None]], fixturefunc
            )
            generator = fixturefunc(**kwargs)
            try:
                fixture_result = next(generator)
            except StopIteration:
                raise ValueError(f"{request.fixturename} did not yield a value") from None
            finalizer = functools.partial(_teardown_yield_fixture, fixturefunc, generator)
            request.addfinalizer(finalizer)
        else:
            fixturefunc = cast(Callable[..., FixtureValue], fixturefunc)
>           fixture_result = fixturefunc(**kwargs)

../../../..../pypoetry/virtualenvs/uk-bin-collection-EwS6Gn8s-py3.12/lib/python3.12.../site-packages/_pytest/fixtures.py:898: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.../tests/step_defs/test_validate_council.py:101: in scrape_step
    context.parse_result = CollectData.run()
uk_bin_collection/uk_bin_collection/collect_data.py:101: in run
    return self.client_code(
uk_bin_collection/uk_bin_collection/collect_data.py:121: in client_code
    return get_bin_data_class.template_method(address_url, **kwargs)
uk_bin_collection/uk_bin_collection/get_bin_data.py:61: in template_method
    bin_data_dict = self.get_and_parse_data(this_url, **kwargs)
uk_bin_collection/uk_bin_collection/get_bin_data.py:82: in get_and_parse_data
    bin_data_dict = self.parse_data(page, url=address_url, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <HarboroughDistrictCouncil.CouncilClass object at 0x7f97707291f0>
page = <Response [200]>
kwargs = {'council_module_str': 'HarboroughDistrictCouncil', 'dev_mode': False, 'headless': True, 'local_browser': False, ...}
user_uprn = '100030489072', bindata = {'bins': []}
URI = 'https://harborough.fccenvironment.co.uk/detail-address'
headers = {'Content-Type': 'application/json', 'Referer': 'https://harborough.fccenvironment.co.uk/', 'User-Agent': 'Mozilla/5.0'}
params = {'Uprn': '100030489072'}, response = <Response [502]>

    def parse_data(self, page: str, **kwargs) -> dict:
    
        user_uprn = kwargs.get("uprn")
        check_uprn(user_uprn)
        bindata = {"bins": []}
    
        URI = "https://harborough.fccenvironment.co.uk/detail-address"
    
        headers = {
            "Content-Type": "application/json",
            "User-Agent": "Mozilla/5.0",
            "Referer": "https://harborough.fccenvironment.co.uk/",
        }
        params = {"Uprn": user_uprn}
        response = requests.post(URI, headers=headers, data=params, verify=False)
    
        # Check for service errors
        if response.status_code == 502:
>           raise ValueError(
                f"The FCC Environment service is currently unavailable (502 Bad Gateway). "
                f"This is a temporary issue with the council's waste collection system. "
                f"Please try again later."
            )
E           ValueError: The FCC Environment service is currently unavailable (502 Bad Gateway). This is a temporary issue with the council's waste collection system. Please try again later.

.../uk_bin_collection/councils/HarboroughDistrictCouncil.py:38: ValueError

To view more test analytics, go to the Test Analytics Dashboard
_{📋 Got 3 mins? Take this short survey to help us improve Test Analytics.}