Update Fareham Borough Council JSON interface and parsing#1723
Conversation
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the WalkthroughThe Fareham Borough Council integration is updated to use a 2025 API endpoint variant with revised request parameters. The bin collection data extraction key is changed, and Garden waste bin handling is refactored from inline creation to a separate detection pass over the API response. Changes
Sequence Diagram(s)sequenceDiagram
participant API as Fareham API
participant Parser as FarehamBoroughCouncil
participant Data as Bin Data
Parser->>API: Request DomesticBinCollections2025on<br/>(Road or Postcode)
API-->>Parser: BinCollectionInformation response
rect rgb(230, 245, 255)
Note over Parser: Parse Regular Bins
loop For each bin in rows[0]
Parser->>Data: Extract bin type & date
Parser->>Data: Add to bins list
end
end
rect rgb(240, 255, 240)
Note over Parser: Detect Garden Waste
alt GardenWasteBinDay present
Parser->>Data: Parse Garden date
Parser->>Data: Add Garden bin entry
end
end
Parser->>Data: Sort bins by collectionDate
Parser-->>Parser: Return parsed bins
Estimated code review effort🎯 2 (Simple) | ⏱️ ~12 minutes
Suggested reviewers
Poem
Pre-merge checks and finishing touches❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #1723 +/- ##
=======================================
Coverage 86.79% 86.79%
=======================================
Files 9 9
Lines 1136 1136
=======================================
Hits 986 986
Misses 150 150 ☔ View full report in Codecov by Sentry. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
uk_bin_collection/uk_bin_collection/councils/FarehamBoroughCouncil.py(3 hunks)
🧰 Additional context used
🪛 Ruff (0.14.4)
uk_bin_collection/uk_bin_collection/councils/FarehamBoroughCouncil.py
59-59: re may be undefined, or defined from star imports
(F405)
60-60: datetime may be undefined, or defined from star imports
(F405)
63-63: date_format may be undefined, or defined from star imports
(F405)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
- GitHub Check: Parity Check (3.12, 1.8.4)
- GitHub Check: Run Integration Tests (3.12, 1.8.4)
- GitHub Check: Run Unit Tests (3.12, 1.8.4)
🔇 Additional comments (1)
uk_bin_collection/uk_bin_collection/councils/FarehamBoroughCouncil.py (1)
38-38: Response key updated to match new API.The extraction key has been changed from
"DomesticBinDay"to"BinCollectionInformation", which aligns with the updated API endpoint.
| "list": "DomesticBinCollections2025on", | ||
| "Road or Postcode": user_postcode, |
There was a problem hiding this comment.
🧩 Analysis chain
Verify the updated API parameters work correctly.
The API list parameter has been updated to "DomesticBinCollections2025on" and the postcode parameter renamed to "Road or Postcode". Please ensure these changes match the council's actual API requirements and test with real postcodes to confirm the endpoint returns valid data.
Run the following script to test the API response:
🏁 Script executed:
#!/bin/bash
# Description: Test the Fareham Borough Council API with sample postcode
# Test with a sample Fareham postcode
curl -s "https://www.fareham.gov.uk/internetlookups/search_data.aspx?type=JSON&list=DomesticBinCollections2025on&Road%20or%20Postcode=PO14%201AA" | jq '.data.rows[0] | keys'Length of output: 64
🏁 Script executed:
#!/bin/bash
# Get raw API response without jq filtering to see actual structure
echo "=== Raw API Response ==="
curl -s "https://www.fareham.gov.uk/internetlookups/search_data.aspx?type=JSON&list=DomesticBinCollections2025on&Road%20or%20Postcode=PO14%201AA" | head -c 500
echo -e "\n\n=== Response Headers and Status ==="
curl -s -w "HTTP Status: %{http_code}\n" -o /dev/null "https://www.fareham.gov.uk/internetlookups/search_data.aspx?type=JSON&list=DomesticBinCollections2025on&Road%20or%20Postcode=PO14%201AA"Length of output: 528
🏁 Script executed:
#!/bin/bash
# Examine the FarehamBoroughCouncil.py file to understand response structure expectations
cat uk_bin_collection/uk_bin_collection/councils/FarehamBoroughCouncil.pyLength of output: 2674
Two issues require attention: inconclusive API parameter verification and critical error handling gap in garden waste parsing.
The API parameters appear syntactically correct based on response structure, but cannot be fully validated externally without a valid postcode in the system. Please manually test these changes with a real Fareham postcode to confirm the API returns valid bin data.
More critically, the garden waste handling at lines 56-65 has a dangerous bug:
results = re.findall(r'(\d{1,2}/\d{1,2}/\d{4})', value)
collection_date = datetime.strptime(results[0], "%d/%m/%Y") # Line 61: IndexError if regex failsThis will raise IndexError if the regex finds no dates. Fix by adding an empty check like the main parsing logic does at lines 44-49, or skip entries with no dates. Additionally, the garden waste section lacks the "today" handling present in the main collection loop (line 46)—add that logic or document why it's not needed.
| # Look for garden waste key | ||
| for key, value in bin_data["rows"][0].items(): | ||
| if key.startswith("GardenWasteBinDay"): | ||
| results = re.findall(r'(\d{1,2}/\d{1,2}/\d{4})', value) | ||
| collection_date = datetime.strptime(results[0], "%d/%m/%Y") | ||
| garden_data = { | ||
| "type": "Garden", | ||
| "collectionDate": collection_date.strftime(date_format), | ||
| } | ||
| data["bins"].append(garden_data) |
There was a problem hiding this comment.
Add error handling and "today" support for garden waste parsing.
The garden waste parsing logic has several issues:
- Line 60: Accessing
results[0]without checking if the regex matched anything will raise anIndexErrorif the garden waste date format is unexpected. - Line 59: The regex doesn't handle
"today"as a date value, unlike the main parsing loop (lines 44-47), creating inconsistent date handling. - Lines 57-58: If multiple keys start with
"GardenWasteBinDay", multiple garden entries will be appended without validation.
Apply this diff to add proper error handling and consistent date parsing:
# Look for garden waste key
for key, value in bin_data["rows"][0].items():
if key.startswith("GardenWasteBinDay"):
- results = re.findall(r'(\d{1,2}/\d{1,2}/\d{4})', value)
- collection_date = datetime.strptime(results[0], "%d/%m/%Y")
- garden_data = {
- "type": "Garden",
- "collectionDate": collection_date.strftime(date_format),
- }
- data["bins"].append(garden_data)
+ results = re.findall(r'(\d{1,2}/\d{1,2}/\d{4}|today)', value)
+ if results:
+ if results[0] == "today":
+ collection_date = datetime.today()
+ else:
+ collection_date = datetime.strptime(results[0], "%d/%m/%Y")
+ garden_data = {
+ "type": "Garden",
+ "collectionDate": collection_date.strftime(date_format),
+ }
+ data["bins"].append(garden_data)
+ break # Only process the first matching key🧰 Tools
🪛 Ruff (0.14.4)
59-59: re may be undefined, or defined from star imports
(F405)
60-60: datetime may be undefined, or defined from star imports
(F405)
63-63: date_format may be undefined, or defined from star imports
(F405)
2 participants
The JSON interface has changed (as well as its input parameters).
Summary by CodeRabbit