Problem
cron.go hard-codes sort: "newest", which means a campaign launched 10+ days ago but currently trending will never reappear in the crawl — it has already scrolled past the 10-page window.
// cron.go
campaigns, err := s.scrapingService.DiscoverCampaigns(catID, "newest", page)
Kickstarter supports the following sort orders:
magic — Kickstarter's recommended/trending mix
newest — most recently launched (current)
end_date — ending soonest (high-urgency campaigns)
most_backed — by backer count
Expected Behaviour
Run at least 2–3 sort passes per crawl cycle so that popular ongoing campaigns and campaigns ending soon are always kept up-to-date.
Proposed Fix
- Add
magic and end_date sort passes alongside newest in the cron loop
- Keep page depth at 3–5 for secondary sorts (to control credit usage)
- Upsert deduplication already handles overlaps
Impact
Captures hot/trending and expiring campaigns that the current strategy misses entirely. Combined with issue #12 (subcategories), coverage could reach 60%+.
Problem
cron.gohard-codessort: "newest", which means a campaign launched 10+ days ago but currently trending will never reappear in the crawl — it has already scrolled past the 10-page window.Kickstarter supports the following sort orders:
magic— Kickstarter's recommended/trending mixnewest— most recently launched (current)end_date— ending soonest (high-urgency campaigns)most_backed— by backer countExpected Behaviour
Run at least 2–3 sort passes per crawl cycle so that popular ongoing campaigns and campaigns ending soon are always kept up-to-date.
Proposed Fix
magicandend_datesort passes alongsidenewestin the cron loopImpact
Captures hot/trending and expiring campaigns that the current strategy misses entirely. Combined with issue #12 (subcategories), coverage could reach 60%+.