Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 20 additions & 24 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -1,38 +1,34 @@
name: CI

on:
pull_request:
branches: [main]
push:
branches: [main]
pull_request:
Comment thread
coderabbitai[bot] marked this conversation as resolved.

permissions:
contents: read

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

jobs:
build-test:
build:
runs-on: ubuntu-latest
permissions:
contents: read
env:
CARGO_TERM_COLOR: always
BUILD_PROFILE: debug
steps:
- uses: actions/checkout@v4
- name: Setup Rust
uses: leynos/shared-actions/.github/actions/setup-rust@v1.1.0
- name: Format
- uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4
- uses: dtolnay/rust-toolchain@master
with:
toolchain: nightly-2025-06-26
components: rustfmt, clippy, llvm-tools-preview
- uses: Swatinem/rust-cache@7939da402645ba29a2df566723491a2c856e8f8a # v2
- name: Check formatting
run: make check-fmt
- name: Lint
run: make lint
- name: Test
run: make test
- name: Install cargo-tarpaulin
run: cargo install cargo-tarpaulin
- name: Run coverage
run: cargo tarpaulin --out lcov
- name: Upload coverage data to CodeScene
if: ${{ secrets.CS_ACCESS_TOKEN }}
uses: leynos/shared-actions/.github/actions/upload-codescene-coverage@v1.1.0
with:
format: lcov
access-token: ${{ secrets.CS_ACCESS_TOKEN }}
installer-checksum: ${{ vars.CODESCENE_CLI_SHA256 }}

- name: Lint Markdown
run: make markdownlint
- name: Validate diagrams
run: make nixie
10 changes: 9 additions & 1 deletion Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

16 changes: 10 additions & 6 deletions Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,11 +1,15 @@
[package]
name = "wildside-engine"
version = "0.1.0"
[workspace]
members = [
"wildside-core",
"wildside-data",
"wildside-cli",
]
resolver = "2"

[workspace.package]
edition = "2024"

[dependencies]

[lints.clippy]
[workspace.lints.clippy]
pedantic = { level = "warn", priority = -1 }

# 1. hygiene
Expand Down
40 changes: 24 additions & 16 deletions docs/roadmap.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,15 +7,24 @@ core data structures of the engine.

- **Set up Project Structure**

- [ ] Run `cargo new --lib wildside-engine` to create the workspace root.
- [ ] Within the `wildside-engine` directory, create the initial crates:
`cargo new --lib core`, `cargo new --lib data`, and `cargo new --bin cli`.
- [ ] Configure the root `Cargo.toml` to define the workspace members (`core`,
`data`, `cli`).
- [x] Create the repository root directory `wildside-engine` and initialise a
virtual workspace:

```bash
mkdir wildside-engine && cd wildside-engine
git init
cargo init --vcs git
```

- [x] Replace the root `Cargo.toml` with a virtual workspace manifest (no
`[package]`), defining members: `cargo new --lib wildside-core`,
`cargo new --lib wildside-data`, and `cargo new --bin wildside-cli`.
- [x] Configure the root `Cargo.toml` to define the workspace members
(`wildside-core`, `wildside-data`, `wildside-cli`) and set `resolver = "2"`.

- **Define Core Domain Model**

- [ ] In `wildside-engine-core`, define the public struct `PointOfInterest`
- [ ] In `wildside-core`, define the public struct `PointOfInterest`
with essential fields like `id`, `location: geo::Coord`, and
`tags: HashMap<String, String>`.
- [ ] Define the `InterestProfile` struct to hold a user's selected themes and
Expand All @@ -26,8 +35,7 @@ core data structures of the engine.
`get_pois_in_bbox(&self, bbox: &geo::Rect) -> Vec<PointOfInterest>`.
- [ ] Define the `TravelTimeProvider` trait with an `async` method
<!-- markdownlint-disable-next-line MD013 -->
`get_travel_time_matrix(&self, pois: &[PointOfInterest]) -> Result<Vec<Vec<Duration>>, Error>`
.
`get_travel_time_matrix(&self, pois: &[PointOfInterest]) -> Result<Vec<Vec<Duration>>, Error>`.
- [ ] Define the `Scorer` trait with a
`score(&self, poi: &PointOfInterest, profile: &InterestProfile) -> f32`
method.
Expand All @@ -36,7 +44,7 @@ core data structures of the engine.

- **Implement OSM PBF Ingestion**

- [ ] In `wildside-engine-data`, add `osmpbf` and `geo` as dependencies.
- [ ] In `wildside-data`, add `osmpbf` and `geo` as dependencies.
- [ ] Create a public function `ingest_osm_pbf(path: &Path)` that uses
`osmpbf::par_map_reduce` to process a PBF file in parallel.
- [ ] Implement the logic to filter for relevant OSM elements (e.g., nodes and
Expand All @@ -55,7 +63,7 @@ core data structures of the engine.

- **Build Wikidata ETL Pipeline**

- [ ] In `wildside-engine-data`, add `wikidata-rust`, `simd-json`, and
- [ ] In `wildside-data`, add `wikidata-rust`, `simd-json`, and
`rusqlite` dependencies.
- [ ] Write a script that downloads the latest Wikidata JSON dump.
- [ ] Implement a parser that iterates through the dump, filters for entities
Expand All @@ -66,7 +74,7 @@ core data structures of the engine.

- **Develop Initial CLI**

- [ ] In `wildside-engine-cli`, use the `clap` crate to define an `ingest`
- [ ] In `wildside-cli`, use the `clap` crate to define an `ingest`
command with arguments for the OSM PBF and Wikidata dump file paths.
- [ ] Implement the command's handler to orchestrate the full pipeline: call
`ingest_osm_pbf`, then the Wikidata ETL process, and finally
Expand All @@ -78,7 +86,7 @@ This phase implements the core logic that gives the engine its intelligence.

- **Implement Global Popularity Scorer**

- [ ] Create the `wildside-engine-scorer` crate.
- [ ] Create the `wildside-scorer` crate.
- [ ] Implement an offline process that iterates through `pois.db`, calculates
a popularity score for each POI based on its sitelink count and heritage
status, and normalises the scores.
Expand All @@ -96,7 +104,7 @@ This phase implements the core logic that gives the engine its intelligence.

- **Define Stable API**

- [ ] In `wildside-engine-core`, define the `SolveRequest` struct with public
- [ ] In `wildside-core`, define the `SolveRequest` struct with public
fields for `start: geo::Coord`, `duration_minutes: u16`,
`interests: InterestProfile`, and a `seed: u64` for reproducible results.
- [ ] Define the `SolveResponse` struct to include the final `Route`, the total
Expand All @@ -109,7 +117,7 @@ This phase tackles the complex route-finding algorithm.

- **Implement Native VRP Solver**

- [ ] Create the `wildside-engine-solver-vrp` crate with a dependency on
- [ ] Create the `wildside-solver-vrp` crate with a dependency on
`vrp-core`.
- [ ] Create a `VrpSolver` struct that implements the `Solver` trait from the
core crate.
Expand All @@ -130,7 +138,7 @@ This phase tackles the complex route-finding algorithm.

- **Integrate Solver into CLI**

- [ ] Add a `solve` command to `wildside-engine-cli` that accepts a path to a
- [ ] Add a `solve` command to `wildside-cli` that accepts a path to a
JSON file.
- [ ] The command will deserialise the JSON into a `SolveRequest`, instantiate
the necessary components (store, scorer, solver), call the solver, and print
Expand Down Expand Up @@ -167,7 +175,7 @@ This phase ensures the engine is robust, reliable, and ready for integration.

- **(Optional) Implement OR-Tools Solver**

- [ ] Create a `wildside-engine-solver-ortools` crate, conditionally compiled
- [ ] Create a `wildside-solver-ortools` crate, conditionally compiled
via the `ortools` feature flag.
- [ ] Add a dependency on a suitable OR-Tools wrapper crate.
- [ ] Implement the `Solver` trait using the CP-SAT solver, mapping the
Expand Down
55 changes: 28 additions & 27 deletions docs/wildside-engine-design.md
Original file line number Diff line number Diff line change
Expand Up @@ -256,7 +256,7 @@ the core logic.
A Rust workspace will be used to manage the engine's components, enforcing
clear boundaries while allowing for atomic changes across crates when necessary.

- `wildside-engine-core`: This crate is the heart of the engine. It contains
- `wildside-core`: This crate is the heart of the engine. It contains
the pure domain model and traits, with no I/O or specific framework
dependencies (i.e., it is `#![no_std]` compatible where possible, without
`tokio` or `actix`).
Expand All @@ -271,30 +271,30 @@ clear boundaries while allowing for atomic changes across crates when necessary.
- This crate is deterministic and side-effect free, making it easy to test
rigorously with property-based testing and fuzzing.

- `wildside-engine-data`: Contains the ETL logic and data adapters.
- `wildside-data`: Contains the ETL logic and data adapters.

- Implements the `PoiStore` trait from the core crate.

- Handles OSM PBF ingestion (using `osmpbf`), Wikidata dump parsing, and
building the data artifacts (e.g., SQLite/RocksDB stores and the `rstar`
index).

- `wildside-engine-scorer`: Implements the `Scorer` trait.
- `wildside-scorer`: Implements the `Scorer` trait.

- Contains the logic for both the offline pre-computation of global
popularity scores and the per-request calculation of user relevance.

- `wildside-engine-solver-vrp`: The default, native Rust implementation of the
- `wildside-solver-vrp`: The default, native Rust implementation of the
`Solver` trait, using the `vrp-core` library.

- `wildside-engine-solver-ortools`: An optional implementation of the `Solver`
- `wildside-solver-ortools`: An optional implementation of the `Solver`
trait, using bindings to Google's CP-SAT solver. This would be enabled via a
feature flag for users who require its specific performance characteristics
and are willing to manage the C++ dependency.

- `wildside-engine-cli`: A small command-line application for operational tasks.
- `wildside-cli`: A small command-line application for operational tasks.

- `ingest`: Runs the full ETL pipeline from `wildside-engine-data` to build
- `ingest`: Runs the full ETL pipeline from `wildside-data` to build
the necessary data artifacts.

- `score`: Triggers the batch computation of global popularity scores.
Expand Down Expand Up @@ -339,7 +339,7 @@ benchmarking, and diagnosing production incidents.
A strict separation between offline preparation and online serving is essential
for performance and scalability.

- **Offline Path:** The `wildside-engine-cli` is used to execute the idempotent
- **Offline Path:** The `wildside-cli` is used to execute the idempotent
ETL process. This process takes raw OSM and Wikidata data and produces a set
of optimized, read-only artifacts:

Expand Down Expand Up @@ -369,11 +369,12 @@ specific implementation a configurable choice.

### 4.1. The `Solver` Trait: A Common Interface

The `wildside-engine-core` crate will define a `Solver` trait. This trait will
have a single primary method,
`solve(request: &SolveRequest) -> Result<SolveResponse, Error>`, which
encapsulates the entire process of finding an optimal route. This abstraction
is the key to making the engine flexible and future-proof.
The `wildside-core` crate will define a `Solver` trait. This trait will have a
single primary method,
`solve(request: &SolveRequest) -> Result<SolveResponse, core::Error>`, which
encapsulates the entire process of finding an optimal route. The trait is
object-safe and keeps the solver synchronous for embeddability. This
abstraction is the key to making the engine flexible and future-proof.

### 4.2. Recommended Native Rust Solution with `vrp-core`

Expand All @@ -391,14 +392,14 @@ will be configured to maximize the total collected `Score(POI)` from visited
powerful built-in metaheuristics will efficiently find a high-quality route
within the required few-second timeframe.15 This implementation will live in the

`wildside-engine-solver-vrp` crate.
`wildside-solver-vrp` crate.

### 4.3. Optional High-Performance Backend: `wildside-engine-solver-ortools`
### 4.3. Optional High-Performance Backend: `wildside-solver-ortools`

To allow for future performance comparisons or to meet extreme optimization
requirements, a second implementation of the `Solver` trait can be provided in
the `wildside-engine-solver-ortools` crate. This would use bindings to Google's
highly optimized CP-SAT solver, such as the `cp_sat` crate.
the `wildside-solver-ortools` crate. This would use bindings to Google's highly
optimized CP-SAT solver, such as the `cp_sat` crate.

This approach offers potentially world-class performance but comes at the cost
of significant build and deployment complexity, requiring a C++ compiler and a
Expand All @@ -413,10 +414,10 @@ A critical prerequisite for any VRP solver is the travel time matrix. The
solver itself is an abstract mathematical engine; it requires an external
component to provide the walking time between every pair of candidate POIs.

This is handled by the `TravelTimeProvider` trait defined in
`wildside-engine-core`. This trait defines the async boundary for the entire
library. While the core solver logic remains synchronous and embeddable, an
implementation of this trait can be asynchronous.
This is handled by the `TravelTimeProvider` trait defined in `wildside-core`.
This trait forms the asynchronous boundary for the library with the signature
`async fn get_travel_time_matrix(...) -> Result<..., core::Error>`. Keeping the
solver synchronous preserves object safety and makes the core embeddable.

The recommended implementation will be an adapter that makes API calls to an
external, open-source routing engine like OSRM or Valhalla, running as a
Expand All @@ -434,7 +435,7 @@ plan covering packaging, testing, and versioning.

The engine will be structured for robust dependency management and deployment.

- **Licensing:** All engine crates (`wildside-engine-*`) will be licensed under
- **Licensing:** All engine crates (`wildside-*`) will be licensed under
the permissive **ISC license**, satisfying the project's legal requirements
while being clear and concise.

Expand Down Expand Up @@ -493,14 +494,14 @@ own repository with no code churn, as the boundaries are already established.
The migration from an initial "engine-in-app" prototype to the final library
structure follows a clear path:

1. Extract all domain types into `wildside-engine-core` and update the
1. Extract all domain types into `wildside-core` and update the
application to import from the new crate.

2. Move scoring and solver logic into the `wildside-engine-scorer` and
`wildside-engine-solver-vrp` crates, implementing the traits from `core`.
The application code becomes a thin adapter.
2. Move scoring and solver logic into the `wildside-scorer` and
`wildside-solver-vrp` crates, implementing the traits from `core`. The
application code becomes a thin adapter.

3. Introduce the `wildside-engine-cli` and integrate it into the CI pipeline
3. Introduce the `wildside-cli` and integrate it into the CI pipeline
for repeatable data ingestion and performance snapshots.

4. Change the application's dependency on the engine crates from a local
Expand Down
10 changes: 10 additions & 0 deletions wildside-cli/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
[package]
name = "wildside-cli"
version = "0.1.0"
edition.workspace = true
publish = false
default-run = "wildside"

[[bin]]
name = "wildside"
path = "src/main.rs"
14 changes: 14 additions & 0 deletions wildside-cli/src/main.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
//! Entry point for the command-line interface.
#![forbid(unsafe_code)]

fn main() {
if let Err(err) = run() {
eprintln!("wildside: {err}");
std::process::exit(1);
}
}

fn run() -> Result<(), Box<dyn std::error::Error>> {
// TODO: parse CLI arguments and dispatch commands.
Ok(())
}
5 changes: 5 additions & 0 deletions wildside-core/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
[package]
name = "wildside-core"
version = "0.1.0"
edition.workspace = true
publish = false
1 change: 1 addition & 0 deletions wildside-core/src/lib.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
//! Core domain types for the Wildside engine.
5 changes: 5 additions & 0 deletions wildside-data/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
[package]
name = "wildside-data"
version = "0.1.0"
edition.workspace = true
publish = false
14 changes: 14 additions & 0 deletions wildside-data/src/lib.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
//! Data access and ingestion logic for the Wildside engine.
//!
//! Responsibilities:
//! - Define repository and source traits for ingestion and query.
//! - Provide adapters for files, HTTP and databases.
//! - Encapsulate serialization formats and schema evolution.
//!
//! Boundaries:
//! - Do not encode domain rules (live in `wildside-core`).
//! - Keep blocking I/O off async executors; prefer async-capable clients.
//!
//! Invariants:
//! - Thread-safe by default where feasible.
//! - No global mutable state.
Loading