Skip to content

Simpler README #6

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ryanorendorff merged 2 commits into from
Oct 15, 2025
Merged

Simpler README #6

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
be7b920
Simpler README
ryanorendorff Oct 15, 2025
9d08a9e
Fix sunscreen-llvm.nix
ryanorendorff Oct 15, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
270 changes: 28 additions & 242 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,40 +1,17 @@
# mdbook-check-code

A configuration-driven mdBook preprocessor that validates code blocks in multiple languages by compiling them with user-specified compilers.
A configuration-driven mdBook preprocessor that validates code blocks by compiling them with user-specified compilers.

## Overview
## Quick Start

`mdbook-check-code` is a preprocessor for [mdBook](https://github.com/rust-lang/mdBook) that automatically extracts and validates code blocks from your documentation. All language behavior is configured in `book.toml` - no languages are built-in. This helps catch errors early and ensures your documentation stays in sync with working code.

## Features

- **Multi-language support**: C, TypeScript, Solidity, or any language you configure
- **Language variants**: Define variants with different compilers (e.g., Parasol C for FHE)
- **Configuration-driven**: All compiler behavior specified in `book.toml` - no hardcoded defaults
- **Environment variable expansion**: Use `${VAR}` syntax in compiler paths
- **No regex**: Uses `pulldown-cmark` for clean markdown parsing
- **Selective compilation**: Supports `ignore` flag to skip specific code blocks
- **Code propagation**: Use the `propagate` flag to share code between blocks (useful for structs and helper functions)
- **mdBook integration**: Works seamlessly with mdBook's build process
- **Clear error messages**: Shows compilation errors with context
- **Extensible**: Add new languages without writing Rust code

## Installation

### Using Nix (Recommended)

This project provides a Nix flake that includes the preprocessor and compilers (Sunscreen LLVM, gcc, TypeScript):
### Using Nix

```bash
# Enter development environment
cd mdbook-check-code
# Enter development environment with all compilers
nix develop

# Build the preprocessor
nix build

# Install to your profile
nix profile install
```

### Using Cargo
Expand All @@ -43,270 +20,79 @@ nix profile install
cargo install --path .
```

**Note**: You'll need to separately install compilers for the languages you want to check:
- Parasol C: [Sunscreen LLVM compiler](https://github.com/Sunscreen-tech/sunscreen-llvm)
- Plain C: gcc or clang
- TypeScript: Node.js and `npm install -g typescript`
Requires separate installation of compilers for enabled languages (gcc, clang, tsc, etc.).

## Usage

### Configure your mdBook

Add the preprocessor and language configurations to your `book.toml`:
Add the preprocessor and language configurations to `book.toml`:

```toml
[book]
title = "My Documentation"
authors = ["Your Name"]

[preprocessor.check-code]
command = "mdbook-check-code"

# C configuration
# C language configuration
[preprocessor.check-code.languages.c]
enabled = true
compiler = "gcc"
flags = ["-fsyntax-only"]

# Parasol variant - uses Sunscreen LLVM for FHE compilation
# Parasol variant for FHE code
[preprocessor.check-code.languages.c.variants.parasol]
compiler = "${CLANG}" # Environment variable expansion
compiler = "${CLANG}"
flags = ["-target", "parasol", "-fsyntax-only"]
preamble = "#include <parasol.h>" # Prepended to all blocks

# TypeScript configuration
[preprocessor.check-code.languages.typescript]
enabled = true
compiler = "tsc"
flags = ["--noEmit", "--skipLibCheck"]

# Solidity configuration
[preprocessor.check-code.languages.solidity]
enabled = true
compiler = "solc"

[output.html]
preamble = "#include <parasol.h>"
```

### Write your documentation

Use standard markdown with code blocks. The fence marker determines which language configuration is used:
Write code blocks with fence markers:

````markdown
# Parasol C Example

```c,variant=parasol
[[clang::fhe_program]] uint8_t add(
[[clang::encrypted]] uint8_t a,
[[clang::encrypted]] uint8_t b
) {
[[clang::fhe_program]] uint8_t add(uint8_t a, uint8_t b) {
return a + b;
}
```

# Plain C Example

```c
#include <stdint.h>

uint32_t multiply(uint32_t a, uint32_t b) {
return a * b;
}
```

# TypeScript Example

```typescript
function greet(name: string): string {
return `Hello, ${name}!`;
}
```
````

### Build your book
Build the book:

```bash
# Set environment variables (if not using Nix)
export CLANG=/path/to/sunscreen-llvm/bin/clang

# Build the book
mdbook build
```

The preprocessor will automatically validate all configured code blocks and report any errors.

## Code Block Flags

### `ignore` - Skip compilation

Use this flag for code that shouldn't be compiled (e.g., pseudocode, incomplete examples):

````markdown
```c,ignore
// This won't be compiled
incomplete_function() {
```
````

### `propagate` - Share code between blocks
The preprocessor validates all code blocks during the build process and reports compilation errors.

Use this flag to make definitions available to subsequent code blocks in the same file:
### Code Block Flags

````markdown
Define a struct:

```c,variant=parasol,propagate
typedef struct Point {
uint16_t x;
uint16_t y;
} Point;
```

Use the struct in later blocks:

```c,variant=parasol
[[clang::fhe_program]] void move_point(
[[clang::encrypted]] Point *p,
uint16_t dx,
uint16_t dy
) {
p->x = p->x + dx;
p->y = p->y + dy;
}
```
````
- `ignore` - Skip compilation for a block
- `propagate` - Make code available to subsequent blocks in the same file

## Configuration

### Language Configuration

Each language requires a full configuration in `book.toml`:

**Required fields**:
- `enabled` (bool): Whether to check this language
- `compiler` (string): Compiler executable (supports `${VAR}` env var expansion)
- `flags` (array): Compiler flags

**Optional fields**:
- `preamble` (string): Code prepended to all blocks (e.g., includes)
- `fence_markers` (array): Custom fence identifiers (e.g., `["ts", "typescript"]`)
All language behavior is configured in `book.toml`. Each language requires:
- `enabled` (bool) - Whether to check this language
- `compiler` (string) - Compiler executable (supports `${VAR}` env var expansion)
- `flags` (array) - Compiler flags

### Language Variants

You can define variants of a language that use different compilers or flags:

```toml
# Base language
[preprocessor.check-code.languages.c]
enabled = true
compiler = "gcc"
flags = ["-fsyntax-only"]

# Parasol variant for FHE code
[preprocessor.check-code.languages.c.variants.parasol]
compiler = "${CLANG}"
flags = ["-target", "parasol", "-fsyntax-only"]
preamble = "#include <parasol.h>"
```

Variants are referenced using the `variant=name` attribute in the fence marker:

````markdown
```c,variant=parasol
// Your Parasol C code here
```
````

### Adding New Languages

Add any language by configuring it in `book.toml`. Example for Python with mypy:

```toml
[preprocessor.check-code.languages.python]
enabled = true
compiler = "mypy"
flags = ["--ignore-missing-imports"]
# Optional: specify custom fence markers
fence_markers = ["python", "py"]
```

By default, the language name is used as the fence marker (e.g., `python`). Use
`fence_markers` to add aliases or override the default behavior.

### Environment Variables

Use `${VAR}` syntax in compiler paths and flags. Common variables:
- `${CLANG}`: Path to Sunscreen LLVM clang binary

### Nix Development Environment

When using `nix develop`, the environment is automatically configured with:
- The Sunscreen LLVM compiler
- Standard gcc compiler
- TypeScript (node + tsc)
- `CLANG` environment variable
- mdBook and other development tools

## How It Works

1. **Integration**: mdBook calls the preprocessor before rendering
2. **Configuration**: Preprocessor loads language configs from `book.toml` and expands env vars
3. **Extraction**: Uses `pulldown-cmark` to extract code blocks (no regex!)
4. **Language matching**: Fence markers matched via simple string equality checks
5. **Compilation**: Each code block is validated with the configured compiler and flags
6. **Error Reporting**: Compilation failures are reported with file names and error messages
7. **Success**: If all code blocks compile, mdBook continues with rendering
Optional:
- `preamble` (string) - Code prepended to all blocks
- `fence_markers` (array) - Custom fence identifiers

## Testing

Test the preprocessor on the included fixtures (includes C, Parasol C variant, TypeScript, and Solidity examples):
Test with included fixtures:

```bash
# Using Nix (recommended - includes all compilers)
# With all compilers (Nix)
nix develop --command bash -c "cd tests/fixtures && mdbook build"

# Or with Cargo (requires compilers to be installed separately)
# Or with Cargo
cargo build --release
cd tests/fixtures
export CLANG=/path/to/sunscreen-llvm/bin/clang
mdbook build
```

## Development

```bash
# Enter development environment (includes all compilers)
nix develop

# Build
cargo build

# Run tests
cargo test

# Format code
cargo fmt

# Run clippy
cargo clippy

# Build with Nix
nix build
cd tests/fixtures && mdbook build
```

## Architecture

- **No regex**: Uses `pulldown-cmark` for markdown parsing and simple string operations
- **Configuration-driven**: All language behavior from `book.toml`, no hardcoded languages
- **Modular**: Clean separation between markdown extraction, configuration, and compilation
- **Extensible**: Add languages via config without touching Rust code

## License

This project is licensed under the GNU Affero General Public License v3.0 (AGPLv3).

See the [LICENSE](LICENSE) file for details.

## Related Projects

- [mdBook](https://github.com/rust-lang/mdBook) - The book generator
Expand Down
9 changes: 5 additions & 4 deletions sunscreen-llvm.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
{ lib, stdenv, fetchurl, autoPatchelfHook, zlib }:

let
version = "2025-09-30";
version = "2025.09.30";
fileVersion = builtins.replaceStrings [ "." ] [ "-" ] version;
urlBase =
"https://github.com/Sunscreen-tech/sunscreen-llvm/releases/download/v${version}";

Expand All @@ -11,17 +12,17 @@ in stdenv.mkDerivation rec {

src = if stdenv.isDarwin then
fetchurl {
url = "${urlBase}/parasol-compiler-macos-aarch64-${version}.tar.gz";
url = "${urlBase}/parasol-compiler-macos-aarch64-${fileVersion}.tar.gz";
sha256 = "0ra93mji3j9km7ia21gsqswn49a3abwc1ml1xq643hzq4xigyqjd";
}
else if stdenv.isAarch64 then
fetchurl {
url = "${urlBase}/parasol-compiler-linux-aarch64-${version}.tar.gz";
url = "${urlBase}/parasol-compiler-linux-aarch64-${fileVersion}.tar.gz";
sha256 = "197fybbjvimnyqwwn3q7s9yrljbqp57s42n9znpckmnbcbp8p373";
}
else
fetchurl {
url = "${urlBase}/parasol-compiler-linux-x86-64-${version}.tar.gz";
url = "${urlBase}/parasol-compiler-linux-x86-64-${fileVersion}.tar.gz";
sha256 = "1p0418nqzs6a2smrbqiyrxj34pimm6qzj7k29l4ys226cz6kfz2r";
};

Expand Down
Loading