Documentation Maintenance

This guide explains how the paiOS documentation system is structured and how to maintain it. Understanding this is crucial for keeping our documentation synchronized with the code.

Architecture Overview

The project uses a Hybrid Documentation Strategy for the codebase to provide the best developer experience:

1. Native Starlight Integration: Rustdoc JSON is converted to MDX files. This provides a consistent look and feel, native search, and integrated navigation.
2. Standard rustdoc HTML (GitHub Pages): The traditional cargo doc output is generated via GitHub Actions and hosted on GitHub Pages at aurintex.github.io/pai-os/rustdoc/. This provides the deep-dive technical details Rust developers are familiar with, including private/internal items.
3. Protocol Buffer Documentation: Proto files are automatically parsed and converted to MDX, providing integrated gRPC API documentation with the same look and feel as the rest of the documentation.

Directory Structure

• docs/scripts/: Contains the logic for generating documentation.
  • rustdoc-gen.mjs: Converts Rustdoc JSON (unstable) to MDX.
  • proto-doc-gen.mjs: Parses Protocol Buffer files and generates MDX documentation.
• .github/workflows/rustdoc.yml: GitHub Actions workflow that generates standard rustdoc HTML and deploys to GitHub Pages.
• docs/src/content/docs/reference/rust/: Where the generated Rust MDX files are stored.
• docs/src/content/docs/reference/api.mdx: Where the generated Protocol Buffer documentation is stored.

Local Development

To work on the documentation site locally:

# Install dependencies
npm install

# Start development server (available at http://localhost:4321)
npm run dev

# Build for production
npm run build

# Preview production build
npm run preview

If the build fails with Received protocol 'astro:' (Node 24+), use Node 20 LTS for the docs build. A docs/.node-version file is provided for fnm/nvm (fnm install then fnm use in docs/).

Common Tasks

Generating Documentation

Documentation generation is automatically part of the main build process:

# From the docs directory
npm run build

To run only the documentation generation:

# Generate Rust MDX documentation (for Starlight integration)
npm run gen:rustdoc

# Generate Protocol Buffer documentation
npm run gen:proto

# Generate all local documentation
npm run gen:rustdoc && npm run gen:proto

Note: Standard rustdoc HTML (with private items) is generated automatically via GitHub Actions on push to main and deployed to GitHub Pages. See .github/workflows/rustdoc.yml.

Updating the Generation Logic

For Rust Documentation:

If you add new Rust modules or change the crate structure:

1. Check docs/scripts/rustdoc-gen.mjs to ensure the module hierarchy is still handled correctly.
2. The script uses the nightly toolchain for JSON export. You need to set up your environment:

# Install nightly toolchain
rustup toolchain install nightly

# Add the required JSON documentation component
rustup component add rust-docs-json --toolchain nightly

Note: Documentation generation will automatically use the +nightly modifier when running through our scripts.

For Protocol Buffer Documentation:

If you add or modify .proto files:

1. Place your .proto files in api/grpc/.
2. The proto-doc-gen.mjs script will automatically detect and parse them.
3. The script supports two modes:
   • protoc-gen-doc (if installed): Uses the external tool for better formatting
   • Built-in parser (fallback): Parses proto files directly without external dependencies
4. Generated documentation is written to docs/src/content/docs/reference/api.mdx.

To install protoc-gen-doc for enhanced formatting (optional):

go install github.com/pseudomuto/protoc-gen-doc/cmd/protoc-gen-doc@latest

Routing and Redirects

To provide clean URLs like /reference/rustdoc/engine/index, we use Astro’s native redirect system in astro.config.mjs. These redirects point to the external GitHub Pages site where the standard rustdoc HTML is hosted.

Important:

• Standard rustdoc HTML is generated via GitHub Actions and deployed to aurintex.github.io/pai-os/rustdoc/.
• The redirects in astro.config.mjs point to this external URL.

Deployment

The documentation can be deployed to:

• GitHub Pages: Recommended for separate hosting from the main website. Configure the repository settings to deploy from the docs/ directory.
• Vercel/Netlify: Can be integrated with the main website (aurintex.com).

CI/CD Behavior

The documentation build is designed to work in different environments:

• Local Development: Requires Rust nightly toolchain. Run npm run gen:rustdoc to generate MDX from rustdoc JSON.
• Vercel/CI: Rustdoc generation is automatically skipped when Cargo is not available. The pre-generated MDX files in docs/src/content/docs/reference/rust/ are used instead.
• GitHub Actions: Standard rustdoc HTML is generated on push to main and deployed to GitHub Pages.

For Contributors: You only need Rust installed if you’re modifying the engine crate and want to regenerate the Rust API documentation. Otherwise, the existing MDX files will be used.

Preventing Regressions (AI Agents)

When working with AI agents like Cursor or TaskMaster:

• Ensure they are aware of the docs-maintenance rule set.
• If an agent modifies the documentation structure, verify that npm run build completes successfully.
• Verify that api.mdx is generated correctly when proto files are present.
• Standard rustdoc HTML is generated separately via GitHub Actions, not during the main build.