architecture

Purpose

Enable publishing Lattice research documents to the web via lattice publish <topic>, generating a static site powered by Astro Starlight and deployed to lattice.zabaca.com via Netlify.

Goals

Simple workflow: lattice publish tryhackme publishes a topic
Astro-powered: Use Starlight for beautiful, accessible documentation
Selective publishing: Choose which topics to make public
Netlify integration: Match existing zabaca.com deployment workflow
Preserve graph data: Include entity relationships in published content

Architecture Overview

┌─────────────────────────────────────────────────────────────────────────┐
│                         lattice publish <topic>                          │
└─────────────────────────────────────────────────────────────────────────┘
                                    │
                                    ▼
┌─────────────────────────────────────────────────────────────────────────┐
│  Phase 1: Export                                                         │
│  ─────────────────                                                       │
│  • Read ~/.lattice/docs/<topic>/*.md                                    │
│  • Query graph for entities & relationships                              │
│  • Mark topic as "published" in database                                 │
│  • Copy/transform markdown files                                         │
└─────────────────────────────────────────────────────────────────────────┘
                                    │
                                    ▼
┌─────────────────────────────────────────────────────────────────────────┐
│  Phase 2: Generate                                                       │
│  ─────────────────                                                       │
│  • Write to Astro content collection (src/content/docs/)                │
│  • Generate navigation sidebar config                                    │
│  • Add entity/relationship metadata as frontmatter                       │
│  • Update site index with new topic                                      │
└─────────────────────────────────────────────────────────────────────────┘
                                    │
                                    ▼
┌─────────────────────────────────────────────────────────────────────────┐
│  Phase 3: Build (optional, with --build or --preview)                   │
│  ─────────────────                                                       │
│  • Run: astro build                                                      │
│  • Output to dist/                                                       │
│  • Or: astro dev for preview                                             │
└─────────────────────────────────────────────────────────────────────────┘
                                    │
                                    ▼
┌─────────────────────────────────────────────────────────────────────────┐
│  Phase 4: Deploy (optional, with --deploy)                              │
│  ─────────────────                                                       │
│  • Git commit & push to zabaca/lattice-site                             │
│  • Or: netlify deploy --prod                                             │
│  • Netlify auto-deploys to lattice.zabaca.com                           │
└─────────────────────────────────────────────────────────────────────────┘

URL Structure

Domain Setup

Domain	Purpose
`www.zabaca.com`	Main website (existing)
`lattice.zabaca.com`	Published research (new subdomain)

URL Patterns

lattice.zabaca.com/
├── /                           # Homepage (all published topics)
├── /tryhackme/                 # Topic index (from README.md)
│   ├── /overview/              # Individual document
│   └── /career-readiness/      # Individual document
├── /macos-window-management/
│   ├── /minimized-window-restore/
│   └── /jetbrains-cmd-backtick-fix/
└── /search/                    # Full-text search (Pagefind)

Slug Derivation

Source File	Published URL
`~/.lattice/docs/tryhackme/README.md`	`/tryhackme/`
`~/.lattice/docs/tryhackme/overview.md`	`/tryhackme/overview/`
`~/.lattice/docs/tryhackme/career-readiness.md`	`/tryhackme/career-readiness/`

Technology Stack

Core

Component	Technology	Why
Static Site Generator	Astro 5.x	Content-focused, excellent Markdown support
Documentation Theme	Starlight	Built-in search, dark mode, sidebar, accessibility
Hosting	Netlify	Already using for zabaca.com, free tier, auto-deploy
Search	Pagefind	Built into Starlight, static search index

Astro Starlight Features Used

Content Collections: Type-safe Markdown handling
Automatic Sidebar: Generated from file structure
Pagefind Search: Full-text search, no server needed
Dark/Light Mode: Toggle built-in
Table of Contents: Auto-generated from headings
Responsive Design: Mobile-friendly out of box
Frontmatter Validation: Zod schemas for metadata

File Structure

Lattice Side (source)

~/.lattice/
├── docs/                         # Research documents (source of truth)
│   ├── tryhackme/
│   │   ├── README.md
│   │   ├── overview.md
│   │   └── career-readiness.md
│   └── macos-window-management/
│       └── ...
├── lattice.duckdb               # Graph database
└── .publish-manifest.json       # Tracks what's published

Astro Site (generated)

~/.lattice/site/                  # Or: ~/zabaca/lattice-site/
├── astro.config.mjs
├── package.json
├── src/
│   ├── content/
│   │   └── docs/                 # Starlight content collection
│   │       ├── tryhackme/
│   │       │   ├── index.md      # From README.md
│   │       │   ├── overview.md
│   │       │   └── career-readiness.md
│   │       └── index.md          # Homepage
│   └── assets/                   # Images, custom CSS
├── public/
│   └── favicon.ico
└── dist/                         # Built output (gitignored)

Markdown Transformation

Input (Lattice format)

# TryHackMe - Platform Overview

## What is TryHackMe?

TryHackMe is a global cybersecurity training platform...

## Sources

1. [TryHackMe vs HackTheBox](https://tryhackme.com/...)

Output (Starlight format)

---
title: TryHackMe - Platform Overview
description: Comprehensive overview of the TryHackMe cybersecurity training platform
sidebar:
  order: 1
  label: Platform Overview
lastUpdated: 2025-12-25
editUrl: false
# Lattice metadata
lattice:
  topic: tryhackme
  entities:
    - { type: "Technology", name: "TryHackMe" }
    - { type: "Concept", name: "Cybersecurity Training" }
  relationships:
    - { from: "TryHackMe", to: "HackTheBox", type: "COMPARED_TO" }
---

## What is TryHackMe?

TryHackMe is a global cybersecurity training platform...

## Sources

1. [TryHackMe vs HackTheBox](https://tryhackme.com/...)

Transformation Rules

Source	Target
First H1 heading	`title` frontmatter
First paragraph	`description` frontmatter
File order in dir	`sidebar.order`
Filename	`sidebar.label` (humanized)
Graph entities	`lattice.entities` frontmatter
Graph relationships	`lattice.relationships` frontmatter

CLI Commands

Primary Command

lattice publish <topic> [options]

Options

Flag	Description	Default
`--all`	Publish all topics	false
`--preview`	Start local dev server	false
`--build`	Build static site	false
`--deploy`	Deploy to Netlify	false
`--dry-run`	Show what would be published	false
`--force`	Republish even if unchanged	false
`--output <path>`	Custom output directory	`~/.lattice/site/`

Examples

# Publish tryhackme topic to content collection
lattice publish tryhackme

# Preview locally (runs astro dev)
lattice publish tryhackme --preview

# Build and deploy
lattice publish --all --build --deploy

# See what would be published
lattice publish tryhackme --dry-run

# List published topics
lattice published

# Unpublish a topic
lattice unpublish tryhackme

# Check publish status
lattice status --published

Implementation Plan

Phase 1: Foundation (MVP)

Goal: Basic lattice publish that copies markdown to Astro site

Tasks:

Create PublishCommand in src/commands/publish.command.ts
Create PublishService in src/publish/publish.service.ts
Add frontmatter transformation logic
Generate sidebar navigation
Create Astro site scaffold (one-time init)

Files to create:

src/
├── commands/
│   └── publish.command.ts
├── publish/
│   ├── publish.module.ts
│   ├── publish.service.ts
│   └── markdown-transformer.service.ts

Phase 2: Astro Site

Goal: Starlight-based site that renders published content

Tasks:

Create Astro Starlight project in ~/.lattice/site/
Configure content collection for docs
Customize Starlight theme (branding, colors)
Add homepage template
Configure Netlify deployment

Astro config:

import { defineConfig } from 'astro/config';
import starlight from '@astrojs/starlight';

export default defineConfig({
  site: 'https://lattice.zabaca.com',
  integrations: [
    starlight({
      title: 'Lattice',
      description: 'Research knowledge base',
      social: {
        github: 'https://github.com/Zabaca/lattice',
      },
      sidebar: [
        // Auto-generated from content structure
        { label: 'Topics', autogenerate: { directory: '.' } },
      ],
      search: {
        provider: 'local', // Pagefind
      },
    }),
  ],
});

Phase 3: Graph Integration

Goal: Include entity/relationship data in published pages

Tasks:

Query graph for topic entities
Add entities as frontmatter metadata
Create “Related Topics” component
Add entity type badges to pages
Generate knowledge graph visualization (optional)

Phase 4: Deployment

Goal: Automated deployment workflow

Tasks:

Create GitHub repo for site (zabaca/lattice-site)
Configure Netlify for subdomain
Add --deploy flag to command
Implement git commit/push workflow
Add CI/CD via Netlify

Netlify Configuration

DNS Setup

Add subdomain in Netlify:

Go to Domain settings for zabaca.com
Add subdomain: lattice.zabaca.com
Point to zabaca/lattice-site deployment

netlify.toml

[build]
  command = "npm run build"
  publish = "dist"

[build.environment]
  NODE_VERSION = "20"

# SPA fallback for client-side routing (if needed)
[[redirects]]
  from = "/*"
  to = "/404"
  status = 404

# Cache static assets
[[headers]]
  for = "/_astro/*"
  [headers.values]
    Cache-Control = "public, max-age=31536000, immutable"

Database Schema Additions

Published Topics Tracking

-- Add to nodes table or create new table
CREATE TABLE IF NOT EXISTS published_topics (
    topic_name VARCHAR PRIMARY KEY,
    published_at TIMESTAMP DEFAULT NOW(),
    last_published TIMESTAMP DEFAULT NOW(),
    url VARCHAR,
    content_hash VARCHAR,  -- To detect changes
    version INTEGER DEFAULT 1
);

.publish-manifest.json (Alternative)

{
  "version": 1,
  "published": {
    "tryhackme": {
      "publishedAt": "2025-12-25T12:00:00Z",
      "lastPublished": "2025-12-25T12:00:00Z",
      "url": "https://lattice.zabaca.com/tryhackme/",
      "contentHash": "abc123...",
      "documents": [
        { "slug": "overview", "hash": "def456..." },
        { "slug": "career-readiness", "hash": "ghi789..." }
      ]
    }
  }
}

Content Policies

What Gets Published

Content	Published	Controllable
Markdown content	Yes	Per-topic
Entity metadata	Yes	Global option
Cross-topic links	Yes	Auto-resolved
Images	Yes	Copied to public/
Source links/citations	Yes	Preserved
Embeddings	No	N/A
Internal notes	No	Use `<!-- -->` comments

Exclusion Patterns

---
publish: false  # Exclude this document
---

Or directory-level:

~/.lattice/docs/
├── tryhackme/           # Published
├── personal-notes/      # Not published (prefix or config)
└── _drafts/             # Not published (underscore prefix)

Future Enhancements

Phase 5+

Knowledge Graph Visualization
- D3.js or Cytoscape graph of entities
- Interactive exploration
- /graph page
RSS Feed
- Auto-generated from published topics
- Updates when new content added
API Endpoints (via Netlify Functions)
- /api/search?q=... - Programmatic search
- /api/graph/:topic - Graph data as JSON
Multi-Author Support
- Vanity URLs: lattice.zabaca.com/:author/:topic
- Author attribution
Versioning
- Track document versions
- Show “last updated” dates
- Changelog per topic
Comments/Discussions
- GitHub Discussions integration
- Or Giscus for page-level comments

Security Considerations

No sensitive data in published content
- Review before publishing
- Exclude patterns for private directories
No embeddings published
- Vector data stays local
Source attribution
- Maintain source links
- Respect copyright in research
Access control (future)
- Could add authentication for private topics
- Netlify Identity or similar

Cost Analysis

Component	Cost
Netlify hosting	Free (100GB/month bandwidth)
Custom subdomain	Free (already own zabaca.com)
Astro/Starlight	Free (open source)
Pagefind search	Free (static, client-side)
Total	$0/month

Success Criteria

lattice publish tryhackme works end-to-end
Site accessible at lattice.zabaca.com/tryhackme/
Search works across all published topics
Mobile-friendly responsive design
Fast (< 2s page load)
Accessible (Lighthouse score > 90)

Next Steps

Create Astro site scaffold in ~/.lattice/site/ or ~/zabaca/lattice-site/
Implement PublishCommand in Lattice
Test with tryhackme topic as first publish
Configure Netlify subdomain
Deploy and verify

architecture

Purpose

Goals

Architecture Overview

URL Structure

Domain Setup

URL Patterns

Slug Derivation

Technology Stack

Core

Astro Starlight Features Used

File Structure

Lattice Side (source)

Astro Site (generated)

Markdown Transformation

Input (Lattice format)

Output (Starlight format)

Transformation Rules

CLI Commands

Primary Command

Options

Examples

Related Commands

Implementation Plan

Phase 1: Foundation (MVP)

Phase 2: Astro Site

Phase 3: Graph Integration

Phase 4: Deployment

Netlify Configuration

DNS Setup

netlify.toml

Database Schema Additions

Published Topics Tracking

.publish-manifest.json (Alternative)

Content Policies

What Gets Published

Exclusion Patterns

Future Enhancements

Phase 5+

Security Considerations

Cost Analysis

Success Criteria

Next Steps

Sources