Project HELIOS: Overall System Architecture

Project HELIOS: Chapter 4 — Overall System Architecture

4.1 The HELIOS Architectural Stack

The HELIOS Operating System is divided into three distinct operational layers: The Orchestrator (Intent), the OS Core (Memory & Routing), and the Execution Engine (Compilation & Rendering).

+-----------------------------------------------------------------------------------+
|                            NATURAL LANGUAGE + RAW MEDIA                           |
+-----------------------------------------------------------------------------------+
                                          |
                                          v
+------------------------------------+          +-----------------------------------+
|  Orchestrator LLM (Planner Agent)  | <------> |       Milvus Vector Database      |
+------------------------------------+          +-----------------------------------+
                                          |
                                   EIR Generation
                                          v
+-----------------------------------------------------------------------------------+
|                          EIR COMPILER & OPTIMIZER                                 |
+-----------------------------------------------------------------------------------+
      |                                   |                                   |
      v                                   v                                   v
+-----------------------+   +---------------------------+   +-----------------------+
|  Custom VFX Core      |   | Vision Agent              |   | Audio Agent           |
|  (Node Graph)         |   | (SAM 2.1 / DINO)          |   | (DeepFilterNet3)      |
+-----------------------+   +---------------------------+   +-----------------------+
      |                                   |                                   |
      +-----------------------------------+-----------------------------------+
                                          |
                                          v
+-----------------------------------------------------------------------------------+
|                     GPU RENDER QUEUE (PyTorch / GLSL Shaders)                     |
+-----------------------------------------------------------------------------------+

1. The Orchestrator Layer (Intent)

Contains a local 70B reasoning model coupled to a Milvus vector database storing clip embeddings, transcripts, and temporal scene metadata. It interprets natural language prompts and generates valid EIR DAGs.

2. The OS Core Layer (Memory & Routing)

Manages the EIR Compiler, AST Optimizer, and GPU VRAM topology. It resolves asset dependencies, optimizes rendering paths, and manages pinned memory ring buffers.

3. The Execution Engine Layer (Compilation & Rendering)

A C++23/Rust kernel that parses the EIR DAG, invokes specialized vision and audio agents via Model Context Protocol (MCP), executes fragment shaders, and encodes the final H.264/H.265 video stream.

---

4.2 Data Flow & Execution Pipeline

The execution engine processes media through a deterministic 5-stage pipeline:

1. Ingestion: Raw media files are decoded using hardware GPU decoders (NVDEC, Apple VDA). Audio is transcribed via Whisper.
2. Semantic Planning: The Orchestrator LLM analyzes transcripts and prompt intent to construct a semantic timeline.
3. Agent Delegation: Tasks (e.g., "Rotoscope subject X from frame 100 to 200") are dispatched to specialized vision and audio agents.
4. EIR Compilation: All agent outputs are serialized into a single EIR JSON-based DAG.
5. Rendering: The GPU engine parses the EIR DAG, applies GLSL fragment shaders, evaluates C++ AST physics, and encodes the final render.