Project HELIOS: Core Editing Engine Technical Specification

Project HELIOS: Chapter 5 — Core Editing Engine Architectural Specification

5.1 Introduction & Post-GUI Engine Philosophy

The HELIOS Core Engine is a high-throughput, headless compositing and rendering kernel written in C++23 and Rust, designed specifically to operate without a Graphical User Interface (GUI). Unlike traditional Non-Linear Editors (NLEs) like Adobe Premiere Pro, DaVinci Resolve, or Adobe After Effects—which are built around interactive event loops and manual timeline viewports—the HELIOS Core Engine executes programmatically as an explicit compilation target for AI Agent swarms.

The engine unifies 2D NLE timelines, 2.5D layer compositing (After Effects parity), 3D spatial simulation (Blender/USD parity), and neural execution queues into a single Directed Acyclic Execution Graph (DAG).

---

5.2 The Core Composition Stack & Layer Hierarchy

The compositing pipeline models scenes as hierarchical Directed Acyclic Graphs (DAGs). Every frame evaluation traverses a tree of Layer instances, evaluating spatial transformations, blending math, and shader sub-graphs.

5.2.1 Layer Primitives

The engine natively instantiates eight discrete layer primitives:

1. Media Tracks (Video/Image/Audio): Memory-mapped GPU textures sourced from hardware-accelerated decoders (NVDEC, Apple VDA, VAAPI).
2. Solid & Shape Layers: Procedural 2D vector path primitives evaluated via GPU-accelerated path rasterizers (using Skia/ModernGL backends).
3. Text Layers: Dynamic typography nodes with full sub-pixel glyph layouts, animatable glyph transforms, and expression-bound formatting.
4. Adjustment Layers: Non-rendering spatial containers that pass down input framebuffers through a chain of fragment shaders to all lower layers.
5. Pre-Compositions (Sub-Graphs): Nested composition execution trees with isolated render targets, framerates, and spatial boundaries.
6. Null Objects: Non-rendering $4 \times 4$ transformation anchors used as parent nodes for kinematic chains.
7. 3D Camera Layers: Virtual cameras specifying intrinsic matrices $K$ and extrinsic matrices $[R \mid t]$ for perspective projections.
8. 3D Light Layers: Point, Spot, Directional, and Area light nodes emitting volumetric metadata for shadow mapping and PBR shading.

---

5.2.2 Track Mattes & Alpha Compositing

The Engine supports advanced alpha and luminance masking across any layer pair. Let $C_{src}$ be the source color tensor, $A_{src}$ the source alpha, and $C_{matte}, A_{matte}$ the matte color/alpha tensors. The track matte operations are mathematically evaluated per-pixel:

• Alpha Matte: $A_{\text{out}} = A_{\text{src}} \cdot A_{\text{matte}}$
• Alpha Inverted Matte: $A_{\text{out}} = A_{\text{src}} \cdot (1.0 - A_{\text{matte}})$
• Luma Matte: $A_{\text{out}} = A_{\text{src}} \cdot (0.2126 \cdot C_{\text{matte},r} + 0.7152 \cdot C_{\text{matte},g} + 0.0722 \cdot C_{\text{matte},b})$
• Luma Inverted Matte: $A_{\text{out}} = A_{\text{src}} \cdot (1.0 - (0.2126 \cdot C_{\text{matte},r} + 0.7152 \cdot C_{\text{matte},g} + 0.0722 \cdot C_{\text{matte},b}))$

---

5.2.3 Programmatic Blending Mode Engine

The compositing engine implements all 28 industrial blending modes on the GPU in float32 linear color space before tone-mapping. For normalized source color $S$ and destination background $B$:

• Multiply: $f(B, S) = B \cdot S$
• Screen: $f(B, S) = 1.0 - (1.0 - B) \cdot (1.0 - S)$
• Overlay:

• Color Dodge: $f(B, S) = \frac{B}{1.0 - S + \epsilon}$
• Difference: $f(B, S) = |B - S|$

---

5.3 3D Engine Architecture: Headless USD & Blender Integration

To achieve full parity with 3D suites (such as Blender), HELIOS embeds a dual-tier 3D engine: a low-latency C++ Vulkan/Metal rasterizer for simple 3D motion graphics, and a headless background daemon interface wrapping Blender’s bpy and Pixar’s Universal Scene Description (USD) runtime for photorealistic path-tracing (Cycles) and EEVEE-Next renders.

5.3.1 Camera Solving & Motion Reconstruction

When integrating 3D assets into existing raw footage, the engine computes camera intrinsics and extrinsics via automated structure-from-motion (SfM). Keypoints tracked by the Vision Agent yield a projection matrix $P = K[R \mid t]$.

Points $X \in \mathbb{R}^4$ in 3D world space map to image coordinates $x \in \mathbb{R}^3$ via:

This allows generated 3D meshes, particle emitters, or extrusions to lock rigidly to tracked physical features in raw video plates.

---

5.4 Programmatic Expression Engine & Physics

The HELIOS Expression Engine replaces manual UI keyframing with executable Abstract Syntax Trees (ASTs). The Agent writes mathematical expressions directly into the EIR, which are compiled JIT (Just-In-Time) via LLVM to machine instructions or evaluated per-frame in C++.

5.4.1 Damped Harmonic Oscillator (Inertial Bounce)

To evaluate organic, physically reactive UI and motion graphic animations programmatically without manual keyframes, property values $y(t)$ are calculated using the damped spring equation:

where $\zeta$ is the damping ratio, $\omega_n$ is undamped natural frequency, and $\omega_d = \omega_n \sqrt{1 - \zeta^2}$ is damped frequency.

5.4.2 C++ AST Expression Evaluator Code Snippet

#include <cmath>
#include <iostream>

struct Keyframe {
  float frame;
  float value;
  float in_tangent_x, in_tangent_y;
  float out_tangent_x, out_tangent_y;
};

class InertialBounceEvaluator {
public:
  static float EvaluateBounce(float current_frame, float key_frame, float amp, float freq, float decay) {
    if (current_frame < key_frame) return 0.0f;
    float t = (current_frame - key_frame) / 30.0f; // 30 FPS norm
    return amp * std::sin(freq * t * 2.0f * M_PI) * std::exp(-decay * t);
  }
};

// C++ Engine Loop called by Frame Scheduler
extern "C" float EvaluateLayerScale(float frame, float base_scale) {
  float bounce = InertialBounceEvaluator::EvaluateBounce(frame, 15.0f, 0.25f, 5.0f, 8.0f);
  return base_scale + bounce;
}

---

5.5 GPU Memory Architecture & Frame Cache Engine

Running multi-modal AI inference concurrently with 4K video rendering introduces immense VRAM pressure. HELIOS solves this with a Zero-Copy Shared Frame Pool and an aggressive cache eviction strategy based on DAG Hash Invalidation.

5.5.1 Memory Topology

• System VRAM Ring Buffer: Pre-allocates pinned GPU memory pools for raw frame textures, preventing dynamic reallocation overhead during runtime.
• Unified Memory Architecture (UMA) Optimization: On Apple Silicon platforms, the engine utilizes zero-copy MTLBuffer sharing between PyTorch tensors and Metal render passes.
• NVMe Fast Swap Cache: Unmodified intermediate layers (e.g., rotoscoped alpha masks from SAM 2.1) are encoded directly to disk as 16-bit half-float EXR or ProRes 4444 sequences.

5.5.2 DAG Invalidation Hashing

Every node $N_i$ in the render graph generates a temporal hash $H(N_i, t)$:

If $H(N_i, t)$ matches the GPU cache key, rendering for node $N_i$ at frame $t$ is bypassed entirely.

---

5.6 GLSL Transition & Motion Shader Subsystem

Transitions in HELIOS are implemented as programmatic GLSL fragment shaders executed on dual framebuffers (texA, texB) over progress parameter $t \in [0, 1]$.

5.6.1 GLSL Optical Zoom-Glitch Transition Code

#version 330 core
out vec4 FragColor;
in vec2 uv;

uniform sampler2D texA; // Outgoing Clip
uniform sampler2D texB; // Incoming Clip
uniform float progress; // Normalized [0.0 -> 1.0]
uniform float zoomIntensity;

vec2 zoom(vec2 uv_in, float amount) {
  vec2 center = vec2(0.5);
  return (uv_in - center) * (1.0 - amount) + center;
}

void main() {
  float p = smoothstep(0.0, 1.0, progress);

  // Calculate Chromatic Aberration & Zoom Offset
  vec2 uvA = zoom(uv, p * zoomIntensity);
  vec2 uvB = zoom(uv, (1.0 - p) * zoomIntensity);

  // Sample Channel Split for Motion Blur Glitch
  float rA = texture(texA, uvA + vec2(p * 0.03, 0.0)).r;
  float gA = texture(texA, uvA).g;
  float bA = texture(texA, uvA - vec2(p * 0.03, 0.0)).b;
  vec4 colA = vec4(rA, gA, bA, 1.0);

  vec4 colB = texture(texB, uvB);

  // Crossfade Alpha Blend
  FragColor = mix(colA, colB, p);
}

---

5.7 Conclusion

The HELIOS Core Editing Engine provides a deterministic, programmatic foundation for autonomous AI editing. By abstracting After Effects' compositing hierarchy, expressions, and blending math alongside Blender's 3D spatial intelligence into a zero-GUI execution DAG, the system enables AI Agent swarms to execute professional-grade post-production pipelines entirely offline.