Truthful outcomes
Executor results and durable checkpoints are revalidated; failed, rejected, skipped, partial, and unavailable states remain visible.
v0.13.0 Verified Autonomy and Runtime Hardening
The Open-Source AI System Control Agent
Control your entire computer with natural language, voice, and hand gestures. An open-source, privacy-first AI agent that plans, executes, and verifies complex multi-step tasks — running locally on your hardware.
157
Actions on main
30+
Gestures
21
Agents
Tier 5
Security
Intuitive Command. Infinite Power.
Control your entire digital ecosystem with zero friction through four core modalities.
mic
Voice & Speech
"Hey Heliox" ambient wake-word with VAD-based endpoints, barge-in support, and real-time push-free dispatch for frictionless execution.
front_hand
30+ Hand Gestures
Static poses (Palm, Pinch) and motion gestures (Two-Finger Swipe) with kinematic smoothing, a 3D hand-geometry filter, and optional gaze tracking fusion.
visibility
Always-On Awareness
Continuous computer-vision loop taking screenshots every 3s, detecting active apps, and maintaining a rolling context buffer fed into the LLM planner.
bolt
Autonomous Jobs
Fire-and-forget background tasks that decompose, execute, and verify completely independent of the UI or main event loop.
psychology v0.13.0 — Verified Multi-Agent
JARVIS Autonomy
Heliox OS combines reactive commands with opt-in proactive and background capabilities — all running through the same permission and verification pipeline.
Brain Load: 84%
arrow_right_alt
Cognitive Offload
🧠
Adaptive Proactive Suggestions
Pattern-matches local screen context and surfaces visible, optional help before you ask. Accept/dismiss feedback tunes each pattern's timing and priority, and temporarily suppresses repeatedly rejected suggestions.
🤝
Interactive Execution Companion
Independently reviews proposed plans, can warn, revise, or stop work that drifts from the request, accepts typed or spoken corrections while a task is running, and offers grounded next ideas after verified results.
⚡
Fire-and-Forget Jobs
Spawn complex multi-step background tasks that decompose, execute, and verify completely independent of the UI or main event loop.
👁️
Always-On Screen Awareness
Auto-bootstrapped computer vision tracks your contextual state cross-platform, natively bridging exactly what you see into the LLM planner.
🎤
Continuous Voice Listener
"Hey Heliox" ambient wake-word with VAD-based speech endpoints and barge-in — start talking and Heliox stops mid-sentence to listen.
🤚
30+ Hand Gestures & Air Drawing
Static poses (Palm, Pinch) and motion gestures (Two-Finger Swipe) with kinematic smoothing. Opt-in 3D hand-geometry filtering via MediaPipe HandLandmarker and coarse gaze-tracking fusion; this is separate from the learned-risk world model.
🖱️
Gesture Cursor Control
Point to move the real OS cursor, pinch to click — opt in via Settings. Open palm always exits instantly. Off by default for safety.
🎯
Adaptive Calibration
On-device continual-learning loop personalizes pinch/thumb thresholds and wake-word matching from implicit signals — no retraining, bounded, resettable.
🌀
Arc Reactor UI & Ambient HUD
Animated, immersive Tauri overlays responding contextually to system actions with real-time visual feedback and status indicators.
neurology Zero-Dependency · On-Device
Cognitive Engine
A lightweight, dependency-free engine estimating attention, stress, and cognitive load from local interaction-history heuristics. Zero external models, no network, no license restrictions.
monitor_heart
Cognitive HUD
Real-time attention/stress/load estimates mapped onto the Svelte UI.
record_voice_over
Dynamic TTS Pacing
JARVIS automatically slows voice generation during high cognitive-load tasks.
shield
Stress-Aware Gate
High-risk actions evaluate cognitive stress first. If distracted, a 10-second auditory confirmation gate holds.
fingerprint
Persona Fingerprint
Subconscious background loop learns cognitive engagement patterns and encodes them to persona.md.
notifications
Smart Notifications
Notification pipeline buffers trivial alerts until a low-load resting state is detected.
speed
Bounded Confidence
Attention and stress estimates report capped confidence from recent signal diversity. Heuristics may add caution but never grant authority.
auto_awesome
JARVIS Intent Classifier
Fully IntegratedNative intent fusion classifying spoken commands against current workload intensity for context-aware execution routing.
Modular Cognitive Architecture
A governed plan–execute–verify system with a modular multi-agent orchestrator — not a simple command runner or an exposed chain-of-thought pipeline.
01
Gateway Hub
LLM evaluates persistent memory context before reasoning
02
Planner
Converts natural language into structured multi-step action plan
03
Orchestrator
Routes each action to 21 specialists across 20 source-scoped roles
04
Specialists
21 domain specialists execute through reviewed OS, browser, and integration adapters
05
Verifier
Normalizes executor results; 18 actions also check observed postconditions
06
Reflector
Self-improvement engine learns from successes and failures
07
Security
Five-tier permission system with confirmation gates & rollback
21 Specialist Agents
terminal
System
folder
File Ops
inventory_2
Package
settings
Service
desktop_windows
Desktop
code
Code
merge
Git
account_tree
Workflow
language
Web
wifi
Network
hub
Integration
extension
Plugin
mail
Comm
forward_to_inbox
Email
calendar_month
Calendar
rss_feed
RSS
visibility
Vision
monitoring
Monitor
search
Forensics
manage_search
Semantic
laptop
SSH
account_tree
Task Decomposition
Complex goals auto-broken into dependency-aware subtask trees with parallel execution paths.
science
Simulation Sandbox
Pre-execution impact reports for dangerous commands showing risk level, affected files, and reversibility.
trending_up
Self-Improving Prompts
Successful reasoning chains stored and reused with keyword-indexed templates and success/failure rates.
labs
Evolution Harness
Detached worktrees evaluated in a network-disabled, credential-free Docker runner — no automatic code promotion.
157 Action Types on Main
The v0.13.0 planner schema declares 157 action types with 157/157 provider coverage across 21 executable specialists. Availability still depends on OS, dependencies, credentials, integrations, and security policy; 18 actions currently have a separate observed post-condition verifier and the other 139 rely on the executor result.
v0.13 capability explorer
Trace every action to its specialist and verification depth.
Filter the canonical 157-action catalog without treating provider coverage as independent outcome verification.
Representative action map. The explorer above reads the complete machine-generated catalog.
folder File Operations
file_readfile_writefile_deletefile_movefile_copyfile_listfile_searchdirectory_summarydirectory_sizefile_hashfile_comparefile_permissions
memory Process Management
process_listprocess_killprocess_info
terminal Shell & Code
shell_commandshell_scriptcode_execute
language Browser & Web
browser_navigatebrowser_clickbrowser_typebrowser_extractbrowser_extract_tablebrowser_extract_links
screenshot_monitor Screen & Vision
screenshotscreen_ocrscreen_analyzescreen_detect_elements
inventory_2 Package Management
package_installpackage_removepackage_updatepackage_search
info System Info
system_infocpu_usagememory_usagedisk_usagenetwork_infobattery_info
web_asset Window Management
window_listwindow_focuswindow_closewindow_minimizewindow_maximize
merge Git
git_statusgit_diffgit_loggit_branchgit_stagegit_commitgit_pushgit_resolve
extension Plugins · WASM · Skills
plugin_callwasm_callskill_run
power_settings_new Power Management
power_shutdownpower_restartpower_sleeppower_lockpower_logout
volume_up Audio / Volume
volume_getvolume_setvolume_mute
brightness_6 Display / Screen
brightness_getbrightness_setscreenshot
wifi Network / WiFi
wifi_listwifi_connectwifi_disconnect
content_paste Clipboard
clipboard_readclipboard_write
schedule Scheduled Tasks
schedule_createschedule_listschedule_deletetrigger_create
data_object Environment Variables
env_getenv_setenv_list
dns Service Mgmt (Linux)
service_startservice_stopservice_restartservice_enableservice_disableservice_status
devices Platform-Specific
gnome_setting_readgnome_setting_writedbus_callregistry_readregistry_write
open_in_new Open · Launch · Notify
open_urlopen_applicationnotifydownload_file
Plugin Ecosystem
Two extension paths: auto-discovered local plugins with Ed25519 signature verification, and a reviewed GitHub marketplace with CI-enforced moderation.
cloud
weather
MarketplaceGet real-time weather data for any location via voice or text commands.
music_note
spotify-control
MarketplaceControl Spotify playback, search tracks, and manage playlists via gestures or voice.
home
home-assistant
MarketplaceBridge your smart home devices into the Heliox agent for unified automation.
verified
Ed25519 Signature Verification
Every plugin is signed. Unsigned, untrusted, or tampered packages are rejected before their manifest or code is loaded. Planner-triggered plugin actions route through the normal permission system.
Built for Privacy & Security
Run entirely on local hardware for total privacy, or seamlessly connect to frontier cloud models. All AI outputs pass through structured schema validation before execution.
Hybrid
Local & Cloud LLMs
AES-256
Vault Encryption
HMAC
Audit Chain
shield
Five-Tier Permission System
Read-only through root-level with confirmation gates and per-action granular approve/deny.
science
Simulation Sandbox
Pre-execution impact reports for dangerous commands with risk assessment and reversibility tracking.
psychology
LLM Safety Critic
Secondary LLM independently reviews Tier 3-4 plans before the confirmation gate fires.
model_training
Learned Risk World Model
Bundled MLP predicts action impact from training samples. Can interrupt but never remove rule-based warnings.
history
Snapshot Rollback
Btrfs/Timeshift on Linux, Windows Restore Points. If backend unavailable, destructive action doesn't run.
record_voice_over
Pocket TTS (Open-Source Voice)
Free, CPU-only local speech model — no API key, no cost. Falls back to platform TTS if not installed.
stop_circle
Mid-Flight Cancellation
Stop button genuinely kills in-flight subprocess (proc.kill()) instead of only stopping the next action.
key
Encrypted Key Storage
API keys stored via platform keyring (GNOME Keyring / Windows Credential Manager). Never logged or sent to local LLMs.
new_releases
Public release · v0.13.0
Verified Autonomy and Runtime Hardening
Heliox now validates terminal outcomes more strictly, applies live settings transactionally, authenticates opt-in peer collaboration, coordinates multimodal inputs, and releases idle runtime resources more predictably.
Transactional settings
Supported live settings validate before commit, apply atomically, and roll back when a runtime rejects the update.
Authenticated peers
Opt-in LAN collaboration uses authenticated setup, bounded timeouts, constrained delegation, and normal permission gates.
Input arbitration
Voice, gesture, gaze, cursor, approval, and staged neural signals coordinate shared resources more reliably.
Bounded resources
Shutdown quiesces background services, while idle local TTS and model workers release heavy memory outside the daemon.
Measured quality
Step budgets, deterministic satisfied-action skips, fast local routes, and refreshed benchmark evidence make execution quality inspectable.
Current main local validation
1,989 Python tests passed · 299 frontend tests passed · 26 visual regressions passed · 0 npm vulnerabilities
The live CI link remains authoritative. Physical microphone, camera, gaze, gesture, audible TTS, and live-EEG accuracy remains device-dependent.
Measured software evidence · 27 August 2026
Benchmarks with boundaries.
The public evidence bundle records distributions, regression controls, raw data, reproduction commands, and known exclusions. These are software-path measurements on one Windows host—not universal device, provider, network, or human-accuracy claims.
Guarded local path
26.476 ms
Median across 100 ready-daemon CPU-status requests; 27.999 ms p95 and zero model calls.
Intent controls
59 / 59
A fixed deterministic routing regression set, not population-level language accuracy.
Shared-loop health
66 ticks
Concurrent scheduler heartbeats during a real one-second CPU sample; 16.575 ms maximum gap.
Learned risk
36k / 5.4k
Training and temporal-validation samples for a bounded disk/process predictor; deterministic policy remains authoritative.
Codex CLI planning
3 / 3
Fixed planning-only cases at 14.708 s median; one account, no action execution, and zero destructive actions.
What these numbers exclude
Model-provider and network latency, browser page loads, UI rendering, microphone capture, audible TTS, camera, gesture, gaze, live EEG, and human accuracy all require separate evidence.
Get Started
Download the compiled desktop app or build from source for development.
Windows
1.
Download Heliox-OS_0.13.0_x64-setup.exe or the .msi from the latest published release.
2.
Install and open Heliox OS.
3.
Enter your API key (Gemini, OpenAI, OpenRouter, Claude, or Meta) in Settings. OpenRouter users can enter an exact catalog model ID, including a current DeepSeek ID.
macOS
1.
Download for Apple Silicon: Heliox-OS_0.13.0_aarch64.dmg
2.
Download for Intel: Heliox-OS_0.13.0_x64.dmg
3.
Mount, drag to Applications, and configure your API key in Settings.
Linux
1.
Choose your format: .AppImage, .deb, or .rpm
2.
Install and run. Requires Python 3.11+.
3.
Configure API key in Settings. First launch may need time for env initialization.
# Clone and install daemon
git clone https://github.com/VyomKulshrestha/Heliox-OS.git
cd Heliox-OS/daemon
pip install -e ".[all,dev]"
# Run the daemon
python -m pilot.server
# Run the frontend (separate terminal)
cd tauri-app/ui && npm ci && npm run dev
Requires Python 3.11+. The desktop app starts the local daemon automatically. Browser-only UI development requires a manually started daemon.
Try It
Natural language commands for anything on your system — from simple queries to complex multi-step automations.
"Show me my system info"
"Kill the process using the most CPU"
"Install Firefox"
"Set my volume to 50%"
"Create a Python project with tests"
"Go to Wikipedia and summarize AI"
"Monitor CPU above 80%"
"List all .py files on Desktop"
1
import heliox
2
from heliox.core import Agent
3
4
# Initialize with 21 specialist agents
5
agent = Agent(name="Heliox-Alpha", mode="Autonomous")
6
agent.spawn_agent(permissions=["sys_control", "vision"])
7
8
agent.execute_loop(prompt="Organize my workspace")
9
>> Status: Agent Active. 21 specialists online.
Capability guides
What Heliox can do—without the hype
Plain-language guides explain the real workflow, safety boundary, hardware requirements, and known limits for every major interaction mode.
01
Voice control
Continuous listening, speech, approvals, and device limits.
02Browser + apps
Target resolution, visible actions, and verified outcomes.
03Hands-free access
Multiple input paths with independent stop controls.
04Gesture + gaze
On-device signals, temporal checks, and calibration.
05Autonomous work
Durable jobs and bounded execution—not unlimited authority.
06Plugin marketplace
Reviewed submissions, integrity checks, and capabilities.
07 · RESEARCH BOUNDARYRecorded-EEG research
Synthetic and recorded signal validation, clearly separated from any live brain-control claim.
08 · PROVIDER BOUNDARYExisting AI subscriptions
Official CLI login, model selection, bounded token evidence, and Heliox-owned approvals.
Creator Profile
About Me
Hi, I'm Vyom Kulshrestha, a pre-final year Computer Science student at VIT, passionate about building next-generation AI systems and intelligent developer experiences.
I'm building Heliox OS — a futuristic AI-powered operating system interface designed to combine autonomous agents, voice interaction, multimodal intelligence, and system-level automation into a seamless computing experience.
Whether you're a developer, open-source contributor, recruiter, or someone excited about the future of human-computer interaction, I'd love to connect.
Ready to supercharge your OS?
An open-source project built solo, growing with every contributor. Be part of what comes next.