To-do list
"Add call the bank to my list", "what's on my list" and "mark call the bank as done". Open items show in the Today tab, the daily brief and the exported report.
v0.16.0 To-dos, Watchers, Habits and Quick Answers
The Open-Source AI System Control Agent
Control your entire computer with natural language, voice, and hand gestures. An open-source, privacy-first AI agent that plans, executes, and verifies complex multi-step tasks — running locally on your hardware.
162
Actions on main
30+
Gestures
21
Agents
Tier 5
Security
Intuitive Command. Infinite Power.
Control your entire digital ecosystem with zero friction through four core modalities.
mic
Voice & Speech
"Hey Heliox" ambient wake-word with VAD-based endpoints, barge-in support, and real-time push-free dispatch for frictionless execution.
front_hand
30+ Hand Gestures
Static poses (Palm, Pinch) and motion gestures (Two-Finger Swipe) with kinematic smoothing, a 3D hand-geometry filter, and optional gaze tracking fusion.
visibility
Always-On Awareness
Continuous computer-vision loop taking screenshots every 3s, detecting active apps, and maintaining a rolling context buffer fed into the LLM planner.
bolt
Autonomous Jobs
Fire-and-forget background tasks that decompose, execute, and verify completely independent of the UI or main event loop.
psychology v0.16.0 — Verified Multi-Agent
JARVIS Autonomy
Heliox OS combines reactive commands with opt-in proactive and background capabilities — all running through the same permission and verification pipeline.
Brain Load: 84%
arrow_right_alt
Cognitive Offload
🧠
Adaptive Proactive Suggestions
Pattern-matches local screen context and surfaces visible, optional help before you ask. Accept/dismiss feedback tunes each pattern's timing and priority, and temporarily suppresses repeatedly rejected suggestions.
🤝
Interactive Execution Companion
Independently reviews proposed plans, can warn, revise, or stop work that drifts from the request, accepts typed or spoken corrections while a task is running, and offers grounded next ideas after verified results.
⚡
Fire-and-Forget Jobs
Spawn complex multi-step background tasks that decompose, execute, and verify completely independent of the UI or main event loop.
👁️
Always-On Screen Awareness
Auto-bootstrapped computer vision tracks your contextual state cross-platform, natively bridging exactly what you see into the LLM planner.
🎤
Continuous Voice Listener
"Hey Heliox" ambient wake-word with VAD-based speech endpoints and barge-in — start talking and Heliox stops mid-sentence to listen.
🤚
30+ Hand Gestures & Air Drawing
Static poses (Palm, Pinch) and motion gestures (Two-Finger Swipe) with kinematic smoothing. Opt-in 3D hand-geometry filtering via MediaPipe HandLandmarker and coarse gaze-tracking fusion; this is separate from the learned-risk world model.
🖱️
Gesture Cursor Control
Point to move the real OS cursor, pinch to click — opt in via Settings. Open palm always exits instantly. Off by default for safety.
🎯
Adaptive Calibration
On-device continual-learning loop personalizes pinch/thumb thresholds and wake-word matching from implicit signals — no retraining, bounded, resettable.
🌀
Arc Reactor UI & Ambient HUD
Animated, immersive Tauri overlays responding contextually to system actions with real-time visual feedback and status indicators.
neurology Zero-Dependency · On-Device
Cognitive Engine
A lightweight, dependency-free engine estimating attention, stress, and cognitive load from local interaction-history heuristics. Zero external models, no network, no license restrictions.
monitor_heart
Cognitive HUD
Real-time attention/stress/load estimates mapped onto the Svelte UI.
record_voice_over
Dynamic TTS Pacing
JARVIS automatically slows voice generation during high cognitive-load tasks.
shield
Stress-Aware Gate
High-risk actions evaluate cognitive stress first. If distracted, a 10-second auditory confirmation gate holds.
fingerprint
Persona Fingerprint
Subconscious background loop learns cognitive engagement patterns and encodes them to persona.md.
notifications
Smart Notifications
Notification pipeline buffers trivial alerts until a low-load resting state is detected.
speed
Bounded Confidence
Attention and stress estimates report capped confidence from recent signal diversity. Heuristics may add caution but never grant authority.
auto_awesome
JARVIS Intent Classifier
Fully IntegratedNative intent fusion classifying spoken commands against current workload intensity for context-aware execution routing.
Modular Cognitive Architecture
A governed plan–execute–verify system with a modular multi-agent orchestrator — not a simple command runner or an exposed chain-of-thought pipeline.
01
Gateway Hub
LLM evaluates persistent memory context before reasoning
02
Planner
Converts natural language into structured multi-step action plan
03
Orchestrator
Routes each action to 21 specialists across 20 source-scoped roles
04
Specialists
21 domain specialists execute through reviewed OS, browser, and integration adapters
05
Verifier
Normalizes executor results; 22 actions also check observed postconditions
06
Reflector
Self-improvement engine learns from successes and failures
07
Security
Five-tier permission system with confirmation gates & rollback
21 Specialist Agents
terminal
System
folder
File Ops
inventory_2
Package
settings
Service
desktop_windows
Desktop
code
Code
merge
Git
account_tree
Workflow
language
Web
wifi
Network
hub
Integration
extension
Plugin
mail
Comm
forward_to_inbox
Email
calendar_month
Calendar
rss_feed
RSS
visibility
Vision
monitoring
Monitor
search
Forensics
manage_search
Semantic
laptop
SSH
account_tree
Task Decomposition
Complex goals auto-broken into dependency-aware subtask trees with parallel execution paths.
science
Simulation Sandbox
Pre-execution impact reports for dangerous commands showing risk level, affected files, and reversibility.
trending_up
Self-Improving Prompts
Successful reasoning chains stored and reused with keyword-indexed templates and success/failure rates.
labs
Evolution Harness
Detached worktrees evaluated in a network-disabled, credential-free Docker runner — no automatic code promotion.
162 Action Types on Main
The v0.16.0 planner schema declares 162 action types with 162/162 provider coverage across 21 executable specialists. Availability still depends on OS, dependencies, credentials, integrations, and security policy; 22 actions currently have a separate observed post-condition verifier and the other 140 rely on the executor result.
v0.16 capability explorer
Trace every action to its specialist and verification depth.
Filter the canonical 162-action catalog without treating provider coverage as independent outcome verification.
Representative action map. The explorer above reads the complete machine-generated catalog.
folder File Operations
file_readfile_writefile_deletefile_movefile_copyfile_listfile_searchdirectory_summarydirectory_sizefile_hashfile_comparefile_permissions
memory Process Management
process_listprocess_killprocess_info
terminal Shell & Code
shell_commandshell_scriptcode_execute
language Browser & Web
browser_navigatebrowser_clickbrowser_typebrowser_extractbrowser_extract_tablebrowser_extract_links
screenshot_monitor Screen & Vision
screenshotscreen_ocrscreen_analyzescreen_detect_elements
inventory_2 Package Management
package_installpackage_removepackage_updatepackage_search
info System Info
system_infocpu_usagememory_usagedisk_usagenetwork_infobattery_info
web_asset Window Management
window_listwindow_focuswindow_closewindow_minimizewindow_maximize
merge Git
git_statusgit_diffgit_loggit_branchgit_stagegit_commitgit_pushgit_resolve
extension Plugins · WASM · Skills
plugin_callwasm_callskill_run
power_settings_new Power Management
power_shutdownpower_restartpower_sleeppower_lockpower_logout
volume_up Audio / Volume
volume_getvolume_setvolume_mute
brightness_6 Display / Screen
brightness_getbrightness_setscreenshot
wifi Network / WiFi
wifi_listwifi_connectwifi_disconnect
content_paste Clipboard
clipboard_readclipboard_write
schedule Scheduled Tasks
schedule_createschedule_listschedule_deletetrigger_create
data_object Environment Variables
env_getenv_setenv_list
dns Service Mgmt (Linux)
service_startservice_stopservice_restartservice_enableservice_disableservice_status
devices Platform-Specific
gnome_setting_readgnome_setting_writedbus_callregistry_readregistry_write
open_in_new Open · Launch · Notify
open_urlopen_applicationnotifydownload_file
Plugin Ecosystem
Two extension paths: auto-discovered local plugins with Ed25519 signature verification, and a reviewed GitHub marketplace with CI-enforced moderation.
cloud
weather
MarketplaceGet real-time weather data for any location via voice or text commands.
music_note
spotify-control
MarketplaceControl Spotify playback, search tracks, and manage playlists via gestures or voice.
home
home-assistant
MarketplaceBridge your smart home devices into the Heliox agent for unified automation.
verified
Ed25519 Signature Verification
Every plugin is signed. Unsigned, untrusted, or tampered packages are rejected before their manifest or code is loaded. Planner-triggered plugin actions route through the normal permission system.
Built for Privacy & Security
Run entirely on local hardware for total privacy, or seamlessly connect to frontier cloud models. All AI outputs pass through structured schema validation before execution.
Hybrid
Local & Cloud LLMs
AES-256
Vault Encryption
HMAC
Audit Chain
shield
Five-Tier Permission System
Read-only through root-level with confirmation gates and per-action granular approve/deny.
science
Simulation Sandbox
Pre-execution impact reports for dangerous commands with risk assessment and reversibility tracking.
psychology
LLM Safety Critic
Secondary LLM independently reviews Tier 3-4 plans before the confirmation gate fires.
model_training
Learned Risk World Model
Bundled MLP predicts action impact from training samples. Can interrupt but never remove rule-based warnings.
history
Snapshot Rollback
Btrfs/Timeshift on Linux, Windows Restore Points. If backend unavailable, destructive action doesn't run.
undo
Per-Task Undo
Restores only what Heliox recorded, re-checks each item first and leaves anything you changed since alone. Computed in code, never by a model, and every undo needs your click.
receipt_long
Signed Task Receipts
Ed25519-signed, hash-chained record of each task that ran actions. Check an exported receipt anywhere with python -m pilot.workflows.task_receipts verify.
record_voice_over
Pocket TTS (Open-Source Voice)
Free, CPU-only local speech model — no API key, no cost. Falls back to platform TTS if not installed.
stop_circle
Mid-Flight Cancellation
Stop button genuinely kills in-flight subprocess (proc.kill()) instead of only stopping the next action.
key
Encrypted Key Storage
API keys stored via platform keyring (GNOME Keyring / Windows Credential Manager). Never logged or sent to local LLMs.
new_releases
Public release · v0.16.0
To-dos, Watchers, Habits and Quick Answers
v0.16.0 handles the small things you ask a computer all day. Keep a to-do list by voice, ask Heliox to tell you when an app closes or a download finishes, set reminders that repeat or wait for an app, track habits, and get sums, conversions and world time on the spot. All 91 built-in commands are answered in code, without a model, and none of the new features can act on their own.
Tell me when
"Tell me when Chrome closes", "let me know when my download finishes" or "tell me when my battery is full". Watchers check every few seconds, say once what happened and give up after a day. They only look.
Repeating and app reminders
"Remind me every hour to stand up" repeats, and interval reminders skip 10 PM to 7 AM. "Remind me to submit my timesheet when I open Outlook" waits for that app to come to the front.
Good morning and good night
"Good morning" gives the date, today's reminders, your list, habit streaks to keep and a nearly full drive or low battery. "Good night" sums up the day with focus time, finished to-dos and tomorrow's first reminder.
Habits and focus history
"Log a workout", "how many workouts this week" and "what's my workout streak". "I did …" only logs habits you already track. Focus sessions are logged, so "how long did I focus this week" has an answer.
Quick answers and helpers
Sums, percentages, unit conversions, date countdowns and the time in about 60 cities are worked out exactly; arithmetic is parsed, never run as code. "Generate a password" puts one on the clipboard and nowhere else. Coin flips, dice and a stopwatch need no model.
Your files and this computer
"What did I download today" and "where is my budget spreadsheet" search your own folders, never inside code repositories. "Why is my computer slow" reads the processor, memory, disks and biggest apps. Heliox only reports; closing anything still asks first.
Clipboard history and snippets
Opt-in clipboard history stays in memory only and never keeps passwords, keys or private windows. "Save that as my signature" reuses text, and "during focus, tell me if I open YouTube" gives one heads-up per session and blocks nothing.
Clipboard history stays off until you turn it on. To-dos, habits, focus history, snippets and app reminders stay on this computer, appear in Settings → Your data with a Clear button each, and are included in backups. Closing an app, opening a file or anything else that changes the computer is an ordinary request with the usual approvals. v0.16.0 keeps v0.15.0's per-task undo, signed receipts, opt-in activity timeline, phone requests and fail-closed approvals.
Versioned evidence
Read the dated benchmark results and current CI results.
Physical microphone, camera, gaze, gesture, audible TTS, and live-EEG accuracy remains device-dependent.
About Heliox OS
What is Heliox OS? Current release, benchmarks, and research limits.
Is Heliox OS the same product as Heliox IDE?
No. Heliox OS is the desktop automation agent at helioxos.dev and VyomKulshrestha/Heliox-OS. Heliox IDE is a different project. Heliox OS is also unrelated to heliox helium-oxygen medical gas.
Which operating systems does Heliox OS support?
Heliox OS publishes Windows, macOS, and Linux release targets. Windows 10/11 is its primary hardware-development platform, and device-dependent behavior still needs validation on the target system.
What are the most important current Heliox OS limitations?
Hardware-dependent features need device-specific validation, operating-system permissions and action availability vary by platform, and external model behavior, availability, quotas, and terms remain provider-owned.
Measured software evidence · 26 October 2026
Benchmarks with boundaries.
The public evidence bundle records distributions, regression controls, raw data, reproduction commands, and known exclusions. These are software-path measurements on one Windows host—not universal device, provider, network, or human-accuracy claims.
Heliox OS assistance co-adaptation is a desktop-software study, unrelated to heliox medical gas. It passed 3/3 policy agency audits on simulated tuning families and changes presentation only. The adaptive engine does not beat the plain interface for every simulated family, and these results do not establish an improvement for real people. Read the simulation evidence.
Guarded local path
25.234 ms
Median across 100 ready-daemon CPU-status requests; 26.930 ms p95 and zero model calls.
Intent controls
59 / 59
A fixed deterministic routing regression set, not population-level language accuracy.
Shared-loop health
65 ticks
Concurrent scheduler heartbeats during a real one-second CPU sample; 16.618 ms maximum gap.
Built-in commands
91 / 91
Command-guide examples handled in code; 1.384 ms median and 14.111 ms p95 per reply across 1,820 replies, zero model calls. Control path only, not speech recognition or voice latency.
Learned risk
25.2k / 5.4k
Training and temporal-validation samples for a bounded disk/process predictor; deterministic policy remains authoritative.
Codex CLI planning
3 / 3
Fixed planning-only cases at 14.708 s median; one account, no action execution, and zero destructive actions.
What these numbers exclude
Model-provider and network latency, browser page loads, UI rendering, microphone capture, audible TTS, camera, gesture, gaze, live EEG, and human accuracy all require separate evidence.
Get Started
Download the compiled desktop app or build from source for development.
Windows
1.
Download Heliox-OS_0.16.0_x64-setup.exe or the .msi from the latest published release.
2.
Install and open Heliox OS.
3.
Enter your API key (Gemini, OpenAI, OpenRouter, Claude, or Meta) in Settings. OpenRouter users can enter an exact catalog model ID, including a current DeepSeek ID.
macOS
1.
Download for Apple Silicon: Heliox-OS_0.16.0_aarch64.dmg
2.
Download for Intel: Heliox-OS_0.16.0_x64.dmg
3.
Mount, drag to Applications, and configure your API key in Settings.
Linux
1.
Choose your format: .AppImage, .deb, or .rpm
2.
Install and run. Requires Python 3.11+.
3.
Configure API key in Settings. First launch may need time for env initialization.
# Clone and install daemon
git clone https://github.com/VyomKulshrestha/Heliox-OS.git
cd Heliox-OS/daemon
pip install -e ".[all,dev]"
# Run the daemon
python -m pilot.server
# Run the frontend (separate terminal)
cd tauri-app/ui && npm ci && npm run dev
Requires Python 3.11+. The desktop app starts the local daemon automatically. Browser-only UI development requires a manually started daemon.
Try It
Natural language commands for anything on your system — from simple queries to complex multi-step automations.
"Show me my system info"
"Kill the process using the most CPU"
"Install Firefox"
"Set my volume to 50%"
"Create a Python project with tests"
"Go to Wikipedia and summarize AI"
"Monitor CPU above 80%"
"List all .py files on Desktop"
"Undo that"
"What did you do today?"
"Focus for 45 minutes"
"Add call the bank to my list"
"Let me know when my download finishes"
"What's 15 percent of 240?"
"Why is my computer slow?"
"Good morning"
1
import heliox
2
from heliox.core import Agent
3
4
# Initialize with 21 specialist agents
5
agent = Agent(name="Heliox-Alpha", mode="Autonomous")
6
agent.spawn_agent(permissions=["sys_control", "vision"])
7
8
agent.execute_loop(prompt="Organize my workspace")
9
>> Status: Agent Active. 21 specialists online.
Capability guides
What Heliox can do—without the hype
Plain-language guides explain the real workflow, safety boundary, hardware requirements, and known limits for every major interaction mode.
01
Voice control
Continuous listening, speech, approvals, and device limits.
02Browser + apps
Target resolution, visible actions, and verified outcomes.
03Hands-free access
Multiple input paths with independent stop controls.
04Gesture + gaze
On-device signals, temporal checks, and calibration.
05Autonomous work
Durable jobs and bounded execution—not unlimited authority.
06Plugin marketplace
Reviewed submissions, integrity checks, and capabilities.
07 · RESEARCH BOUNDARYRecorded-EEG research
Synthetic and recorded signal validation, clearly separated from any live brain-control claim.
08 · PROVIDER BOUNDARYExisting AI subscriptions
Official CLI login, model selection, bounded token evidence, and Heliox-owned approvals.
Creator Profile
About Me
Hi, I'm Vyom Kulshrestha, a pre-final year Computer Science student at VIT, passionate about building next-generation AI systems and intelligent developer experiences.
I'm building Heliox OS — a futuristic AI-powered operating system interface designed to combine autonomous agents, voice interaction, multimodal intelligence, and system-level automation into a seamless computing experience.
Whether you're a developer, open-source contributor, recruiter, or someone excited about the future of human-computer interaction, I'd love to connect.
Ready to supercharge your OS?
An open-source project built solo, growing with every contributor. Be part of what comes next.