Interactive generative systems that respond to user input in real time have advanced rapidly, moving beyond fixed video clips toward environments that can be navigated and modified on the fly.
Most early examples of such models prioritized visual rendering and control, leaving acoustic elements either absent or added after the fact through separate processes. This separation often produced mismatches between what the user saw and what they heard, particularly when the viewpoint changed.
On October 12, 2026, Noiz AI and researchers at the Hong Kong University of Science and Technology released what they call the 'Helix World,' an interactive model that generates video and spatial stereo audio together from a starting image.
Noiz Users navigate with standard keyboard inputs: WASD keys for movement and arrow keys for looking around.
An optional text prompt can describe or reshape the scene, with changes taking effect in subsequent generation chunks of roughly 1.3 seconds.
Output runs at 768 by 512 resolution and 24 frames per second, accompanied by 48 kHz stereo audio whose spatial characteristics shift with camera motion so that sound sources pan as the viewpoint turns.
Builds on top of the open-source LTX-2.3's audio-visual model, Helix World uses continuous 6-DoF camera trajectories that are injected into the visual attention mechanism via projective relative positional encoding, while discrete keyboard actions are incorporated through adaptive layer normalization on diffusion timesteps.
Cross-modal attention then binds visual geometry to acoustic features, transferring ego-motion into stereo panning.
Training proceeds in stages: a bidirectional teacher model is first trained on curated data, then distilled into a few-step causal student that generates block by block with a sliding key-value cache.
An online trajectory distillation loss is combined with distribution matching to reduce drift during longer rollouts, followed by long-horizon streaming fine-tuning.
Data curation began with about 4,100 hours of raw footage drawn from real-world video, game recordings, and open-source sets.
A four-stage pipeline applied heuristic filtering to remove dual-mono audio, cuts, overlays, and static scenes; standardized clips to 121 frames at the target resolution and frame rate; used a multimodal language model for semantic filtering to retain diegetic sound; and recovered metric camera poses while generating separate visual, audio, and joint captions.
The result was roughly 3,000 hours of trainable material across 2.1 million clips.
On the WBench navigation split, Helix World scores 79.9, placing it near the strongest silent models while achieving a real-time factor of 0.77 on a single NVIDIA H800 GPU (including decoding).
It is the only model in the reported comparison that both runs faster than real time and produces native audio.
On the accompanying HelixBench of 1,015 human-verified cases, which measures spatial-acoustic consistency by correlating on-screen sound source positions with stereo panning, the causal student reaches 41.8, compared with 12.6 for EchoWM, the other native-audio world model in the set.
Cascaded pipelines that re-dub Helix World video with separate video-to-audio models produce negative spatial scores, indicating that post-hoc audio does not track camera motion.
In a user study, participants preferred full audio-visual rollouts from Helix World over EchoWM and the re-dubbed variants; when video was muted, preference shifted toward EchoWM in some comparisons.
A preview checkpoint, inference code on GitHub, and browser demo are available, with the technical report on arXiv.
Social media discussion has noted the joint generation of video and camera-aligned stereo as a practical step for immersive exploration, while also observing that reported speeds apply to data-center hardware and that visual quality trails the highest-scoring silent models when audio is disregarded.
Genie 3 from Google DeepMind, accessible experimentally through Project Genie, generates navigable environments at 720p and roughly 20-24 frames per second from text or image prompts, with keyboard controls and reported consistency spanning several minutes along with roughly one minute of visual memory for previously observed elements.
It supports modes such as walking, riding, or driving and can be grounded in Street View imagery, but produces no audio.
However, access remains restricted to Google AI Ultra subscribers, and public prototype sessions are typically limited to about 60 seconds.
Runway’s GWM Worlds 2 on the other hand, is able to stream continuous 720p video at 24 frames per second together with 48 kHz audio that includes speech, sound effects, and ambience, with no fixed session length.
Users define an environment, subjects, visual style, and rules, then issue free-form text actions directed at characters or the scene itself alongside continuous camera input; dialogue can include lip synchronization, and agents can be given objectives.
The system is presented as a research preview available via contact form rather than a public demo or open weights.
In comparison to the two, Helix World is probably the weakest in generating pure visual fidelity or high-resolution rendering. Its resolution is lower, and its visual benchmark scores lag behind.
However, it trades that visual compute to generate real-time stereo sound.
Long story short, Helix World occupies a narrower band focused on navigation from a single image with native spatial stereo that tracks the camera, operating at lower resolution while remaining the only reported real-time model that combines native audio with faster-than-real-time performance on a single data-center GPU.
Genie 3 emphasizes longer visual consistency and photorealistic rendering without sound, whereas GWM Worlds 2 provides broader control through arbitrary text actions and continuous multi-modal audio including speech, at the cost of restricted access.
Each approach trades resolution, control expressiveness, audio spatial fidelity, and availability differently.
