Text
Actions
Vision-Language Model
Expert
States
Instructions
Dynin-Robotics
Dynin-Robotics is an omnimodal unified diffusion vision-language-action model that formulates language, visual dynamics, goal states, and robot actions as conditional targets of a single masked-diffusion backbone.
Model and code will be released soon.
State
Instruction
Action
Goal
Sensor
Dynin-Robotics
State
Instruction
Action
Goal
Sensor
Put the glue stick inside the open drawer0.01, 0.13, 0.63, 0.40, -0.25, -0.05
Generation
Overview
Dynin-Robotics is an omnimodal vision-language-action model built on the masked-diffusion model (MDM) backbone of Dynin-Omni. It extends a shared discrete token space to language, visual observations and future states, robot actions, and optional sensor or metadata context, representing them as variables of one partially observed trajectory. Through unified objective training, a single model learns policy generation, action-conditioned world modeling, instruction understanding, and goal-state prediction by changing only which spans are visible and which are denoised. At inference time, this shared interface supports six operating modes that combine block-wise action generation with optional goal prediction, joint world-action denoising, or world-model-based candidate reranking. Across LIBERO, LIBERO-Plus, VLABench, and real-world FR3 manipulation, Dynin-Robotics achieves strong performance and robustness, while an accelerated dInfer path delivers up to 29.15× higher effective action-token throughput with minimal prediction quality degradation.
Why Unified Models?
Language-oriented policies understand instructions, while video and world models predict how scenes change. Dynin-Robotics brings both abilities—and action generation—into one shared model.
Vision-Language Model(π0.5, GR00T)
Next states
Actions
Video Generation Model
Expert
States
Instructions
Video Generation Model(Cosmos Policy, Mimic Video)
Text
Future states
Actions
Unified Model
States
Instructions
Action
Unified Model(Dynin-Robotics)
- Text
- Vision
- Action
- Noise / Mask
- Latent condition
Four Capabilities for Robotics,
One Unified Model
Dynin-Robotics unifies policy generation, world modeling, task understanding, and goal-state prediction in a single masked-diffusion backbone. Each capability is realized as a different conditional denoising query over the same multimodal robot trajectory.
Policy
Generate robot action sequences from the current visual state and task instruction.
- Input
- Current statesInstructionsGoal stateSensor
- Output
- Action sequence
InstructionPut the glue stick inside the open drawer
Action sequence
-0.80-0.550.180.04-0.150.41
World Modeling
Predict future visual states from observed frames and robot actions.
- Input
- Current statesInstructionsActions
- Output
- Next states
InstructionTake the purple plush toy out of the bowl
Action
0.010.130.630.40-0.25-0.05
Goal-State Prediction
Generate a goal state from an initial observation and language instruction.
- Input
- Initial stateInstructions
- Output
- Goal state
InstructionUnfold the white towel on the table
Task Understanding
Decode an observed task trajectory into a natural-language task description.
- Input
- Task video frames
- Output
- Task description
Task video frames
Task descriptionPush the faucet of the sink slightly to the left
One Backbone,
Multiple Parallel Token Pathways
A shared bidirectional Transformer reconstructs masked text, image/video, and action tokens through three parallel modality pathways.
Ground truthPredictedMask
Dynin-RoboticsMasked Diffusion Language Model
TextDetokenizer
ImageDetokenizer
ActionDetokenizer
Omnimodal Unified Objective Training
A single Dynin-Robotics backbone learns policy, world modeling, goal-state prediction, and instruction understanding by changing which trajectory tokens are visible, optional, or masked for prediction.
Visible conditionOptional
Policy
Generate robot actions from the observed state and instruction, with optional goal-state context and sensor input.
ObjectiveStateInstructionAActionGoal stateSensor
Dynin-Robotics
POObjectiveVStateTInstructionActionVGoal stateSSensor
World Modeling
Predict the next visual state from the current state, with language and action provided when available.
ObjectiveStateInstructionActionVNext stateSensor
Dynin-Robotics
WMObjectiveVCurrent stateTInstructionAActionStateSensor
Goal-State Prediction
Generate a visual goal state from the initial observation and task instruction.
ObjectiveInitial stateInstructionActionVGoal stateSensor
Dynin-Robotics
GPObjectiveVInitial stateTInstructionActionStateSensor
Task Understanding
Recover the task instruction from an observed sequence of robot states.
ObjectiveState sequenceTInstructionActionStateSensor
Dynin-Robotics
TUObjectiveVState sequenceInstructionActionStateSensor
Composable Unified Inference
One post-trained model composes policy, goal-state, and world-model predictions into six inference strategies, ranging from direct action decoding to goal-guided joint denoising and dynamics-aware candidate reranking.
Default Policy
Directly denoise an action chunk from the current state and instruction.
ObjectiveStateInstructionActionStateText
Dynin-Robotics
ObjectiveStateInstructionActionStateText
ObjectiveStateInstructionAActionStateText
Dynin-Robotics
POObjectiveVStateTInstructionActionStateText
ObjectiveStateInstructionActionStateText
Dynin-Robotics
ObjectiveStateInstructionActionStateText
Demonstrations
Demonstration rollouts across LIBERO, LIBERO+, and real-world manipulation tasks on the Franka Research 3 platform.
LIBERO
Spatial
Object
Goal
Long
LIBERO+
Camera
Robot
Language
Light
Background
Noise
Layout
Real-World Manipulation
2× · autonomous
Examples
Qualitative results show how Dynin-Robotics predicts visual futures, imagines instruction-conditioned goal states, and reconstructs task language from frame sequences.
World Modeling
Predict future visual states from five observed frames and a six-dimensional robot action.
Action
0.010.01-0.12-0.020.040.00
Goal-State Prediction
An initial state and a language instruction condition the generated goal state.
Instruction
Take the pen out of the cup and place it on the counter
Instruction
place apple into middle drawer
Task Understanding
An ordered frame sequence plays as a short clip and is decoded into a task instruction.
Generated task description
Put the purple object in the bowl
Generated task description
Fold the cloth on the table
Performance
The selected comparison rows below retain the paper’s model-family grouping. LIBERO is close to saturation, while LIBERO-Plus more clearly exposes robustness to camera, embodiment, language, lighting, background, noise, and layout shifts. The throughput results also summarize action-decoding gains from the modified dInfer framework.
LIBERO
Success rate (%)
| Model | Spatial | Object | Goal | Long | AVG↑ |
|---|---|---|---|---|---|
| Vision-Language Model | |||||
| π0.5 | 98.8 | 98.2 | 98.0 | 92.4 | 96.9 |
| ABot-M0 | 98.8 | 99.8 | 99.0 | 96.6 | 98.6 |
| Video Generation Model | |||||
| Cosmos Policy | 98.1 | 100.0 | 98.2 | 97.6 | 98.5 |
| LingBot-VA | 98.5 | 99.6 | 97.2 | 98.5 | 98.5 |
| Unified Model | |||||
| MMaDA-VLA | 98.8 | 99.8 | 98.0 | 95.2 | 98.0 |
| Dynin-Robotics | 98.9 | 99.8 | 97.8 | 95.8 | 98.1 |
LIBERO-Plus
Success rate (%)
| Model | Camera | Robot | Language | Light | Background | Noise | Layout | AVG↑ |
|---|---|---|---|---|---|---|---|---|
| Vision-Language Model | ||||||||
| OpenVLA-OFT | 56.4 | 31.9 | 79.5 | 88.7 | 93.3 | 75.8 | 74.2 | 69.6 |
| π0 | 13.8 | 6.0 | 58.8 | 85.0 | 81.4 | 79.0 | 68.9 | 53.6 |
| π0-FAST | 65.1 | 21.6 | 61.0 | 73.2 | 73.2 | 74.4 | 68.8 | 61.6 |
| RIPT-VLA | 55.2 | 31.2 | 77.6 | 88.4 | 91.6 | 73.5 | 74.2 | 68.4 |
| ABot-M0 | 60.4 | 67.9 | 86.4 | 96.2 | 91.6 | 86.4 | 82.6 | 80.5 |
| Unified Model | ||||||||
| UniVLA | 1.8 | 46.2 | 69.6 | 69.0 | 81.0 | 21.2 | 31.9 | 42.9 |
| Dynin-Robotics | 59.8 | 48.2 | 85.0 | 83.5 | 84.6 | 78.2 | 71.8 | 73.0 |
Ablation Study
We ablate both unified post-training objectives and inference strategies on VLABench to examine how complementary supervision and inference-time composition affect robustness to out-of-distribution instructions. ID and OOD results are shown together with the reported OOD gap and effective throughput.
Training Objective Ablation
| Training variant | ID | OOD | Gap ↓ |
|---|---|---|---|
| Policy only | 47.15 | 33.88 | 13.27 |
| + World Modeling | 46.01 | 39.54 | 6.47 |
| + Task Understanding | 49.38 | 45.29 | 4.09 |
| + Goal-State Prediction | 49.61 | 47.28 | 2.33 |
Inference Ablation Study
| Inference variant | ID | OOD | Gap ↓ | Throughput ↑ |
|---|---|---|---|---|
| (a) Default Policy | 45.8 | 41.4 | 4.4 | 9.238 |
| (b) Action/World Model Joint Denoise | 46.4 | 40.5 | 5.9 | 8.805 |
| (c) Goal-State Guided Policy | 45.6 | 38.2 | 7.1 | 9.208 |
| (d) Action Candidate Reranking | 46.3 | 42.0 | 4.3 | 4.904 |
| (e) Goal-State Guided + Action/World Model Joint Denoise | 49.6 | 47.2 | 2.4 | 8.709 |
| (f) Goal-State Guided + Action Candidate Reranking | 48.9 | 47.7 | 1.4 | 4.828 |
VLM and Video Model Analysis
Random-instruction diagnostics reveal complementary language and visual-dynamics priors.
Under random instructions, π0.5 loses 0.16 success on InsertFlower and 0.30 on SelectFruit, showing clear language sensitivity. Mimic-Video changes by +0.10 and +0.02, respectively, indicating stronger reliance on visual and task-level action priors.
π0.5Language-oriented prior
stronger semantic sensitivity
Mimic-VideoDynamics-oriented prior
stronger task-level visual prior
This diagnostic explains complementary priors; it is not a standalone ranking of overall policy quality.
Contributors
- Hoeun Lee§ ¶Project Leader
- Jaeik Kim¶Core Contributor
- Jusang Oh¶Core Contributor
- Jinhyeok KimAcceleration
- Geon ChoiEvaluation
- Hyeonggeun KimInfrastructure
- Jaeyoung Do†Supervisor
§ Project lead
¶ Core contributors
† Supervision and Corresponding author