ACT
68.3%41 successes / 60 trials
ROBOTICS · A CONTROLLED COMPARISON
ACT vs.
SmolVLA
Same robot. Same local data. Different strengths.
A physical comparison on the SO-101 using 150 demonstrations, two camera views, and 120 autonomous trials across familiar and shifted conditions.
Explore the findings↘THE FINDING
ACT led overall.
The workspace tells a more complicated story.
ACT was more dependable when the task stayed close to the demonstrations. SmolVLA did slightly better under unfamiliar positions, and both policies tied under darker lighting.
+6.7percentage points overall
SmolVLA
61.7%37 successes / 60 trials01 / CONTROLLED SETUP
One task. Two practical systems.
What stayed fixed—and what did not.
150demonstrations
88,162frames
2synchronized cameras
100Ksteps per policy
15trials per condition
ACT
Task-specific policy
No prior ACT policy checkpoint. Its ResNet-18 visual encoder began with ImageNet weights.
- Action chunk
- 100
- Temporal ensembling
- Off
- Observation
- Two RGB views
SmolVLA
Pre-trained VLA, fine-tuned locally
Initialized from the released lerobot/smolvla_base checkpoint, then fine-tuned on the same 150 episodes.
- Action chunk
- 50
- Inference
- Synchronous
- Observation
- Two RGB views + fixed prompt
Interpretation rule. This compares two selected practical training configurations—not architecture alone. Initialization, preprocessing, chunk length, learning-rate schedule, and trainable components differ.
02 / FOUR CONDITIONS
Change one thing at a time.
15 trials per model in every condition.
Familiar
Familiar objects · familiar cells 3, 4, 6, 7 · three lamps on
Unfamiliar object
A rigid purple block changes identity, shape, color, and material.
Unfamiliar position
Familiar objects begin in unseen cells 1, 2, 5, 8, or 9.
Unfamiliar environment
The three workspace lamps are switched off; everything else stays fixed.
03 / RESULTS
Absolute results before interpretation.
Each bar represents 15 real-world trials.
ACT led in two conditions, SmolVLA in one, with one tie.
ACT was 13.3 points higher in the familiar condition and 20.0 points higher with the unfamiliar object. SmolVLA was one trial higher at unfamiliar positions. Both reached 66.7% under darker lighting.
04 / SPATIAL PATTERN
Similar totals.
Different local strengths.
FarthestMiddleClosest
↑ SO-101 positioned below Cell 8
05 / TRAINING COST
ACT reached its evaluated checkpoint
2.5 hours sooner.
Same computer · 100,000 steps
ACT training≈ 12 h
SmolVLA fine-tuning≈ 14.5 h
Wall-clock time compares the actual procedures used. It is not an isolated measure of architectural efficiency.
06 / READ THE BOUNDARIES
A careful result is a bounded result.
The experiment is descriptive. It does not claim statistical significance or universal model superiority.
Scope +
One SO-101 arm, one pick-and-place task, one 150-episode dataset, and one final checkpoint per model.
Trial design +
Fifteen trials per model-condition pair. Objects and positions were randomized independently rather than matched trial-for-trial.
Order effects +
All ACT trials were completed before all SmolVLA trials, with conditions tested in a fixed order.
Measurement +
Outcomes were binary. Completion time, joint trajectories, end-effector paths, memory, and energy were not recorded.
THE LARGER LESSON
More demonstrations are not automatically more experience.
A dataset can contain many episodes while showing the robot a narrow world. Variety and balance across objects, positions, backgrounds, and lighting may matter as much as total quantity.
07 / PAPER & EVALUATION
Read the full record.
Methods, trial protocol, results, limitations, and independent academic evaluation.