This case study uses a deliberately narrow experiment: a compact 619,952-parameter vision-language-action policy for one LIBERO Panda close-the-drawer task, trained first with supervised fine-tuning (SFT) and then with 24 accepted GRPO updates. Training ran sequentially on one RTX 3060, with one random seed.
The objective is not to prove that GRPO improves VLA systems in general. It is to show how to make a local improvement claim auditable—and how to stop the claim exactly where the evidence stops.
1. Define “improvement” before reading the score
An RL stage can improve a policy on one difficult slice while leaving an easier benchmark unchanged. That is not contradictory. It means the two evaluations answer different questions.
- Full-task episodes ask whether the policy can complete the whole behavior from the benchmark start state.
- Recovery episodes ask whether the policy can recover from a controlled, more failure-prone state.
The comparison therefore froze the task, simulator setup, recovery-state definition, evaluation count, and success criterion. Only the evaluated policy artifact changed from the SFT checkpoint to the post-GRPO checkpoint.
2. Report the paired result, including saturation
| Evaluation slice | SFT | After GRPO | Supported interpretation |
|---|---|---|---|
| Recovery episodes | 14 / 30 | 20 / 30 | Higher success on this fixed recovery slice |
| Full-task episodes | 3 / 3 | 3 / 3 | No measurable separation in a saturated tiny sample |
Across both policies, the evidence set contains 60 paired recovery episodes and 6 full-task episodes: 66 episode runs in total. The recovery result is the useful signal. The 3/3 versus 3/3 full-task result is a warning that the easier slice is too small and saturated to distinguish the checkpoints.
A defensible sentence is: “After 24 accepted GRPO updates, the policy improved from 14/30 to 20/30 successes on a fixed recovery evaluation while both checkpoints remained 3/3 on the small full-task check.”
It would not be defensible to write “GRPO increased general VLA success rate,” because there is one task, one seed, one simulator setup, and no broad benchmark or physical-robot evaluation.
3. Audit artifact identity before interpreting behavior
A clean score table is useless if both rows accidentally evaluated the same weights. Before running episodes, record enough identity evidence to distinguish the artifacts:
- checkpoint paths and creation timestamps;
- parameter count and a deterministic weight checksum;
- the exact evaluation command and configuration;
- episode-level outcomes, not only the aggregate rate;
- at least one recorded successful rollout for behavior inspection.
This converts “I think I loaded the GRPO model” into a verifiable chain from checkpoint to command to episode output.
4. Keep simulator evidence inside the simulator boundary
LIBERO is a benchmark for lifelong robot learning, and its simulated tasks are useful for controlled comparisons. But a simulator result does not establish camera robustness, calibration tolerance, actuator behavior, latency tolerance, or safety on a physical robot.
This experiment is best described as a narrow mechanism validation. It does not demonstrate multi-task language generalization, real-world deployment, or multi-GPU scaling. Those require separate evidence.
5. A reusable evaluation checklist
- Freeze the comparison. Keep the environment, start states, success rule, and episode budget fixed.
- Separate evaluation slices. Do not merge easy full-task runs with targeted recovery cases.
- Verify artifact identity. Record paths, hashes, configs, and commands.
- Preserve episode evidence. Store each outcome and representative rollouts.
- Report denominators. “20/30” is more informative than “66.7%” alone.
- State the boundary. Name the task, seed count, simulator, hardware, and missing tests.