Policy stress in simulation

A policy that clears the simulator's nominal conditions has not been shown to survive a world that is heavier, lighter or slipperier than the one it was tuned for. This run asks where a scripted controller stops clearing a gate defined relative to its own nominal rate.

Robosuite manipulation primitives driven by a scripted OSC_POSE position controller. The controller is scripted on purpose. The comparison is then between stress tiers rather than between policies, and a difference cannot be a difference in learning.

Every task is run at its nominal condition and at a ladder of stress tiers that scale mass and friction. Seeds are paired across tiers, so a tier-to-tier difference is a condition difference and not a sampling difference. The gate is relative: a tier passes when it retains a fixed fraction of that task's own nominal success rate.

Scripted OSC_POSE controller, four tasks, four tiers, 50 episodes per cell. Lift and PickPlaceCan hold at 1.000 across every tier. Stack moves 0.900 to 0.820. Door moves 0.800 to 0.760 with the moderate tier at 0.660, which is non-monotonic and is reported as such.

What this establishes: the ladder applies, the scoring measures it, and the harness reproduces. What it does not establish: policy robustness. That run needs a learned checkpoint, and it runs on a partner's own. The relative gate passes a weak policy at 0.8x its own weak baseline; an absolute floor is not yet defined — the gate tier is the only threshold.

The headline figures below are the run's own. The video shows the same run as the pipeline produced it.

This page. It is a scripted OSC_POSE position controller in robosuite — not a learned policy, and no real robot — stressed by tiered mass and friction, and scored against a pre-registered gate that is relative to each task's own nominal rate.

An earlier policy-stress ladder circulated with a different set of numbers. It was produced by a scripted controller too, on burned seeds under the first version's harsher perturbation ranges, and it is not reproducible from any stored artifact — so it is not published here, and no page of this site carries it.

Two of the four tasks are insensitive controls: Lift, Lift, Lift, Lift, Stack, Stack, Stack, Stack, PickPlaceCan, PickPlaceCan, PickPlaceCan, PickPlaceCan, Door, Door, Door, Door held their nominal rate through every tier, so they measure the harness's reproducibility rather than its sensitivity. The sensitive tasks are none.

Where does the degradation begin, and is the tier where it begins a tier the policy was ever tuned for?

• Pillar 1 is simulation only: a scripted OSC_POSE position controller in robosuite. No learned policy, no real robot, no real footage.
• The gate is relative to nominal, not absolute: the effective threshold for a task is 0.80 times that task's own nominal rate.
• robosuite is not bit-deterministic, so the reproducibility figure is a measured band rather than a promise of exact replay.
• At n=50 per tier a rate near its threshold is often inconclusive: the Wilson interval straddles the threshold more often than not.
• Only Stack, Door moved under perturbation; Lift, PickPlaceCan held its nominal rate through every tier, so they measure the harness's reproducibility, not its sensitivity.