Physical AI sim-to-real verification asks which simulation results support a claim about the actual robot, task and conditions. Define the claim, compare simulator and physical configurations, pair representative evaluations and record the effect of domain gaps before using simulated evidence in a release review. Simulation can support development and bounded performance arguments, but it does not automatically demonstrate the physical application or its safety. Keep the remaining physical checks and the authority for evidence use explicit.
This guide is for US manufacturing robot teams choosing how to use simulation in an acceptance or release decision. Its output is a simulation-to-physical applicability matrix. The buyer's question is whether the requirements workflow can explain which evidence transfers, under what conditions and what still needs examination on the installed system.
What does simulation-to-real verification need to establish?
Begin with the specific physical claim. Comparing candidate policies for one manipulation task is different from demonstrating an installed cell's response or accepting a broader object family. A simulator can be useful for the first decision while leaving the second or third unresolved.
Describe the intended embodiment and configuration: robot, tooling, sensors, preprocessing, policy, action adapter, control implementation, fixture and relevant task conditions. Then identify which of those elements the simulator represents and how. Similar names or geometry do not establish that observations, action semantics and physical effects match.
NASA's verification guidance includes modelling and simulation within analytical methods and calls for controlled requirements, product configuration, procedures and work products. Its validation guidance identifies problems arising from boundary conditions, approximations and insufficient geometric, physical or temporal fidelity. These are transferable engineering principles, not a commercial-robot release standard. (NASA, 5.3 Product Verification, NASA, 5.4.1.2.3, Validation Deficiencies)
NIST's physical AI research programme highlights the relationship between AI algorithms, robot systems and manufacturing tasks, including the gap between academic demonstrations and industrial use. Use that distinction to keep the task-level physical question visible. The programme does not certify a simulator or supply a universal transfer threshold. (NIST Physical AI project, Summary and Technical Idea)
What does the SIMPLER research demonstrate, and what are its limits?
Xuanlin Li and colleagues' paper, Evaluating Real-World Robot Manipulation Policies in Simulation, introduces SIMPLER environments for evaluating manipulation policies. Its PMLR 2025 abstract reports over 1,500 paired simulation-and-real evaluations across two embodiments and eight task families, with strong correlation in the reported setups. It identifies visual and control disparities as key challenges and proposes methods to reduce those gaps. (Li and colleagues, PMLR 270, 2025, abstract)
The original paper lists contributors from UC San Diego, Stanford, UC Berkeley and Google DeepMind. Its demonstrated scope concerns the manipulation setups, policies and comparisons investigated by that work. Those results are the authors' research, not Arc tests or evidence for a reader's robot. (Original paper, author affiliations and abstract)
The useful lesson is to investigate the relationship between the simulated and physical task, including consequential visual and control differences. The research does not establish that a new simulator is a reliable proxy for every robot, warehouse condition or safety function. A team needs evidence for its own intended inference.
The existing learned-policy acceptance guide defines episodes, outcomes, populations and interpretation. Use that contract for the paired comparison. This article owns the separate question of why evidence from one medium can support a physical claim.
How should the paired evaluation be defined before results exist?
Choose the physical task and decision first, then define matched cases in simulation and on the robot. Keep endpoint criteria, interventions, reset rules and conditions comparable. If a difference is intentional, record its purpose and how it limits the inference.
Retain important task slices. A favourable aggregate comparison can conceal disagreement for a consequential object, starting placement or visual condition. Select the relevant slices before execution and specify how discrepancies will affect the evidence disposition. Do not change the pairing after seeing an inconvenient physical result without preserving the original evaluation and revised question.
Control the policy and adapters. The same model checkpoint can receive different camera preprocessing or action conversion in simulation and the physical robot. A comparison with different wrappers may assess the complete pipelines, but it cannot be described as a clean comparison of simulator fidelity alone.
Record uncertainty and criteria appropriate to the claim. The project must decide what agreement means and what result leaves transfer unresolved. This guide supplies no universal correlation coefficient, trial count or acceptable performance gap. A statistical relationship for one population is also not a guarantee for unseen conditions.
What belongs in a simulation-to-physical applicability matrix?
The matrix connects a proposed physical claim, simulator representation, known gap, paired evidence and permitted use. A report should have its own identity and actual configuration. The matrix is an editorial method, with criteria and acceptance authority supplied by the project.
The following table describes a fictional US manufacturing fixture-loading proposal, MAPLE-9. Policy PL-5 would place an approved part in fixture FX-3. All evaluations are planned, none is executed and no transfer claim is accepted. The matrix contains no invented outcome values.
| Physical claim | Simulator representation and gap | Planned paired evidence | Evidence-use decision still needed |
|---|---|---|---|
| Approved part reaches the fixture endpoint | Simulator models the nominal part and fixture, contact/friction applicability is unresolved | Matched part placements and endpoint measurements with the same completion rule | Determine whether the comparison supports the declared part/task scope or requires a narrower claim |
| Perception supports the selected visual conditions | Rendered images differ from the physical camera, lighting and part finish | Paired visual-condition slices with controlled preprocessing and observation references | Identify unsupported conditions rather than treating aggregate agreement as complete coverage |
| Commands produce the required motion contribution | Simulated action adapter may differ from the deployed controller | Compare action meanings, limits and relevant motion response in the declared configurations | Resolve control differences before using simulated behaviour for the physical claim |
| Interruption has the required response | Simulator represents an interruption signal but not all physical equipment or retained load | Separate physical response evaluation for the integrated arrangement | Simulated signal handling cannot close the complete installed response claim |
| New gripper option remains within scope | Simulator and physical gripper have different geometry or compliance assumptions | Review affected contacts and task cases, plan new comparisons as needed | Existing evidence remains attached to the old option until continued applicability is assessed |
Read the fourth row carefully. It allows a simulator result to support a narrow signal-handling claim while keeping the physical interruption response open. That is a useful evidence disposition. A single pass/fail field would make the supported and unsupported questions harder to see.
Connect each gap to an owner and closure method. A rendering mismatch may need data or revised case selection, while an action-semantic mismatch needs interface and implementation work. Asking for more simulation runs would not resolve an incorrect action adapter.
Which domain gaps matter for a robot release claim?
Investigate differences consequential to the intended task. They can concern observation appearance, geometry, sensor placement, timing, control response, contact, friction, deformation, object variation and surrounding equipment. This is an engineering prompt, not a requirement to model every physical detail perfectly.
Distinguish an identified gap from an unexamined assumption. If the physical surface finish differs from the simulated material, record the difference. If nobody has characterised its influence, keep that influence unresolved. A high-fidelity description or photorealistic image cannot supply the missing evidence.
Also identify differences in experimental practice. Resetting a physical object by a person can produce a different starting population from a simulator's reset distribution. Human interventions, discarded attempts and repeated failures must follow the agreed contract. Otherwise the comparison can reflect evaluation rules rather than transfer capability.
The operating-boundary guide explains the distinction between intended conditions, tested conditions and conditions the deployed arrangement can recognise. The matrix should state which of those conditions each simulated and physical result addresses.
What physical evidence remains necessary?
Choose physical work against the claims left unresolved by the simulation comparison. For a manufacturing cell, that can include actual sensor and control integration, tooling and load behaviour, consequential interfaces, interruption and recovery, and representative task conditions. The applicable safety and assurance process determines the evidence needed for its safety claims.
A task-performance comparison does not independently establish machinery guarding, energy control or complete protective-function adequacy. Keep those requirements and verification threads linked to the release scope. The safety-function traceability guide addresses the separate function, target and validation authority.
NIST AI RMF MEASURE 2.3 concerns criteria demonstrated under conditions similar to deployment, and 2.5 addresses generalisability limitations. These voluntary outcomes support retaining the differences and restrictions on evidence use. They do not authorise a simulation-only safety release. (AI RMF 1.0, Table 3, MEASURE 2.3 and 2.5)
What failure should hold the transfer claim?
In the fictional MAPLE-9 proposal, the simulated action adapter interprets the policy output as a position increment, while the deployed controller expects a differently defined command. The proposed paired plan compares only final task completion. It contains no controlled comparison of action semantics or motion behaviour.
The unresolved interface difference could make apparent task agreement misleading and leave consequential movement differences unexamined. More runs under the same mismatch would not establish that the intended physical claim has been tested. The team needs a controlled action contract and evidence for the deployed interpretation.
A proposed disposition would hold the transfer argument for the affected motion claim, resolve the adapter agreement and revise the paired comparison. The physical interruption claim would remain separate. No discrepancy or test has occurred in this example. Its useful output is the specific work required before evidence from simulation can be accepted for a declared physical use.
The robotics configuration-evidence guide supplies the result and applicability record. This matrix adds medium identity, matched physical comparison and the reason a simulation result is or is not admissible for the particular claim.
How can Arc connect simulation evidence to release review?
Arc's verification records link procedures, configurations, results and evidence to requirements. Its programme model can hold configured medium, gap, pairing and applicability records with owners and relationships.
Configuration views and review decisions can preserve the proposed release scope and evidence disposition. The team defines the evaluation, maintains the links and judges the inference. Arc does not run the simulator, measure robot transfer or certify that simulated evidence is sufficient.
Sources and method
Prepared on 2 November 2026 using Li and colleagues' SIMPLER paper and PMLR record, NIST's physical AI research description and selected AI RMF outcomes, NASA's verification and validation guidance, and Arc's canonical capability-page source. Research counts belong to the cited study. The applicability matrix and MAPLE-9 scenario are fictional editorial methods with no executed evaluations. No universal transfer threshold or simulation-only safety approval is asserted. This develops roadmap R28's simulation-to-real evidence purpose. Arc sells requirements management and systems engineering software.
Frequently asked questions
Can simulation evidence support a physical robot release decision?
It can support defined claims when the team justifies the simulator's relevance, configuration, conditions and limits with appropriate evidence. Record the physical comparison and remaining checks. A useful development result should not automatically become evidence for every installed-system claim.
Does the SIMPLER study prove transfer for every physical AI robot?
The paper reports correlated performance in its evaluated manipulation setups, policies, two embodiments and eight task families. It identifies visual and control gaps. Those findings do not establish a universal proxy or evidence for a different robot, task or safety function. (Li and colleagues, PMLR abstract)
Must the simulated and physical robots be perfect digital twins?
The required fidelity depends on the claim and consequential differences. Identify what the simulator represents, which gaps matter and how evidence addresses them. Appearance or a digital-twin label does not establish applicability, while a bounded model can still be useful for a narrower claim.
What should be matched in paired evaluations?
Match the defined task, outcome rules, relevant conditions, policy and adapters, episode/reset rules, interventions and configuration scope. Record intentional differences and their effect on the inference. Report consequential slices and unresolved gaps rather than relying only on an aggregate result.
Does a good task-performance correlation close safety requirements?
It addresses the defined performance comparison. Safety-related requirements need their applicable risk, design, integration and validation evidence. Keep those claims separate and identify which physical checks remain before the responsible authority considers release.