Physical AI test evidence in CI becomes useful for engineering review when the run, report, requirement revision, procedure and assessed configuration remain connected. Preserve the external evidence, resolve the target record identities and prepare a proposal against an inspected revision. Then reconcile submission state and let the authorised owners decide applicability and acceptance.
A finished job is the start of an evidence handoff
A CI job finishes and leaves a report. The test team wants to attach it to a requirement. Before that handoff, the integration needs to know which requirement revision the activity assessed, which artifacts the rig used and whether the intended proposal was actually recorded. A green job badge does not answer those questions.
Use a workflow that preserves each state: job completed, test run recorded, report stored or referenced, proposal prepared, proposal reviewed and evidence accepted for a defined claim. A timeout or rejected review can occur between those states. The recovery record should make the next step clear without rewriting the earlier result.
Agree the integration contract before choosing operations
The method in this article is conceptual and system-agnostic. Implementing it needs a current contract for the chosen requirements system: supported records and operations, field and relationship meanings, revision handling, responses/errors and permissions for the intended environment. No executable endpoint, payload or current Arc capability is supplied by this method.
Begin with the meanings the integration must preserve. Then map them to supported representations. If the system has a suitable report-reference field, use it under the agreed contract. If a required representation is unavailable, keep that part unresolved with its owner. The need to connect evidence does not establish an upload or specialised result operation.
NASA's requirements-management guidance supports maintained relationships between requirements, design information and test plans or procedures. Its configuration-management guidance supports retained configuration and change status. Those engineering principles inform the handoff, rather than defining an API.
Map a candidate without inventing a run
WR-01 is a fictional supervised indoor warehouse replay bench. Its identifiers, 80 ms criterion and evidence states are teaching choices, with no customer result, stopping-distance meaning, physical operating release or safety acceptance. Actual execution, measurement and reviewer decisions are not supplied.
Accepted historical teaching CFG-A = H1/L1/F1.4/M1/CAL-07/MAP-03. CR-014 proposes candidate CFG-B = H1/L1/F1.4/M2/CAL-07/MAP-03, with only M1 → M2 declared to change.
| Manifest element | Historical teaching CFG-A | Candidate teaching CFG-B |
|---|---|---|
| Hardware | H1 | H1 |
| Lidar | L1 | L1 |
| Firmware | F1.4 | F1.4 |
| Perception model | M1 | M2 |
| Calibration | CAL-07 | CAL-07 |
| Map | MAP-03 | MAP-03 |
The proposed activity would assess SYS-PER-01 v1 using TEST-LAT-01 v1:
For DSET-07 v1 under LOAD-02, elapsed time from replayed sensor-frame arrival to detection delivery at IF-DET-01 v1 shall not exceed 80 ms.
EVID-LAT-A is an assumed fictional historical pass for CFG-A, without a measured value or execution date. EVID-LAT-B remains Pending for CFG-B, with no candidate run, report or result supplied. RUN-042 and CR-014 are teaching labels only, rather than real execution or Arc identifiers. Unchanged labels alone do not prove artifact identity, dependency independence or evidence applicability.
The eight-group evidence-proposal mapping — This table specifies information to resolve, rather than a request payload. Keep the robot's perception model M2 separate from any programme schema or record model used by the requirements system. Real accountable owners below are roles to assign.
| Mapping group | WR-01 teaching context | Required real mapping or owner action |
|---|---|---|
| External report and run | RUN-042 is a teaching label. No real producing system, execution, raw data, report or digest is supplied. | CI/evidence owner preserves producing-system/run identity, execution time, raw/report references and digests where used. Record report production and accessibility separately from review. |
| Requirement | SYS-PER-01 v1 names the teaching obligation. | Requirement owner resolves the actual item and inspected revision, preserving criterion and source identity. A title or teaching label is not a production ID. |
| Programme model, type, field and relationship IDs | Actual target schema and record IDs are unknown. The programme model is distinct from perception model M2. | Schema/record owner resolves permitted schema identity, item/type IDs, fields, relationship types and endpoint item IDs, including their meanings. |
| Procedure and configuration | TEST-LAT-01 v1 with planned CFG-B: H1/L1/F1.4/M2/CAL-07/MAP-03, DSET-07 v1, LOAD-02 and IF-DET-01 v1. | Verification/configuration owners identify the actual assessed artifacts and procedure/condition revisions, linking the programme baseline separately and reconciling the planned state. |
| Units, outcome and anomalies | Invented 80 ms criterion. EVID-LAT-B Pending. Actual clocks/resolution/events/frames, dataset/load content and outcome/anomalies unknown. | Verification owner maps actual values/units, event pairing, clock basis, result interpretation and scoped issues. Keep unknown and unreviewed states distinct from accepted coverage. |
| Supported evidence representation | No chosen evidence field, relationship, storage or upload operation is supplied. | Integration owner selects only representations supported by the reviewed contract. Hold an unsupported mapping with its evidence/schema owner. |
| Branch and revision context | No actual source, destination, working branch or head revision supplied. | Integration owner retains destination and source identity, inspected working revision, proposal/review context and the supported revision constraint. Refresh a mapping if its source obligation changes. |
| Review and authority | No proposal, reviewer/date, decision or baseline acceptance supplied. | Record the actual supported proposal/review reference and authorised verification/baseline decisions, with inspected revisions, limits and rejected reasoning. |
The mapping should preserve the question the evidence answers. Linking a report to the wrong revision changes that question. Relabelling the rig as the planned candidate changes the assessed configuration. Converting an unknown outcome into pass changes the result. Each needs an explicit resolution before the proposal can support review.
NASA's product-verification guidance supports checking data integrity, recording discrepancies and tracing results to product requirements. NIST's voluntary MEASURE guidance supports documented test sets, metrics and tools. They inform what the mapping preserves. The robotics evidence ledger carries the detailed applicability record.
Move from evidence to review in controlled steps
Treat the sequence as a design to check against the chosen system's contract. A step needs a supported operation and permitted context before implementation.
- Preserve the external run and report, with the requirement, procedure and actual configuration they claim to assess. Record the job state and the test outcome separately. EVID-LAT-B stays Pending in WR-01 because no candidate execution or result exists in the supplied example.
- Resolve the real programme schema, record/type/field/relationship identities and authoritative source revisions through permitted reads. Keep that context beside the mapping.
- Identify the review destination and supported working route, such as a branch or proposal. Record the revision inspected and prepare the evidence references and relationships the contract supports.
- Inspect the proposed changes through the supported preview or review mechanism. Check requirement revision, configuration, units, report identity and any returned effects actually defined by the contract.
- Confirm that source context and working revision still match the inspected proposal. Use the supported revision-control mechanism for a controlled application. Changed context requires a new mapping and inspection.
- Reconcile target state through supported readback. Record confirmed application, confirmed non-application or an unresolved outcome, including report-reference accessibility.
- Present the reconciled proposal for engineering review. The verification owner assesses applicability and anomalies. Baseline acceptance and implementation confirmation follow their own authority and records.
A client sending an operation establishes an attempt. Supported readback establishes what can be confirmed about the target state. The reviewed proposal then establishes the precise change the owners considered. Keeping those records separate makes a failure recoverable and prevents a completed transfer from acquiring an implied acceptance decision.
The failure and reconciliation table — Use the table to define recovery with the integration owner. Preserve non-secret context, including what is known about the action state. Each dependent step waits for the affected uncertainty to be resolved.
| Failure or uncertainty | Preserve | Required disposition before continuing |
|---|---|---|
| Wrong or unresolved record, model/type, field or relationship ID | Run/report identity, inspected source revision, mapping and the conflicting identity. | Schema/record owner resolves IDs and meanings. Re-read and inspect the corrected mapping. |
| Missing, unreachable or incomplete report/provenance | Producing-system/run reference, available raw/report identity, accessibility finding and missing fields. | CI/evidence owner repairs production or access through the authorised route. Verification owner then assesses sufficiency. |
| Invalid units, events, clocks or configuration | Original values/units, definitions, criterion, planned/actual manifests and exact conflict. | Verification/configuration owners resolve the affected claim. Preserve the original observation and inspect any corrected mapping. |
| Authentication or permission denial | Operation context, non-secret response/error, inspected revisions and confirmed action state. | Stop dependent actions and use the authorised owner/access route. Recheck permitted context when resolved. |
| Stale working revision or changed source requirement | Source/working revisions, inspected proposal context and conflicting revision evidence. | Read authoritative context, determine mapping validity and regenerate/inspect the proposal before another controlled attempt. |
| Rejected preview or unsupported representation/operation | Intended change, inspected contract version and rejection or unsupported part. | Contract/schema owner supplies a supported route. Keep the affected mapping held while unresolved. |
| Ambiguous application, timeout or unknown response | Run/proposal identity, inspected revisions, sent operation context and known/unknown outcome. | Reconcile through supported readback before retry. Hold with the integration owner if applied versus not applied cannot be established. |
| Duplicate run or evidence submission | Producing-system/run identity, report identity/digest where used and prior proposal/readback. | Reconcile prior state and follow the owner-approved submission-identity policy. Automatic deduplication or harmless repetition is not assumed. |
| Review rejects or limits applicability | Source evidence, proposed relationship, inspected revision, reviewer reasoning and covered/excluded claims. | Preserve result and rejection/limit separately. Revise the proposal or obtain supplementary/repeated evidence through the controlled route. |
A timeout leaves a question to reconcile — Suppose an implementation attempts to apply an evidence proposal and the connection times out. The client knows it sent the request. Its response record may not establish whether the target changed. Preserve the attempted proposal and inspected revisions, then use the contract's supported readback to determine the state.
Retry policy follows that evidence. If the outcome is still unknown, keep the action held for the integration owner. A repeated operation could create a duplicate or act against changed context, depending on the actual contract. Idempotency, deduplication and retry semantics need explicit contract and client-policy decisions.
The same approach helps with stale source revisions. Even if the destination is unchanged, a requirement revised after mapping may need a different evidence relationship or comparison. Refresh both the engineering context and the proposal before continuing.
Give each part of the handoff an owner
The CI maintainer owns report production and execution context. Integration and schema owners own identities, representations, supported operations and reconciliation. The verification owner judges evidence meaning, anomalies and applicability. The authorised baseline owner decides what enters the accepted programme state.
Agree access and retention requirements with their actual owners. The information-authority guide explains the wider exchange contract. This article's artifact focuses on the external-run-to-proposal mapping and recovery states, while the model-change dossier covers the wider dependency review.
One question about accepted coverage
Can CI establish accepted requirement coverage automatically?
A valid automated activity can produce a measured result for a defined criterion and context. The evidence mapping proposes its use in a requirements record. Applicability, anomaly disposition and authorised baseline acceptance remain distinct decisions under the programme's process. A green job state should preserve those distinctions. EVID-LAT-B has no supplied candidate run or result, so it remains Pending.
Review one mapping and its recovery plan
Take one external report and complete the eight groups with the CI, integration, schema and verification owners. Use the current contract to resolve its representations, then review the timeout and changed-revision cases before implementing the client. That gives a workflow evaluation, including an Arc evaluation, a concrete handoff to assess in an authorised environment.