Arc Skills / Verification and validation
Assess whether test results demonstrate a requirement has been met
Arc Skills reviews measurements, configuration, and decision rules for each test obligation. It returns a traceable result assessment that keeps anomalies and uncertainty visible instead of averaging away failures.
Use this skill
Use Arc Skills to assess the supplied test results against approved acceptance criteria.
Inputs:
- Requirement revision, exact obligation, and approved decision rule
- Raw Run data, tested article and configuration, procedure, and calibration records
- Anomaly log and measurement uncertainty policy
Return per-observation calculations, evidence locators, uncertainty, and requirement-level verdict.
Do not average away an applicable failing observation or invent a guard bandWhat you provide and what you get
| What you have | How it is used | What you get |
|---|---|---|
| Requirement and criterion | Fixes the actual comparator and conditions | Decision boundary |
| Raw Run record | Supplies timestamps, article, and observations | Calculated elapsed times |
| Anomalies and uncertainty | Tests result validity and boundary judgment | Pass, fail, or indeterminate verdict |
One passing and one failing valve observation
Illustrative engineering example.
Two illustrative runs use the same applicable valve build. The approved limit is 2 s. Timestamps come from the same logged timebase; uncertainty policy and the second-run anomaly review are pending.
R-4: Valve shall close within 2 s after an accepted close command.
V-4/01: t0=10.0 s, t1=11.8 s.
V-4/02: t0=20.0 s, t1=22.3 s.
Anomaly A-7 opened on V-4/02; no authorized exclusion.| Run / calculation | Nominal result | Assessment |
|---|---|---|
| V-4/01: 11.8 − 10.0 | 1.8 s ≤ 2 s | Nominal pass; confirm timestamp and uncertainty rule. |
| V-4/02: 22.3 − 20.0 | 2.3 s > 2 s | Nominal fail on applicable build; A-7 investigation open. |
| R-4 overall | One result exceeds limit | Do not close as verified; retain A-7 and decide valid-run status. |
The arithmetic is direct: the two elapsed times are 1.8 s and 2.3 s. Averaging them to 2.05 s would not satisfy a criterion that applies to each close command.
The 2.3 s observation is contrary evidence unless the anomaly investigation justifies exclusion under the project’s rules. The first run may also need a boundary decision if measurement uncertainty can cross 2 s.
Keep observation and disposition separate
- Confirm the requirement, criterion, Run article, and configuration match.
- Calculate each measured quantity from preserved raw records.
- Apply only the approved comparison and uncertainty rule.
- Separate anomalous-run investigation from the current technical verdict.
Questions about this task
Can one passing run offset one failing run?
Not under a per-command ≤2 s criterion. Each applicable observation must be judged against the agreed rule.
Does an open anomaly make V-4/02 disappear?
No. Keep it visible until an authorized investigation establishes whether the run was invalid and why.
Sources and further reading
- NASA Systems Engineering Handbook: Lifecycle and requirements guidance; the example decisions are illustrative.