Robot latency verification starts with two observable events that match the system requirement, then measures their elapsed time under the stated configuration and load. Allocate the intervening timing path to accountable owners, establish clocks and measurement validity, and plan the coverage and decision rule before comparing a result with the limit. Component benchmarks contribute evidence only for the segments they actually assess.
For a verification or control lead, this is the decision to settle when a sensor, runtime or controller changes before acceptance. A fast model benchmark and a responsive actuator test can both be correct while leaving sampling delay, queueing or command delivery outside the claim.
Define the physical interval before choosing instruments
State the triggering condition, start event, physical response threshold and observation locations. “Latency” might describe time from exposure to inference, input callback to command issue or physical cue to actuator movement. Those intervals have different meanings. A test must observe the one the requirement actually controls.
VIM’s measurand definition identifies the quantity intended to be measured and explains the importance of its state and conditions. Applied to timing, define the two events, their association and the conditions under which their difference represents the intended response.
NASA’s product-verification guidance connects verification to a specified requirements baseline, plan, product and enabling products. It calls for analysing data quality, integrity, correctness, consistency and validity. A completed timing script is therefore one activity state. It does not settle whether its observations verify the system interval.
A fictional sorting robot with a physical response requirement
SORT-23 is an invented supervised bench robot that moves a sorting paddle when an eligible marked part crosses a sensor datum. Its configuration SORT-C1 declares sensor SENS-23 v1, estimator EST-23 v1, runtime RT-23 v1, controller CT-23 v1, drive DRV-23 and paddle assembly PAD-23. SET-23 v1 defines the eligible part, datum, paddle state, supply, temperature and load profile. All values and plans below are fictional and unexecuted.
SYS-LAT-230 v1: Under each specified SET-23 v1 assessment condition, elapsed time from an eligible part crossing datum D-23 to the paddle reference point first reaching 2.0 mm displacement in the commanded direction from its defined initial position shall not exceed 120 ms.
An independent datum sensor is the proposed start observation. A displacement instrument is the proposed end observation. Their signals would be captured by an agreed timing instrument. Software timestamps along the path help explain where delay occurs, but they do not replace the physical start and end events.
The 2.0 mm threshold describes first movement to a defined displacement. It does not establish final paddle position, successful sorting, motion-path safety or a stopping distance. The invented 120 ms limit has no customer rationale. A real programme must justify the requirement and any wider operating claims separately.
Allocate the complete timing path
The prospective budget identifies contiguous events t0 through t6 for one associated stimulus. The planned segment limits add to 100 ms, leaving 20 ms reserve against SYS-LAT-230. None is an observed component or system result.
| Segment and boundary | Proposed allocation | Owner and verification question |
|---|---|---|
| t0 → t1: datum crossing to sensor acquisition | 16 ms, SEN-LAT-230 | Sensor owner checks sampling phase, exposure/acquisition meaning and eligibility detection. A processing-start benchmark excludes this wait. |
| t1 → t2: acquisition to usable estimator input | 8 ms, IO-LAT-230 | Interface owner includes transfer, preprocessing and queue entry at the defined usable-input boundary. |
| t2 → t3: usable input to associated decision | 24 ms, EST-LAT-230 | Perception/runtime owner includes scheduling and decision generation under the agreed load. |
| t3 → t4: decision to command transmission | 12 ms, CT-LAT-230 | Control owner defines consumption, control-cycle wait and the command observation point. |
| t4 → t5: transmission to accepted drive command | 10 ms, DRV-LAT-230 | Drive/interface owners define actual acceptance. A transport acknowledgement is not assumed to mean physical command acceptance. |
| t5 → t6: accepted command to 2.0 mm displacement | 30 ms, ACT-LAT-230 | Actuator owner includes drive initiation and mechanical movement at the specified load and initial state. |
| Parent timing and reserve | 16 + 8 + 24 + 12 + 10 + 30 = 100 ms. Reserve 20 ms. | Systems and verification owners retain t0 → t6 closure. Unobserved segments or interactions remain open. |
The arithmetic is useful only when boundaries line up. A segment ending at queue entry cannot be followed by a segment starting at inference execution unless the intervening wait is included. If work overlaps, summing separate benchmarks may double count elapsed time. Define the actual event chain and concurrency before choosing the allocation model.
NASA’s requirements-management guidance supports traceable allocation and checking that the lower-level set covers its parent. A real timing budget should therefore show each child criterion, conditions and contribution, with the end-to-end requirement retained.
Write the clock and measurement-validity contract
MEAS-23 v1 should define how the two physical observations become an elapsed-time value. Record instrument identity, firmware, acquisition settings, timebase calibration applicability, trigger detection and threshold algorithm. Preserve raw signals so a reviewer can check which crossing produced the start and which displacement crossing ended the same response.
A common monotonic timing reference is a proposed option for the physical capture. If separate clocks are used, record their offset, rate relationship, corrections and validity through the assessment. A one-off synchronisation label does not bound drift or timestamp error. Capture resolution also differs from timebase accuracy and event-detection uncertainty.
The ROS 2 Clock and Time design document distinguishes SystemTime, SteadyTime and ROSTime and describes simulated time that can pause or jump. Log-clock timestamps must be related to the physical measurement basis before they support the same interval. The design description supplies no measured synchronisation tolerance for SORT-23.
| Validity item | Record before execution | Decision if unresolved |
|---|---|---|
| Paired event identity | Stimulus ID, part eligibility, initial paddle state and association through software and physical capture. | Do not use an unpaired observation to close SYS-LAT-230. |
| Observation definition | Datum crossing, displacement direction/threshold, signal processing, interpolation and missing-crossing rule. | Reconcile the observed events with the requirement or revise the procedure through review. |
| Clock and units | Capture tick basis, scale to seconds, reported milliseconds, corrections and any cross-clock relationship. | An unknown clock relationship leaves the elapsed-time claim unresolved. |
| Instrument applicability | Serial, configuration, calibration reference, range, timebase and environmental conditions. | Investigate affected observations and evidence scope before accepting their values. |
| Measurement uncertainty and rule | Event detection, threshold/reference uncertainty, timebase behaviour, quantisation, corrections and comparison rule. | A nominal value below the limit is insufficient if the agreed uncertainty treatment is missing. |
| Load and capture effect | Actual processor/network/control load and evidence that instrumentation does not materially alter the assessed timing. | Identify the supported condition and any missing representative assessment. |
Keep the measurement decision separate from budget reserve
VIM defines measurement uncertainty as a parameter characterising the dispersion of values attributed to the measurand. It is not the same as a subsystem timing limit or unspent performance reserve. The verification owner needs a measurement model and an agreed comparison rule.
For teaching arithmetic only, MEAS-23 proposes absolute error allowances of 0.3 ms for start-event detection, 0.4 ms for end-event detection, 0.8 ms for timebase contribution and 0.5 ms for threshold processing. Their conservative sum is 2.0 ms. These are invented bounded-error allowances, not measured uncertainties, standard deviations or a statistical confidence interval. Their bounds and completeness would need evidence.
The example decision rule is measured elapsed time plus the justified 2.0 ms allowance ≤120 ms for every valid assessed event, with all other coverage and validity conditions met. If a hypothetical observation were 116.5 ms, the comparison would be 118.5 ms. A hypothetical 119.0 ms would compare at 121.0 ms and would not close the requirement under that rule. Neither number is a result from SORT-23.
A programme may choose another justified rule. Preserve its authority and rationale before testing. If the measurement uncertainty cannot be made small enough for the acceptance need, improve the method or seek an authorised decision. Do not spend the 20 ms subsystem reserve to disguise an unknown measurement contribution.
Plan coverage and keep failures distinct from invalid measurements
PLAN-LAT-23 should include sampling-phase variation, defined processor/network load, control-cycle phase, eligible stimulus variation and the specified actuator load/initial state. Start with the operating envelope that changes the timing argument. Record each condition, its rationale and planned activity instead of pooling incompatible observations into one headline value.
Report valid measured responses, late responses, genuine missing responses and invalid instrument records separately. If the robot never reaches the displacement threshold, the response outcome needs assessment. If the instrument fails to retain an end channel, that observation may be invalid. Neither can disappear from the record as a discarded outlier.
A sample maximum supports the conditions and events actually assessed. It does not prove an absolute worst-case deadline across all possible executions. If a deterministic bound is required, the programme may need additional justified analysis and timing evidence. A percentile target must be an authorised requirement rather than a convenient substitute for the stated maximum.
Changing SENS-23 to a different sampling mode or RT-23 to a new runtime raises specific questions about t0 → t3, load and event association. Use the existing change-impact dossier for the broader dependency sweep and the configuration-evidence ledger for each reuse decision. The result here is the timing-specific verification plan and validity argument.
Evaluate Arc with a timing requirement and incomplete clock evidence
Arc’s verification and test capability connects plans, executions, results and evidence to requirements and configurations. Its traceability and configuration capability provide linked context for reviewing the timing claim.
In a demo, represent SYS-LAT-230, its segment allocations, MEAS-23 and a planned sensor-mode change. Link a clearly fictional report that meets the nominal limit but lacks clock applicability evidence. Ask the verification reviewer to keep measurement closure unresolved, identify the owner and preserve the comparison rule and source configuration. Inspect whether that distinction remains clear in the review record.
The team’s instruments and timing tools produce the observations. Arc’s public capability reference does not establish physical capture, a native timing analyser or automatic acceptance. Agree the representation and permitted integration before treating a demonstration as evidence of your workflow.
Questions about timing evidence
Can a model inference benchmark verify robot response latency?
It can support the inference allocation when its events, configuration and load match that segment. A sensor-event-to-actuator requirement also includes sampling, transport, queues, control, drive behaviour and physical movement. The benchmark alone does not verify that complete interval.
Must all timestamps use the same clock?
A common measurement clock can simplify the argument. Different clocks can be used when their relationship, corrections, uncertainty and validity over the assessed interval are established. Subtracting timestamps from clocks with an unknown relationship does not provide a valid elapsed-time claim.
Can a 99th-percentile result close a maximum-latency requirement?
No. A percentile and a maximum are different claims. Use the authorised requirement’s comparison rule and keep the number of valid events, late responses, missing responses and invalid measurements visible. Revise the obligation through the agreed authority if a statistical criterion is intended.
Should missing actuator responses be discarded from the timing sample?
A genuine missing response is a behaviour outcome to assess against the requirement, not a convenient exclusion. A missing instrument record may instead make that event’s measurement invalid. Preserve the distinction, event identity and recovery action before interpreting the remaining data.
Does passing replay establish physical actuator timing?
A replay can assess the behaviour and timing boundaries represented by its setup. It does not establish physical sensor sampling, drive or actuator response unless those elements and observation points are actually included with applicable evidence. State the supported claim and remaining integrated checks.