Test Project is the service-level owner of a validation run's identity, recording, and report preferences. It has no packet pads and does not generate or forward traffic. Its presence binds the service's supported recording-capable components to one result database for that execution.
Return to Validation Tools for the component list and shared lifecycle. See Validation Models for the Testcases, Traffic Patterns, WAN profiles, and Validation Rules used by the design.
Placement And Responsibilities
Include exactly one Test Project per service using TestPort or Test Hosts. Place it near the workload diagram for readability, without drawing packet connections to it. Other packet services can run without a project, but do not receive this project's recorded TestDB lifecycle implicitly.
| Object | Owns |
|---|---|
| Test Project | Project identity, sampling, recording policy, completion grace, selected acceptance rules, and result publication. |
| Testcase / Traffic Pattern | Activity schedule, traffic behavior, and action-level expectations. |
| Test Hosts / generator / benchmark | Runtime workload behavior and its measurements. |
| TestPort | Physical attachment and authoritative port observations. |
| Validation Rules | Reusable acceptance intent and grading/report choices. |
An HTTP action's expected response, for example, belongs to its Traffic Pattern. The overall test's acceptance criteria belong to the Validation Rules selected for the report.
Identity And Report Settings
| Setting | Use |
|---|---|
| Project Name | Required name shown in the report; initially Test Project. |
| External Project ID | Optional customer, test-plan, or tracking reference. It is not the generated run ID. |
| Description | Purpose, scope, and conditions of this test. |
| Labels | Optional classifications such as release, feature, or site. |
| Report Title / Report Notes | Optional title override and static context included with the recorded run. |
| Scenario Seed | Reproducible activity participation and population assignments; initially 1. It does not guarantee identical network timing across runs. |
Use specific names and record the test conditions needed for later comparison. Run IDs, timestamps, device facts, and actual outcomes are generated evidence, rather than editable project literals.
Sampling, Recording, And Completion
Sampling Interval (ms) sets the recording interval for supported test components. It defaults to 1000 ms and accepts 100–60,000 ms. An interval is a measurement requirement, not permission to duplicate a cached observation as a new sample. Review coverage and actual collection timestamps in the result.
Recording Policy has two modes:
- Required, the default, stops the test on recording failure.
- Best Effort permits traffic execution to continue with incomplete recording. It does not turn missing evidence into an acceptance pass.
With Stop when complete enabled, Test Project waits for finite workload sources, keeps packet processing active for Completion Grace Period (s), and then finalizes recording and shuts down the service. Grace defaults to one second and accepts 0–3600 seconds. Choose a value sufficient for the intended receive drain and any WAN delay/backlog.
A WAN timeline is an intermediate processing timeline, not an independent finite source. Its last phase does not by itself finish a service. Continuous listeners or hosted services may require an explicit stop. Manual stop can interrupt the completion grace period.
Select Validation Rules
Create a rule set under Models → Validation Models → Validation Rules. Choose RFC-2544 or Stateful Traffic to match the workload. A rule selected in the service wizard or Test Project is a report preference; traffic startup records measurements without loading, binding, or evaluating the checks.
Open the finalized test.testdb, click Generate Report, and explicitly select Validation Rule for that report. The report job freezes the selected rule definition and evaluates it against the recorded evidence. Selecting None skips validation. Rule edits do not require a traffic restart and do not rewrite the source TestDB.
Follow Validation and Report for rule creation, KPI settings, missing-data and acceptance policies, report generation, and opening the resulting PDF. Review execution, recording completeness, and the report's acceptance verdict separately; a completed run alone is not a pass.
Configure A Recorded Test
- Build a valid finite workload, for example the paired Test Hosts design or a RFC2544 Benchmark.
- Add one Test Project and enter a meaningful Project Name, description, and any external reference or report context.
- Leave Recording Policy at Required for a baseline run. Choose a sampling interval that matches the desired observation granularity and a completion grace that covers the path's drain requirements.
- Save a suitable Validation Rules set and select it if acceptance is required. Review its enabled checks for the measurements this design records. The assessment is performed when generating the report.
- Save and verify the diagram. Start through the service workflow with Stop when complete selected for the finite test.
- Inspect preparation, workload progress, port measurements, and recording status. Let the workload and grace period finish.
- Open the recorded result and review execution state, recording completeness, and measurements. Generate a report with the selected rule to review acceptance results and scores.
Locate And Interpret Results
Each execution creates a new read-only result location beside the saved NIO
design: Test_Results/<device-local timestamp>/test.testdb. Result Location
is derived, rather than an arbitrary output path entered on the component.
The run retains its resolved configuration and measurements. Later traffic
model or project edits do not rewrite that evidence. Report-time assessment
uses the rule snapshot selected for each report job.
Check the following before accepting a result:
- Did the execution complete, fail, or get cancelled?
- Is recording complete and finalized, or incomplete with a recorded reason?
- Were the intended required criteria enabled and evaluated with usable data?
- Do workload, physical-port, and independent receiver observations agree?
Troubleshooting
| Symptom | What to check |
|---|---|
| Verification requires a project or reports multiple projects. | Include one Test Project for the service's TestPorts/Test Hosts. |
| No TestDB result is produced. | Project presence, saved workspace design, access/storage, and the recorded startup error. |
| Execution stops because recording failed. | Required recording policy and the recorder's failure details. |
| The report has no acceptance verdict. | Select a Validation Rule in Generate Report and review its enabled checks. |
| A report check is unavailable. | Recorded producer/metric support, selected entity binding, and missing-data policy. |
| Delayed traffic is missing from final counts. | Completion grace, backlog, and whether manual stop interrupted drain. |
Field Reference
Test Project field reference and initial values
Initial values describe a newly added component. Saved designs keep their configured values. Dependent fields appear when their controlling option is selected.
| Setting | Purpose | Initial value |
|---|---|---|
| Scenario Seed | Reproducible AppSim track participation and population assignments | 1 |
| Project Name | Name shown on the test report cover and summary | Test Project |
| External Project ID | Optional project or tracking ID shown in reports | Not set |
| Description | Purpose and scope of this test | Not set |
| Labels | Optional comma- or line-separated labels frozen into each recorded run | Not set |
| Completion Grace Period (s) | Keep packet processing active after all finite workloads complete, then finalize results and stop the service | 1 |
| Report Title | Optional title override for the generated report | Not set |
| Report Notes | Optional notes displayed in the generated report | Not set |
| Sampling Interval (ms) | Monitoring interval for all recording-capable test components | 1000 |
| Recording Policy | Required stops the test on recording failure; best effort continues with incomplete recording | required |
| Result Location | Each execution creates Test_Results/<device-local timestamp>/test.testdb beside the saved NIO file | Test_Results/<DateTime>/test.testdb |
| Validation Rules | Optional reusable passing criteria; leave unselected to run and record without acceptance criteria | Not set |