OBSERVE. ANALYZE. ACT.
Support Login

FlowMagic NextIO Components

FlowMagic Test Project

Define test identity, recording and completion policy, select Validation Rules, and interpret execution, evidence completeness, and acceptance results.

UI Path: NextIO Diagram Editor → Components → Validation Tools → Test Project ID: nextio-flowmagic-test-project

Test Project is the service-level owner of a validation run's identity, recording, and report preferences. It has no packet pads and does not generate or forward traffic. Its presence binds the service's supported recording-capable components to one result database for that execution.

Return to Validation Tools for the component list and shared lifecycle. See Validation Models for the Testcases, Traffic Patterns, WAN profiles, and Validation Rules used by the design.

Placement And Responsibilities

Include exactly one Test Project per service using TestPort or Test Hosts. Place it near the workload diagram for readability, without drawing packet connections to it. Other packet services can run without a project, but do not receive this project's recorded TestDB lifecycle implicitly.

Object Owns
Test Project Project identity, sampling, recording policy, completion grace, selected acceptance rules, and result publication.
Testcase / Traffic Pattern Activity schedule, traffic behavior, and action-level expectations.
Test Hosts / generator / benchmark Runtime workload behavior and its measurements.
TestPort Physical attachment and authoritative port observations.
Validation Rules Reusable acceptance intent and grading/report choices.

An HTTP action's expected response, for example, belongs to its Traffic Pattern. The overall test's acceptance criteria belong to the Validation Rules selected for the report.

Identity And Report Settings

Setting Use
Project Name Required name shown in the report; initially Test Project.
External Project ID Optional customer, test-plan, or tracking reference. It is not the generated run ID.
Description Purpose, scope, and conditions of this test.
Labels Optional classifications such as release, feature, or site.
Report Title / Report Notes Optional title override and static context included with the recorded run.
Scenario Seed Reproducible activity participation and population assignments; initially 1. It does not guarantee identical network timing across runs.

Use specific names and record the test conditions needed for later comparison. Run IDs, timestamps, device facts, and actual outcomes are generated evidence, rather than editable project literals.

Sampling, Recording, And Completion

Sampling Interval (ms) sets the recording interval for supported test components. It defaults to 1000 ms and accepts 100–60,000 ms. An interval is a measurement requirement, not permission to duplicate a cached observation as a new sample. Review coverage and actual collection timestamps in the result.

Recording Policy has two modes:

  • Required, the default, stops the test on recording failure.
  • Best Effort permits traffic execution to continue with incomplete recording. It does not turn missing evidence into an acceptance pass.

With Stop when complete enabled, Test Project waits for finite workload sources, keeps packet processing active for Completion Grace Period (s), and then finalizes recording and shuts down the service. Grace defaults to one second and accepts 0–3600 seconds. Choose a value sufficient for the intended receive drain and any WAN delay/backlog.

A WAN timeline is an intermediate processing timeline, not an independent finite source. Its last phase does not by itself finish a service. Continuous listeners or hosted services may require an explicit stop. Manual stop can interrupt the completion grace period.

Select Validation Rules

Create a rule set under Models → Validation Models → Validation Rules. Choose RFC-2544 or Stateful Traffic to match the workload. A rule selected in the service wizard or Test Project is a report preference; traffic startup records measurements without loading, binding, or evaluating the checks.

Open the finalized test.testdb, click Generate Report, and explicitly select Validation Rule for that report. The report job freezes the selected rule definition and evaluates it against the recorded evidence. Selecting None skips validation. Rule edits do not require a traffic restart and do not rewrite the source TestDB.

Follow Validation and Report for rule creation, KPI settings, missing-data and acceptance policies, report generation, and opening the resulting PDF. Review execution, recording completeness, and the report's acceptance verdict separately; a completed run alone is not a pass.

Configure A Recorded Test

  1. Build a valid finite workload, for example the paired Test Hosts design or a RFC2544 Benchmark.
  2. Add one Test Project and enter a meaningful Project Name, description, and any external reference or report context.
  3. Leave Recording Policy at Required for a baseline run. Choose a sampling interval that matches the desired observation granularity and a completion grace that covers the path's drain requirements.
  4. Save a suitable Validation Rules set and select it if acceptance is required. Review its enabled checks for the measurements this design records. The assessment is performed when generating the report.
  5. Save and verify the diagram. Start through the service workflow with Stop when complete selected for the finite test.
  6. Inspect preparation, workload progress, port measurements, and recording status. Let the workload and grace period finish.
  7. Open the recorded result and review execution state, recording completeness, and measurements. Generate a report with the selected rule to review acceptance results and scores.

Locate And Interpret Results

Each execution creates a new read-only result location beside the saved NIO design: Test_Results/<device-local timestamp>/test.testdb. Result Location is derived, rather than an arbitrary output path entered on the component. The run retains its resolved configuration and measurements. Later traffic model or project edits do not rewrite that evidence. Report-time assessment uses the rule snapshot selected for each report job.

Check the following before accepting a result:

  1. Did the execution complete, fail, or get cancelled?
  2. Is recording complete and finalized, or incomplete with a recorded reason?
  3. Were the intended required criteria enabled and evaluated with usable data?
  4. Do workload, physical-port, and independent receiver observations agree?

Troubleshooting

Symptom What to check
Verification requires a project or reports multiple projects. Include one Test Project for the service's TestPorts/Test Hosts.
No TestDB result is produced. Project presence, saved workspace design, access/storage, and the recorded startup error.
Execution stops because recording failed. Required recording policy and the recorder's failure details.
The report has no acceptance verdict. Select a Validation Rule in Generate Report and review its enabled checks.
A report check is unavailable. Recorded producer/metric support, selected entity binding, and missing-data policy.
Delayed traffic is missing from final counts. Completion grace, backlog, and whether manual stop interrupted drain.

Field Reference

Test Project field reference and initial values

Initial values describe a newly added component. Saved designs keep their configured values. Dependent fields appear when their controlling option is selected.

SettingPurposeInitial value
Scenario Seed Reproducible AppSim track participation and population assignments 1
Project Name Name shown on the test report cover and summary Test Project
External Project ID Optional project or tracking ID shown in reports Not set
Description Purpose and scope of this test Not set
Labels Optional comma- or line-separated labels frozen into each recorded run Not set
Completion Grace Period (s) Keep packet processing active after all finite workloads complete, then finalize results and stop the service 1
Report Title Optional title override for the generated report Not set
Report Notes Optional notes displayed in the generated report Not set
Sampling Interval (ms) Monitoring interval for all recording-capable test components 1000
Recording Policy Required stops the test on recording failure; best effort continues with incomplete recording required
Result Location Each execution creates Test_Results/<device-local timestamp>/test.testdb beside the saved NIO file Test_Results/<DateTime>/test.testdb
Validation Rules Optional reusable passing criteria; leave unselected to run and record without acceptance criteria Not set