Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[uss_qualifier] extract some fragments for scenarios injecting flights #892

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# Flight injection test step fragment

This test step fragment documents the injection of test flights.

In this step, uss_qualifier injects a single nominal flight into each SP under test, usually with a start time in the future. Each SP is expected to queue the provided telemetry and later simulate that telemetry coming from an aircraft at the designated timestamps.

## 🛑 Successful injection check

Per **[interuss.automated_testing.rid.injection.UpsertTestSuccess](../../../../../requirements/interuss/automated_testing/rid/injection.md)**, the injection attempt of the valid flight should succeed for every NetRID Service Provider under test.

**[astm.f3411.v19.NET0500](../../../../../requirements/astm/f3411/v19.md)** requires a Service Provider to provide a persistently supported test instance of their implementation.
This check will fail if the flight was not successfully injected.

## 🛑 Identifiable flights check

This particular test requires each flight to be uniquely identifiable by its 2D telemetry position; the same (lat, lng) pair may not appear in two different telemetry points, even if the two points are in different injected flights. This should generally be achieved by injecting appropriate data.

## Valid flight check

TODO: Validate injected flights, especially to make sure they contain the specified injection IDs

Per **[interuss.automated_testing.rid.injection.UpsertTestResult](../../../../../requirements/interuss/automated_testing/rid/injection.md)**, the NetRID Service Provider under test should only make valid modifications to the injected flights. This includes:
* A flight with the specified injection ID must be returned.
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# Service provider polling test step fragment

uss_qualifier acts as a Display Provider to query Service Providers under test in this step.

## ⚠️ ISA query check

**[interuss.f3411.dss_endpoints.SearchISAs](../../../../../requirements/interuss/f3411/dss_endpoints.md)** requires a USS providing a DSS instance to implement the DSS endpoints of the OpenAPI specification. If uss_qualifier is unable to query the DSS for ISAs, this check will fail.

## ⚠️ Successful flight details query check

**[astm.f3411.v19.NET0710,2](../../../../../requirements/astm/f3411/v19.md)** and **[astm.f3411.v19.NET0340](../../../../../requirements/astm/f3411/v19.md) require a Service Provider to implement the GET flight details endpoint. This check will fail if uss_qualifier cannot query that endpoint (specified in the ISA present in the DSS) successfully.

## [Flight presence checks](../display_data_evaluator_flight_presence.md)
Original file line number Diff line number Diff line change
Expand Up @@ -28,42 +28,14 @@ If specified, uss_qualifier will act as a Display Provider and check a DSS insta

## Nominal flight test case

### Injection test step
### [Injection test step](./fragments/flight_injection.md)

In this step, uss_qualifier injects a single nominal flight into each SP under test, usually with a start time in the future. Each SP is expected to queue the provided telemetry and later simulate that telemetry coming from an aircraft at the designated timestamps.

#### 🛑 Successful injection check

Per **[interuss.automated_testing.rid.injection.UpsertTestSuccess](../../../../requirements/interuss/automated_testing/rid/injection.md)**, the injection attempt of the valid flight should succeed for every NetRID Service Provider under test.

**[astm.f3411.v19.NET0500](../../../../requirements/astm/f3411/v19.md)** requires a Service Provider to provide a persistently supported test instance of their implementation.
This check will fail if the flight was not successfully injected.

#### Valid flight check

TODO: Validate injected flights, especially to make sure they contain the specified injection IDs

Per **[interuss.automated_testing.rid.injection.UpsertTestResult](../../../../requirements/interuss/automated_testing/rid/injection.md)**, the NetRID Service Provider under test should only make valid modifications to the injected flights. This includes:
* A flight with the specified injection ID must be returned.

#### 🛑 Identifiable flights check

This particular test requires each flight to be uniquely identifiable by its 2D telemetry position; the same (lat, lng) pair may not appear in two different telemetry points, even if the two points are in different injected flights. This should generally be achieved by injecting appropriate data.

### Service Provider polling test step
### [Service Provider polling test step](./fragments/sp_polling.md)

If a DSS was provided to this test scenario, uss_qualifier acts as a Display Provider to query Service Providers under test in this step.

#### ⚠️ ISA query check

**[interuss.f3411.dss_endpoints.SearchISAs](../../../../requirements/interuss/f3411/dss_endpoints.md)** requires a USS providing a DSS instance to implement the DSS endpoints of the OpenAPI specification. If uss_qualifier is unable to query the DSS for ISAs, this check will fail.

#### ⚠️ Area too large check

**[astm.f3411.v19.NET0250](../../../../requirements/astm/f3411/v19.md)** requires that a NetRID Service Provider rejects a request for a very large view area with a diagonal greater than *NetMaxDisplayAreaDiagonal*. If such a large view is requested and a 413 error code is not received, then this check will fail.

#### [Flight presence checks](./display_data_evaluator_flight_presence.md)

#### ⚠️ Flights data format check

**[astm.f3411.v19.NET0710,1](../../../../requirements/astm/f3411/v19.md)** and **[astm.f3411.v19.NET0340](../../../../requirements/astm/f3411/v19.md)** require a Service Provider to implement the P2P portion of the OpenAPI specification. This check will fail if the response to the /flights endpoint does not validate against the OpenAPI-specified schema.
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# Flight injection test step fragment

This test step fragment documents the injection of test flights.

In this step, uss_qualifier injects a single nominal flight into each SP under test, usually with a start time in the future. Each SP is expected to queue the provided telemetry and later simulate that telemetry coming from an aircraft at the designated timestamps.

## 🛑 Successful injection check

Per **[interuss.automated_testing.rid.injection.UpsertTestSuccess](../../../../../requirements/interuss/automated_testing/rid/injection.md)**, the injection attempt of the valid flight should succeed for every NetRID Service Provider under test.

**[astm.f3411.v22a.NET0500](../../../../../requirements/astm/f3411/v22a.md)** requires a Service Provider to provide a persistently supported test instance of their implementation.
This check will fail if the flight was not successfully injected.

## 🛑 Identifiable flights check

This particular test requires each flight to be uniquely identifiable by its 2D telemetry position; the same (lat, lng) pair may not appear in two different telemetry points, even if the two points are in different injected flights. This should generally be achieved by injecting appropriate data.


## Valid flight check

TODO: Validate injected flights, especially to make sure they contain the specified injection IDs

Per **[interuss.automated_testing.rid.injection.UpsertTestResult](../../../../../requirements/interuss/automated_testing/rid/injection.md)**, the NetRID Service Provider under test should only make valid modifications to the injected flights. This includes:
* A flight with the specified injection ID must be returned.
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# Service provider polling test step fragment

uss_qualifier acts as a Display Provider to query Service Providers under test in this step.

## ⚠️ ISA query check

**[interuss.f3411.dss_endpoints.SearchISAs](../../../../../requirements/interuss/f3411/dss_endpoints.md)** requires a USS providing a DSS instance to implement the DSS endpoints of the OpenAPI specification. If uss_qualifier is unable to query the DSS for ISAs, this check will fail.

## ⚠️ Successful flight details query check

**[astm.f3411.v22a.NET0710,2](../../../../../requirements/astm/f3411/v22a.md)** and **[astm.f3411.v22a.NET0340](../../../../../requirements/astm/f3411/v22a.md) require a Service Provider to implement the GET flight details endpoint. This check will fail if uss_qualifier cannot query that endpoint (specified in the ISA present in the DSS) successfully.

## [Flight presence checks](../display_data_evaluator_flight_presence.md)
Original file line number Diff line number Diff line change
Expand Up @@ -24,20 +24,7 @@ A [`DSSInstanceResource`](../../../../resources/astm/f3411/dss.py) is required f

## Unauthenticated requests test case

### Injection test step

In this step, uss_qualifier injects a single nominal flight into each SP under test, usually with a start time in the future. Each SP is expected to queue the provided telemetry and later simulate that telemetry coming from an aircraft at the designated timestamps.

#### 🛑 Successful injection check

Per **[interuss.automated_testing.rid.injection.UpsertTestSuccess](../../../../requirements/interuss/automated_testing/rid/injection.md)**, the injection attempt of the valid flight should succeed for every NetRID Service Provider under test.

**[astm.f3411.v22a.NET0500](../../../../requirements/astm/f3411/v22a.md)** requires a Service Provider to provide a persistently supported test instance of their implementation.
This check will fail if the flight was not successfully injected.

#### 🛑 Identifiable flights check

This particular test requires each flight to be uniquely identifiable by its 2D telemetry position; the same (lat, lng) pair may not appear in two different telemetry points, even if the two points are in different injected flights. This should generally be achieved by injecting appropriate data.
### [Injection test step](./fragments/flight_injection.md)

### Unauthenticated requests test step

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -28,42 +28,16 @@ If specified, uss_qualifier will act as a Display Provider and check a DSS insta

## Nominal flight test case

### Injection test step
### [Injection test step](./fragments/flight_injection.md)

In this step, uss_qualifier injects a single nominal flight into each SP under test, usually with a start time in the future. Each SP is expected to queue the provided telemetry and later simulate that telemetry coming from an aircraft at the designated timestamps.

#### 🛑 Successful injection check

Per **[interuss.automated_testing.rid.injection.UpsertTestSuccess](../../../../requirements/interuss/automated_testing/rid/injection.md)**, the injection attempt of the valid flight should succeed for every NetRID Service Provider under test.

**[astm.f3411.v22a.NET0500](../../../../requirements/astm/f3411/v22a.md)** requires a Service Provider to provide a persistently supported test instance of their implementation.
This check will fail if the flight was not successfully injected.

#### Valid flight check

TODO: Validate injected flights, especially to make sure they contain the specified injection IDs

Per **[interuss.automated_testing.rid.injection.UpsertTestResult](../../../../requirements/interuss/automated_testing/rid/injection.md)**, the NetRID Service Provider under test should only make valid modifications to the injected flights. This includes:
* A flight with the specified injection ID must be returned.

#### 🛑 Identifiable flights check

This particular test requires each flight to be uniquely identifiable by its 2D telemetry position; the same (lat, lng) pair may not appear in two different telemetry points, even if the two points are in different injected flights. This should generally be achieved by injecting appropriate data.

### Service Provider polling test step
### [Service Provider polling test step](./fragments/sp_polling.md)

If a DSS was provided to this test scenario, uss_qualifier acts as a Display Provider to query Service Providers under test in this step.

#### ⚠️ ISA query check

**[interuss.f3411.dss_endpoints.SearchISAs](../../../../requirements/interuss/f3411/dss_endpoints.md)** requires a USS providing a DSS instance to implement the DSS endpoints of the OpenAPI specification. If uss_qualifier is unable to query the DSS for ISAs, this check will fail.

#### ⚠️ Area too large check

**[astm.f3411.v22a.NET0250](../../../../requirements/astm/f3411/v22a.md)** requires that a NetRID Service Provider rejects a request for a very large view area with a diagonal greater than *NetMaxDisplayAreaDiagonal*. If such a large view is requested and a 413 error code is not received, then this check will fail.

#### [Flight presence checks](./display_data_evaluator_flight_presence.md)

#### ⚠️ Flights data format check

**[astm.f3411.v22a.NET0710,1](../../../../requirements/astm/f3411/v22a.md)** and **[astm.f3411.v22a.NET0340](../../../../requirements/astm/f3411/v22a.md)** requires a Service Provider to implement the P2P portion of the OpenAPI specification. This check will fail if the response to the /flights endpoint does not validate against the OpenAPI-specified schema.
Expand All @@ -82,10 +56,6 @@ This implies that any recent position outside the area must be either preceded o

(This check validates NET0270 b and c).

#### ⚠️ Successful flight details query check

**[astm.f3411.v22a.NET0710,2](../../../../requirements/astm/f3411/v22a.md)** and **[astm.f3411.v22a.NET0340](../../../../requirements/astm/f3411/v22a.md) require a Service Provider to implement the GET flight details endpoint. This check will fail if uss_qualifier cannot query that endpoint (specified in the ISA present in the DSS) successfully.

#### ⚠️ Flight details data format check

**[astm.f3411.v22a.NET0710,2](../../../../requirements/astm/f3411/v22a.md)** and **[astm.f3411.v22a.NET0340](../../../../requirements/astm/f3411/v22a.md) require a Service Provider to implement the P2P portion of the OpenAPI specification. This check will fail if the response to the flight details endpoint does not validate against the OpenAPI-specified schema.
Expand Down
7 changes: 1 addition & 6 deletions monitoring/uss_qualifier/suites/astm/netrid/f3411_19.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
<th><a href="../../README.md#checked-in">Checked in</a></th>
</tr>
<tr>
<td rowspan="77" style="vertical-align:top;"><a href="../../../requirements/astm/f3411/v19.md">astm<br>.f3411<br>.v19</a></td>
<td rowspan="76" style="vertical-align:top;"><a href="../../../requirements/astm/f3411/v19.md">astm<br>.f3411<br>.v19</a></td>
<td><a href="../../../requirements/astm/f3411/v19.md">DSS0010</a></td>
<td>Implemented</td>
<td><a href="../../../scenarios/astm/netrid/v19/dss/token_validation.md">ASTM NetRID DSS: Token Validation</a></td>
Expand Down Expand Up @@ -246,11 +246,6 @@
<td>TODO</td>
<td><a href="../../../scenarios/astm/netrid/v19/aggregate_checks.md">ASTM F3411-19 NetRID aggregate checks</a></td>
</tr>
<tr>
<td><a href="../../../requirements/astm/f3411/v19.md">NET0250</a></td>
<td>Implemented</td>
<td><a href="../../../scenarios/astm/netrid/v19/nominal_behavior.md">ASTM NetRID nominal behavior</a></td>
</tr>
<tr>
<td><a href="../../../requirements/astm/f3411/v19.md">NET0260,NearRealTime</a></td>
<td>Implemented</td>
Expand Down
2 changes: 1 addition & 1 deletion monitoring/uss_qualifier/suites/astm/netrid/f3411_22a.md
Original file line number Diff line number Diff line change
Expand Up @@ -550,7 +550,7 @@
<tr>
<td><a href="../../../requirements/interuss/automated_testing/rid/injection.md">UpsertTestResult</a></td>
<td>TODO</td>
<td><a href="../../../scenarios/astm/netrid/v22a/nominal_behavior.md">ASTM NetRID nominal behavior</a></td>
<td><a href="../../../scenarios/astm/netrid/v22a/misbehavior.md">ASTM NetRID SP clients misbehavior handling</a><br><a href="../../../scenarios/astm/netrid/v22a/nominal_behavior.md">ASTM NetRID nominal behavior</a></td>
</tr>
<tr>
<td><a href="../../../requirements/interuss/automated_testing/rid/injection.md">UpsertTestSuccess</a></td>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -546,7 +546,7 @@
<tr>
<td><a href="../../requirements/interuss/automated_testing/rid/injection.md">UpsertTestResult</a></td>
<td>TODO</td>
<td><a href="../../scenarios/astm/netrid/v22a/nominal_behavior.md">ASTM NetRID nominal behavior</a></td>
<td><a href="../../scenarios/astm/netrid/v22a/misbehavior.md">ASTM NetRID SP clients misbehavior handling</a><br><a href="../../scenarios/astm/netrid/v22a/nominal_behavior.md">ASTM NetRID nominal behavior</a></td>
</tr>
<tr>
<td><a href="../../requirements/interuss/automated_testing/rid/injection.md">UpsertTestSuccess</a></td>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -969,7 +969,7 @@
<tr>
<td><a href="../../requirements/interuss/automated_testing/rid/injection.md">UpsertTestResult</a></td>
<td>TODO</td>
<td><a href="../../scenarios/astm/netrid/v22a/nominal_behavior.md">ASTM NetRID nominal behavior</a></td>
<td><a href="../../scenarios/astm/netrid/v22a/misbehavior.md">ASTM NetRID SP clients misbehavior handling</a><br><a href="../../scenarios/astm/netrid/v22a/nominal_behavior.md">ASTM NetRID nominal behavior</a></td>
</tr>
<tr>
<td><a href="../../requirements/interuss/automated_testing/rid/injection.md">UpsertTestSuccess</a></td>
Expand Down
Loading