diff --git a/monitoring/uss_qualifier/scenarios/astm/utm/dss/fragments/oir/crud/read.md b/monitoring/uss_qualifier/scenarios/astm/utm/dss/fragments/oir/crud/read_correct.md similarity index 82% rename from monitoring/uss_qualifier/scenarios/astm/utm/dss/fragments/oir/crud/read.md rename to monitoring/uss_qualifier/scenarios/astm/utm/dss/fragments/oir/crud/read_correct.md index e02a640657..11cc62c3de 100644 --- a/monitoring/uss_qualifier/scenarios/astm/utm/dss/fragments/oir/crud/read.md +++ b/monitoring/uss_qualifier/scenarios/astm/utm/dss/fragments/oir/crud/read_correct.md @@ -2,9 +2,9 @@ This test step fragment validates that operational intent references can be read -## 🛑 Get operational intent reference by ID check +## [Read query succeeds](./read_query.md) -If an operational intent reference cannot be queried using its ID, the DSS is failing to meet **[astm.f3548.v21.DSS0005,1](../../../../../../../requirements/astm/f3548/v21.md)**. +Check query succeeds. ## 🛑 Get operational intent reference response format conforms to spec check diff --git a/monitoring/uss_qualifier/scenarios/astm/utm/dss/fragments/oir/crud/search.md b/monitoring/uss_qualifier/scenarios/astm/utm/dss/fragments/oir/crud/search_correct.md similarity index 77% rename from monitoring/uss_qualifier/scenarios/astm/utm/dss/fragments/oir/crud/search.md rename to monitoring/uss_qualifier/scenarios/astm/utm/dss/fragments/oir/crud/search_correct.md index e026d4a94a..e2f2b176b3 100644 --- a/monitoring/uss_qualifier/scenarios/astm/utm/dss/fragments/oir/crud/search.md +++ b/monitoring/uss_qualifier/scenarios/astm/utm/dss/fragments/oir/crud/search_correct.md @@ -2,9 +2,9 @@ This test step fragment validates that operational intent references can be searched for -## 🛑 Successful operational intent reference search query check +## [Search query succeeds](./search_query.md) -If the DSS fails to let us search in the area for which the OIR was created, it is failing to meet **[astm.f3548.v21.DSS0005,1](../../../../../../../requirements/astm/f3548/v21.md)**. +Check query succeeds. ## 🛑 Search operational intent reference response format conforms to spec check diff --git a/monitoring/uss_qualifier/scenarios/astm/utm/dss/fragments/oir/sync.md b/monitoring/uss_qualifier/scenarios/astm/utm/dss/fragments/oir/sync.md index d6623f85e4..b008fcc16f 100644 --- a/monitoring/uss_qualifier/scenarios/astm/utm/dss/fragments/oir/sync.md +++ b/monitoring/uss_qualifier/scenarios/astm/utm/dss/fragments/oir/sync.md @@ -7,6 +7,13 @@ This test step fragment validates that operational intent references are properl If the previously created or mutated operational intent reference cannot be found at a DSS, either one of the instances at which the operational intent reference was created or the one that was queried, may be failing to implement **[astm.f3548.v21.DSS0210,2a](../../../../../../requirements/astm/f3548/v21.md)**. +## 🛑 Propagated operational intent reference general area is synchronized check + +When querying a secondary DSS for operational intents in the planning area that contains the propagated operational +intent, if the propagated operational intent is not contained in the response, then the general area in which the +propagated operational intent is located is not synchronized across DSS instances. +As such, either the primary or the secondary DSS fails to properly implements **[astm.f3548.v21.DSS0210,2e](../../../../../../requirements/astm/f3548/v21.md)**. + ## ⚠️ Propagated operational intent reference contains the correct manager check If the operational intent reference returned by a DSS to which the operational intent reference was synchronized to does not contain the correct manager, diff --git a/monitoring/uss_qualifier/scenarios/astm/utm/dss/synchronization/op_intent_ref_synchronization.md b/monitoring/uss_qualifier/scenarios/astm/utm/dss/synchronization/op_intent_ref_synchronization.md index 4f88a90fe8..19037bfdf7 100644 --- a/monitoring/uss_qualifier/scenarios/astm/utm/dss/synchronization/op_intent_ref_synchronization.md +++ b/monitoring/uss_qualifier/scenarios/astm/utm/dss/synchronization/op_intent_ref_synchronization.md @@ -49,25 +49,31 @@ Verify that an operational intent reference can be created on the primary DSS. Verify that the operational intent reference returned by the DSS under test is properly formatted and contains the expected content. -### Query newly created OIR test step +### Retrieve newly created OIR test step -Query the created operational intent at every DSS provided in `dss_instances`. +Retrieve and validate synchronization of the created operational intent at every DSS provided in `dss_instances`. -#### [Get OIR](../fragments/oir/crud/read.md) +#### [Get OIR query](../fragments/oir/crud/read_query.md) -Confirms that each DSS provides access to the created operational intent reference, +Check that read query succeeds. #### [OIR is synchronized](../fragments/oir/sync.md) +Confirms that each DSS provides direct access to the created operational intent reference. Confirm that the operational intent reference that was just created is properly synchronized across all DSS instances. -#### [OIR Content is correct](../fragments/oir/validate/correctness.md) +### Search for newly created OIR test step -Verify that the operational intent reference returned by every DSS is correctly formatted and corresponds to what was created earlier. +Search for and validate synchronization of the created operational intent at every DSS provided in `dss_instances`. -#### [OIR Versions are correct](../fragments/oir/validate/non_mutated.md) +#### [Search OIR](../fragments/oir/crud/search_query.md) -Verify that the operational intent reference's version fields are as expected. +Check that search query succeeds. + +#### [OIR is synchronized](../fragments/oir/sync.md) + +Confirms that each DSS returns the operational intent in relevant search results. +Confirm that the operational intent reference that was just created is properly synchronized across all DSS instances. ### Mutate OIR test step @@ -86,25 +92,31 @@ Verify that the operational intent reference returned by the DSS is properly for Verify that the operational intent reference's version fields have been updated. -### Query updated OIR test step +### Retrieve updated OIR test step -Query the updated operational intent reference at every DSS provided in `dss_instances`. +Retrieve and validate synchronization of the updated operational intent at every DSS provided in `dss_instances`. + +#### [Get OIR query](../fragments/oir/crud/read_query.md) + +Check that read query succeeds. #### [OIR is synchronized](../fragments/oir/sync.md) -Confirm that the operational intent reference that was just mutated is properly synchronized across all DSS instances. +Confirms that each DSS provides direct access to the updated operational intent reference. +Confirm that the operational intent reference that was just updated is properly synchronized across all DSS instances. -#### [Get OIR](../fragments/oir/crud/read.md) +### Search for updated OIR test step -Confirms that the operational intent reference that was just mutated can be retrieved from any DSS. +Search for and validate synchronization of the updated operational intent at every DSS provided in `dss_instances`. -#### [Validate OIR](../fragments/oir/validate/correctness.md) +#### [Search OIR](../fragments/oir/crud/search_query.md) -Verify that the operational intent reference returned by every DSS is correctly formatted and corresponds to what was mutated earlier. +Check that search query succeeds. -#### [OIR Versions are correct](../fragments/oir/validate/non_mutated.md) +#### [OIR is synchronized](../fragments/oir/sync.md) -Verify that the operational intent reference's version fields are as expected. +Confirms that each DSS returns the operational intent in relevant search results. +Confirm that the operational intent reference that was just updated is properly synchronized across all DSS instances. ### Delete OIR test step diff --git a/monitoring/uss_qualifier/scenarios/astm/utm/dss/synchronization/op_intent_ref_synchronization.py b/monitoring/uss_qualifier/scenarios/astm/utm/dss/synchronization/op_intent_ref_synchronization.py index a7aa6f53b8..19a0ec1511 100644 --- a/monitoring/uss_qualifier/scenarios/astm/utm/dss/synchronization/op_intent_ref_synchronization.py +++ b/monitoring/uss_qualifier/scenarios/astm/utm/dss/synchronization/op_intent_ref_synchronization.py @@ -1,8 +1,6 @@ from datetime import datetime, timedelta from typing import List, Optional -from implicitdict import StringBasedDateTime -from uas_standards.astm.f3548.v21 import api from uas_standards.astm.f3548.v21.api import ( OperationalIntentReference, PutOperationalIntentReferenceParameters, @@ -11,7 +9,7 @@ ) from uas_standards.astm.f3548.v21.constants import Scope -from monitoring.monitorlib.fetch import QueryError +from monitoring.monitorlib.fetch import QueryError, Query from monitoring.monitorlib.geotemporal import Volume4D, Volume4DCollection from monitoring.prober.infrastructure import register_resource_type from monitoring.uss_qualifier.resources.astm.f3548.v21 import PlanningAreaResource @@ -134,18 +132,26 @@ def run(self, context: ExecutionContext): self._create_oir_with_params(self._oir_params) self.end_test_step() - self.begin_test_step("Query newly created OIR") + self.begin_test_step("Retrieve newly created OIR") self._query_secondaries_and_compare(self._oir_params) self.end_test_step() + self.begin_test_step("Search for newly created OIR") + self._search_secondaries_and_compare(self._oir_params) + self.end_test_step() + self.begin_test_step("Mutate OIR") self._test_mutate_oir_shift_time() self.end_test_step() - self.begin_test_step("Query updated OIR") + self.begin_test_step("Retrieve updated OIR") self._query_secondaries_and_compare(self._oir_params) self.end_test_step() + self.begin_test_step("Search for updated OIR") + self._search_secondaries_and_compare(self._oir_params) + self.end_test_step() + self.begin_test_step("Delete OIR") self._test_delete_sub() self.end_test_step() @@ -228,34 +234,99 @@ def _query_secondaries_and_compare( self, expected_oir_params: PutOperationalIntentReferenceParameters ): for secondary_dss in self._dss_read_instances: - self._validate_oir_from_secondary( - secondary_dss=secondary_dss, - expected_oir_params=expected_oir_params, - involved_participants=list( - {self._primary_pid, secondary_dss.participant_id} - ), + with self.check( + "Get operational intent reference by ID", + secondary_dss.participant_id, + ) as check: + try: + oir, q = secondary_dss.get_op_intent_reference(self._oir_id) + self.record_query(q) + except QueryError as e: + self.record_queries(e.queries) + check.record_failed( + summary="GET for operational intent reference failed", + details=f"Query for operational intent reference failed: {e.msg}", + query_timestamps=e.query_timestamps, + ) + + involved_participants = list( + {self._primary_pid, secondary_dss.participant_id} ) + with self.check( + "Operational intent reference can be found at every DSS", + involved_participants, + ) as check: + if q.status_code == 404: + check.record_failed( + summary="Requested operational intent was not found at secondary DSS.", + details=f"Query for operational intent reference {self._oir_id} failed: {e.msg}", + query_timestamps=e.query_timestamps, + ) + + self._validate_oir_from_secondary( + oir=oir, + q=q, + expected_oir_params=expected_oir_params, + involved_participants=involved_participants, + ) + + def _search_secondaries_and_compare( + self, expected_oir_params: PutOperationalIntentReferenceParameters + ): + for secondary_dss in self._dss_read_instances: + with self.check( + "Successful operational intent reference search query", + [secondary_dss.participant_id], + ) as check: + try: + oirs, q = secondary_dss.find_op_intent(self._planning_area_volume4d) + self.record_query(q) + except QueryError as qe: + self.record_queries(qe.queries) + check.record_failed( + summary="Failed to search for operational intent references", + details=f"Failed to query operational intent references: got response code {qe.cause_status_code}: {qe.msg}", + query_timestamps=qe.query_timestamps, + ) + + involved_participants = list( + {self._primary_pid, secondary_dss.participant_id} + ) + + with self.check( + "Propagated operational intent reference general area is synchronized", + involved_participants, + ) as check: + oir: Optional[OperationalIntentReference] = None + for _oir in oirs: + if _oir.id == self._oir_id: + oir = _oir + + if oir is None: + check.record_failed( + summary="Propagated OIR not found", + details=f"OIR {self._oir_id} was not found in the secondary DSS when searched for its expected geo-temporal extent", + query_timestamps=[q.request.timestamp], + ) + + self._validate_oir_from_secondary( + oir=oir, + q=q, + expected_oir_params=expected_oir_params, + involved_participants=involved_participants, + ) + + # TODO: craft a search with an area of interest that does not intersect with the planning area, + # but whose convex hull intersects with the planning area + def _validate_oir_from_secondary( self, - secondary_dss: DSSInstance, + oir: OperationalIntentReference, + q: Query, expected_oir_params: PutOperationalIntentReferenceParameters, involved_participants: List[str], ): - with self.check( - "Operational intent reference can be found at every DSS", - involved_participants, - ) as check: - try: - oir, q = secondary_dss.get_op_intent_reference(self._oir_id) - self.record_query(q) - except QueryError as e: - self.record_queries(e.queries) - check.record_failed( - summary="GET for operational intent reference failed", - details=f"Query for operational intent reference failed: {e.msg}", - query_timestamps=e.query_timestamps, - ) with self.check( "Propagated operational intent reference contains the correct manager", @@ -323,24 +394,6 @@ def _validate_oir_from_secondary( query_timestamps=[q.request.timestamp], ) - with self.check( - "Get operational intent reference response content is correct", - [secondary_dss.participant_id], - ) as check: - # Do a full validation of the OIR as a sanity check - OIRValidator( - main_check=check, - scenario=self, - expected_manager=self._expected_manager, - participant_id=[secondary_dss.participant_id], - oir_params=expected_oir_params, - ).validate_fetched_oir( - expected_oir_id=self._oir_id, - fetched_oir=q, - expected_version=self._current_oir.version, - expected_ovn=self._current_oir.ovn, - ) - def _test_mutate_oir_shift_time(self): """Mutate the OIR by adding 10 seconds to its start and end times. This is achieved by updating the first and last element of the extents. diff --git a/monitoring/uss_qualifier/scenarios/astm/utm/dss/validators/oir_validator.py b/monitoring/uss_qualifier/scenarios/astm/utm/dss/validators/oir_validator.py index 07566e7fa7..1f604da1e5 100644 --- a/monitoring/uss_qualifier/scenarios/astm/utm/dss/validators/oir_validator.py +++ b/monitoring/uss_qualifier/scenarios/astm/utm/dss/validators/oir_validator.py @@ -415,7 +415,7 @@ def validate_searched_oir( QueryOperationalIntentReferenceResponse ) - by_id = {oir: oir.id for oir in resp_parsed.operational_intent_references} + by_id = {oir.id: oir for oir in resp_parsed.operational_intent_references} with self._scenario.check( "Created operational intent reference is in search results", self._pid diff --git a/monitoring/uss_qualifier/suites/astm/utm/dss_probing.md b/monitoring/uss_qualifier/suites/astm/utm/dss_probing.md index 5b0a847279..74db9b71e0 100644 --- a/monitoring/uss_qualifier/suites/astm/utm/dss_probing.md +++ b/monitoring/uss_qualifier/suites/astm/utm/dss_probing.md @@ -27,7 +27,7 @@ Checked in - astm
.f3548
.v21
+ astm
.f3548
.v21
DSS0005,1 Implemented ASTM F3548-21 UTM DSS Operational Intent Reference Access Control
ASTM SCD DSS: Interfaces authentication
ASTM SCD DSS: Operational Intent Reference Key Validation
ASTM SCD DSS: Operational Intent Reference Synchronization
ASTM SCD DSS: Subscription Simple
ASTM SCD DSS: Subscription Synchronization
ASTM SCD DSS: Subscription Validation
ASTM SCD DSS: Subscription and entity deletion interaction
ASTM SCD DSS: Subscription and entity interaction @@ -132,6 +132,11 @@ Implemented ASTM SCD DSS: Operational Intent Reference Synchronization + + DSS0210,2e + Implemented + ASTM SCD DSS: Operational Intent Reference Synchronization + DSS0210,2f Implemented diff --git a/monitoring/uss_qualifier/suites/astm/utm/f3548_21.md b/monitoring/uss_qualifier/suites/astm/utm/f3548_21.md index f6ac43a509..b2a757f791 100644 --- a/monitoring/uss_qualifier/suites/astm/utm/f3548_21.md +++ b/monitoring/uss_qualifier/suites/astm/utm/f3548_21.md @@ -35,7 +35,7 @@ Checked in - astm
.f3548
.v21
+ astm
.f3548
.v21
DSS0005,1 Implemented ASTM F3548 flight planners preparation
ASTM F3548-21 UTM DSS Operational Intent Reference Access Control
ASTM SCD DSS: Interfaces authentication
ASTM SCD DSS: Operational Intent Reference Key Validation
ASTM SCD DSS: Operational Intent Reference Synchronization
ASTM SCD DSS: Subscription Simple
ASTM SCD DSS: Subscription Synchronization
ASTM SCD DSS: Subscription Validation
ASTM SCD DSS: Subscription and entity deletion interaction
ASTM SCD DSS: Subscription and entity interaction
Nominal planning: not permitted conflict with equal priority
Off-Nominal planning: down USS
Off-Nominal planning: down USS with equal priority conflicts not permitted @@ -145,6 +145,11 @@ Implemented ASTM SCD DSS: Operational Intent Reference Synchronization + + DSS0210,2e + Implemented + ASTM SCD DSS: Operational Intent Reference Synchronization + DSS0210,2f Implemented diff --git a/monitoring/uss_qualifier/suites/faa/uft/message_signing.md b/monitoring/uss_qualifier/suites/faa/uft/message_signing.md index 89feb92231..b3cb9eb381 100644 --- a/monitoring/uss_qualifier/suites/faa/uft/message_signing.md +++ b/monitoring/uss_qualifier/suites/faa/uft/message_signing.md @@ -18,7 +18,7 @@ Checked in - astm
.f3548
.v21
+ astm
.f3548
.v21
DSS0005,1 Implemented ASTM F3548 flight planners preparation
ASTM F3548-21 UTM DSS Operational Intent Reference Access Control
ASTM SCD DSS: Interfaces authentication
ASTM SCD DSS: Operational Intent Reference Key Validation
ASTM SCD DSS: Operational Intent Reference Synchronization
ASTM SCD DSS: Subscription Simple
ASTM SCD DSS: Subscription Synchronization
ASTM SCD DSS: Subscription Validation
ASTM SCD DSS: Subscription and entity deletion interaction
ASTM SCD DSS: Subscription and entity interaction
Nominal planning: not permitted conflict with equal priority
Off-Nominal planning: down USS
Off-Nominal planning: down USS with equal priority conflicts not permitted @@ -128,6 +128,11 @@ Implemented ASTM SCD DSS: Operational Intent Reference Synchronization + + DSS0210,2e + Implemented + ASTM SCD DSS: Operational Intent Reference Synchronization + DSS0210,2f Implemented diff --git a/monitoring/uss_qualifier/suites/interuss/dss/all_tests.md b/monitoring/uss_qualifier/suites/interuss/dss/all_tests.md index 3f87043968..4c4def593a 100644 --- a/monitoring/uss_qualifier/suites/interuss/dss/all_tests.md +++ b/monitoring/uss_qualifier/suites/interuss/dss/all_tests.md @@ -408,7 +408,7 @@ ASTM NetRID DSS: Concurrent Requests
ASTM NetRID DSS: ISA Expiry
ASTM NetRID DSS: ISA Subscription Interactions
ASTM NetRID DSS: Simple ISA
ASTM NetRID DSS: Submitted ISA Validations
ASTM NetRID DSS: Subscription Simple
ASTM NetRID DSS: Subscription Validation
ASTM NetRID DSS: Token Validation - astm
.f3548
.v21
+ astm
.f3548
.v21
DSS0005,1 Implemented ASTM F3548-21 UTM DSS Operational Intent Reference Access Control
ASTM SCD DSS: Interfaces authentication
ASTM SCD DSS: Operational Intent Reference Key Validation
ASTM SCD DSS: Operational Intent Reference Synchronization
ASTM SCD DSS: Subscription Simple
ASTM SCD DSS: Subscription Synchronization
ASTM SCD DSS: Subscription Validation
ASTM SCD DSS: Subscription and entity deletion interaction
ASTM SCD DSS: Subscription and entity interaction @@ -513,6 +513,11 @@ Implemented ASTM SCD DSS: Operational Intent Reference Synchronization + + DSS0210,2e + Implemented + ASTM SCD DSS: Operational Intent Reference Synchronization + DSS0210,2f Implemented diff --git a/monitoring/uss_qualifier/suites/uspace/flight_auth.md b/monitoring/uss_qualifier/suites/uspace/flight_auth.md index 539cea93f6..5ee68ba652 100644 --- a/monitoring/uss_qualifier/suites/uspace/flight_auth.md +++ b/monitoring/uss_qualifier/suites/uspace/flight_auth.md @@ -19,7 +19,7 @@ Checked in - astm
.f3548
.v21
+ astm
.f3548
.v21
DSS0005,1 Implemented ASTM F3548 flight planners preparation
ASTM F3548-21 UTM DSS Operational Intent Reference Access Control
ASTM SCD DSS: Interfaces authentication
ASTM SCD DSS: Operational Intent Reference Key Validation
ASTM SCD DSS: Operational Intent Reference Synchronization
ASTM SCD DSS: Subscription Simple
ASTM SCD DSS: Subscription Synchronization
ASTM SCD DSS: Subscription Validation
ASTM SCD DSS: Subscription and entity deletion interaction
ASTM SCD DSS: Subscription and entity interaction
Nominal planning: not permitted conflict with equal priority
Off-Nominal planning: down USS
Off-Nominal planning: down USS with equal priority conflicts not permitted @@ -129,6 +129,11 @@ Implemented ASTM SCD DSS: Operational Intent Reference Synchronization + + DSS0210,2e + Implemented + ASTM SCD DSS: Operational Intent Reference Synchronization + DSS0210,2f Implemented diff --git a/monitoring/uss_qualifier/suites/uspace/required_services.md b/monitoring/uss_qualifier/suites/uspace/required_services.md index f5a8dce453..0dccb79125 100644 --- a/monitoring/uss_qualifier/suites/uspace/required_services.md +++ b/monitoring/uss_qualifier/suites/uspace/required_services.md @@ -454,7 +454,7 @@ ASTM NetRID DSS: Concurrent Requests
ASTM NetRID DSS: ISA Expiry
ASTM NetRID DSS: ISA Subscription Interactions
ASTM NetRID DSS: Simple ISA
ASTM NetRID DSS: Submitted ISA Validations
ASTM NetRID DSS: Subscription Simple
ASTM NetRID DSS: Subscription Validation
ASTM NetRID DSS: Token Validation - astm
.f3548
.v21
+ astm
.f3548
.v21
DSS0005,1 Implemented ASTM F3548 flight planners preparation
ASTM F3548-21 UTM DSS Operational Intent Reference Access Control
ASTM SCD DSS: Interfaces authentication
ASTM SCD DSS: Operational Intent Reference Key Validation
ASTM SCD DSS: Operational Intent Reference Synchronization
ASTM SCD DSS: Subscription Simple
ASTM SCD DSS: Subscription Synchronization
ASTM SCD DSS: Subscription Validation
ASTM SCD DSS: Subscription and entity deletion interaction
ASTM SCD DSS: Subscription and entity interaction
Nominal planning: not permitted conflict with equal priority
Off-Nominal planning: down USS
Off-Nominal planning: down USS with equal priority conflicts not permitted @@ -564,6 +564,11 @@ Implemented ASTM SCD DSS: Operational Intent Reference Synchronization + + DSS0210,2e + Implemented + ASTM SCD DSS: Operational Intent Reference Synchronization + DSS0210,2f Implemented