Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pydantic2 migration made integration tests green #6719

Open
wants to merge 39 commits into
base: pydantic_v2_migration_do_not_squash_updates
Choose a base branch
from

Conversation

GitHK
Copy link
Contributor

@GitHK GitHK commented Nov 14, 2024

What do these changes do?

Bonus:

  • fixed issues with missing common-library across multiple tests and services
  • disabled openentelemtry when putting up simcore stack for tests

Related issue/s

How to test

Dev-ops checklist

Copy link

codecov bot commented Nov 14, 2024

Codecov Report

Attention: Patch coverage is 78.94737% with 4 lines in your changes missing coverage. Please review.

Project coverage is 82.58%. Comparing base (a2d3742) to head (b968a23).
Report is 1 commits behind head on pydantic_v2_migration_do_not_squash_updates.

Additional details and impacted files
@@                               Coverage Diff                               @@
##           pydantic_v2_migration_do_not_squash_updates    #6719      +/-   ##
===============================================================================
+ Coverage                                        81.15%   82.58%   +1.43%     
===============================================================================
  Files                                             1531     1524       -7     
  Lines                                            63884    63980      +96     
  Branches                                          2229     2229              
===============================================================================
+ Hits                                             51846    52840     +994     
+ Misses                                           11705    10804     -901     
- Partials                                           333      336       +3     
Flag Coverage Δ *Carryforward flag
integrationtests 53.35% <ø> (-0.01%) ⬇️ Carriedforward from 00f906b
unittests 85.35% <78.94%> (+0.22%) ⬆️

*This pull request uses carry forward flags. Click here to find out more.

Components Coverage Δ
api ∅ <ø> (∅)
pkg_aws_library 93.32% <ø> (ø)
pkg_dask_task_models_library 96.61% <100.00%> (ø)
pkg_models_library 91.23% <100.00%> (ø)
pkg_notifications_library 84.57% <ø> (ø)
pkg_postgres_database 87.30% <ø> (ø)
pkg_service_integration 70.00% <ø> (ø)
pkg_service_library 76.37% <80.00%> (-0.08%) ⬇️
pkg_settings_library 90.90% <ø> (ø)
pkg_simcore_sdk 78.21% <0.00%> (-0.06%) ⬇️
agent 97.01% <ø> (ø)
api_server 89.66% <ø> (-0.18%) ⬇️
autoscaling ∅ <ø> (∅)
catalog 90.57% <ø> (ø)
clusters_keeper 98.73% <ø> (ø)
dask_sidecar 91.26% <ø> (+0.39%) ⬆️
datcore_adapter 93.17% <ø> (ø)
director 58.48% <ø> (+0.09%) ⬆️
director_v2 76.29% <ø> (-0.01%) ⬇️
dynamic_scheduler 96.59% <ø> (ø)
dynamic_sidecar 88.19% <ø> (+28.40%) ⬆️
efs_guardian 90.12% <ø> (ø)
invitations 93.49% <ø> (ø)
osparc_gateway_server 85.74% <ø> (ø)
payments 92.77% <ø> (ø)
resource_usage_tracker 90.61% <ø> (-0.16%) ⬇️
storage 89.66% <100.00%> (ø)
webclient ∅ <ø> (∅)
webserver 80.05% <100.00%> (-0.02%) ⬇️

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a2d3742...b968a23. Read the comment docs.

@GitHK GitHK changed the title Pr osparc pydantic2 integration libraries2 Fix integration tests simcore-sdk and dynamic-sidecar Nov 14, 2024
@GitHK GitHK self-assigned this Nov 14, 2024
@GitHK GitHK added the t:maintenance Some planned maintenance work label Nov 14, 2024
@GitHK GitHK added this to the Event Horizon milestone Nov 14, 2024
@GitHK GitHK marked this pull request as ready for review November 14, 2024 10:34
Andrei Neagu added 2 commits November 14, 2024 13:24
…quash_updates' into pr-osparc-pydantic2-integration-libraries2
@GitHK GitHK changed the title Fix integration tests simcore-sdk and dynamic-sidecar Fixed most of the integration tests Nov 14, 2024
Copy link
Member

@sanderegg sanderegg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a few things

@GitHK GitHK changed the title Fixed most of the integration tests pydantic2 migration made integration tests green Nov 15, 2024
Copy link

sonarcloud bot commented Nov 15, 2024

Copy link
Member

@sanderegg sanderegg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please check my comments, there are a few left and I think you forgot all the dev.txt and ci.txt files

@@ -205,7 +205,10 @@ class ServiceUpdate(ServiceMetaDataEditable, ServiceAccessRights):
class ServiceGet(
ServiceMetaDataPublished, ServiceAccessRights, ServiceMetaDataEditable
): # pylint: disable=too-many-ancestors
owner: LowerCaseEmailStr | None
owner: LowerCaseEmailStr | None = Field(
None,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this change seems wrong. the ServiceGet comes from the catalog. this changes the API are you sure?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know we have the following scenario:
-> push service to osparc registry who's email is not part of the users
-> this owner filed is set to None

This should cover this case where. Since an integration test was failing this helped with fixing it. I'm not 100% sure if it might break anything though.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, the intent of the schema is that owner field is nullable but REQUIRED, here you made it NOT required.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is the API of the catalog so I guess this should not be changed

@@ -261,7 +261,7 @@ def jupyter_service(docker_registry: str, node_meta_schema: dict) -> dict[str, A
)


@pytest.fixture(scope="session", params=["2.0.4"])
@pytest.fixture(scope="session", params=["2.0.7"])
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: should this not go into upstream master?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no we don't use it there for anything. It's OK like this, since this is used for the CI.

services/agent/requirements/prod.txt Show resolved Hide resolved
@GitHK GitHK requested a review from sanderegg November 15, 2024 12:53
| list[Any]
| dict[str, Any]
| None,
Field(union_mode="left_to_right"),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the annotation is interesting :-)

@@ -205,7 +205,10 @@ class ServiceUpdate(ServiceMetaDataEditable, ServiceAccessRights):
class ServiceGet(
ServiceMetaDataPublished, ServiceAccessRights, ServiceMetaDataEditable
): # pylint: disable=too-many-ancestors
owner: LowerCaseEmailStr | None
owner: LowerCaseEmailStr | None = Field(
None,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, the intent of the schema is that owner field is nullable but REQUIRED, here you made it NOT required.

@@ -94,13 +94,37 @@ def testing_environ_vars(env_devel_file: Path) -> EnvVarsDict:
# ensure we do not use the bucket of simcore or so
env_devel["S3_BUCKET_NAME"] = "pytestbucket"

# ensure OpenTelemetry is not enabled
env_devel |= {
x: "null"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

x? are you Ellon's supporter? env_name or e or n at least means something ... otherwise _ ;-)

env_devel |= {
x: "null"
for x in (
"AGENT_TRACING",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So what is you stating here is that by default all tracing are always disabled. Fine.
Cons: when we have a new service we will most probably forget to disable it.

Other approaches would be:

  • .env-devel disables tracing by default (bad if we want to us it locally)
  • Every services explicitly in his conftest.py overriding it in the app_environment (my preferred one)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
t:maintenance Some planned maintenance work
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants