Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Dataset integration tests - Table Columns #1548

Merged
merged 54 commits into from
Sep 24, 2024

Conversation

dlpzx
Copy link
Contributor

@dlpzx dlpzx commented Sep 13, 2024

Feature or Bugfix

  • Feature: Testing

Detail

Follow-up of #1391

  • Implement Table Column tests

Relates

Security

Please answer the questions below briefly where applicable, or write N/A. Based on
OWASP 10.

  • Does this PR introduce or modify any input fields or queries - this includes
    fetching data from storage outside the application (e.g. a database, an S3 bucket)?
    • Is the input sanitized?
    • What precautions are you taking before deserializing the data you consume?
    • Is injection prevented by parametrizing queries?
    • Have you ensured no eval or similar functions are used?
  • Does this PR introduce any functionality or component that requires authorization?
    • How have you ensured it respects the existing AuthN/AuthZ mechanisms?
    • Are you logging failed auth attempts?
  • Are you using or adding any cryptographic features?
    • Do you use a standard proven implementations?
    • Are the used keys controlled by the customer? Where are they stored?
  • Are you introducing any new policies/roles/users?
    • Have you used the least-privilege principle? How?

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

dlpzx and others added 30 commits July 1, 2024 19:08
…egration-tests-datasets-pt2

# Conflicts:
#	tests_new/integration_tests/core/environment/global_conftest.py
#	tests_new/integration_tests/modules/s3_datasets/global_conftest.py
#	tests_new/integration_tests/modules/s3_datasets/test_s3_dataset.py
@dlpzx
Copy link
Contributor Author

dlpzx commented Sep 13, 2024

Tested locally:
image

…egration-tests-datasets-pt4

# Conflicts:
#	tests_new/integration_tests/modules/s3_datasets/queries.py
#	tests_new/integration_tests/modules/s3_datasets/test_s3_tables_columns.py
@dlpzx dlpzx marked this pull request as ready for review September 16, 2024 10:12
@dlpzx
Copy link
Contributor Author

dlpzx commented Sep 16, 2024

Testing in real CICD pipeline - I will update this comment with the results

@dlpzx dlpzx self-assigned this Sep 17, 2024
dataset = request.getfixturevalue(dataset_fixture_name)
tables = request.getfixturevalue(tables_fixture_name)
table_uri = tables[0].tableUri
assert_that(sync_dataset_table_columns).raises(GqlError).when_called_with(client2, table_uri).contains(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it actually catcher GqlError? When I test locally (with AWS deployment) the same thing returns just Exception(

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you test it in your deployment (not locally)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, and the tests for the table_columns pass:

image

There is an issue for the update_persisten_env which I am looking into. It should not be related to this, but I am waiting to merge until I get the issue
image

Copy link
Contributor Author

@dlpzx dlpzx Sep 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The error on the teardown of resources for worksheet might be related to the metadata forms delete:
image
The error in the test of the persistent environment is a Timeout
Uploading image.png…

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Worksheet is not yet connected to MFs. In any way. That's strange

response = update_dataset_table_column(
client1, column_uri, {'description': f'{session_id} new updated description'}
)
assert_that(response.description).is_equal_to(f'{session_id} new updated description')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I would minimize risk of accidental typos, if we are to change this

new_desc = f'{session_id} new updated description'
response = update_dataset_table_column(
        client1, column_uri, {'description': new_desc}
    )
assert_that(response.description).is_equal_to(new_desc)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done!

Copy link
Contributor

@SofiaSazonova SofiaSazonova left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple of nits.

@dlpzx dlpzx merged commit 59f0b26 into main Sep 24, 2024
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

3 participants