Skip to content

Commit

Permalink
[test] Add integration test for accessing sd sttr in dc (flyteorg#2969)
Browse files Browse the repository at this point in the history
* test: Add integration test for attr access of sd

Signed-off-by: JiaWei Jiang <waynechuang97@gmail.com>

* Correct file path

Signed-off-by: JiaWei Jiang <waynechuang97@gmail.com>

* test: Support interaction with minio s3 bucket

1. Upload a local parquet file to minio s3 bucket
2. Access StructuredDataset attr from a dataclass
3. Open StructuredDataset from a remote path

Signed-off-by: JiaWei Jiang <waynechuang97@gmail.com>

* Delete an unmerged integration test

Signed-off-by: JiaWei Jiang <waynechuang97@gmail.com>

* Try imagespec with commit sha of corresponding fix

Signed-off-by: JiaWei Jiang <waynechuang97@gmail.com>

* Remove redundant test

Signed-off-by: JiaWei Jiang <waynechuang97@gmail.com>

* Remove default_factory and create sd dc from input uri

Signed-off-by: JiaWei Jiang <waynechuang97@gmail.com>

* refactor: Clean test logic

1. Remove redundant prints
2. Use `mock.patch.dict` to setup `os.environ` for the current test fn
    * Avoid contaminating other tests running in the same process

Signed-off-by: JiaWei Jiang <waynechuang97@gmail.com>

* Remove redundant minio env var setup and add test comments

Signed-off-by: JiaWei Jiang <waynechuang97@gmail.com>

* Support uploading tmp pqt file

Signed-off-by: JiaWei Jiang <waynechuang97@gmail.com>

* Udpate deprecated module

Signed-off-by: JiaWei Jiang <waynechuang97@gmail.com>

* Remove redundant and unused imports

Signed-off-by: JiaWei Jiang <waynechuang97@gmail.com>

---------

Signed-off-by: JiaWei Jiang <waynechuang97@gmail.com>
Signed-off-by: Shuying Liang <shuying.liang@gmail.com>
  • Loading branch information
JiangJiaWei1103 authored and shuyingliang committed Jan 21, 2025
1 parent c0eb9ed commit 6b19e98
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion tests/flytekit/integration/remote/test_remote.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
from urllib.parse import urlparse
import uuid
import pytest
import mock
from unittest import mock

from flytekit import LaunchPlan, kwtypes, WorkflowExecutionPhase
from flytekit.configuration import Config, ImageConfig, SerializationSettings
Expand Down

0 comments on commit 6b19e98

Please sign in to comment.