Skip to content

connects to basin w3s service #28

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Feb 9, 2024
Merged

connects to basin w3s service #28

merged 5 commits into from
Feb 9, 2024

Conversation

brunocalza
Copy link
Collaborator

@brunocalza brunocalza commented Feb 6, 2024

This PR updates the provider to make a call to https://github.com/tablelandnetwork/basin-w3s when a file is uploaded

It also does some cleaning up:

  • removes old RPC cold
  • and any unnecessary old web3 storage code

Signed-off-by: Bruno Calza <brunoangelicalza@gmail.com>
@@ -45,7 +45,7 @@ spec:
"--evm-chain-id", "$(EVM_CHAIN_ID)",
"--bind-address", "$(BIND_ADDRESS)",
"--bind-health-address", "$(BIND_HEALTH_ADDRESS)",
"--w3s-token", "$(W3S_TOKEN)",
"--basin-w3s-endpoint", "$(BASIN_W3S_ENDPOINT)",
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

new flag to indicate where the server is located. right now it's a private server inside kubernetes reachable at basin-w3s.basin-staging.svc.cluster.local:8080

}

#[async_trait]
impl Web3StorageClient for Web3StorageImpl {
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changes the web3 storage client to make a call to the service. It returns a UploadResponse object

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one suggestion on naming. what do you think about naming the trait as Web3Storage and the struct as Web3StorageClient?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sounds good!

@brunocalza brunocalza force-pushed the bcalza/w3s branch 3 times, most recently from d7885a1 to 8e4ec18 Compare February 6, 2024 20:55
Signed-off-by: Bruno Calza <brunoangelicalza@gmail.com>
@brunocalza brunocalza marked this pull request as ready for review February 6, 2024 21:14
@brunocalza brunocalza self-assigned this Feb 6, 2024
@brunocalza brunocalza requested a review from avichalp February 6, 2024 21:14

let result_cids = next_uploader
.finish_results()
let res = w3s_client
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Copy link
Contributor

@avichalp avichalp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good

Signed-off-by: Bruno Calza <brunoangelicalza@gmail.com>
Signed-off-by: Bruno Calza <brunoangelicalza@gmail.com>
@@ -530,7 +530,16 @@ pub async fn write_event<W: Web3StorageClient>(
}
}

let cid_bytes = match upload_w3s(gcs_client, w3s_client, &filename).await {
let mut retries = 0;
let cid_bytes = match loop {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe good to move the retry part into a different function? because this expressions looks quite complicated to read. For example:

pub async fn upload_w3s_with_retry<W: Web3Storage>(
    filename: &String,
    gcs_client: GcsClient,
    w3s_client: W,
) -> basin_common::errors::Result<Vec<u8>> {
    let mut retries = 0;
    loop {
        let result = upload_w3s(gcs_client.clone(), w3s_client.clone(), &filename).await;
        if result.is_ok() || retries > 3 {
            break result;
        } else {
            retries += 1;
            tokio::time::sleep(std::time::Duration::from_millis(100)).await;
        }
    }
}
 let cid_bytes = match upload_w3s_with_retry(&filename, gcs_client, w3s_client).await {
        .....
    };

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Totally! This should be in the web3 client actually, transparent to this part of the code, but was having a hard time getting the code to compile haha anyway, this was just a test to see if retrying would fix the problem of uploading big files, but I actually found out what the problem was. I'm going to remove it, actually

Signed-off-by: Bruno Calza <brunoangelicalza@gmail.com>
@brunocalza brunocalza merged commit 0dab234 into main Feb 9, 2024
@brunocalza brunocalza deleted the bcalza/w3s branch February 9, 2024 17:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants