-
Notifications
You must be signed in to change notification settings - Fork 0
connects to basin w3s service #28
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Signed-off-by: Bruno Calza <brunoangelicalza@gmail.com>
7cb6f31
to
3f0ca93
Compare
@@ -45,7 +45,7 @@ spec: | |||
"--evm-chain-id", "$(EVM_CHAIN_ID)", | |||
"--bind-address", "$(BIND_ADDRESS)", | |||
"--bind-health-address", "$(BIND_HEALTH_ADDRESS)", | |||
"--w3s-token", "$(W3S_TOKEN)", | |||
"--basin-w3s-endpoint", "$(BASIN_W3S_ENDPOINT)", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
new flag to indicate where the server is located. right now it's a private server inside kubernetes reachable at basin-w3s.basin-staging.svc.cluster.local:8080
lib/worker/src/web3storage.rs
Outdated
} | ||
|
||
#[async_trait] | ||
impl Web3StorageClient for Web3StorageImpl { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
changes the web3 storage client to make a call to the service. It returns a UploadResponse
object
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
one suggestion on naming. what do you think about naming the trait as Web3Storage
and the struct as Web3StorageClient
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
sounds good!
d7885a1
to
8e4ec18
Compare
Signed-off-by: Bruno Calza <brunoangelicalza@gmail.com>
8e4ec18
to
1e94fd8
Compare
|
||
let result_cids = next_uploader | ||
.finish_results() | ||
let res = w3s_client |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good
Signed-off-by: Bruno Calza <brunoangelicalza@gmail.com>
Signed-off-by: Bruno Calza <brunoangelicalza@gmail.com>
lib/worker/src/routes/vaults.rs
Outdated
@@ -530,7 +530,16 @@ pub async fn write_event<W: Web3StorageClient>( | |||
} | |||
} | |||
|
|||
let cid_bytes = match upload_w3s(gcs_client, w3s_client, &filename).await { | |||
let mut retries = 0; | |||
let cid_bytes = match loop { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe good to move the retry part into a different function? because this expressions looks quite complicated to read. For example:
pub async fn upload_w3s_with_retry<W: Web3Storage>(
filename: &String,
gcs_client: GcsClient,
w3s_client: W,
) -> basin_common::errors::Result<Vec<u8>> {
let mut retries = 0;
loop {
let result = upload_w3s(gcs_client.clone(), w3s_client.clone(), &filename).await;
if result.is_ok() || retries > 3 {
break result;
} else {
retries += 1;
tokio::time::sleep(std::time::Duration::from_millis(100)).await;
}
}
}
let cid_bytes = match upload_w3s_with_retry(&filename, gcs_client, w3s_client).await {
.....
};
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Totally! This should be in the web3 client actually, transparent to this part of the code, but was having a hard time getting the code to compile haha anyway, this was just a test to see if retrying would fix the problem of uploading big files, but I actually found out what the problem was. I'm going to remove it, actually
Signed-off-by: Bruno Calza <brunoangelicalza@gmail.com>
This PR updates the provider to make a call to https://github.com/tablelandnetwork/basin-w3s when a file is uploaded
It also does some cleaning up: