refactor rclone and mention --disable-http2

storj · Feb 9, 2024 · fb77f42 · fb77f42
1 parent 2881b89
commit fb77f42
Show file tree

Hide file tree

Showing 4 changed files with 76 additions and 435 deletions.
diff --git a/app/dcs/getting-started/page.md b/app/dcs/getting-started/page.md
@@ -171,14 +171,23 @@ Next we'll upload a file. Here is an example image of a tree growing hard drives
 
 Copy the file to your bucket.
 
-{% code-group %}
+{% tabs %}
+{% tab label="rclone" %}
+
+{% callout type="info"  %}
+Using `--disable-http2` with rclone for Storj is recommended for increased transfer speeds by avoiding HTTP/2 specific issues.
+{% /callout %}
 
 ```shell {% title="rclone" %}
 # link[8:11] https://rclone.org/commands/rclone_copy/
 # terminal
-rclone copy ~/Downloads/storj-tree.png storj:my-bucket/
+rclone copy --disable-http2 ~/Downloads/storj-tree.png storj:my-bucket/
 ```
 
+{% /tab %}
+
+{% tab label="aws cli" %}
+
 ```shell {% title="aws cli" %}
 # focus
 # terminal
@@ -187,26 +196,39 @@ aws s3 --endpoint-url=https://gateway.storjshare.io cp ~/Downloads/storj-tree.pn
 upload: Downloads/storj-tree.png to s3://my-bucket/storj-tree.png
 ```
 
-{% /code-group %}
+{% /tab %}
+
+{% /tabs %}
 
 ## Download file
 
 To retrieve the file, use the same command as upload but reverse the order of the arguments
 
-{% code-group %}
+{% tabs %}
+{% tab label="rclone" %}
+
+{% callout type="info"  %}
+Using `--disable-http2` with rclone for Storj is recommended for increased transfer speeds by avoiding HTTP/2 specific issues.
+{% /callout %}
 
 ```shell {% title="rclone" %}
 # terminal
 # link[8:11] https://rclone.org/commands/rclone_copy/
-rclone copy storj:my-bucket/ ~/Downloads/storj-tree-2.png
+rclone copy --disable-http2 storj:my-bucket/ ~/Downloads/storj-tree-2.png
 ```
 
+{% /tab %}
+
+{% tab label="aws cli" %}
+
 ```shell {% title="aws cli" %}
 # terminal
 aws s3 --endpoint-url=https://gateway.storjshare.io cp s3://my-bucket/ ~/Downloads/storj-tree-2.png
 ```
 
-{% /code-group %}
+{% /tab %}
+
+{% /tabs %}
 
 ## List files
 

diff --git a/app/dcs/third-party-tools/rclone/page.md b/app/dcs/third-party-tools/rclone/page.md
@@ -13,56 +13,26 @@ metadata:
     different integration patterns for optimized performance and security.
 ---
 
-## Introduction
+Follow the [Getting Started guide](docId:AsyYcUJFbO1JI8-Tu8tW3) to setup Rclone.
 
-These 'Getting Starting' tutorials will showcase the process for configuring [Rclone](https://rclone.org) with Storj DCS. We will cover only some of the basic features in this guide.
+The following is more details about the 2 ways you can use Rclone with Storj.
 
-To make the most of Rclone, look at the complete Rclone [command reference](https://rclone.org/commands/).
+## S3 Compatible
 
-{% callout type="danger"  %}
-These guides are experimental. The main functionality appears to work, but there are expected to be undiscovered issues (including issues around connection timeouts). Please report any issues you may run into on this [forum thread](https://forum.storj.io/t/two-more-tech-previews-rclone-and-restic/6072).
-{% /callout %}
+Use our [S3 compatible API](docId:eZ4caegh9queuQuaazoo) to increase upload performance and reduce the load on your systems and network. A 1GB upload will result in only 1GB of data being uploaded
 
-You will need one of the following:
-
-- [](docId:Ch4vLynsEqyT2-3qDEBiy) that someone else shared with you, or
-
-- [](docId:OXSINcFRuVMBacPvswwNU) (access token) of a Storj DCS project you are a member of.
-
-## Selecting an Integration Pattern
+- Faster upload
+- Reduction in network load
+- Server-side encryption
 
 ## Native
 
 Use our native integration pattern to take advantage of client-side encryption as well as to achieve the best possible download performance. Uploads will be erasure-coded [](docId:Pksf8d0TCLY2tBgXeT18d), thus a 1GB upload will result in 2.68GB of data being uploaded to storage nodes across the network.
 
-### Use this pattern for
-
-- The strongest security
-
-- The best download speed
-
-{% quick-links %}
-{% quick-link %}
-[](docId:Mk51zylAE6xmqP7jUYAuX)
-{% /quick-link %}
-{% /quick-links %}
-
-## Hosted Gateway
-
-Use our S3 compatible Hosted Gateway integration pattern to increase upload performance and reduce the load on your systems and network. Uploads will be encrypted and erasure-coded [](docId:hf2uumViqYvS1oq8TYbeW), thus, a 1GB upload will result in only 1GB of data being uploaded to the Hosted Gateway.
-
-### Use this pattern for
-
-- Reduced upload time
-
-- Reduction in network load
-
-{% callout type="info"  %}
-By selecting this integration pattern, you are opting in to server-side encryption.
-{% /callout %}
+- End-to-end encryption
+- Faster download speed
 
 {% quick-links %}
-{% quick-link %}
-[](docId:WayQo-4CZXkITaHiGeQF_)
-{% /quick-link %}
+{% quick-link title="Rclone S3 compatible" href="docId:AsyYcUJFbO1JI8-Tu8tW3" /%}
+{% quick-link title="Rclone native" href="docId:Mk51zylAE6xmqP7jUYAuX" /%}
 {% /quick-links %}
diff --git a/app/dcs/third-party-tools/rclone/rclone-native/page.md b/app/dcs/third-party-tools/rclone/rclone-native/page.md
@@ -18,7 +18,6 @@ Use our native integration pattern to take advantage of client-side encryption a
 ## Use this pattern for
 
 - The strongest security
-
 - The best download speeds
 
 ## Setup
@@ -180,158 +179,4 @@ q) Quit config
 e/n/d/r/c/s/q> q
 ```
 
-{% callout type="info"  %}
-For additional security, you should consider using the (s) option
-
-`Set configuration password` option. It will encrypt the `rclone.conf` configuration file. This way secrets like the [](docId:OXSINcFRuVMBacPvswwNU) (access token), the encryption passphrase, and the access grant won't be stolen if an attacker get access to your configuration file.
-{% /callout %}
-
-## Create a Bucket
-
-Use the `mkdir` command to create new bucket, e.g. `mybucket`.
-
-```yaml
-rclone mkdir waterbear:mybucket
-```
-
-## List All Buckets
-
-Use the `lsf` command to list all buckets.
-
-```yaml
-rclone lsf waterbear:
-```
-
-{% callout type="info"  %}
-Note the colon (`:`) character at the end of the command line.
-{% /callout %}
-
-## Delete a Bucket
-
-Use the `rmdir` command to delete an empty bucket.
-
-```yaml
-rclone rmdir waterbear:mybucket
-```
-
-Use the `purge` command to delete a non-empty bucket with all its content.
-
-```yaml
-rclone purge waterbear:mybucket
-```
-
-## Upload Objects
-
-Use the `copy` command to upload an object.
-
-```yaml
-rclone copy --progress ~/Videos/myvideo.mp4 waterbear:mybucket/videos/
-```
-
-{% callout type="info"  %}
-The `--progress` flag is for displaying progress information. Remove it if you don't need this information.
-{% /callout %}
-
-Use a folder in the local path to upload all its objects.
-
-```yaml
-rclone copy --progress ~/Videos/ waterbear:mybucket/videos/
-```
-
-{% callout type="info"  %}
-Only modified files will be copied.
-{% /callout %}
-
-## List Objects
-
-Use the `ls` command to list recursively all objects in a bucket.
-
-```yaml
-rclone ls waterbear:mybucket
-```
-
-Add the folder to the remote path to list recursively all objects in this folder.
-
-```yaml
-rclone ls waterbear:mybucket/videos/
-```
-
-Use the `lsf` command to list non-recursively all objects in a bucket or a folder.
-
-```yaml
-rclone lsf waterbear:mybucket/videos/
-```
-
-## Download Objects
-
-Use the `copy` command to download an object.
-
-```yaml
-rclone copy --progress waterbear:mybucket/videos/myvideo.mp4 ~/Downloads/
-```
-
-{% callout type="info"  %}
-The `--progress` flag is for displaying progress information. Remove it if you don't need this information.
-{% /callout %}
-
-Use a folder in the remote path to download all its objects.
-
-```yaml
-rclone copy --progress waterbear:mybucket/videos/ ~/Downloads/
-```
-
-## Delete Objects
-
-Use the `deletefile` command to delete a single object.
-
-```yaml
-rclone deletefile waterbear:mybucket/videos/myvideo.mp4
-```
-
-Use the `delete` command to delete all object in a folder.
-
-```yaml
-rclone delete waterbear:mybucket/videos/
-```
-
-## Print the Total Size of Objects
-
-Use the `size` command to print the total size of objects in a bucket or a folder.
-
-```yaml
-rclone size waterbear:mybucket/videos/
-```
-
-## Sync Two Locations
-
-Use the `sync` command to sync the source to the destination, changing the destination only. Doesn’t transfer unchanged files, testing by size and modification time or MD5SUM. Destination is updated to match source, including deleting files if necessary.
-
-```yaml
-rclone sync --progress ~/Videos/ waterbear:mybucket/videos/
-```
-
-{% callout type="info"  %}
-The `--progress` flag is for displaying progress information. Remove it if you don't need this information.
-{% /callout %}
-
-{% callout type="info"  %}
-Since this can cause data loss, test first with the `--dry-run` flag to see exactly what would be copied and deleted.
-{% /callout %}
-
-The sync can be done also from Storj DCS to the local file system.
-
-```yaml
-rclone sync --progress waterbear:mybucket/videos/ ~/Videos/
-```
-
-Or between two Storj DCS buckets.
-
-```yaml
-rclone sync --progress waterbear-us:mybucket/videos/ waterbear-europe:mybucket/videos/
-```
-
-Or even between another cloud storage and Storj DCS.
-
-```yaml
-rclone sync --progress s3:mybucket/videos/ waterbear:mybucket/videos/
-```
+For additional commands you can do, see [](docId:WayQo-4CZXkITaHiGeQF_).