Add past blog articles & release notes

blog/2022-07-22-sg3w-is-in-town.md

@@ -0,0 +1,26 @@
+---
+title: Contain your excitement, the s3gw is in town
+description: We've developed a standalone RADOS Gateway. Welcome to the s3gw Project.
+slug: s3gw-rados-gateway-standalone
+authors:
+  - name: The s3gw team
+tags: [blog]
+hide_table_of_contents: false
+---
+
+The Aquarist Labs team is back with an open source and cloud-native S3 service. After spending months investigating a storage appliance built on Ceph, the team identified the need to complement the Rancher storage portfolio with their unique skillsets and lessons learned from their time developing Aquarium.
+
+<!--truncate-->
+
+Introducing the [s3gw project][1], an S3-compatible gateway running a standalone [RADOS Gateway (RGW)][2] implementation, using a non-RADOS backend for standalone usage. This new project provides the required infrastructure to build a container able to run on a Kubernetes cluster with S3-compatible endpoints to applications.
+
+This is the first publicly available iteration of the s3gw project. We expect (and welcome!) bugs and we know our performance is not optimal (yet!). This release is meant for testing and feedback gathering and it is not recommended for production use. See our release notes for more details.
+
+## Call to action
+
+We would love to hear from you about what you'd like to see on our roadmap. What would enable you best to use s3gw in your environment?
+
+Reach out to us at <s3gw@suse.com> or our [Slack channel](https://aquaristlabs.slack.com/archives/C03RFG0BES0). You can also join [our mailing list](https://lists.suse.com/mailman/listinfo/s3gw) or have a look at our [GitHub repository](https://github.com/aquarist-labs/s3gw) -- feature requests are welcome! 🙂
+
+[1]:https://github.com/aquarist-labs/s3gw
+[2]:https://docs.ceph.com/en/quincy/radosgw/

blog/2022-07-28-pv-s3-access.md

@@ -0,0 +1,283 @@
+---
+title: Does your PV need S3 access? We’ve got you covered
+description: In a cloud-native environment, it is important to offer storage systems that can interact with clients using a standard protocol.
+slug: does-your-pv-need-s3-access
+authors:
+  - name: The s3gw team
+tags: [blog, s3gw, Rancher, Longhorn]
+hide_table_of_contents: false
+---
+
+Increased demand for cloud storage solutions has become a crucial topic in recent years: companies are requiring data to be made more readily available for their cloud-native applications.
+
+<!--truncate-->
+
+Increased demand for cloud storage solutions has become a crucial topic in recent years: companies are requiring data to be made more readily available for their cloud-native applications.
+
+In a cloud-native environment, it is important to offer storage systems that can interact with clients using a standard protocol.
+
+## Simple Storage Service
+
+![S3 logo](https://www.suse.com/c/wp-content/uploads/2022/07/s3.png)
+
+[Simple Storage Service,](https://docs.aws.amazon.com/AmazonS3/latest/userguide/Welcome.html) or S3, is a protocol designed by Amazon that launched in the United States market in 2006. S3 is a vast protocol that covers key concepts such as *buckets*, *objects*, *keys*, *versioning*, *ACLs* and *regions*.
+
+To continue with this article, you need to know that the S3 API can be invoked with REST calls and that you can just store your objects inside holders called buckets. For more information, there are other resources available online.
+
+##  K3s and Rancher
+
+This article explores the use of [K3s](https://k3s.io/) and [Rancher](https://rancher.com/) as foundations for experiments with an S3 gateway.
+
+![K3S logo](https://www.suse.com/c/wp-content/uploads/2022/05/k3s-icon-color-1-300x245.png)
+
+K3s is a lightweight Kubernetes distribution that runs smoothly on Edge low resource devices. Rancher is a graphical user manager that simplifies the underlying complexity of a Kubernetes cluster.
+
+![Rancher logo](https://www.suse.com/c/wp-content/uploads/2022/05/rancher-logo-cow-black.png)
+
+With Rancher, you can manage a cluster in a user-friendly fashion, regardless of the Kubernetes version being used.
+
+## Longhorn
+
+![Longhorn logo](https://www.suse.com/c/wp-content/uploads/2022/07/longhorn.png)
+
+A Kubernetes cluster and manager alone are not sufficient when dealing with cloud storage. You could use the primitive resources offered by a standard Kubernetes cluster such as the basic persistent volume types, but we recommend installing a component that takes care of providing your pods with some advanced storage resource.
+
+It is desirable to have a system that can take care of your data securely and in a redundant fasion and expose volumes with the standard Kubernetes interfaces. [Longhorn](https://longhorn.io/) is the right system for this kind of need. Built from scratch to work natively with Kubernetes, Longhorn allows pods to obtain highly available persistent volumes. The portion of storage managed by Longhorn is replicated so that a hardware failure does not compromise user's data.
+
+## S3 Gateway
+
+![S3GW logo](https://www.suse.com/c/wp-content/uploads/2022/07/logo-s3gw-300x256.png)
+
+Having Longhorn deployed on your cluster allows the persistent volumes to be consumed by internal applications deployed on Kubernetes. If you want to give access to the data to external clients, you need an S3 gateway.
+
+External clients can store and read data to and from the cluster using the S3 API. For this role, we are going to employ [s3gw.](https://aquarist-labs.io/s3gw/)
+
+s3gw is being developed on the foundations of [Ceph](https://ceph.com/en/) S3 gateway: radosgw. Even if s3gw is still in its infancy and still in an early stage of development, it can already be used to test and play with S3 functionalities.
+
+## Let's start cooking ingredients
+
+Now that you have identified all the pieces, you are ready to start building your environment.\
+For this tutorial, we are installing K3s on an OpenSUSE Linux OS. For the sake of simplicity, because Kubernetes needs to have certain networking resources available, it can be worth completely disabling the system firewall.
+
+If you prefer to keep your firewall on, have a look [here](https://rancher.com/docs/k3s/latest/en/installation/installation-requirements/#networking).
+
+### Stop Firewall
+
+From a shell prompt, run the following command:
+
+```bash
+sudo systemctl stop firewalld.service
+```
+
+### Install K3s
+
+From a shell prompt, run the following command:
+
+```bash
+$ curl -sfL <https://get.k3s.io> | INSTALL_K3S_VERSION=v1.23.9+k3s1 sh -
+```
+
+After installation has terminated, you can check that the cluster is running with:
+
+```bash
+$ sudo kubectl get nodes
+```
+
+If everything is ok, you should see something similar:
+
+```bash
+NAME STATUS ROLES                AGE   VERSION
+suse Ready  control-plane,master 56s   v1.23.8+k3s1
+```
+
+If you prefer using K3s with your regular user and not as root, you can run:
+
+```bash
+$ sudo cp /etc/rancher/k3s/k3s.yaml ~/.kube/config && chown $USER ~/.kube/config && chmod 600 ~/.kube/config && export KUBECONFIG=~/.kube/config
+```
+
+After this, you will be able to operate on K3s with your user.
+
+### Install Helm
+
+![Helm logo](https://www.suse.com/c/wp-content/uploads/2022/07/helm-260x300.png)
+
+We are going to install Rancher using an [Helm](https://helm.sh/) chart, so you must first install Helm on the system:
+
+```bash
+$ sudo zypper install helm
+```
+
+### Deploy Rancher
+
+Let's begin to install Rancher adding the its latest repo to Helm:
+
+```bash
+$ helm repo add rancher-latest <https://releases.rancher.com/server-charts/latest>
+```
+
+After this, you must define a new Kubernetes namespace where Rancher will install:
+
+```bash
+$ kubectl create namespace cattle-system
+```
+
+As the official [documentation](https://rancher.com/docs/rancher/v2.6/en/installation/install-rancher-on-k8s/#install-the-rancher-helm-chart) dictates, this must be named *cattle-system.*
+
+Because Rancher management server is designed to be secure by default and requires SSL/TLS configuration you must deploy some additional resource:
+
+```bash
+$ kubectl apply -f <https://github.com/cert-manager/cert-manager/releases/download/v1.7.1/cert-manager.crds.yaml>
+
+$ helm repo add jetstack <https://charts.jetstack.io>
+
+$ helm install cert-manager jetstack/cert-manager\
+ --namespace cert-manager\
+ --create-namespace\
+ --version v1.7.1
+```
+
+Let's check that the cert-manager has successfully deployed and the related pods are running:
+
+```bash
+$ kubectl get pods --namespace cert-manager
+```
+
+```bash
+NAME                                       READY   STATUS    RESTARTS   AGE
+cert-manager-5c6866597-zw7kh               1/1     Running   0          2m
+cert-manager-cainjector-577f6d9fd7-tr77l   1/1     Running   0          2m
+cert-manager-webhook-787858fcdb-nlzsq      1/1     Running   0          2m
+```
+
+Now, you are required to define an hostname in /etc/hosts pointing to the ip address of one of the host's physical interfaces, for example:
+
+10.0.0.2 rancher.local
+
+After this, you can finally launch the Rancher installation command:
+
+```bash
+helm install rancher rancher-latest/rancher\
+ --namespace cattle-system\
+ --set hostname=rancher.local\
+ --set bootstrapPassword=admin
+```
+
+When Rancher's pods have booted up,  with your browser you can navigate to: <https://rancher.local> and complete the initial setup:
+
+![Screenshot 1](https://www.suse.com/c/wp-content/uploads/2022/07/rancher1-1024x943.png)
+
+Once you have completed the step, and saved the password, you can start exploring your local cluster with the graphical manager:
+
+![Screenshot 2](https://www.suse.com/c/wp-content/uploads/2022/07/rancher.local_-1024x614.png)
+
+Depending on what you have deployed on the cluster, you could see more or less resource consumption.
+
+### Deploy Longhorn
+
+You can now deploy Longhorn using the Charts chooser under the Apps section on the left of the Rancher's dashboard:
+
+![](https://www.suse.com/c/wp-content/uploads/2022/07/rancher3-1024x727.png)
+
+The installation is pretty straightforward and you don't need to change any default value of the chart.\
+In the end, if everything has gone well, you should see the Rancher's console showing:
+
+![Screenshot 3](https://www.suse.com/c/wp-content/uploads/2022/07/ranch-longh-install-1024x596.png)
+
+After Longhorn has been installed, you can simply click on the Longhorn entry on the left Rancher's menu to be redirected to the Longhorn's dashboard:
+
+![](https://www.suse.com/c/wp-content/uploads/2022/07/long-dash-1024x651.png)
+
+A freshly installation of Longhorn shows that still no application is using a persistent volume.
+
+### Deploy s3gw
+
+You are now ready to add the last ingredient to your system: s3gw, the S3 gateway.\
+Rancher does not bundle with the [Helm repository of s3gw](https://github.com/aquarist-labs/s3gw-charts) set by default, so you must add it from the dashboard:
+
+![Screenshot 4](https://www.suse.com/c/wp-content/uploads/2022/07/s3gw-repo-add-1024x608.png)
+
+You can choose an unique name: for example: s3gw.\
+For the Target field you have to choose Git repository and the repository URL:
+
+<https://github.com/aquarist-labs/s3gw-charts>
+
+In the Git Branch field put the latest available release:
+
+v0.2.0
+
+You can now click on Create button.\
+On Apps section, you can now find the s3gw chart:
+
+![Screenshot 5](https://www.suse.com/c/wp-content/uploads/2022/07/s3gw-chart-1024x630.png)
+
+So let's proceed with installation; you can choose a namespace and a name for s3gw:
+
+![Screenshot 6](https://www.suse.com/c/wp-content/uploads/2022/07/s3gw-ns1-1024x657.png)
+
+You don't need to customize the chart, so you can leave the bottom checkbox unchecked.\
+Once the installation completed, if everything has gone well, you should see the Rancher's console showing:
+
+![Screenshot 7](https://www.suse.com/c/wp-content/uploads/2022/07/s3gw-installed-1024x774.png)
+
+On Longhorn dashboard, you can verify that the application is using a Longhorn persistent volume:
+
+![Screenshot 8](https://www.suse.com/c/wp-content/uploads/2022/07/lh-s3gw-1024x480.png)
+
+### Test the S3 gateway
+
+By default the s3gw chart configures an ingress resource pointing to the S3 gateway with the FQDN: s3gw.local .\
+Thus, you must define s3gw.local in /etc/hosts pointing to the ip address of one of the host's physical interfaces, for example:
+
+10.0.0.2 s3gw.local
+
+For testing the S3 gateway you can rely on [s3cmd](https://github.com/s3tools/s3cmd) that is a popular command line S3 client.\
+You can install it choosing a method listed [here](https://github.com/s3tools/s3cmd/blob/master/INSTALL.md).\
+Once you have installed it, you can take the s3cmd configuration file from [here](https://raw.githubusercontent.com/aquarist-labs/s3gw-core/main/env/s3cmd.cfg) and use that as it is against s3gw.\
+All you need to to is to create a directory, put s3cmd.cfg inside that and finally invoke s3cmd.
+
+#### Create a bucket
+
+```bash
+$ s3cmd -c s3cmd.cfg mb s3://foo
+```
+
+#### Put some objects in the bucket
+
+Let's create a 1mb file filled with random data and put it in the  bucket:
+
+```bash
+$ dd if=/dev/random bs=1k count=1k of=obj.1mb.bin
+$ s3cmd -c s3cmd.cfg put obj.1mb.bin s3://foo
+```
+
+Let's create a 10mb file filled with random data and put it in the bucket:
+
+```bash
+$ dd if=/dev/random bs=1k count=10k of=obj.10mb.bin
+$ s3cmd -c s3cmd.cfg put obj.10mb.bin s3://foo
+```
+
+#### List objects contained in a bucket
+
+```bash
+$ s3cmd -c s3cmd.cfg ls s3://foo
+2022-07-26 15:03 10485760  s3://foo/obj.10mb.bin
+2022-07-26 15:01 1048576   s3://foo/obj.1mb.bin
+```
+
+#### Delete an object
+
+```bash
+$ s3cmd -c s3cmd.cfg rm s3://foo/obj.10mb.bin
+```
+
+### In summary
+
+In this tutorial, you've seen how to set up a K3s cluster, manage it with Rancher, install Longhorn and finally enrich the system with a S3 gateway. K3s, Rancher and Longhorn are powerful tools to set up an environment providing resilient and performing storage capabilities. If you need to expose the storage to external clients, you can choose to install s3gw with a near zero effort.
+
+## Call to action
+
+We would love to hear from you about what you'd like to see on our roadmap. What would enable you best to use s3gw in your environment? 
+
+Reach out to us at <s3gw@suse.com> or our [Slack channel](https://aquaristlabs.slack.com/archives/C03RFG0BES0). You can also join [our mailing list](https://lists.suse.com/mailman/listinfo/s3gw) or have a look at our [GitHub repository](https://github.com/aquarist-labs/s3gw) -- feature requests are welcome! 🙂

blog/2022-11-21-introduction-to-s3gw.md

@@ -0,0 +1,55 @@
+---
+title: Introduction to s3gw
+description: Introductory blog post on the s3gw Project, a standalone S3 service based in the RADOS Gateway project.
+slug: introduction-to-s3gw
+authors:
+  - name: The s3gw team
+tags: [blog, s3gw, Rancher, introduction]
+hide_table_of_contents: false
+---
+
+# What is s3gw?
+
+s3gw is an S3-compatible service focused on deployments in a Kubernetes environment backed by any PVC, including [Longhorn](https://longhorn.io/). Since its inception, the primary focus has been on cloud native deployments. However, the s3gw can be deployed in a myriad of scenarios, provided some form of storage is attached.
+
+<!--truncate-->
+
+s3gw is based on Ceph's RADOSGW (RGW) but runs as a stand--alone service without the RADOS cluster and relies on a storage backend still under heavy development by the storage team at SUSE. The s3gw team is also developing a web-based UI for management and an object explorer. 
+
+## The s3gw service
+
+Distributed as a small container, the s3gw service runs RGW and exposes an S3-compatible API. Instead of requiring a full Ceph cluster deployment, we leverage RGW's standalone capabilities and keep data on a local storage volume. Although the focus is on running within a Kubernetes environment with on-premise storage provided to the container, s3gw can consume any storage type with a filesystem on it. This can be PVC in Kubernetes or a local directory on a development machine. 
+
+As the container consumes the storage volume, the object data is kept in a hash tree of directories and the metadata is kept in an SQLite database. This allows us to leverage the ACID properties of SQLite to ensure the state is committed atomically while keeping large blobs of data on the filesystem and away from SQLite's path. 
+
+In the future, we will release a blog post describing the s3gw service's data store in more depth. 
+
+However, it should be noted that we don't support all RGW's S3 APIs. Some components are still under development and other features not yet included that are required for proper operation. 
+
+For example, while deleting objects and buckets is currently supported, we don't support lifecycle management. Lifecycle policies and IAM are some of the things that we will be working on shortly. 
+
+## The s3gw web UI
+
+Also distributed as a small container, the s3gw web UI provides an intuitive way of interacting with the s3gw service. This includes user and bucket management, as well as an object explorer. 
+
+We have a few screenshots of the current UI version below, but please keep in mind that we are still actively developing it and it is not feature-complete.
+
+![s3gw login page screenshot](https://www.suse.com/c/wp-content/uploads/2022/11/s3gw-login-1-1024x747.png)
+![s3gw dashboard screenshot](https://www.suse.com/c/wp-content/uploads/2022/11/s3gw-dashboard-1-1024x747.png)
+![s3gw bucket creation](https://www.suse.com/c/wp-content/uploads/2022/11/Screenshot-2022-11-21-at-15.49.46-1024x751.png)
+![s3gw bucket list dashboard screeenshot](https://www.suse.com/c/wp-content/uploads/2022/11/s3gw-list-buckets-1024x747.png)
+![s3gw file explorer](https://www.suse.com/c/wp-content/uploads/2022/11/s3gw-file-explorer-1-1024x747.png)
+
+##  Installing
+
+You may find our Helm chart helpful if you have a Kubernetes cluster, whichever flavor that might be. You'll be able to find it on [ArtifactHub](https://artifacthub.io/packages/helm/s3gw/s3gw), and our [documentation](https://s3gw-docs.readthedocs.io/en/latest/helm-charts/) can provide important insights as to available configuration values. 
+
+Alternatively, if you are using Rancher, you may find s3gw available in the Partner repository, as depicted below:
+
+![Partner repository](https://www.suse.com/c/wp-content/uploads/2022/11/Screenshot-2022-11-21-at-16.04.15-1024x372.png)
+
+## Call for action
+
+We would love to hear from you about what you'd like to see on our roadmap. What would enable you best to use s3gw in your environment? 
+
+Reach out to us at <s3gw@suse.com> or our [Slack channel](https://aquaristlabs.slack.com/archives/C03RFG0BES0). You can also join [our mailing list](https://lists.suse.com/mailman/listinfo/s3gw) or have a look at our [GitHub repository](https://github.com/aquarist-labs/s3gw) -- feature requests are welcome! 🙂