Skip to content

Commit

Permalink
WIP
Browse files Browse the repository at this point in the history
  • Loading branch information
fmassot committed Oct 28, 2024
1 parent 01c3dd0 commit 2d13b2d
Show file tree
Hide file tree
Showing 4 changed files with 215 additions and 10 deletions.
202 changes: 202 additions & 0 deletions docs/deployment/kubernetes/aws-marketplace.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,202 @@
---
title: AWS Marketplace Install
sidebar_label: AWS Markeplace
sidebar_position: 5
---

This guide will help you install Quickwit on EKS from the AWS marketplace.

## Prerequisites
- Running Elastic Kubernetes cluster (EKS)
- `kubectl`
- Permission to create the IAM role and Policies
- AWS CLI
- `eksctl` if you don't have an IAM OIDC provider for your cluster.

## Target platforms
Quickwit containers can be run in ECS (including Fargate), or EKS.

## Set up

Let's use the following environment variables:

```bash
export NAMESPACE=qw-tutorial
export EKS_CLUSTER=qw-cluster
export S3_BUCKET={your-bucket}
export SERVICE_ACCOUNT_NAME=qw-sa
export REGION={your-region}
export CLUSTER_ID={your-cluster-id}
```

Create the namespace for our playground:

```bash
kubectl create ns ${NAMESPACE}
```

And set this namespace as the default one:

```bash
kubectl config set-context --current --namespace=${NAMESPACE}
```


### Create IAM OIDC provider if you don't have one

To check if you have one provider for your EKS cluster, just run:

```bash
aws iam list-open-id-connect-providers
```

If you have one, you will get a response similar to this one:

```json
{
"OpenIDConnectProviderList": [
{
"Arn": "arn:aws:iam::(some-ID):oidc-provider/oidc.eks.{your-region}.amazonaws.com/id/{your-cluster-id}"
}
]
}
```

If you don't, run the following command:

```bash
eksctl utils associate-iam-oidc-provider --cluster ${EKS_CLUSTER} --approve
```

You can run again `aws iam list-open-id-connect-providers` to get the ARN of the provider.

### Create an IAM policy

You need to set the following policy to allow Quickwit to access your S3 bucket.

Then create the policy using the AWS CLI:

```bash
cat > s3-policy.json <<EOF
{
"Statement": [
{
"Action": [
"s3:ListBucket"
],
"Effect": "Allow",
"Resource": [
"arn:aws:s3:::${S3_BUCKET}"
]
},
{
"Action": [
"s3:AbortMultipartUpload",
"s3:DeleteObject",
"s3:GetObject",
"s3:ListMultipartUploadParts",
"s3:PutObject"
],
"Effect": "Allow",
"Resource": [
"arn:aws:s3:::${S3_BUCKET}/*"
]
}
],
"Version": "2012-10-17"
}
EOF
```

```bash
aws iam create-policy --policy-name qw-s3-policy --policy-document file://s3-policy.json
```

### Create an IAM Role and attach the policy

```bash
cat > s3-role.json << EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Federated": "arn:aws:iam::${IAM_ID}:oidc-provider/oidc.eks.${REGION}.amazonaws.com/id/${CLUSTER_ID}"
},
"Action": "sts:AssumeRoleWithWebIdentity",
"Condition": {
"StringEquals": {
"oidc.eks.${REGION}.amazonaws.com/id/${CLUSTER_ID}:aud": "sts.amazonaws.com",
"oidc.eks.${REGION}.amazonaws.com/id/${CLUSTER_ID}:sub": "system:serviceaccount:${NAMESPACE}:${SERVICE_ACCOUNT_NAME}"
}
}
}
]
}
EOF
```

```bash
aws iam create-role --role-name s3-role --assume-role-policy-document file://s3-role.json
```

And then attach the policy to the role:

```bash
aws iam attach-role-policy --role-name s3-role --policy-arn=arn:aws:iam::${IAM_ID}:policy/s3-policy
```

## Install Quickwit using Helm

We are now ready to install Quickwit on EKS. If you'd like to know more about Helm, consult our [comprehensive guide](./helm.md) for installing Quickwit on Kubernetes.

```bash
helm repo add quickwit https://helm.quickwit.io
helm repo update quickwit
```

Let's set Quickwit `values.yaml`:

```yaml
image:
repository: quickwit/quickwit
pullPolicy: Always

serviceAccount:
create: true
name: ${SERVICE_ACCOUNT_NAME}
annotations:
eks.amazonaws.com/role-arn: arn:aws:iam::${ARN_ID}:role/${SERVICE_ACCOUNT_NAME}

config:
default_index_root_uri: s3://${S3_BUCKET}/qw-indexes
metastore_uri: s3://${S3_BUCKET}/qw-indexes

```

We're ready to deploy:

```bash
helm install <deployment name> quickwit/quickwit -f values.yaml
```

## Check that Quickwit is running

It should take a few seconds for the cluster to start. During the startup process, individual pods might restart themselves several times.

To access the UI, you can run the following command and then open your browser at [http://localhost:7280](http://localhost:7280):

```
kubectl port-forward svc/quickwit-searcher 7280:7280
```

## Uninstall the deployment

Run the following Helm command to uninstall the deployment

```bash
helm uninstall <deployment name>
```

And don't forget to clean your bucket, Quickwit should have stored 3 files in `s3://${S3_BUCKET}/qw-indexes`.
14 changes: 12 additions & 2 deletions docs/deployment/kubernetes/eks.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
---
title: EKS + Helm
title: Install Quickwit on AWS EKS
sidebar_label: AWS EKS
sidebar_position: 3
---

Expand Down Expand Up @@ -76,6 +77,15 @@ Then create the policy using the AWS CLI:
cat > s3-policy.json <<EOF
{
"Statement": [
{
"Action": [
"s3:ListBucket"
],
"Effect": "Allow",
"Resource": [
"arn:aws:s3:::${S3_BUCKET}"
]
},
{
"Action": [
"s3:AbortMultipartUpload",
Expand Down Expand Up @@ -177,7 +187,7 @@ It should take a few seconds for the cluster to start. During the startup proces
To access the UI, you can run the following command and then open your browser at [http://localhost:7280](http://localhost:7280):

```
kubectl port-forward svc/{release-name}-quickwit-searcher 7280:7280
kubectl port-forward svc/quickwit-searcher 7280:7280
```

## Uninstall the deployment
Expand Down
7 changes: 0 additions & 7 deletions docs/deployment/kubernetes/gke.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,15 +65,8 @@ image:
tag: edge

serviceAccount:
<<<<<<< HEAD
create: false
name: quickwit-sa
=======
create: true
name: ${GKE_SERVICE_ACCOUNT}
annotations:
iam.gke.io/gcp-service-account: ${GCP_SERVICE_ACCOUNT}@${PROJECT_ID}.iam.gserviceaccount.com
>>>>>>> 99c372cec (Add EKS tutorial with service account setup.)

config:
default_index_root_uri: gs://${BUCKET}/qw-indexes
Expand Down
2 changes: 1 addition & 1 deletion docs/deployment/kubernetes/glasskube.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: Install Quickwit with Glasskube
sidebar_label: Glasskube
sidebar_position: 3
sidebar_position: 4
---

[Glasskube](https://glasskube.dev) is a package manager for Kubernetes that empowers you to effortlessly install, upgrade, configure, and manage your Kubernetes cluster packages, all while streamlining repetitive and cumbersome maintenance tasks.
Expand Down

0 comments on commit 2d13b2d

Please sign in to comment.