Skip to content

Feat/zsh #32

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 5 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 2 additions & 3 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ ENV VERSION_LOKI="2.9.5"
# Update the system and install required packages
RUN apt-get update -y && \
apt-get upgrade -y && \
apt-get install -y curl unzip groff-base less gnupg2 git jq && \
apt-get install -y curl unzip groff-base less gnupg2 git jq zsh && \
rm -rf /var/lib/apt/lists/*

# Install specific AWS CLI version
Expand Down Expand Up @@ -46,12 +46,11 @@ ADD rclone.conf /root/.config/rclone/rclone.conf
RUN mkdir /app

ADD github-backup.sh /usr/bin/backup-github
ADD gdrive-backup.sh /usr/bin/backup-gdrive
ADD loki-logcli-backup.sh /usr/bin/backup-loki-logs-as-json
ADD vault-backup.sh /usr/bin/backup-vault
ADD s3-backup.sh /usr/bin/s3-backup


WORKDIR /app

CMD ["bash"]
CMD ["zsh"]
10 changes: 5 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@

- AWS S3 Credentials. Regardless of whether you backup google team drives or github repos, you will need these environment variables to be set:

```bash
```zsh
export AWS_DEFAULT_REGION=us-west-2 # must be the same region as the bucket
export S3_BUCKET_NAME="bucket-name"
export AWS_ACCESS_KEY_ID=XXXXXXXXXXXXXXXXXXXXX
Expand All @@ -14,7 +14,7 @@ export AWS_SECRET_ACCESS_KEY=XXXXXXXXXXXXXXXXXXXXX

Following variables must be set

```bash
```zsh
export GITHUB_ORG_TO_BACKUP="GlueOps" # Set this to the organization you want to backup. The GITHUB_TOKEN must have read access to all the repos in this organization.
export GITHUB_TOKEN="" # GitH needs to have read access to all repositories within the organization. We use the fine grained access tokens (beta feature)
```
Expand All @@ -25,7 +25,7 @@ export GITHUB_TOKEN="" # GitH needs to have read access to all repositories with

- Example backup

```bash
```zsh
docker build . -t backup && docker run -it backup
# Export ALL the variables required as mentioned in this README.md and then run:
backup-github
Expand All @@ -40,14 +40,14 @@ backup-github

Following variables must be set

```bash
```zsh
export RCLONE_DRIVE_SERVICE_ACCOUNT_CREDENTIALS='<<json-without \n (newlines)>>' # Get this from the IAM user in the rclone google cloud service account project and remove all newlines \n
export RCLONE_DRIVE_TEAM_DRIVE="XXXXXXXXXXXXXX" # team drive id ex. `0ZZH9DD53YuyEaYU7sqb`
```

- Example to run a download of the team drive to local

```bash
```zsh
docker build . -t backup && docker run -it backup
# Export ALL the variables required as mentioned in this README.md and then run:
./gdrive-backup.sh
Expand Down
27 changes: 0 additions & 27 deletions gdrive-backup.sh

This file was deleted.

5 changes: 3 additions & 2 deletions github-backup.sh
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
#!/bin/bash
#!/bin/zsh

set -e
setopt ERR_EXIT
setopt PIPE_FAIL

# Check if required variables are set
if [[ -z "${GITHUB_ORG_TO_BACKUP}" ]]; then
Expand Down
7 changes: 4 additions & 3 deletions loki-logcli-backup.sh
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
#!/bin/bash
set -e
#!/bin/zsh
setopt ERR_EXIT
setopt PIPE_FAIL

cleanup() {
echo "Cleaning up..."
Expand Down Expand Up @@ -93,4 +94,4 @@ done
if [ "$ERRORS" -gt 0 ]; then
echo "ERROR: Found $ERRORS errors. Exiting the script."
exit 1
fi
fi
5 changes: 3 additions & 2 deletions s3-backup.sh
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
#!/bin/bash
set -e
#!/bin/zsh
setopt ERR_EXIT
setopt PIPE_FAIL

# Ensure all required environment variables are set
if [ -z "$SRC_AWS_ACCESS_KEY_ID" ] || [ -z "$SRC_AWS_SECRET_ACCESS_KEY" ] || [ -z "$DST_AWS_ACCESS_KEY_ID" ] || [ -z "$DST_AWS_SECRET_ACCESS_KEY" ] || [ -z "$SRC_BUCKET" ] || [ -z "$DST_BUCKET" ]; then
Expand Down
12 changes: 6 additions & 6 deletions test-vault-snapshot/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,8 @@ If you make an error in the process then it's recommended to delete everything,

Start the container with:

```bash
docker run -it -p 8200:8200 -v `pwd`/data:/data backup bash
```zsh
docker run -it -p 8200:8200 -v `pwd`/data:/data backup zsh
```

Run the following script to:
Expand All @@ -31,7 +31,7 @@ Run the following script to:
- Unseal vault
- Login to vault with the root token

```bash
```zsh
mkdir -p /data/raft
cat > /data/raft/peers.json << EOF
[
Expand Down Expand Up @@ -60,21 +60,21 @@ vault login $root_token

Create an S3 PRESIGNED URL to the backup that we want to restore. Limit the time to 10mins or whatever you feel is appropriate. And then export the url as a variable in the same terminal session from above.

```bash
```zsh
S3_PRESIGNED_DOWNLOAD_URL="https://time-sensitive-and-authenticated-url-to-download-backup-from-s3"
curl -o backup_to_restore.snap $S3_PRESIGNED_DOWNLOAD_URL
vault operator raft snapshot restore -force backup_to_restore.snap
```

# In the same terminal session unseal with the unseal token for the backup you just restored

```bash
```zsh
vault operator unseal 55ebb6859b269cd1ce501989ebba821baf84076c28d84008474aa3fddc0a24b3
```

# In the same terminal session, login with the root token for the backup you unsealed/restored.

```bash
```zsh
vault login hvs.fm0DOOSsPTwqB7rFFNbJgCle
```

Expand Down
6 changes: 3 additions & 3 deletions vault-backup.sh
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
#!/bin/bash

set -e
#!/bin/zsh

setopt ERR_EXIT
setopt PIPE_FAIL

date=$(date '+%Y-%m-%d')
echo "Starting Vault backup...@ ${date}"
Expand Down