Creating staging instances on Amazon

To save costs, we don't keep our staging system online all the time. Instead, we make it reasonable to be able to spin it up through scripts.

It's a one-liner.

The one line

./script/launch-conglomerate.sh staging

And test away, at https://staging.overviewdocs.com

Shutting down

Run script/staging-stop.sh.

This will terminate all staging instances and associated volumes.

Prerequisites

Hehe. There are a few.

1. You must be in the latest checkout of aws-overview-tools.
2. You need to set some environment variables: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_KEYPAIR_NAME, AWS_DEFAULT_REGION and OVERVIEW_MANAGE_HOST. You should have a script somewhere that sets all five, and you should source it whenever you're about to interact with Overview.
3. The manage and logstash instances must exist.
4. There must be snapshots called [database] Daily backup and [searchindex] Daily backup.
5. There must be the security group staging-conglomerate.
6. There must be secrets in the overview-staging-secrets S3 bucket, and the staging-conglomerate role must be able to read them.
7. There must be a staging.zip in the overview-builds S3 bucket. (Use overview-manage publish to create it.)

Differences between staging and production

• The cookie domain is different. We don't have any cookies on overviewdocs.com: we only have them on www.overviewdocs.com and staging.overviewdocs.com. That ensures no overlaps.
• API keys to external services (Google Analytics, Intercom) are different.
• Production uses the "official" production-database and production-searchindex volumes. Staging loads their latest snapshots.
• Production reads production.zip and Staging reads staging.zip from the overview-builds S3 bucket.

Security concerns

• Our users' sensitive data is on the database-staging instance. If that database is hacked, we expose real data.

Shut down the staging environment when it isn't in use with script/staging-stop.sh