Gratipay helps companies pay for open source, in order to cultivate an economy of gratitude, generosity, and love.

Scope	Documentation
company policies, procedures, etc.	http://inside.gratipay.com
product customer-facing pages	https://gratipay.com/about
software installation	← You are here!
python library	https://gratipay.readthedocs.io/

Table of Contents

• Quick Start
• Installing
  • Satisfying Dependencies
  • Setting up a Database
  • Building
  • Launching
  • Help!
• Installing with Vagrant
• Installing with Docker
• Configuring
• Developing
  • Codebase Overview
  • Modifying CSS and JavaScript
  • Modifying the Database
• Testing
• API
  • Implementations
• Glossary
• License

Quick Start

Thanks for hacking on Gratipay! Be sure to review CONTRIBUTING as well if that's what you're planning to do.

Unix-like

Given Python 2.7, Postgres 9.6, and a C/make toolchain:

git clone https://github.com/gratipay/gratipay.com.git
cd gratipay.com
createdb gratipay
make schema fake

Now make run to boot the app or make test to run the tests.

Read more.

Vagrant

Given VirtualBox 4.3 and Vagrant 1.7.x:

vagrant up

Read more.

Docker

Given some version(?) of Docker:

docker build -t gratipay .
docker run -p 8537:8537 gratipay

Read more.

Installing

Satisfying Dependencies

Building, launching, developing and testing gratipay.com requires several pieces of software:

• a C/make toolchain,
• Python version 2.7,
• Postgres version 9.6, and
• Firefox and geckodriver for testing.

Unix-like operating systems (Ubuntu, macOS, etc.) generally include a C/make toolchain. If you're on Windows, your best bet is to use Vagrant or Docker.

All Python dependencies are bundled in our repo (under vendor/), but some include C extensions with additional operating-system level dependencies that need to be met. Here are notes for psycopg2. Other candidates for trouble are libsass and cryptography. Good luck!

Debian/Ubuntu

Maybe try scripts/bootstrap-debian.sh?

macOS

If make env gives you an Operation not permitted error from shutil.copytree then you're probably using the system Python and you should try Homebrew instead:

brew install python

Here are the installation options for Postgres.

If you are getting an error about unknown argument: '-mno-fused-madd' when running make, then add Wno-error=unused-command-line-argument-hard-error-in-future to your ARCHFLAGS environment variable and run make clean env again (see this Stack Overflow answer for more information):

ARCHFLAGS=-Wno-error=unused-command-line-argument-hard-error-in-future make clean env

Setting up a Database

The best version of Postgres to use is 9.6.2, because that's what we're using in production at Heroku. You need at least 9.5 to support the features we depend on, along with the pg_stat_statements and pg_trgm extensions.

To setup Postgres for Gratipay's needs run:

sudo -u postgres createuser --superuser $USER
createdb gratipay
createdb gratipay-test

You can speed up the test suite when using a regular HDD by running:

psql -q gratipay-test -c 'alter database "gratipay-test" set synchronous_commit to off'

Schema

Once Postgres is set up, run:

make schema

Which populates the database named by DATABASE_URL with the schema from sql/schema.sql.

Example data

The gratipay database created in the last step is empty. To populate it with some fake data, so that more of the site is functional, run this command:

make fake

Building

All Python dependencies (including virtualenv) are bundled with Gratipay in the vendor/ directory. Gratipay is designed so that you don't manage its virtualenv (a Python-specific sandboxing mechanism) directly and you don't download its dependencies at build time but rather at clone time. To create a virtualenv with all Python dependencies installed:

make env

If you haven't run Gratipay for a while, you can reinstall the dependencies:

make clean env

Add the necessary schemas and insert dummy data into postgres:

make schema
make fake

Launching

Once you've installed Python and Postgres and set up a database, you can use make to build and launch Gratipay:

make run

If you don't have make, look at the Makefile to see what steps you need to perform to build and launch Gratipay. The Makefile is pretty simple and straightforward.

If Gratipay launches successfully it will look like this:

$ make run
PATH=env/bin:{lots-more-of-your-own-PATH} env/bin/honcho run -e defaults.env,local.env web
[2017-08-25 15:05:18 -0400] [18093] [INFO] Starting gunicorn 19.7.1
[2017-08-25 15:05:18 -0400] [18093] [INFO] Listening at: http://0.0.0.0:8537 (18093)
[2017-08-25 15:05:18 -0400] [18093] [INFO] Using worker: sync
[2017-08-25 15:05:18 -0400] [18096] [INFO] Booting worker with pid: 18096
pid-18096 thread-140736833041344 (MainThread) Instantiating Application from gunicorn_entrypoint
pid-18096 thread-140736833041344 (MainThread) Reading configuration from defaults, environment, and kwargs.
pid-18096 thread-140736833041344 (MainThread)   base_url                                              default                 
pid-18096 thread-140736833041344 (MainThread)   changes_reload         False                          default                 
pid-18096 thread-140736833041344 (MainThread)   changes_reload         True                           environment variable ASPEN_CHANGES_RELOAD=yes
pid-18096 thread-140736833041344 (MainThread)   charset_dynamic        UTF-8                          default                 
pid-18096 thread-140736833041344 (MainThread)   charset_static         None                           default                 
pid-18096 thread-140736833041344 (MainThread)   colorize_tracebacks    True                           default                 
pid-18096 thread-140736833041344 (MainThread)   indices                [u'index.html', u'index.json', u'index', u'index.html.spt', u'index.json.spt', u'index.spt'] default                 
pid-18096 thread-140736833041344 (MainThread)   list_directories       False                          default                 
pid-18096 thread-140736833041344 (MainThread)   logging_threshold      0                              default                 
pid-18096 thread-140736833041344 (MainThread)   media_type_default     text/plain                     default                 
pid-18096 thread-140736833041344 (MainThread)   media_type_json        application/json               default                 
pid-18096 thread-140736833041344 (MainThread)   project_root           None                           default                 
pid-18096 thread-140736833041344 (MainThread)   project_root           .                              environment variable ASPEN_PROJECT_ROOT=.
pid-18096 thread-140736833041344 (MainThread)   renderer_default       stdlib_percent                 default                 
pid-18096 thread-140736833041344 (MainThread)   show_tracebacks        False                          default                 
pid-18096 thread-140736833041344 (MainThread)   show_tracebacks        True                           environment variable ASPEN_SHOW_TRACEBACKS=yes
pid-18096 thread-140736833041344 (MainThread)   www_root               None                           default                 
pid-18096 thread-140736833041344 (MainThread)   www_root               www/                           environment variable ASPEN_WWW_ROOT=www/
pid-18096 thread-140736833041344 (MainThread) project_root is relative to CWD: '.'.
pid-18096 thread-140736833041344 (MainThread) project_root set to /Users/whit537/personal/gratipay/gratipay.com.
pid-18096 thread-140736833041344 (MainThread) Found plugin for renderer 'jinja2'
pid-18096 thread-140736833041344 (MainThread) Renderers (*ed are unavailable, CAPS is default):
pid-18096 thread-140736833041344 (MainThread)   json_dump        
pid-18096 thread-140736833041344 (MainThread)   jsonp_dump       
pid-18096 thread-140736833041344 (MainThread)   stdlib_template  
pid-18096 thread-140736833041344 (MainThread)   stdlib_format    
pid-18096 thread-140736833041344 (MainThread)   jinja2           
pid-18096 thread-140736833041344 (MainThread)   STDLIB_PERCENT   
pid-18096 thread-140736833041344 (MainThread) Won't log to Sentry (SENTRY_DSN is empty).
pid-18096 thread-140736833041344 (MainThread) AWS SES is not configured! Mail will be dumped to the console here.
pid-18096 thread-140736833041344 (MainThread) Cron: not installing update_cta.
pid-18096 thread-140736833041344 (MainThread) Cron: not installing self_check.
pid-18096 thread-140736833041344 (MainThread) Cron: not installing <lambda>.
pid-18096 thread-140736833041344 (MainThread) Cron: not installing flush.
pid-18096 thread-140736833041344 (MainThread) Cron: not installing log_metrics.

You should then find this in your browser at http://localhost:8537/:

Congratulations! Now enter a dollar amount less than 2000 (ironically), and submit the form to complete the basic flow:

You're off and running! At some point, try running the test suite.

Help!

If you get stuck somewhere along the way, make an issue here on GitHub.

Thanks for installing Gratipay! 😃

Installing with Vagrant

Vagrant provides a convenient interface to VirtualBox to run and test Gratipay in virtual machine. This may be handy if you're on Windows.

You will need Vagrant and VirtualBox installed. On Linux you may need to install nfs-kernel-server as well.

With Vagrant, you can run Gratipay by running vagrant up from the project directory. Please note that if you ever switch between running Gratipay on your own machine to Vagrant or vice versa, you will need to run make clean.

The Vagrantfile will download a pristine Ubuntu image (base box), save it, and create a virtual machine (VM) in VirtualBox. Then it will set up Gratipay prerequisites (the process is known as "provisioning") and show a welcome message.

The next time you run vagrant up, it will reuse the VM. Vagrant uses SSH based authentication. To login to VM, use the vagrant ssh command. If you're prompted for a password when logging in, please use vagrant.

Mac users: If you're prompted for a password during initial installation, it's sudo and you should enter your Mac OS password.

Ubuntu users: If you experience problems, please see this issue. As mentioned there, you will also need to be wary of projects that are nested in encrypted directories.

Installing with Docker

You can also install/run Gratipay with Docker.

Build it with the included Dockerfile:

$ git clone https://github.com/gratipay/gratipay.com.git
$ cd gratipay.com
$ docker build -t gratipay .

Once you've built the image, you can launch a container:

$ docker run -d -p 8537:8537 gratipay

Check it out at localhost:8537!

To edit files and have those changes reflect in the running container, mount your local folder when you execute the run command:

$ docker run -d -v $PWD:/srv/gratipay.com -p 8537:8537 gratipay

You can get the running container's ID with docker ps. With that, you can

• view the logs:

$ docker logs [container_id]

• run commands within the project root:

$ docker exec [container_id] make schema
$ docker exec [container_id] make fake

Once you're done, kill the running container:

$ docker kill [container_id]

Configuring

Gratipay's default configuration lives in defaults.env. If you'd like to override some settings, create a file named local.env to store them.

The following explains some of the content of that file:

The GITHUB_* keys are for a gratipay-dev application in the Gratipay organization on Github. It points back to localhost:8537, which is where Gratipay will be running if you start it locally with make run. Similarly with the TWITTER_* keys, but there they required us to spell it 127.0.0.1.

If you are running Gratipay somewhere other than localhost:8537, then you'll need to set BASE_URL, but your options are limited because we use proprietary fonts from Typography.com, and they filter by Referer. You won't get the right fonts unless you use an approved domain. We've configured gratipay.dev as well as localhost, so if you don't want to run on localhost then configure gratipay.dev in your /etc/hosts file and set this in local.env:

BASE_URL=http://gratipay.dev:8537
GITHUB_CLIENT_ID=ca4a9a35c161af1d024d
GITHUB_CLIENT_SECRET=8744f6333d51b5f4af38d46cf035ecfcf34c671e
GITHUB_CALLBACK=http://gratipay.dev:8537/on/github/associate

If you wish to use a different username or database name for the database, you should override the DATABASE_URL in local.env using the following format:

DATABASE_URL=postgres://<username>@localhost/<database name>

We use Amazon Web Services' Simple Email Service (AWS SES) for sending emails. In development, we dump outbound mail to the console by default. This is fine if all you need to do is, e.g., copy/paste verification links. If you need to receive emails within a proper mail client during development, then sign up for AWS's free tier and override the AWS_* credentials from defaults.env in your local.env. You'll have to verify the email addresses you want to receive email with on SES.

Developing

Codebase Overview

Directory	Frontend	Backend	Description
www	✅		web requests land here, e.g., https://gratipay.com/on/npm/express hits www/on/npm/%platform.spt (a simplate)
js scss	✅		JavaScript (w/ jQuery) and SCSS files, dynamically pipelined via endpoints at www/assets/gratipay.js.spt and .css.spt
templates emails	✅		templating files for web and email, respectively, using Jinja
gratipay		✅	a Python library with app wiring, models, and business logic
sql		✅	SQL files, the main one is schema.sql, changes go in a branch.sql, but there's also lots of raw SQL in Python strings throughout gratipay and even www
tests	✅	✅	test scripts, tests/ttw run "through the web" on a real browser, tests/py simulate HTTP calls and exercise Python APIs
gratipay/testing	✅	✅	submodule for infrastructure used by test scripts

Modifying CSS and JavaScript

We use SCSS, with files stored in scss/. All of the individual files are combined in scss/gratipay.scss which itself is compiled by libsass in www/assets/gratipay.css.spt on each request (it's behind a CDN in production).

We use a similar pattern for JavaScript. Individual files are in js/, and they're concatenated on the fly (and put behind a CDN in production) in www/assets/gratipay.js.spt.

Modifying the Database

We write SQL, specifically the PostgreSQL variant. To make schema or data changes, use deploy hooks.

Testing

Run Gratipay's test suite with:

make test

This invokes the pyflakes linter and then the pytest test runner with four layers of configuration (last wins): defaults.env, local.env, tests/defaults.env, tests/local.env. To run a subset of the test suite or otherwise influence the test run, pass arguments to py.test using an ARGS environment variable like so:

ARGS="tests/py/test_billing_payday.py -k notify -vvv" make test

The tests in tests/ttw ("through the web") require Firefox and geckodriver. The tests in tests/py do not.

Be careful! The test suite deletes data in all tables in the public schema of the database configured in your testing environment.

API

The Gratipay API is comprised of these four endpoints:

/about/charts.json (source)—public—Returns an array of objects, one per week, showing aggregate numbers over time. The stats page uses this.

/about/paydays.json (source)—public—Returns an array of objects, one per week, showing aggregate numbers over time. The old charts page used to use this.

/~username/public.json (example, source)—public—Returns an object with these keys:

• "taking"—an estimate of the amount the given participant will take from Teams this week

• "elsewhere"—participant's connected accounts elsewhere; returns an object with these keys:

  • "bitbucket"—participant's Bitbucket account; possible values are:
    • undefined (key not present)—no Bitbucket account connected
    • https://bitbucket.org/api/1.0/users/%bitbucket_username
  • "github"—participant's GitHub account; possible values are:
    • undefined (key not present)—no GitHub account connected
    • https://api.github.com/users/%github_username
  • "twitter"—participant's Twitter account; possible values are:
    • undefined (key not present)—no Twitter account connected
    • https://api.twitter.com/1.1/users/show.json?id=%twitter_immutable_id&include_entities=1
  • "openstreetmap"—participant's OpenStreetMap account; possible values are:
    • undefined (key not present)—no OpenStreetMap account connected
    • http://www.openstreetmap.org/user/%openstreetmap_username

/~username/payment-instructions.json (source)—private—Responds to GET with an array of objects representing your current payment instructions. A payment instruction is created when a ~user instructs Gratipay to make voluntary payments to a Team. Pass a team_slug with GET to fetch payment instruction only for that particular team. POST an array of objects containing team_slug and amount to bulk upsert payment instructions (make sure to set Content-Type to application/json). The amount must be encoded as a string rather than a number. In case the upsert is not successful for any object, there will be an error attribute in the response explaining the error along with the team_slug to identify the object for which the error occurred.

This endpoint requires authentication. Look up your user ID and API key on your account page and pass them using basic auth.

E.g.: Request

curl https://gratipay.com/~username/payment-instructions.json \
    -u $userid:$api_key \
    -X POST \
    -d '[{"amount": "1.00", "team_slug": "foobar"}]' \
    -H "Content-Type: application/json"

Response

[
    {
        "amount": "1.00",
        "ctime": "2016-01-30T12:38:00.182230+00:00",
        "due": "0.00",
        "mtime": "2016-02-06T14:37:28.532508+00:00",
        "team_name": "Foobar team",
        "team_slug": "foobar"
    }
]

API Implementations

Below are some projects that use the Gratipay APIs, that can serve as inspiration for your project!

Renamed to Gratipay

• Ruby: gratitude: a simple ruby wrapper for the Gratipay API

• php-curl-class: a php class to tip using the Gratipay API

• gratipay-twisted: Gratipay client for the Twisted framework

• WordPress: WP-Gratipay: a simple way to show a Gratipay widget on your WordPress site

Still Using Gittip

These probably still work, but are using our old name:

• Drupal: Gittip: Includes a Gittip giving field type to let you implement the Khan academy model for users on your Drupal site. (ticket)

• Node.js: Node-Gittip (also see Khan Academy's setup) (ticket)

• hubot-gittip: A Hubot script for interacting with a shared Gratipay account. (ticket)

• gittip-collab: A Khan-style tool for managing a Gittip account as a team. (ticket)

• WWW::Gittip: A Perl module implementing the Gittip API more or less (ticket)

Glossary

Account Elsewhere - An entity's registration on a platform other than Gratipay (e.g., Twitter).

Entity - An entity.

Participant - An entity registered with Gratipay.

User - A person using the Gratipay website. Can be authenticated or anonymous. If authenticated, the user is guaranteed to also be a participant.

License

MIT