PGAdapter supports SQLAlchemy 2.0 with the psycopg driver. This document shows how to use this sample application, and lists the limitations when working with SQLAlchemy 2.x with PGAdapter.

The sample.py file contains a sample application using SQLAlchemy 2.x with PGAdapter. Use this as a reference for features of SQLAlchemy 2.x that are supported with PGAdapter. This sample assumes that the reader is familiar with SQLAlchemy 2.x, and it is not intended as a tutorial for how to use SQLAlchemy in general.

See Limitations for a full list of known limitations when working with SQLAlchemy 2.x.

You can run the sample directly on the Spanner emulator with the following command. It will automatically start both PGAdapter and the Spanner emulator. This requires Docker on the local machine to work:

python run_sample.py

You can also connect to a real Spanner instance instead of the emulator by running the sample like this. The database must exist. The sample will automatically create the tables that are needed for the sample:

python run_sample.py \
  --project my-project \
  --instance my-instance \
  --database my-database \
  --credentials /path/to/credentials.json

You can also start PGAdapter before you run the sample and connect to that PGAdapter instance. The following command shows how to start PGAdapter using the pre-built Docker image and then run the sample against that instance. See Running PGAdapter for more information on other options for how to run PGAdapter.

export GOOGLE_APPLICATION_CREDENTIALS=/path/to/credentials.json
docker pull gcr.io/cloud-spanner-pg-adapter/pgadapter
docker run \
  -d -p 5432:5432 \
  -v ${GOOGLE_APPLICATION_CREDENTIALS}:${GOOGLE_APPLICATION_CREDENTIALS}:ro \
  -e GOOGLE_APPLICATION_CREDENTIALS \
  -v /tmp:/tmp \
  gcr.io/cloud-spanner-pg-adapter/pgadapter \
  -p my-project -i my-instance \
  -x
python run_sample.py \
  --host localhost \
  --port 5432 \
  --database my-database

The sample data model is created automatically by the sample script.

The sample data model contains example tables that cover all supported data types the Cloud Spanner PostgreSQL dialect. It also includes an example for how interleaved tables can be used with SQLAlchemy. Interleaved tables is a Cloud Spanner extension of the standard PostgreSQL dialect.

The corresponding SQLAlchemy model is defined in model.py.

You can also create the sample data model manually using for example psql. Run the following command in this directory. Replace the host, port and database name with the actual host, port and database name for your PGAdapter and database setup.

psql -h localhost -p 5432 -d my-database -f create_data_model.sql

You can also drop an existing data model using the drop_data_model.sql script:

psql -h localhost -p 5432 -d my-database -f drop_data_model.sql

Cloud Spanner supports the following data types in combination with SQLAlchemy 2.x.

The following limitations are currently known:

Spanner supports the serial data type. Columns with this data type use a backing bit-reversed sequence to generate unique values that are safe to use as primary key values in Spanner. The values are not monotonically increasing.

You must set a default_sequence_kind for your database before you can create a column with data type serial. The create_data_model.sql file includes a statement to set the default.

The TicketSale model in this sample application uses a serial for auto-generated primary keys:

1. See model.py for the model definition.
2. See create_data_model.sql for the table definition.

See https://cloud.google.com/spanner/docs/primary-key-default-value#serial-auto-increment for more information on the serial data type in Spanner.

Python model definition:

class Singer(Base):
  id = Column(Integer, primary_key=True)
  name = Column(String(100))

singer = Singer(name="Alice")
session.add(singer)
session.commit()

Database option and table definition:

alter database db set spanner.default_sequence_kind='bit_reversed_positive';
create table if not exists singers (
  id   serial primary key,
  name varchar not null
);

INSERT ... ON CONFLICT ... are not supported by Cloud Spanner and should not be used. Trying to use https://docs.sqlalchemy.org/en/20/dialects/postgresql.html#sqlalchemy.dialects.postgresql.Insert.on_conflict_do_update or https://docs.sqlalchemy.org/en/20/dialects/postgresql.html#sqlalchemy.dialects.postgresql.Insert.on_conflict_do_nothing will fail.

Rolling back to a SAVEPOINT can fail if the transaction contained at least one query that called a volatile function or if the underlying data that has been accessed by the transaction has been modified by another transaction.

Only SELECT ... FOR UPDATE without any additional options is supported. The NOWAIT and SKIP LOCKED options are not supported.

SQLAlchemy will by default use read/write transactions for all database operations, including for workloads that only read data. This will cause Cloud Spanner to take locks for all data that is read during the transaction. It is recommended to use either autocommit or read-only transactions for workloads that are known to only execute read operations. Read-only transactions do not take any locks. You can create a separate database engine that can be used for read-only transactions from your default database engine by adding the postgresql_readonly=True execution option.

read_only_engine = engine.execution_options(postgresql_readonly=True)

Using isolation level AUTOCOMMIT will suppress the use of (read/write) transactions for each database operation in SQLAlchemy. Using autocommit is more efficient than read/write transactions for workloads that only read and/or that do not need the atomicity that is offered by transactions.

You can create a separate database engine that can be used for workloads that do not need transactions by adding the isolation_level="AUTOCOMMIT" execution option to your default database engine.

autocommit_engine = engine.execution_options(isolation_level="AUTOCOMMIT")

Read-only transactions and database engines using AUTOCOMMIT will by default use strong reads for queries. Cloud Spanner also supports stale reads.

• A strong read is a read at a current timestamp and is guaranteed to see all data that has been committed up until the start of this read. Spanner defaults to using strong reads to serve read requests.
• A stale read is read at a timestamp in the past. If your application is latency sensitive but tolerant of stale data, then stale reads can provide performance benefits.

See also https://cloud.google.com/spanner/docs/reads#read_types

You can create a database engine that will use stale reads in autocommit mode by adding the following to the connection string and execution options of the engine:

conn_string = "postgresql+psycopg://user:password@localhost:5432/my-database" \
              "?options=-c spanner.read_only_staleness='MAX_STALENESS 10s'"
engine = create_engine(conn_string).execution_options(isolation_level="AUTOCOMMIT")