Resource usage, vertical scaling

[notramo]

I ran a simple benchmark on a 2 core machine, but the CPU isn't utilized to 100%.
TypeScript client, query builder.
Query an object, filter_single by randomly selected id. All ids are queried before into a JS array, and selected randomly.
Few thousands of queries are started concurrently, then being await-ed after all has been launched (Promise.all()).

I don't know where the bottleneck is.

Is the compiler single-threaded? Shouldn't be an issue, as I assume EdgeDB caches the query. I use query parameter for the id, so the query can be fully cached.

The EdgeDB instance is running in Docker.

Is there some parameter I could tune to utilize my CPU to 100%?

I am considering to use EdgeDB for a new project that could involve very high system load. Single thread can be a blocker, causing an ORM and Postgres/MariaDB being used for the new project.

Can an EdgeDB Docker image fully utilize a 32 or 64 core CPU?

[elprans]

Take a look at the benchmark setup we have in IMDBench, the methodology is expanded in this blog post.

Without looking at the code it's hard to say where the bottleneck in your benchmark could be. It might be a Docker issue, especially on macOS.

[notramo]

The IMDBench setup doesn't specify any extra Docker configuration. The container is running on a Linux host.

I don't think the bottleneck is in the TypeScript code, but in the EdgeDB server config.

import e from './src/edgeql-js/';
import { createClient } from 'edgedb';

const client = createClient();

// pre-build EdgeQL to avoid wasting time with it in the
// loop
const query = e.params(
  {
    id: e.uuid
  },
  (params) =>
    e.select(e.User, () => ({
      ...e.User['*'],
      filter_single: { id: params.id }
    }))
);

// query a list of all objects
const objectIdList = await e
  .select(e.User, () => ({
    id: true
  }))
  .run(client);

// number of queries used in the benchmark
const len = 10000;

// store randomized ids
const randomizedIdList: string[] = new Array(len);

// fill the id list with randomly selected ids
for (let i = 0; i < len; ++i) {
  randomizedIdList[i] =
    objectIdList[Math.floor(Math.random() * objectIdList.length)].id;
}

const startTime = Date.now();

// start all queries concurrently, without awaiting any
const promises = randomizedIdList.map((id) =>
  query.run(client, { id })
);

// wait for all query to finish
await Promise.all(promises);

console.debug('total time:', (Date.now() - startTime) / 1000, 'seconds');

[elprans]

The first thing I would suggest is running the server without Docker to see if that makes a difference. You could also set the builtin client concurrency to a higher value, e.g createClient({concurrency: 200}).

[notramo]

I tried increasing concurrency, but there was no significant change.
I noticed that the bun process fully saturates a CPU core (probably deserialization), so I tried with 2 Bun process, and the CLI too.

time edgedb > /dev/null < bench.edgeql

bench.edgeql content:

select User { * } filter .id = <uuid>'...';
# this repeated 50 000 times

Despite these efforts, I couldn't get the EdgeDB to fully utilize the CPU.

I looked at the process resources in htop, and I found a Python process, which showed significant CPU usage, but did not fully use the CPU.

/usr/lib/x86_64-linux-gnu/edgedb-server-3/bin/python3 -I /usr/lib/x86_64-linux-gnu/edgedb-server-3/bin/edgedb-server.py

I guess EdgeDB can not effectively utilize a multi-core CPU because it's written in Python.

[elprans]

I noticed that the bun process fully saturates a CPU core (probably deserialization)

Well then that's your bottleneck. I suggest you experiment with imdbench, as it has an apples-to-apples setup, including raw Postgres and different clients.

I guess EdgeDB can not effectively utilize a multi-core CPU because it's written in Python.

You can run multiple EdgeDB processes against the same Postgres cluster and scale it that way if you find query throughput insufficient in practice. For context, here's (a fairly old, but still relevant) benchmark showing throughput of 14,000 queries per second. You can use that as a ballpark of how much a single EdgeDB process can handle before saturating the CPU.

[notramo]

Well then that's your bottleneck. I suggest you experiment with imdbench, as it has an apples-to-apples setup, including raw Postgres and different clients.

I have tried with multiple Bun processes, and also with multiple CLI processes, so this bottleneck is eliminated, but then it starts to slow down too. This time the bottleneck is the EdgeDB process. 2 Bun processes don't saturate the CPU, but it's not 100% utilized either.

You can run multiple EdgeDB processes against the same Postgres cluster and scale it that way if you find query throughput insufficient in practice. […]

Does it mean that EdgeDB don't do internal caching, so all processes can freely mutate the data, and it's synchronized? Where can I find guide for setting it up?

The baseline would be 100 000 queries per second for considering EdgeDB in a particular project. It doesn't matter if it's 1 core or 64 core, or multiple machines, the only criteria is that it have to be achieved somehow.