Skip to content
This repository was archived by the owner on Mar 17, 2025. It is now read-only.

Commit 3fb8dfc

Browse files
daveadsdaveads
committed
Merge branch 'main' into analyze_port
2 parents 9cccb7c + da28f61 commit 3fb8dfc

File tree

9 files changed

+67
-58
lines changed

9 files changed

+67
-58
lines changed

‎.github/workflows/bench.yml‎

Lines changed: 19 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -17,14 +17,24 @@ jobs:
1717
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
1818
strategy:
1919
matrix:
20-
service: [apollo_server, caliban, netflix_dgs, gqlgen, tailcall, async_graphql, hasura, graphql_jit]
20+
service:
21+
[
22+
apollo_server,
23+
caliban,
24+
netflix_dgs,
25+
gqlgen,
26+
tailcall,
27+
async_graphql,
28+
hasura,
29+
graphql_jit,
30+
]
2131
steps:
2232
- name: Checkout (GitHub)
2333
uses: actions/checkout@v4
2434
with:
2535
token: ${{ secrets.GITHUB_TOKEN }}
2636
ref: ${{ github.event_name == 'pull_request_target' && github.event.pull_request.head.sha || github.sha }}
27-
37+
2838
- name: Build devcontainer and run setup and benchmark
2939
uses: devcontainers/ci@v0.3
3040
with:
@@ -37,27 +47,26 @@ jobs:
3747
- name: List benchmark files
3848
run: |
3949
ls -la bench*.txt || echo "No matching files found"
40-
50+
4151
- name: Upload benchmark results
4252
uses: actions/upload-artifact@v3
4353
with:
4454
name: benchmark-results
4555
path: bench*.txt
4656

47-
4857
analyze:
49-
needs: benchmark
58+
needs: build
5059
runs-on: benchmarking-runner
5160
steps:
5261
- name: Checkout (GitHub)
5362
uses: actions/checkout@v4
54-
63+
5564
- name: Download all benchmark results
5665
uses: actions/download-artifact@v3
5766
with:
5867
name: benchmark-results
5968
path: .
60-
69+
6170
- name: List downloaded artifacts
6271
run: ls -la bench*.txt || echo "No matching files found"
6372

@@ -67,19 +76,19 @@ jobs:
6776
6877
- name: Print benchmark results
6978
run: cat ./results.md
70-
79+
7180
- name: Comment benchmark results on PR
7281
if: github.event_name == 'pull_request_target'
7382
uses: peter-evans/commit-comment@v3
7483
with:
7584
sha: ${{ github.event.pull_request.head.sha }}
7685
body-path: "results.md"
7786
reactions: eyes
78-
87+
7988
- name: Commit and push changes (on main branch)
8089
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
8190
uses: stefanzweifel/git-auto-commit-action@v5
8291
with:
8392
branch: main
8493
commit_author: Author <actions@github.com>
85-
commit_message: "Update performance results in README.md"
94+
commit_message: "[ci skip] update performance results in README.md"

‎README.md‎

Lines changed: 24 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -51,32 +51,32 @@ Get started with the benchmarks:
5151
| Query | Server | Requests/sec | Latency (ms) | Relative |
5252
|-------:|--------:|--------------:|--------------:|---------:|
5353
| 1 | `{ posts { id userId title user { id name email }}}` |
54-
|| [Tailcall] | `29,496.60` | `3.38` | `196.74x` |
55-
|| [async-graphql] | `1,805.45` | `55.70` | `12.04x` |
56-
|| [Caliban] | `1,547.91` | `64.33` | `10.32x` |
57-
|| [GraphQL JIT] | `1,335.26` | `74.60` | `8.91x` |
58-
|| [Gqlgen] | `772.80` | `128.41` | `5.15x` |
59-
|| [Netflix DGS] | `358.06` | `177.42` | `2.39x` |
60-
|| [Apollo GraphQL] | `268.25` | `366.16` | `1.79x` |
61-
|| [Hasura] | `149.93` | `549.07` | `1.00x` |
54+
|| [Tailcall] | `29,173.60` | `3.41` | `211.81x` |
55+
|| [async-graphql] | `2,019.59` | `49.49` | `14.66x` |
56+
|| [Caliban] | `1,744.80` | `57.28` | `12.67x` |
57+
|| [GraphQL JIT] | `1,342.93` | `74.16` | `9.75x` |
58+
|| [Gqlgen] | `797.87` | `124.41` | `5.79x` |
59+
|| [Netflix DGS] | `366.18` | `181.49` | `2.66x` |
60+
|| [Apollo GraphQL] | `263.39` | `372.49` | `1.91x` |
61+
|| [Hasura] | `137.74` | `587.80` | `1.00x` |
6262
| 2 | `{ posts { title }}` |
63-
|| [Tailcall] | `58,939.60` | `1.69` | `67.77x` |
64-
|| [Caliban] | `9,238.76` | `11.16` | `10.62x` |
65-
|| [async-graphql] | `9,138.76` | `11.34` | `10.51x` |
66-
|| [Gqlgen] | `2,136.19` | `48.48` | `2.46x` |
67-
|| [Apollo GraphQL] | `1,775.87` | `56.26` | `2.04x` |
68-
|| [Netflix DGS] | `1,593.80` | `69.82` | `1.83x` |
69-
|| [GraphQL JIT] | `1,399.68` | `71.35` | `1.61x` |
70-
|| [Hasura] | `869.69` | `114.81` | `1.00x` |
63+
|| [Tailcall] | `58,485.30` | `1.70` | `67.37x` |
64+
|| [Caliban] | `9,751.71` | `10.62` | `11.23x` |
65+
|| [async-graphql] | `9,444.13` | `10.73` | `10.88x` |
66+
|| [Gqlgen] | `2,181.26` | `47.48` | `2.51x` |
67+
|| [Apollo GraphQL] | `1,760.69` | `56.72` | `2.03x` |
68+
|| [Netflix DGS] | `1,598.09` | `70.06` | `1.84x` |
69+
|| [GraphQL JIT] | `1,392.45` | `71.71` | `1.60x` |
70+
|| [Hasura] | `868.17` | `114.99` | `1.00x` |
7171
| 3 | `{ greet }` |
72-
|| [Caliban] | `67,637.00` | `1.07` | `25.83x` |
73-
|| [Tailcall] | `59,557.80` | `1.69` | `22.75x` |
74-
|| [Gqlgen] | `47,525.20` | `5.21` | `18.15x` |
75-
|| [async-graphql] | `47,150.60` | `2.21` | `18.01x` |
76-
|| [Netflix DGS] | `8,261.44` | `15.02` | `3.16x` |
77-
|| [Apollo GraphQL] | `8,186.12` | `12.54` | `3.13x` |
78-
|| [GraphQL JIT] | `5,273.45` | `18.93` | `2.01x` |
79-
|| [Hasura] | `2,618.18` | `38.13` | `1.00x` |
72+
|| [Caliban] | `67,512.70` | `1.07` | `26.49x` |
73+
|| [Tailcall] | `59,015.70` | `1.71` | `23.16x` |
74+
|| [Gqlgen] | `47,378.10` | `5.25` | `18.59x` |
75+
|| [async-graphql] | `47,361.00` | `2.25` | `18.58x` |
76+
|| [Netflix DGS] | `8,297.54` | `14.39` | `3.26x` |
77+
|| [Apollo GraphQL] | `8,128.40` | `12.43` | `3.19x` |
78+
|| [GraphQL JIT] | `5,241.04` | `19.05` | `2.06x` |
79+
|| [Hasura] | `2,548.54` | `39.16` | `1.00x` |
8080

8181
<!-- PERFORMANCE_RESULTS_END -->
8282

‎assets/latency_histogram1.png‎

249 Bytes
Loading

‎assets/latency_histogram2.png‎

-287 Bytes
Loading

‎assets/latency_histogram3.png‎

-107 Bytes
Loading

‎assets/req_sec_histogram1.png‎

-293 Bytes
Loading

‎assets/req_sec_histogram2.png‎

-380 Bytes
Loading

‎assets/req_sec_histogram3.png‎

-517 Bytes
Loading

‎results.md‎

Lines changed: 24 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -3,31 +3,31 @@
33
| Query | Server | Requests/sec | Latency (ms) | Relative |
44
|-------:|--------:|--------------:|--------------:|---------:|
55
| 1 | `{ posts { id userId title user { id name email }}}` |
6-
|| [Tailcall] | `29,496.60` | `3.38` | `196.74x` |
7-
|| [async-graphql] | `1,805.45` | `55.70` | `12.04x` |
8-
|| [Caliban] | `1,547.91` | `64.33` | `10.32x` |
9-
|| [GraphQL JIT] | `1,335.26` | `74.60` | `8.91x` |
10-
|| [Gqlgen] | `772.80` | `128.41` | `5.15x` |
11-
|| [Netflix DGS] | `358.06` | `177.42` | `2.39x` |
12-
|| [Apollo GraphQL] | `268.25` | `366.16` | `1.79x` |
13-
|| [Hasura] | `149.93` | `549.07` | `1.00x` |
6+
|| [Tailcall] | `29,173.60` | `3.41` | `211.81x` |
7+
|| [async-graphql] | `2,019.59` | `49.49` | `14.66x` |
8+
|| [Caliban] | `1,744.80` | `57.28` | `12.67x` |
9+
|| [GraphQL JIT] | `1,342.93` | `74.16` | `9.75x` |
10+
|| [Gqlgen] | `797.87` | `124.41` | `5.79x` |
11+
|| [Netflix DGS] | `366.18` | `181.49` | `2.66x` |
12+
|| [Apollo GraphQL] | `263.39` | `372.49` | `1.91x` |
13+
|| [Hasura] | `137.74` | `587.80` | `1.00x` |
1414
| 2 | `{ posts { title }}` |
15-
|| [Tailcall] | `58,939.60` | `1.69` | `67.77x` |
16-
|| [Caliban] | `9,238.76` | `11.16` | `10.62x` |
17-
|| [async-graphql] | `9,138.76` | `11.34` | `10.51x` |
18-
|| [Gqlgen] | `2,136.19` | `48.48` | `2.46x` |
19-
|| [Apollo GraphQL] | `1,775.87` | `56.26` | `2.04x` |
20-
|| [Netflix DGS] | `1,593.80` | `69.82` | `1.83x` |
21-
|| [GraphQL JIT] | `1,399.68` | `71.35` | `1.61x` |
22-
|| [Hasura] | `869.69` | `114.81` | `1.00x` |
15+
|| [Tailcall] | `58,485.30` | `1.70` | `67.37x` |
16+
|| [Caliban] | `9,751.71` | `10.62` | `11.23x` |
17+
|| [async-graphql] | `9,444.13` | `10.73` | `10.88x` |
18+
|| [Gqlgen] | `2,181.26` | `47.48` | `2.51x` |
19+
|| [Apollo GraphQL] | `1,760.69` | `56.72` | `2.03x` |
20+
|| [Netflix DGS] | `1,598.09` | `70.06` | `1.84x` |
21+
|| [GraphQL JIT] | `1,392.45` | `71.71` | `1.60x` |
22+
|| [Hasura] | `868.17` | `114.99` | `1.00x` |
2323
| 3 | `{ greet }` |
24-
|| [Caliban] | `67,637.00` | `1.07` | `25.83x` |
25-
|| [Tailcall] | `59,557.80` | `1.69` | `22.75x` |
26-
|| [Gqlgen] | `47,525.20` | `5.21` | `18.15x` |
27-
|| [async-graphql] | `47,150.60` | `2.21` | `18.01x` |
28-
|| [Netflix DGS] | `8,261.44` | `15.02` | `3.16x` |
29-
|| [Apollo GraphQL] | `8,186.12` | `12.54` | `3.13x` |
30-
|| [GraphQL JIT] | `5,273.45` | `18.93` | `2.01x` |
31-
|| [Hasura] | `2,618.18` | `38.13` | `1.00x` |
24+
|| [Caliban] | `67,512.70` | `1.07` | `26.49x` |
25+
|| [Tailcall] | `59,015.70` | `1.71` | `23.16x` |
26+
|| [Gqlgen] | `47,378.10` | `5.25` | `18.59x` |
27+
|| [async-graphql] | `47,361.00` | `2.25` | `18.58x` |
28+
|| [Netflix DGS] | `8,297.54` | `14.39` | `3.26x` |
29+
|| [Apollo GraphQL] | `8,128.40` | `12.43` | `3.19x` |
30+
|| [GraphQL JIT] | `5,241.04` | `19.05` | `2.06x` |
31+
|| [Hasura] | `2,548.54` | `39.16` | `1.00x` |
3232

3333
<!-- PERFORMANCE_RESULTS_END -->

0 commit comments

Comments
 (0)