You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
A FHIR® Store with internal, fast CQL Evaluation Engine
@@ -13,9 +12,9 @@ The goal of this project is to provide a FHIR® Store with an internal CQL Evalu
13
12
14
13
## State
15
14
16
-
The project is currently under active development. Essentially all official [CQL Tests][3] pass. Please report any issues you encounter during evaluation.
15
+
Blaze passes all [Touchstone][12] FHIR 4.0.1 Basic tests and almost all [CQL Tests][3]. Please report any issues you encounter during evaluation.
17
16
18
-
Latest release: [v0.9.0-alpha.10][5]
17
+
Latest release: [v0.8.0][5]
19
18
20
19
## Quick Start
21
20
@@ -25,194 +24,24 @@ In order to run Blaze just execute the following:
25
24
26
25
```bash
27
26
docker volume create blaze-data
28
-
docker run -p 8080:8080 -v blaze-data:/app/data samply/blaze:0.9.0-alpha.27
27
+
docker run -p 8080:8080 -v blaze-data:/app/data samply/blaze:0.8.0
29
28
```
30
29
31
30
Blaze will create multiple directories inside the `blaze-data` volume on its first start and use the same directories on subsequent starts.
32
31
32
+
Please refer to [Docker Deployment](docs/deployment/docker-deployment.md) for the full documentation.
33
33
34
34
### Standalone Java without Docker
35
35
36
-
In case Docker isn't available, Blaze can be run on a machine having OpenJDK 11 installed. Blaze is tested with [AdoptOpenJDK][11].
36
+
Please have a look into [Manual Deployment](docs/deployment/manual-deployment.md).
This will create three directories called `index`, `transaction` and `resource` inside the current working directory, one for each database part used.
61
-
62
-
Logging output should appear which prints the most important settings and system parameters like Java version and available memory.
63
-
64
-
In order to test connectivity, query the health endpoint:
65
-
66
-
```bash
67
-
curl http://localhost:8080/health
68
-
```
69
-
70
-
## Configuration
71
-
72
-
Blaze is configured solely through environment variables. There is a default for every variable. So all are optional.
73
-
74
-
A part of the environment variables depends on the storage variant chosen. The storage variant can be set through the `STORAGE` env var. The default is `in-memory` for the JAR and `standalone` for the Docker image. The third setting is `distributed`. The following tables list the database relavant environment variables by storage variant.
75
-
76
-
### In-memory
77
-
78
-
| Name | Default | Since | Description |
79
-
| :--- | :--- | :--- | :--- |
80
-
| DB\_RESOURCE\_INDEXER\_BATCH\_SIZE | 1 | v0.8 | The number of resources which are indexed in a batch. |
81
-
82
-
### Standalone
83
-
84
-
The three database directories must not exist on the first start of Blaze and will be created by Blaze itself. It's possible to put this three directories on different disks in order to improve performance.
85
-
86
-
| Name | Default | Since | Description |
87
-
| :--- | :--- | :--- | :--- |
88
-
| INDEX\_DB\_DIR | – | v0.8 | The directory were the index database files are stored. |
89
-
| TRANSACTION\_DB\_DIR | – | v0.8 | The directory were the transaction log files are stored. This directory must not exist on the first start of Blaze and will be created by Blaze. |
90
-
| RESOURCE\_DB\_DIR | – | v0.8 | The directory were the resource files are stored. This directory must not exist on the first start of Blaze and will be created by
91
-
| DB\_RESOURCE\_INDEXER\_BATCH\_SIZE | 1 | v0.8 | The number of resources which are indexed in a batch. |
92
-
| DB\_BLOCK\_CACHE\_SIZE | 128 | v0.8 | The size of the [block cache][9] of the DB in MB. |
93
-
| DB\_RESOURCE\_CACHE\_SIZE | 10000 | v0.8 | The size of the resource cache of the DB in number of resources. |
94
-
| DB\_MAX\_BACKGROUND\_JOBS | 4 | v0.8 | The maximum number of the [background jobs][10] used for DB compactions. |
95
-
| DB\_RESOURCE\_INDEXER\_THREADS | 4 | v0.8 | The number threads used for indexing resources. |
96
-
| DB\_RESOURCE\_INDEXER\_BATCH\_SIZE | 1 | v0.8 | The number of resources which are indexed in a batch. |
97
-
98
-
### Distributed
99
-
100
-
The distributed storage variant only uses the index database locally.
101
-
102
-
| Name | Default | Since | Description |
103
-
| :--- | :--- | :--- | :--- |
104
-
| INDEX\_DB\_DIR | – | v0.8 | The directory were the index database files are stored. |
105
-
| DB\_BLOCK\_CACHE\_SIZE | 128 | v0.8 | The size of the [block cache][9] of the DB in MB. |
106
-
| DB\_RESOURCE\_CACHE\_SIZE | 10000 | v0.8 | The size of the resource cache of the DB in number of resources. |
107
-
| DB\_MAX\_BACKGROUND\_JOBS | 4 | v0.8 | The maximum number of the [background jobs][10] used for DB compactions. |
108
-
| DB\_RESOURCE\_INDEXER\_THREADS | 4 | v0.8 | The number threads used for indexing resources. |
109
-
| DB\_RESOURCE\_INDEXER\_BATCH\_SIZE | 1 | v0.8 | The number of resources which are indexed in a batch. |
110
-
| DB\_KAFKA\_BOOTSTRAP\_SERVERS | localhost:9092 | v0.8 | A comma separated list of bootstrap servers for the Kafka transaction log. |
111
-
| DB\_KAFKA\_MAX\_REQUEST\_SIZE | 1048576 | v0.8 | The maximum size of a encoded transaction able to send to the Kafka transaction log in bytes. |
112
-
| DB\_CASSANDRA\_CONTACT\_POINTS | localhost:9042 | v0.8 | A comma separated list of contact points for the Cassandra resource store. |
113
-
| DB\_CASSANDRA\_KEY\_SPACE | blaze | v0.8 | The Cassandra key space were the `resources` table is located. |
114
-
| DB\_CASSANDRA\_PUT\_CONSISTENCY\_LEVEL | TWO | v0.8 | Cassandra consistency level for resource put (insert) operations. Has to be set to `ONE` on a non-replicated keyspace. |
115
-
116
-
117
-
### Other Environment Variables
118
-
119
-
| Name | Default | Since | Description |
120
-
| :--- | :--- | :--- | :--- |
121
-
| PROXY\_HOST | — | v0.6 | The hostname of the proxy server for outbound HTTP requests |
122
-
| PROXY\_PORT | — | v0.6 | Port of the proxy server |
123
-
| PROXY\_USER | — | v0.6.1 | Proxy server user, if authentication is needed. |
124
-
| PROXY\_PASSWORD | — | v0.6.1 | Proxy server password, if authentication is needed. |
125
-
| CONNECTION\_TIMEOUT | 5 s | v0.6.3 | connection timeout for outbound HTTP requests |
126
-
| REQUEST\_TIMEOUT | 30 s | v0.6.3 | request timeout for outbound HTTP requests |
127
-
| TERM\_SERVICE\_URI |[http://tx.fhir.org/r4](http://tx.fhir.org/r4)| v0.6 | Base URI of the terminology service |
128
-
| BASE\_URL |[http://localhost:8080](http://localhost:8080)|| The URL under which Blaze is accessible by clients. The [FHIR RESTful API](https://www.hl7.org/fhir/http.html) will be accessible under `BASE_URL/fhir`. |
129
-
| SERVER\_PORT | 8080 || The port of the main HTTP server |
130
-
| METRICS\_SERVER\_PORT | 8081 | v0.6 | The port of the Prometheus metrics server |
131
-
| LOG\_LEVEL | info | v0.6 | one of trace, debug, info, warn or error |
| FHIR\_OPERATION\_EVALUATE\_MEASURE\_THREADS | 4 | v0.8 | The maximum number of parallel $evaluate-measure executions. Not the same as the number of threads used for measure evaluation which equal to the number of available processors. |
Cassandra can be used as resource storage. You have to create a keyspace and one table for Blaze. You can use the Cassandra Query Language Shell via Docker running the following command:
162
-
163
-
```bash
164
-
docker run -it --rm --network cassandra_default cassandra:3 cqlsh cassandra
CREATE TABLE blaze.resources (hash text PRIMARY KEY, content blob);
177
-
```
178
-
179
-
## Tuning Guide
180
-
181
-
### Common Considerations
182
-
183
-
#### CPU Cores
184
-
185
-
Blaze heavily benefits from vertical scaling (number of CPU cores). Even with single clients, transactions and CQL queries scale with the number of CPU cores. Performance tests show that 32 cores can be utilized easily.
186
-
187
-
#### Memory
188
-
189
-
Blaze uses three main memory regions, namely the JVM heap memory, the RocksDB block cache and the OS page cache. The JVM heap memory is used for general operation and specifically for the resource cache which caches resources in their ready to use object form. The RocksDB block cache is separate from the JVM heap memory because it's implemented as off-heap memory region. The block cache stores resources and index segments in uncompressed blob form. After the resource cache, it functions as a second level cache which stores resources in a more compact form and is not subject to JVM garbage collection. The OS page cache functions as a third level of caching, because it contains all recently access database files. The database files are compressed and so store data in an even more compact form.
190
-
191
-
The first recommendation is to leave half of the available system memory for the OS page cache. For example if you have 16 GB of memory, you should only allocate 8 GB for JVM heap + RocksDB block cache. The ratio between the JVM heap size and the RocksDB block cache can vary. First you should monitor the JVM heap usage and garbage collection activity in your use case. Blaze provides a Prometheus metrics endpoint on port 8081 per default. Using half of the remaining system memory for the JVM heap and half for the RocksDB block cache will be a good start.
192
-
193
-
Depending on the capabilities of your disk I/O system, having enough OS page cache for holding most of the active database file size, can really make a huge difference in performance. In case all your database files fit in memory, you have essentially a in-memory database. Test with up to 128 GB system memory showed that effect.
194
-
195
-
#### Disk I/O
196
-
197
-
Blaze is designed with local (NVMe) SSD storage in mind. That is the reason why Blaze uses RocksDB as underlying storage engine. RocksDB benefits from fast storage. The idea is that latency of local SSD storage is faster than the network. So remote network storage has no longer an advantage.
198
-
199
-
If you have only spinning disks available, a bigger OS page cache can help. Otherwise especially the latency of operations will suffer.
200
-
201
-
### Transactions
202
-
203
-
Blaze maintains indexes for FHIR search parameters and CQL evaluation. The indexing process can be executed in parallel if transactions contain more than one resource. Transaction containing only one resource and direct REST interactions like create, update or delete don't benefit from parallel indexing.
204
-
205
-
Depending on your system, indexing can be I/O or CPU bound. Performance tests show, that at least 20 CPU cores can be utilized for resource indexing. The default number of indexing threads is 4 but can be changed by setting `DB_RESOURCE_INDEXER_THREADS` to a different value. However only transactions with more than `DB_RESOURCE_INDEXER_THREADS` resources will benefit from it.
206
-
207
-
If your transactions contain way more resources as your number of CPU cores available, you could experiment with increasing the `DB_RESOURCE_INDEXER_BATCH_SIZE` which has a default value of 1. With a value of more than one, the resource indexer will create batches of resources which are indexed together by one thread. This batching lowers the coordination overhead. But be aware that a batch size of about your transaction size will prevent any parallelism.
208
-
209
-
### CQL Queries
210
-
211
-
212
-
213
-
## Deployment
38
+
## Documentation
214
39
215
-
In-deep deployment options of Blaze are described in the [Deployment Section][4] of the Blaze documentation.
40
+
*[Importing Data](docs/importing-data.md)
41
+
*[Conformance](docs/conformance.md)
42
+
*[Performance](docs/performance.md)
43
+
*[Tuning Guide](docs/tuning-guide.md)
44
+
*[Tooling](docs/tooling.md)
216
45
217
46
## YourKit Profiler
218
47
@@ -222,7 +51,7 @@ The developers of Blaze uses the YourKit profiler to optimize performance. YourK
222
51
223
52
## License
224
53
225
-
Copyright 2019 The Samply Development Community
54
+
Copyright 2020 The Samply Development Community
226
55
227
56
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
228
57
@@ -232,10 +61,10 @@ Unless required by applicable law or agreed to in writing, software distributed
Copy file name to clipboardExpand all lines: docs/conformance.md
+1-3Lines changed: 1 addition & 3 deletions
Original file line number
Diff line number
Diff line change
@@ -2,15 +2,13 @@
2
2
3
3
Blaze implements the [RESTful FHIR API](https://www.hl7.org/fhir/http.html) and claims its capabilities in a [CapabilityStatement](https://www.hl7.org/fhir/capabilitystatement.html).
4
4
5
-
{% hint style="info" %}
6
5
A current CapabilityStatement can be fetched from [https://blaze.life.uni-leipzig.de/fhir/metadata](https://blaze.life.uni-leipzig.de/fhir/metadata)
7
-
{% endhint %}
8
6
9
7
## Conformance Tests
10
8
11
9
### Touchstone
12
10
13
-
Touchstone test results can be found here: [FHIR 4.0.0 Server / LIFE - Blaze / Jul 2019](https://touchstone.aegis.net/touchstone/analytics/published?month=201907&spec=FHIR%204.0.0&pa=HL7_FHIR_SERVER&testSystem=5cdd3fed0a120e4f98344faf&supportedOnly=true&cb=/HL7_FHIR_4_0_0)
Copy file name to clipboardExpand all lines: docs/cql-queries.md
+2-6Lines changed: 2 additions & 6 deletions
Original file line number
Diff line number
Diff line change
@@ -1,19 +1,15 @@
1
-
---
2
-
description: How to write and execute CQL queries
3
-
---
4
-
5
1
# CQL Queries
6
2
7
3
## Install the Quality Reporting UI
8
4
9
-
The most accesible way to create and execute CQL queries is to use the Quality Reporting UI. The Quality Reporting UI is a desktop application which you can download [here](https://github.com/samply/blaze-quality-reporting-ui).
5
+
The most accessible way to create and execute CQL queries is to use the Quality Reporting UI. The Quality Reporting UI is a desktop application which you can download [here](https://github.com/samply/blaze-quality-reporting-ui).
10
6
11
7
## Run Blaze
12
8
13
9
If you don't already have Blaze running, you can read about how to do it in [Deployment](deployment/). If you have Docker available just run:
14
10
15
11
```
16
-
docker run -p 8080:8080 -v blaze-data:/app/data samply/blaze:0.8.0-beta.3
12
+
docker run -p 8080:8080 -v blaze-data:/app/data samply/blaze:0.8.0
17
13
```
18
14
19
15
Start the Quality Reporting UI. You should see an empty measure list.
Cassandra can be used as resource storage. You have to create a keyspace and one table for Blaze. You can use the Cassandra Query Language Shell via Docker running the following command:
26
+
27
+
```bash
28
+
docker run -it --rm --network cassandra_default cassandra:3 cqlsh cassandra
0 commit comments