Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Faster skipping of JSON values #1214

Merged
merged 1 commit into from
Jan 13, 2025

Conversation

plokhotnyuk
Copy link
Contributor

@plokhotnyuk plokhotnyuk commented Jan 13, 2025

Below are throughput results of benchmarks for two cases:

  • ExtractFieldsReading tests skipping of 1, 10, 100 additional extra fields and their simple numeric values
  • GeoJSONReading tests a GeoJSON format message where the discriminator fields are in the last positions of their JSON objects.

Also, It saves from possible DoS attacks with deeply nested JSON objects or arrays in place of values to skip that could throws stack overflows internally and reported with "Unexpected structure" as a result of parsing.

Tested with Scala 3 and JDK 21 on Intel® Core™ i7-11800H CPU @ 2.3GHz (max 4.6GHz).

Before

Benchmark                     (size)   Mode  Cnt        Score       Error  Units
ExtractFieldsReading.zioJson       1  thrpt    5  5822959.833 ± 97416.143  ops/s
ExtractFieldsReading.zioJson      10  thrpt    5  1316834.542 ± 20668.995  ops/s
ExtractFieldsReading.zioJson     100  thrpt    5   123209.630 ±  1084.177  ops/s
GeoJSONReading.zioJson           N/A  thrpt    5     5219.434 ±    97.488  ops/s

After

Benchmark                     (size)   Mode  Cnt        Score        Error  Units
ExtractFieldsReading.zioJson       1  thrpt    5  7146001.174 ± 256909.348  ops/s
ExtractFieldsReading.zioJson      10  thrpt    5  1408616.469 ±   8237.600  ops/s
ExtractFieldsReading.zioJson     100  thrpt    5   171101.538 ±   3852.582  ops/s
GeoJSONReading.zioJson           N/A  thrpt    5     6871.387 ±     49.618  ops/s

@plokhotnyuk plokhotnyuk requested a review from a team as a code owner January 13, 2025 10:26
@plokhotnyuk plokhotnyuk force-pushed the faster-skipping-of-JSON-values branch 2 times, most recently from 855f9bd to 206a68e Compare January 13, 2025 11:39
@jdegoes jdegoes merged commit e604945 into zio:series/2.x Jan 13, 2025
28 checks passed
@plokhotnyuk plokhotnyuk deleted the faster-skipping-of-JSON-values branch January 13, 2025 14:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants