Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Eliminate the need to have struct and multimap names on the wire schema #24

Closed
tigrannajaryan opened this issue Jan 25, 2025 · 0 comments · Fixed by #29
Closed

Eliminate the need to have struct and multimap names on the wire schema #24

tigrannajaryan opened this issue Jan 25, 2025 · 0 comments · Fixed by #29

Comments

@tigrannajaryan
Copy link
Collaborator

It should be possible to do schema compatibility checks without symbolic names. This should reduce wire size of schema which will help the cases with small STEF payloads.
tigrannajaryan added a commit that referenced this issue Jan 25, 2025
@tigrannajaryan
Contributes to #24
8f589e3
tigrannajaryan added a commit that referenced this issue Jan 29, 2025
@tigrannajaryan
Contributes to #24
f9e21c6
tigrannajaryan added a commit that referenced this issue Jan 29, 2025
@tigrannajaryan
Better schema compat checks
43ed852 
Contributes to #24

Don't require exact name match for Schema compatibility checks
tigrannajaryan added a commit that referenced this issue Jan 30, 2025
@tigrannajaryan
Better schema compat checks
3f9ad54 
Contributes to #24

Don't require exact name match for Schema compatibility checks
tigrannajaryan added a commit that referenced this issue Jan 31, 2025
@tigrannajaryan
Better schema compat checks
9f6a034 
Contributes to #24

Don't require exact name match for Schema compatibility checks
tigrannajaryan added a commit that referenced this issue Feb 4, 2025
@tigrannajaryan
Introduce minimal WireSchema
0015987 
This decouples human-readable Schema that is in JSON
format from WireSchema that is serialized in binary
format for efficiency. WireSchema contains only the
information that is necessary for wire compatibility
checks.

Resolves #24
(Note that we still need struct names for checks).

Here are byte size comparisons between JSON and WIRE
for the Otel/TEF schema with "Metrics" root:

```
JSON:  2976, zstd:  760
WIRE:   115, zstd:  113
```
tigrannajaryan added a commit that referenced this issue Feb 4, 2025
@tigrannajaryan
Introduce minimal WireSchema
a304634 
This decouples human-readable Schema that is in JSON
format from WireSchema that is serialized in binary
format for efficiency. WireSchema contains only the
information that is necessary for wire compatibility
checks.

Resolves #24
(Note that we still need struct names for checks).

Here are byte size comparisons between JSON and WIRE
for the Otel/TEF schema with "Metrics" root:

```
JSON:  2976, zstd:  760
WIRE:   115, zstd:  113
```
tigrannajaryan added a commit that referenced this issue Feb 4, 2025
@tigrannajaryan
Introduce minimal WireSchema
dec8360 
This decouples human-readable Schema that is in JSON
format from WireSchema that is serialized in binary
format for efficiency. WireSchema contains only the
information that is necessary for wire compatibility
checks.

Resolves #24
(Note that we still need struct names for checks).

Here are byte size comparisons between JSON and WIRE
for the Otel/TEF schema with "Metrics" root:

```
JSON:  2976, zstd:  760
WIRE:   115, zstd:  113
```

The following components were refactored to use WireSchema
instead of Schema:
- Generated Reader/Writer
- Encoder/Decoders
- gRPC client/server

The Generator continues using JSON Schema as its input, however
it now generates a WireSchema to be used in the generated code.
@tigrannajaryan tigrannajaryan mentioned this issue Feb 4, 2025
Merged
tigrannajaryan added a commit that referenced this issue Feb 6, 2025
@tigrannajaryan
Introduce minimal WireSchema
bd3a147 
This decouples human-readable Schema that is in JSON
format from WireSchema that is serialized in binary
format for efficiency. WireSchema contains only the
information that is necessary for wire compatibility
checks.

Resolves #24
(Note that we still need struct names for checks).

Here are byte size comparisons between JSON and WIRE
for the Otel/TEF schema with "Metrics" root:

```
JSON:  2976, zstd:  760
WIRE:   115, zstd:  113
```

The following components were refactored to use WireSchema
instead of Schema:
- Generated Reader/Writer
- Encoder/Decoders
- gRPC client/server

The Generator continues using JSON Schema as its input, however
it now generates a WireSchema to be used in the generated code.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Introduce minimal WireSchema splunk/stef
1 participant
@tigrannajaryan