Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow use of dash character (-) in data_stream.namespace #2388

Open
ebeahan opened this issue Sep 25, 2024 · 2 comments
Open

Allow use of dash character (-) in data_stream.namespace #2388

ebeahan opened this issue Sep 25, 2024 · 2 comments
Labels
bug Something isn't working

Comments

@ebeahan
Copy link
Member

ebeahan commented Sep 25, 2024

Summary

Propose ECS removes the restriction of using the dash character, -, in the data_stream.namespace field.

Motivation:

RFC 0009 - Data stream fields includes this restrictions for the data_stream.namespace field:

No longer than 100 chars

The current description for data_stream.namespace also restricts using the dash character:

    - name: namespace
      level: extended
      type: constant_keyword
      example: production
      description: >
        A user defined namespace. Namespaces are useful to allow grouping of data.

        Many users already organize their indices this way, and the data stream naming scheme now provides this
        best practice as a default. Many users will populate this field with `default`. If no value is used, it falls back to `default`.

        Beyond the Elasticsearch index naming criteria noted above, `namespace` value has the additional restrictions:
          * Must not contain `-` <= Not present in RFC
          * No longer than 100 characters
      short: A user defined namespace. Namespaces are useful to allow grouping of data.

Suspect these restrictions may have been mistakenly copied from the same restriction on data_stream.type and data_stream.dataset.

I would not consider the change breaking. It makes data_stream.namespace more permissive and the data_stream.* field set is still considered beta.

@carsonip
Copy link
Member

carsonip commented Oct 2, 2024

I am on board with making data_stream.namespace more permissive in order to not introduce breaking changes in existing implementations.

However, as a counter argument, allowing - in data_stream.namespace makes it a bit tricky to come up with correct index patterns in some edge cases. I'm not sure if that's considered yet.

For example, let's say we have 2 data streams, one called metrics-generic.otel-default, another DS metrics-generic-my.otel-ns where namespace is my.otel-ns. They will both match metrics-*.otel-* index pattern, which may cause some index templates to be applied unintentionally.

cc @felixbarny

@felixbarny
Copy link
Member

I don't see the risk of that being fundamentally different to metrics-*.otel-* matching datasets that happen to end with .otel (for example metrics-apm.app.myservice.otel-default) but aren't actually in the OTel native format.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants