Versioning with upstream projects

[ghost]

If there are no restrictions from technology side, there are two main competing approaches to versioning with upstream:

1. Upstream semantic version - splunk version (suffix, changed for patches)
2. Splunk semantic version - upstream version (as suffix)

Both have advantages and disadvantages.

1. Upstream - splunk suffix

• easy to follow from upstream point of view
• open source (upstream) first
• allows to release splunk patches easily (changing suffix)
• well known to the org (we did this for SFX agent)

2. Splunk - upstream suffix

• allows for Splunk-managed semantic versioning
• harder to follow upstream releases (need to look for suffix)
• easier to manage if we don't want to follow each release of the upstream (no "gaps" in versions)

Before we set this in the spec, I'd like to hear from the rest of the team.

[owais]

 Upstream - splunk suffix

It is impossible to denote a breaking change in Splunk distribution when using this approach. Bumping anything in the suffix will not tell package managers that they should not auto-upgrade the library. On the contrary it'll look like a very minor change to package managers.

Also, this may also not be possible for all languages. For example, Python only allows a handful of suffixes to be published to pypi (dev, alpha, beta, etc). As a result, I think this should be a recommendation and be language specific.

There are other ways to making this information available to users. Users can look at release notes, dependencies, use version CLI arg, etc. I think this is highly language specific and at the minimum we should require the release notes to mention Otel version very loudly. Other than that, we can have language specific recommendations.
Users may want to know which upstream version our lib targets but adding the ver

[iNikem]

My first question would be: do we need the same versioning schema for all-in-one distributions such as javaagent and for "manually add this package to your application" such as python/nodeJS. If yes, this kinda mean that we should use something similar to SemVer for our versions.

I strongly object against have a version string compound of two semvers, otel and splunk. And this is again impossible in case of instrumentations distributed via package managers. Which in reality leaves only one option on the table: we have our own SemVer versioning and upstream Otel versions is marked in documentation and release notes.

[owais]

This makes complete sense to me. As long as we document it, people who care can always look at the docs/release notes. Adding suffixes doesn't really solve anything IMO. Generally suffixes are used for very thin wrappers like OS distributions that want to specify packaging or build changes in suffixes.

[pellared]

Additionally, I have a strong opinion that our version should be always a SemVer 2.0.0 string. It makes doing a lot of automation easier.

[owais]

I wish the industry would move to something better like ComVer but totally with you on following SemVer strictly.

[ivomagi]

When picking a solution I have just one ask. Currently the customers get confused on figuring out the mapping between our agent version and the corresponding Otel version (for example splunk-otel-java 0.10.0 includes Otel SDK/API and instrumentations 1.1.0). The

I would like this mapping to be an easy task, explicitly transparent from just "looking" at the package and not requiring digging through release notes. I am pretty sure this is not a unique issue - what about all the commercial bundles of other OS software, I would assume someone has chosen an elegant solution here.

[owais]

But how often do customers want to do that? I'd imagine once per version usually when they first install or upgrade a version. Do we need to have it front and center in the package name/version for such a rare use case?

[ivomagi]

What I have experienced is that the need arises when problems occur. For example when upgrade from Otel SDK X.Y to Otel SDK X.Z breaks manual instrumentation or a custom propagator. Or when an otel instrumentation update breaks an autoinstrumentation for library W.

[iNikem]

I am pretty sure this is not a unique issue - what about all the commercial bundles of other OS software, I would assume someone has chosen an elegant solution here.

Can you provide any example of any such commercial bundle and any elegant-ish solution? I cannot imagine how can we merge two semantic versions into one while preserving semantic versioning rules and not breaking all package managers.

[ghost]

With regards to python as pointed out by @owais - that's exactly what I meant stating "If there are no restrictions from the technology side". IF package manager does not allow format other than own, there is no discussion :)

With regards to pros vs cons - I look at this from @ivomagi point of view. I've had a chance to speak directly with customers only a handful of times (usually with Nike) but the case "what is the mapping between Splunk and OTEL" came up very often. Keeping it not in plain sight (== in release notes) will increase customer's pain (friction to adopt and use our stuff). So if we can't have two SemVers, as noted by @iNikem, we're left with only one option (where applicable / allowed by the technology) - upstream SemVer + splunk suffix.

[owais]

I'm sure way more customers will end up upgrading to breaking releases if Splunk version is added as a suffix. I think adding splunk or otel suffixes to version numbers themselves is overkill to solve this problem. Adding it to builds like JAR files is a lot more acceptable.

If we really want to have suffixes, we must have otel version as a suffix to splunk version so that we can bump meansure versions to indicate breaking change. Bumping anything in suffix is not going to help IMO.

I personally think we are giving the problem more weight than it deserves. I'm not dismissing the issue but IMO it's rare enough and needs to be explained just once to each customer.

@ivomagi if the question comes up mainly when there is a problem, may be we should add otel and splunk versions both as global spans (if we don't already). It should be pretty obvious which versions are being used by just looking at any trace for the service in question in APM.

[ghost]

The question is how often are we goin to introduce a breaking change in our distro of some upstream project? ;)

[owais]

I don't think that matters at all. In fact if it is very rare, I'd say it becomes even more important to have a clear way to communicate that to people and to package managers both.

[owais]

Also note that for most languages that don't bundle everything up in a single file, it is not even valid to have Otel version as a suffix. This is because our libraries will usually specify a compatible version range for Otel instead of a specific version. For example, Splunk Python lists any version of Otel greater than or equal to 1.0 and lesser than 2.0 as a compatible version. Users can install any Otel API/SDK version in that range and everything will work. If we depends on a feature shipped in Otel 1.2 then we'll list that as the least compatible version. Moreover, Otel API and SDK may have different versions as they'll likely be developed at different pace. This is true for any language that lists Otel as dependencies instead of bundling it.

So does a fixed suffix even mean anything?

[ghost]

I see out of the discussion two contradicting point of views:

• development (SemVer all the way, as it's easier for us to version, introduce breaking changes and write automation)
• customer / product management - much easier to communicate changes / fixes / releases to our customers, leaving us with an option to introduce our changes via suffix

So to re-iterate the problem, we're talking about versioning a release in a situation when:

1. We have a Splunk distribution of some (OpenSource) project
2. Technology does not prevent us from setting our desired versioning schema (so not Python I guess ;) )

With SFX Java agent we did "upstream + suffix". Did we experience any issues with that? I understand that @johnbley can shed some light here.

All in all I'd like to progress with the task to document versioning for Splunk OTEL Java, but in a standardised way, hence this discussion ;)

[owais]

just to clarify

development (SemVer all the way, as it's easier for us to version, introduce breaking changes and write automation)

I'm of this view but not because it is easier for us. IMO it is better for customers. Imagining myself as a customer, I'd hate it if I or my package manager didn't pay much attention to an upgrade and rolled it out only to see it break things. Ability for me and my tools to easily detect breaking changes triumphs the convenience of checking Otel and Splunk versions by looking at the same string.

For this reason I strongly suggest we either don't do suffixes or if we must, we always have otel version as a suffix so we can introduce breaking changes in the main version when something changes in our code or we upgrade to a major otel version.

[ghost]

I guess that the main difference is that I'm looking from the Java Agent point of view, which is not distributed via package manager but downloaded manually from GitHub. Out of my experience with typical enterprise customers:

• they upgrade usually due to an issue (so they have to upgrade even if the next release contains a breaking change)
• they don't care (or even don't know) what additional knowledge does semantic version bring - the upgrade is dictated by business needs so they just go with upgrade path (on the other hand if the stuff works they will stay with even a very ancient version tho ;) )

Given the differences between eg java agent and python-managed packages, I'm wondering if we need / can set up a generic rule at all. IMO we can only if we resign from added value of having upstream version publicised in our version (either by suffix or main part) but this should be "signed off" by folks that are more customer-facing (@ivomagi I'm looking at you, but also at @flands ;) ). Otherwise we can just specify that "exact versioning scheme depends on the project and technology used".

[pellared]

The versioning scheme is very often technology-specific (that is why I think having it in specs would be very hard and may be not worth the effort).

I feel that the version scheme where two versions are provided may not work in any other place than for Java AutoInstr plugin.
Even for .NET where we want to build a similar architecture, I feel that it would be safer to follow .NETs package versioning scheme. I imagine it may possible to add auto instrumentation via .NET package management.

For sure it is good to reference the "upstream" minimum or required version in the "release notes/changelog" like here: https://github.com/signalfx/splunk-otel-java/releases/tag/v0.10.0 👍

[flands]

Note the most similar project to OTel in terms of size and distribution model is K8s. Eventually OTel will likely look like this: https://kubernetes.io/partners/. One very popular distribution is Rancher: https://github.com/rancher/rancher/releases/tag/v2.4.15. They took the approach of having a rancher-specific semver and adding k8s version in release notes.

[flands]

In order to support breaking change on the Splunk side, Splunk-specific semver will be followed. Release notes and logs will contain upstream information as already defined in the specification. This does means that v1.0.0 for Splunk DOES NOT match v1.0.0 of the OTel project (e.g. Java).