-
Notifications
You must be signed in to change notification settings - Fork 153
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
adopt technical addon, as is (#5389)
- Loading branch information
Showing
38 changed files
with
2,508 additions
and
0 deletions.
There are no files selected for viewing
100 changes: 100 additions & 0 deletions
100
internal/buildscripts/packaging/technical-addon/Makefile
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,100 @@ | ||
# Used for building | ||
SOURCE_DIR?=$(dir $(abspath $(lastword $(MAKEFILE_LIST)))) | ||
BUILD_DIR?=./build | ||
OTEL_COLLECTOR_VERSION?=0.104.0 | ||
SPLUNK_OTELCOL_DOWNLOAD_BASE?=https://github.com/signalfx/splunk-otel-collector/releases/download | ||
PLATFORM?=linux | ||
ARCH?=amd64 | ||
|
||
# Used for testing & validation | ||
ORCA_CLOUD?=kubernetes | ||
UF_VERSION?=8.2.7 | ||
SPLUNK_PLATFORM?=x64_centos_7 | ||
SPLUNK_ARCH?=x86_64 | ||
|
||
# Used for releases | ||
SPLUNK_OTEL_VERSION?= | ||
TA_VERSION?= | ||
|
||
|
||
.PHONY: generate-technical-addon | ||
generate-technical-addon: env-guard-all | ||
BUILD_DIR="$(BUILD_DIR)" \ | ||
SOURCE_DIR="$(SOURCE_DIR)" \ | ||
$(SOURCE_DIR)/packaging-scripts/make-buildspace.sh | ||
|
||
.PHONY: env-guard-all | ||
env-guard-all: | ||
SOURCE_DIR="$(SOURCE_DIR)" \ | ||
BUILD_DIR="$(BUILD_DIR)" \ | ||
$(SOURCE_DIR)/packaging-scripts/env/all.sh | ||
|
||
.PHONY: env-guard-ta | ||
env-guard-ta: env-guard-all | ||
PLATFORM="$(PLATFORM)" \ | ||
ARCH="$(ARCH)" \ | ||
OTEL_COLLECTOR_VERSION="$(OTEL_COLLECTOR_VERSION)" \ | ||
$(SOURCE_DIR)/packaging-scripts/env/ta.sh | ||
|
||
|
||
.PHONY: env-guard-verify | ||
env-guard-verify: env-guard-ta | ||
ORCA_CLOUD="$(ORCA_CLOUD)" \ | ||
UF_VERSION="$(UF_VERSION)" \ | ||
$(SOURCE_DIR)/packaging-scripts/env/verify.sh | ||
|
||
.PHONY: download | ||
download: env-guard-ta | ||
BUILD_DIR="$(BUILD_DIR)" \ | ||
SOURCE_DIR="$(SOURCE_DIR)" \ | ||
OTEL_COLLECTOR_VERSION="$(OTEL_COLLECTOR_VERSION)" \ | ||
SPLUNK_OTELCOL_DOWNLOAD_BASE="$(SPLUNK_OTELCOL_DOWNLOAD_BASE)" \ | ||
PLATFORM="$(PLATFORM)" \ | ||
ARCH="$(ARCH)" \ | ||
$(SOURCE_DIR)/packaging-scripts/download-release.sh | ||
|
||
|
||
.PHONY: package | ||
package: env-guard-ta | ||
BUILD_DIR="$(BUILD_DIR)" \ | ||
SOURCE_DIR="$(SOURCE_DIR)" \ | ||
ARCH="$(ARCH)" \ | ||
PLATFORM="$(PLATFORM)" \ | ||
OTEL_COLLECTOR_VERSION="$(OTEL_COLLECTOR_VERSION)" \ | ||
$(SOURCE_DIR)/packaging-scripts/package-ta.sh | ||
|
||
.PHONY: distribute-ta | ||
distribute-ta: generate-technical-addon download package | ||
|
||
.PHONY: verify | ||
verify: env-guard-verify | ||
SOURCE_DIR="$(SOURCE_DIR)" \ | ||
BUILD_DIR="$(BUILD_DIR)" \ | ||
ARCH="$(ARCH)" \ | ||
PLATFORM="$(PLATFORM)" \ | ||
OTEL_COLLECTOR_VERSION="$(OTEL_COLLECTOR_VERSION)" \ | ||
ORCA_CLOUD="$(ORCA_CLOUD)" \ | ||
UF_VERSION="$(UF_VERSION)" \ | ||
SPLUNK_PLATFORM="$(SPLUNK_PLATFORM)" \ | ||
OLLY_ACCESS_TOKEN="$(OLLY_ACCESS_TOKEN)" \ | ||
$(SOURCE_DIR)/packaging-scripts/deploy-and-verify.sh | ||
|
||
.PHONY: update-deps | ||
update-deps: | ||
SOURCE_DIR="$(SOURCE_DIR)" \ | ||
SPLUNK_OTEL_VERSION="$(SPLUNK_OTEL_VERSION)" \ | ||
./packaging-scripts/update-otel-version.sh | ||
TA_VERSION="$(TA_VERSION)" \ | ||
$(SOURCE_DIR)/packaging-scripts/update-ta-version.sh | ||
|
||
.PHONY: release | ||
release: | ||
SOURCE_DIR="$(SOURCE_DIR)" \ | ||
$(SOURCE_DIR)/packaging-scripts/release-ta-version.sh | ||
|
||
.PHONY: update-and-release | ||
update-and-release: update-deps release | ||
|
||
.PHONY: clean | ||
clean: | ||
rm -rf build |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,42 @@ | ||
# Need Help? | ||
See `#otel-ta-community` for help. | ||
|
||
# Developing this TA | ||
See [`CONTRIBUTING.md`](./CONTRIBUTING.md) | ||
|
||
# Configuration | ||
## environment variables | ||
In addition to the typical collector environment variables, we additionally provide | ||
|
||
- `$SPLUNK_HOME` ex `/opt/splunk` | ||
- `$SPLUNK_OTEL_TA_HOME` ex `/opt/splunk/etc/apps/Splunk_TA_Otel` | ||
- `$SPLUNK_OTEL_TA_PLATFORM_HOME` ex `/opt/splunk/etc/apps/Splunk_TA_Otel/linux_x86_64` | ||
|
||
These are useful for setting things in `inputs.conf` such as | ||
- `splunk_bundle_dir=$SPLUNK_OTEL_TA_PLATFORM_HOME/bin/agent-bundle` | ||
- `splunk_collectd_dir=$SPLUNK_OTEL_TA_PLATFORM_HOME/bin/agent-bundle/run/collectd` | ||
|
||
There is a debug environment variable for the TA scripts, which will log verbose | ||
messages if set to anything other than the empty string. | ||
- `SPLUNK_OTEL_TA_DEBUG` | ||
|
||
## parameters | ||
See `Splunk_TA_Otel/README/inputs.conf.spec` or `Splunk_TA_Otel/default/inputs.conf` for configuration values. | ||
|
||
Of note is that we need to read the `access_token` from a file into an environment variable during TA initialization. | ||
By default, this access_token is expected to live in `$SPLUNK_OTEL_TA_HOME/local/access_token` | ||
|
||
|
||
# Installation | ||
|
||
## Reducing the size of the TA | ||
Customers may want to remove unnecessary configuration if they're only using one | ||
operating system (ex windows, linux). | ||
|
||
This can be accomplished by removing either `Splunk_TA_Otel/linux_x86_64` or | ||
`Splunk_TA_Otel/windows_x86_64`, respectively. | ||
|
||
## Maintaining configuration between upgrades | ||
As with all TAs, any changes made to `configs` will be overwritten. | ||
Customers should copy any relevant custom configuration from `configs/` or `defaults/` | ||
to `local/`. |
19 changes: 19 additions & 0 deletions
19
internal/buildscripts/packaging/technical-addon/RELEASE.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,19 @@ | ||
# Making a new release of the TA | ||
|
||
## Update versions for splunk otel collector && TA | ||
1. run `make update-deps` | ||
2. Verify `git diff main -- Makefile Splunk_TA_otel/default/app.conf` shows expected changes | ||
|
||
|
||
## Cut Release | ||
- run `make update-and-release` to automatically upgrade to the latest splunk otel collector release and bump the minor release. | ||
- For more control, you can directly specify the environment variables `SPLUNK_OTEL_VERSION` and `TA_VERSION` | ||
- Example: `SPLUNK_OTEL_VERSION=0.0.0 TA_VERSION=0.0.1 make -e update-and-release` (replacing the version values as desired) | ||
- upload `out/distribution/Splunk_TA_otel.tgz` artifact from `build-all-platforms` job of release branch to splunkbase | ||
|
||
If you have issues with pushing the branch due to duplicate refs or anything, `git tag -d TAG_NAME` can be useful | ||
|
||
## Market release | ||
1. Inform docs team of release (or DIY) | ||
2. Include link to our [latest release notes](https://github.com/signalfx/splunk-otel-collector/releases/) | ||
3. Optionally include link to [latest upstream release notes](https://github.com/open-telemetry/opentelemetry-collector-contrib/releases) |
144 changes: 144 additions & 0 deletions
144
internal/buildscripts/packaging/technical-addon/Splunk_TA_otel/README.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,144 @@ | ||
# Splunk Add-on for OpenTelemetry Collector | ||
|
||
This technical add-on for Splunk Universal Forwarder helps in deploying the OpenTelemetry Collector alongside Universal Forwarder. | ||
|
||
Splunk Add-on for OpenTelemetry Collector is supported on Linux and Windows on `amd64` (`x86_64`) | ||
|
||
## Contents | ||
|
||
The package contains the following folders and files - | ||
|
||
------------| | ||
|------windows_x86_64 | ||
|
||
------------| | ||
|------linux_x86_64 | ||
|
||
------------| | ||
|------README | ||
|
||
------------| | ||
|------default | ||
|
||
------------| | ||
|------configs | ||
|
||
------------| | ||
|------README.md | ||
|
||
|
||
The Windows and Linux folders contain platform specific binaries. The default folder contains the `app.conf` and `inputs.conf` files. The configs folder contains sample configuration files for collecting host metrics and traces using the OpenTelemetry Collector. | ||
|
||
## Install the add-on | ||
|
||
The installation process for the Splunk Add-on for OpenTelemetry Collector differs depending on the deployment method. | ||
|
||
### Splunk Deployment Server | ||
|
||
Follow these steps to install the add-on using Splunk Deployment Server: | ||
|
||
1. Extract the `Splunk_TA_otel.tgz` file to the `$SPLUNK_HOME/etc/deployment-apps` folder. | ||
2. Edit the configuration files. | ||
3. Follow the instructions for deploying apps. See https://docs.splunk.com/Documentation/Splunk/8.1.2/Updating/Updateconfigurations. | ||
4. Make sure to activate the restart of Universal Forwarder after deployment. | ||
|
||
### Splunk Universal Forwarder | ||
|
||
To install the add-on directly on Universal Forwarder, follow these steps: | ||
|
||
1. Log in to Universal Forwarder. | ||
2. Copy the tar file on the server. | ||
3. Extract the package to the `$SPLUNK_HOME/etc/apps` folder: | ||
|
||
``` | ||
tar -zxf Splunk_TA_otel.tgz | ||
``` | ||
|
||
## Configure the add-on | ||
|
||
Before using the add-on, create a `local` folder and write the contents of your Splunk Observability Cloud access token to this file. | ||
**WE STRONGLY RECOMMEND** that you restrict read and write to this token to the minimal possible permissions. | ||
Alternatively, you can set the `SPLUNK_ACCESS_TOKEN` environment variable to avoid needing to write this. | ||
|
||
```sh | ||
cd /opt/splunkforwarder/etc/apps/Splunk_TA_otel/ | ||
mkdir local | ||
cp -R config local | ||
touch local/access_token | ||
# make the contents of the access_token file your o11y access token. | ||
``` | ||
|
||
After all the steps are completed, restart Splunk Universal Forwarder by running `$SPLUNK_HOME/bin/splunk.exe restart` | ||
|
||
## Configure the OpenTelemetry Collector | ||
|
||
To configure the OpenTelemetry Collector within the add-on, follow these steps: | ||
|
||
1. Create a new configuration file in YAML format for the Collector. For more information, see [Configure the Collector](https://docs.splunk.com/Observability/gdi/opentelemetry/configure-the-collector.html) in the Splunk Observability Cloud documentation. | ||
2. Edit the inputs.conf file inside default to point to the new configuration file. | ||
3. Restart Splunk Universal Forwarder. | ||
|
||
Various settings in the `inputs.conf.spec` are "pass through" [environment variables](https://github.com/signalfx/splunk-otel-collector/blob/main/internal/settings/settings.go#L37-L64) to the default splunk distribution of the opentelemetry collector's agent configuration. We do not currently pass through things related to `gateway` nor for the splunk hec related settings. That said, the binary vended in the TA is identical to upstream | ||
|
||
Note that we additionally configure the following environment variables in the hopes of making configuration easier. | ||
- `$SPLUNK_OTEL_TA_HOME` References the location of the TA, ex `/opt/splunk/etc/apps/Splunk_TA_Otel` | ||
- `$SPLUNK_OTEL_TA_PLATFORM_HOME` references the location of the platform-specific configuration for the TA, ex `/opt/splunk/etc/apps/Splunk_TA_Otel/linux_x86_64`. By default the `splunk_bundle_dir` and `splunk_collectd_dir` options in `inputs.conf` references this environment variable. | ||
|
||
Finally, setting `$SPLUNK_OTEL_TA_DEBUG` to anything other than an empty string will provide detailed logging messages during TA start up. | ||
|
||
|
||
## Check operational status | ||
|
||
Both the add-on and the OpenTelemetry Collector generate log files to indicate operational status. | ||
|
||
You can find the log files in the `$SPLUNK_HOME/var/log/splunk/` folder. | ||
|
||
- The Splunk add-on log file is `Splunk_TA_otel.log`. | ||
- By default, the OpenTelemetry Collector log file is `otel.log` (although you can override this path to whatever you please). | ||
|
||
## Explore metrics and traces | ||
|
||
You can browse the metrics and traces collected by the add-on in in Splunk Observability Cloud. See the [Splunk Observability Cloud documentation for more information](https://docs.splunk.com/Observability). | ||
|
||
If you're using the default configuration, you can search [metrics finder](https://docs.splunk.com/observability/en/metrics-and-metadata/metrics-finder-metadata-catalog.html#metrics-finder-and-metadata-catalog) for [`splunk.distribution:otel-ta`](https://app.signalfx.com/#/metrics?sources%5B%5D=splunk.distribution:otel-ta). | ||
|
||
*note*: if you're using your own configuration, *please* continue to include the telemetry in the configuration. | ||
|
||
|
||
Under `processors` | ||
``` | ||
resource/telemetry: | ||
attributes: | ||
- action: insert | ||
key: splunk.distribution | ||
value: otel-ta | ||
``` | ||
|
||
Under `receivers` | ||
``` | ||
# This section is used to collect the OpenTelemetry Collector metrics | ||
# Even if just a Splunk APM customer, these metrics are included | ||
prometheus/internal: | ||
config: | ||
scrape_configs: | ||
- job_name: 'otel-collector' | ||
scrape_interval: 10s | ||
static_configs: | ||
- targets: ["${env:SPLUNK_LISTEN_INTERFACE}:8888"] | ||
metric_relabel_configs: | ||
- source_labels: [ __name__ ] | ||
regex: '.*grpc_io.*' | ||
action: drop | ||
``` | ||
|
||
Under `pipelines`, assuming you're using a `signalfx` exporter (only `otlp` is supported for our internal telemetry metrics) | ||
``` | ||
metrics/telemetry: | ||
receivers: [prometheus/internal] | ||
processors: [memory_limiter, batch, resourcedetection, resource/telemetry] | ||
exporters: [signalfx] | ||
``` | ||
|
||
# Binary File Declaration | ||
`linux_x86_64/bin/otelcol_linux_amd64` | ||
`windows_x86_64/bin/otelcol_windows_amd64.exe` |
55 changes: 55 additions & 0 deletions
55
internal/buildscripts/packaging/technical-addon/Splunk_TA_otel/README/inputs.conf.spec
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,55 @@ | ||
[Splunk_TA_otel://<name>] | ||
|
||
splunk_access_token_file = <value> | ||
* File whose contents store the credentials to be set in `SPLUNK_ACCESS_TOKEN` (used to auth with Splunk Observability Cloud, default `$SPLUNK_OTEL_TA_HOME/local/access_token`). | ||
|
||
splunk_otel_log_file = <value> | ||
* Log file for otel collector | ||
|
||
|
||
# Below are all "pass through" configuration options that will enable environment variables supported in | ||
# https://github.com/signalfx/splunk-otel-collector/blob/main/internal/settings/settings.go#L37-L64 | ||
gomemlimit = <value> | ||
* Value to use for the `GOMEMLIMIT` environment variable | ||
|
||
splunk_api_url = <value> | ||
* Value to use for the `SPLUNK_API_URL` environment variable | ||
|
||
splunk_bundle_dir = <value> | ||
* Value to use for the `SPLUNK_BUNDLE_DIR` environment variable (used in smart agent config, default `$SPLUNK_OTEL_TA_PLATFORM_HOME/bin/agent-bundle`). NOTE: Once extracted, please do not move the folder structure, as some "first-run" patching occurs that depends on the directory it was run in. Instead, allow the TA to re-extract to the new location if changed. | ||
|
||
splunk_collectd_dir = <value> | ||
* Value to use for the `SPLUNK_COLLECTD_DIR` environment variable (used in smart agent config, default `$SPLUNK_OTEL_TA_HOME/bin/agent-bundle/run/collectd`) | ||
|
||
splunk_config = <value> | ||
* Value to use for the `SPLUNK_CONFIG` environment variable (default `$SPLUNK_OTEL_TA_HOME/config/ta_agent_config.yaml`) | ||
|
||
splunk_config_dir = <value> | ||
* Value to use for the `SPLUNK_CONFIG_DIR` environment variable (default `$SPLUNK_OTEL_TA_HOME/config/`) | ||
|
||
splunk_debug_config_server = <value> | ||
* Value to use for the `SPLUNK_DEBUG_CONFIG_SERVER` environment variable | ||
|
||
splunk_config_yaml = <value> | ||
* Value to use for the `SPLUNK_CONFIG_YAML` environment variable | ||
|
||
splunk_listen_interface = <value> | ||
* Value to use for the `SPLUNK_LISTEN_INTERFACE` environment variable | ||
|
||
splunk_gateway_url = <value> | ||
* Value to use for the `SPLUNK_GATEWAY_URL` environment variable | ||
|
||
splunk_memory_limit_mib = <value> | ||
* Value to use for the `SPLUNK_MEMORY_LIMIT_MIB` environment variable | ||
|
||
splunk_memory_total_mib = <value> | ||
* Value to use for the `SPLUNK_MEMORY_TOTAL_MIB` environment variable | ||
|
||
splunk_trace_url = <value> | ||
* Endpoint for `SPLUNK_TRACE_URL` | ||
|
||
splunk_ingest_url = <value> | ||
* Endpoint for `SPLUNK_API_URL` | ||
|
||
splunk_realm = <value> | ||
* Splunk Observability realm to use for the `SPLUNK_REALM` environment variable (ex us0) |
Oops, something went wrong.