Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: README refresh and docs update #157

Merged
merged 5 commits into from
Apr 30, 2024
Merged

docs: README refresh and docs update #157

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
00adc1c
docs: README refresh and docs update
trentm Apr 30, 2024
8d78a9f
some more tweaks
trentm Apr 30, 2024
043b1a4
correct grammar
trentm Apr 30, 2024
f811e34
correfct markdown syntax
trentm Apr 30, 2024
945486b
readme usage section tweaks
trentm Apr 30, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
85 changes: 85 additions & 0 deletions DEVELOPMENT.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
This document contains informal notes to help developers of the Elastic APM
Node.js agent. Developers should feel free to aggressively weed out obsolete
notes. For structured developer and contributing *rules and guidelines*, see
[CONTRIBUTING.md](./CONTRIBUTING.md).


### mockotlpserver OTLP endpoint

For local development, it can be useful to have an OTLP endpoint that is local,
and can show the exact details of data being sent by the OTel SDK. The
[mockotlpserver package](./packages/mockotlpserver/) in this repository
provides a CLI tool for this.

```sh
git clone https://github.com/elastic/elastic-otel-node.git
cd elastic-otel-node/
npm ci
cd packages/mockotlpserver
npm start -- --help # mockotlpserver CLI options
npm start
```

This starts a service listening on the default OTLP/gRPC and OTLP/HTTP ports.
It will print received OTLP request data. By default it shows a raw print of
the protobuf request, e.g.:

```
ExportTraceServiceRequest {
resourceSpans: [
ResourceSpans {
scopeSpans: [
ScopeSpans {
spans: [
Span {
attributes: [
KeyValue { key: 'http.url', value: AnyValue { stringValue: 'http://localhost:3000/' } },
...
name: 'GET',
kind: 2,
...
```

and a "summary" compact representation of the request, e.g.:

```
------ trace 802356 (2 spans) ------
span f06b1a "GET" (15.5ms, SPAN_KIND_CLIENT, GET http://localhost:3000/ -> 200)
+9ms `- span 226bf7 "GET" (4.2ms, SPAN_KIND_SERVER, GET http://localhost:3000/ -> 200)
```

Try it with:

```sh
cd elastic-otel-node/examples
node -r @elastic/opentelemetry-node simple-http-request.js
```

See [the mockotlpserver README](./packages/mockotlpserver#readme) for more details.


# Logging tips

## logging

`OTEL_LOG_LEVEL=verbose` will turn on the most verbose-level logging in the SDK,
including enabling the core OpenTelemetry `diag` logger messages.

This distro's logging is currently in the JSON format used by the
[`luggite`](https://github.com/trentm/node-luggite) library. It be somewhat
pretty-formatted via the [`pino-pretty` tool](https://github.com/pinojs/pino-pretty):

OTEL_LOG_LEVEL=verbose node myapp.js | pino-pretty

One of the important libs in the SDK is [require-in-the-middle](https://github.com/elastic/require-in-the-middle)
for intercepting `require(...)` statements for monkey-patching. You can get
debug output from it via:

DEBUG=require-in-the-middle

And don't forget the node core [`NODE_DEBUG` and `NODE_DEBUG_NATIVE`](https://nodejs.org/api/all.html#cli_node_debug_module)
environment variables:

NODE_DEBUG=*
NODE_DEBUG_NATIVE=*

126 changes: 48 additions & 78 deletions packages/opentelemetry-node/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,103 +1,73 @@
# Elastic OpenTelemetry Node.js Distribution
# Elastic OpenTelemetry Distribution for Node.js

This is the Elastic OpenTelemetry Node.js Distribution. It is a light wrapper
around the OpenTelemetry Node SDK that makes it easier to get started using
OpenTelemetry in your Node.js applications, especially if you are using [Elastic
Observability](https://www.elastic.co/observability) as your observability
solution.
This is the Elastic OpenTelemetry Distribution for Node.js (the "Distro").
It is a light wrapper around the OpenTelemetry Node SDK that makes it easier to
get started using OpenTelemetry in your Node.js applications, especially if you
are using [Elastic Observability](https://www.elastic.co/observability) as your
observability solution.

# Current status

Pre-alpha

# Install

Eventually this will be `npm install @elastic/opentelemetry-node`.
However, while still in early development, this package is not yet published
to npm, so you'll need to access it via git:

git clone https://github.com/elastic/elastic-otel-node.git
cd elastic-otel-node/
npm ci

and then install the package sub-directory:
# Current status

npm install .../elastic-otel-node/packages/opentelemetry-node
The current release is **alpha**, and not yet recommended for production use.
We welcome your feedback! You can reach us either on the [issue tracker](https://github.com/elastic/elastic-otel-node/issues)
or on [Elastic's Discuss forum](https://discuss.elastic.co/tags/c/observability/apm/58/nodejs).

(TODO: update ^^ once published to npm.)
Some limitations / notes:
- We expect to support most every instrumentation included in [`@opentelemetry/auto-instrumentations-node`](https://github.com/open-telemetry/opentelemetry-js-contrib/tree/main/metapackages/auto-instrumentations-node#supported-instrumentations). However, currently only a subset is supported. See [the supported instrumentations here](./docs/supported-technologies.md#instrumentations).


# Usage

To start the SDK, it must be loaded before any of your application code. The
recommended way to do that is via Node.js's [`-r, --require`
option](https://nodejs.org/api/all.html#all_cli_-r---require-module):

node -r @elastic/opentelemetry-node my-app.js

TODO: Link to coming user guide for related topics: ES module support, configuration reference, starting the SDK via


# Configuring your telemetry endpoint

By default the SDK will send telemetry data via OpenTelemetry's protocol (OTLP)
to the configured endpoint (by default it sends to <http://localhost:4317>):

OTEL_EXPORTER_OTLP_ENDPOINT=... \
OTEL_EXPORTER_OTLP_HEADERS=... \
node -r @elastic/opentelemetry-node my-app.js

You can send to any OTLP endpoint, for example: an [OTel Collector](https://opentelemetry.io/docs/collector/),
or directly to an Elastic Observability deployment. Since version 7.14, Elastic
[supports OTLP natively](https://www.elastic.co/blog/native-opentelemetry-support-in-elastic-observability).

# 1. install
npm install --save @elastic/opentelemetry-node

### Elastic Observability endpoint
# 2. configure via OTEL_ envvars, for example:
export OTEL_EXPORTER_OTLP_ENDPOINT=https://{your-otlp-endpoint.example.com}
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer {your-Elastic-secret-token}"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here we are using the generic term oltp endpoint but next line we refer to elastic secret token which is more concrete to our stack. I wonder if we should add a note here telling APM server is an OTLP endpoint also

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll change the example here to this:

# 1. install
npm install --save @elastic/opentelemetry-node

# 2. configure via OTEL_ envvars, for example:
export OTEL_EXPORTER_OTLP_ENDPOINT=https://{your-otlp-endpoint.example.com}
export OTEL_EXPORTER_OTLP_HEADERS="Authorization={authorization-information}"
export OTEL_SERVICE_NAME=my-service

# 3. start
node -r @elastic/opentelemetry-node my-service.js

If using an Elastic Observability deployment
to which to send telemetry data, the OTEL_EXPORTER_* settings will look
something like:

export OTEL_EXPORTER_OTLP_ENDPOINT=https://{deployment-name}.apm.{cloud-region}.cloud.es.io
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer {deployment-secret-token}"

export OTEL_SERVICE_NAME=my-service

First, you will need an Elastic APM deployment. See: https://www.elastic.co/guide/en/apm/guide/current/apm-quick-start.html
You will need two pieces of information: the APM **server URL** (this is the OTLP endpoint) and your APM **secret code** (or **API key** if using API keys).
Then configure your
# 3. start
node -r @elastic/opentelemetry-node my-service.js

```sh
export OTEL_EXPORTER_OTLP_ENDPOINT="${ELASTIC_APM_SERVER_URL}"
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer ${ELASTIC_APM_SECRET_TOKEN}"
node -r @elastic/opentelemetry-node my-app.js
```
The Distro will automatically instrument popular modules (see [supported instrumentations](https://github.com/open-telemetry/opentelemetry-js-contrib/tree/main/metapackages/auto-instrumentations-node#supported-instrumentations)))
used by your service, and send trace, metrics, and logs telemetry data (using
OTLP) to your configured observability backend.

Or if using an API key, then:
The Distro can be configured via `OTEL_*` environment variables, per the
[OpenTelemetry Environment Variable spec](https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/).

```sh
export OTEL_EXPORTER_OTLP_ENDPOINT="${ELASTIC_APM_SERVER_URL}"
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=ApiKey ${ELASTIC_APM_API_KEY}"
node -r @elastic/opentelemetry-node my-app.js
```
See the [Getting Started guide](./docs/getting-started.md) for more details.


### mockotlpserver endpoint
# Documentation

TODO: move this out to dev docs
- [Getting Started](./docs/getting-started.md)
- [Supported Technologies](./docs/supported-technologies.md)
- [Metrics](./docs/metrics.md)

If you don't yet have an OTLP endpoint setup and just want to see the SDK
working, you can run a *mock* OTLP server locally with the `mockotlpserver`
utility in this repository:

```sh
git clone https://github.com/elastic/elastic-otel-node.git
cd elastic-otel-node/
npm ci
cd packages/mockotlpserver
npm start
```
# Why this distribution?

Now running an application with this SDK will send to the mock endpoint, which
prints out any received telemetry data, for example:
As mentioned above, this Distro is a wrapper around the [OpenTelemetry Node
SDK (`@opentelemetry/sdk-node`)](https://github.com/open-telemetry/opentelemetry-js/tree/main/experimental/packages/opentelemetry-sdk-node). So why the separate package?
A few reasons:

```sh
cd elastic-otel-node/examples
node -r @elastic/opentelemetry-node simple-http-request.js
```
- With this separate package we hope to experiment with making it easier to get
started with OpenTelemetry instrumentation in Node.js services. For example,
`@elastic/opentelemetry-node` includes a number of OTel packages as dependencies,
so the user only needs to install/update a single package -- at least for the
default use case. This is similar to the OTel
`@opentelemetry/auto-instrumentations-node` package.

See [the mockotlpserver README](../mockotlpserver#readme) for more details.
- Having a separate package will sometimes allow us to iterate more quickly with
changes in SDK behavior. However, our plan is to upstream any improvements to
the OpenTelemetry JS repositories.

- Should it be necessary, having a separate package would allow us to more
quickly release a fix for a particular issue required by a customer of ours.

- Providing an eventual easy migration path for customers of our current
non-OpenTelemetry-based [Node.js APM agent](https://github.com/elastic/apm-agent-nodejs)
to this SDK may be made easier by having our own package entry point.

128 changes: 100 additions & 28 deletions packages/opentelemetry-node/docs/getting-started.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
# Introduction

This is the Elastic OpenTelemetry Distribution for Node.js (the "Distro"). It is
a Node.js package that provides:
- easy way to instrument your application with OpenTelemetry
- configuration defaults for best usage
The Elastic OpenTelemetry Distribution for Node.js (the "Distro") is a Node.js
package that provides:

- an easy way to instrument your application with OpenTelemetry, and
- configuration defaults for best usage.

Use the Distro to start the OpenTelemetry SDK with your Node.js application to automatically
capture tracing data, performance metrics, and logs. Traces, metrics, and logs are sent
Expand All @@ -16,45 +17,116 @@ and quickly identify root causes of service issues.

This getting started guide will show how to use this Distro to instrument your Node.js application and send OpenTelemetry data to an Elastic Observability deployment. Note, however, that as an OpenTelemetry SDK, it supports sending data to any OTLP endpoint, e.g. an [OpenTelemetry Collector](https://opentelemetry.io/docs/collector/).

An Elastic Observability deployment includes an OTLP endpoint to receive data. That data is processed and stored in Elasticsearch, and Kibana provides a web interface to visualize and analyze the date. If you do not already have
a deployment to use, follow [this Quick Start guide](https://www.elastic.co/blog/getting-started-with-elastic-cloud)
to create a free trial on Elastic's cloud. From this deployment you will need
the APM **`serverUrl`** and a configured **`apmAgentKey`** to use for configuring the SDK distribution.

## Installation

```sh
npm install --save @elastic/opentelemetry-node
```

## Initialization
The Distro is a single package that includes all the OpenTelemetry JS packages
that are needed for most cases.

<!-- TODO: refer to advanced section of "start the SDK" when we have that doc. -->

## Configuration

The Distro is a wrapper around the [OpenTelemetry Node.js SDK](https://github.com/open-telemetry/opentelemetry-js/tree/main/experimental/packages/opentelemetry-sdk-node)
and other OpenTelemetry JS packages. It is typically configured with `OTEL_*`
environment variables defined by the OpenTelemetry spec. The most common
configuration settings are `OTEL_EXPORTER_OTLP_*` to set the endpoint for
sending data and `OTEL_SERVICE_NAME` to identify your service.

The Distro will send telemetry data via OpenTelemetry's protocol (OTLP) to the
configured endpoint (by default it sends to <http://localhost:4317>). The
endpoint can be changed by setting the following environment vars:

It’s important that the agent is started before you require **any** other modules
in your Node.js application - i.e. before express, http, etc.
- `OTEL_EXPORTER_OTLP_ENDPOINT`: full URL of the endpoint where to send the data
- `OTEL_EXPORTER_OTLP_HEADERS`: Comma-separated list of `key=value` pairs which
will be added to the headers of every request. Typically this is used for
authentication information.

The preferred way to get the SDK started is by using the `--require`
Node.js [CLI option](https://nodejs.org/api/cli.html#-r---require-module).
For example, to send telemetry data to your Elastic Observability deployment you
might start the application like this:

```sh
node --require @elastic/opentelemetry-node app.js
export OTEL_EXPORTER_OTLP_ENDPOINT=https://{your-otlp-endpoint.example.com}
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer {your-Elastic-secret-token}"
export OTEL_SERVICE_NAME=my-service
```

## Configuration

By default the SDK will send telemetry data via OpenTelemetry's protocol (OTLP)
to the configured endpoint (by default it sends to <http://localhost:4317>). The
endpoint configuration can be changed by setting the following environment vars:
If you don't have an Elastic Observability deployment or don't have the
endpoint and auth data for your deployment, see the [Elastic Observability setup](#elastic-observability-setup)
section below.

- `OTEL_EXPORTER_OTLP_ENDPOINT`: full URL of the endpoint where to send the data.
- `OTEL_EXPORTER_OTLP_HEADERS`: coma separated list of `key=value` pairs which will
be added to the headers of every request.
See the [OpenTelemetry Environment Variable spec](https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/) for other available configuration settings.

## Initialization

As an example if you want to send telemetry data to your Elastic's APM deployment you
may start the application like this
For the Distro to automatically instrument modules used by your Node.js service,
it must be started before your service code `require`s its dependencies --
e.g. before `express`, `http`, etc. are loaded. The recommended way to get the
Distro started is by using the `-r, --require` Node.js
[CLI option](https://nodejs.org/api/cli.html#-r---require-module).

```sh
export OTEL_EXPORTER_OTLP_ENDPOINT="https://apm-server-url.co"
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=ApiKey VnVhQ2ZHY0JDZGJr..."
node -r @elastic/opentelemetry-node/start.js my-app.js
node --require @elastic/opentelemetry-node my-service.js
```

The Distro will automatically instrument popular modules (see [supported instrumentations](./supported-technologies.md#instrumentations))
used by your service, and send traces, metrics, and logs telemetry data (using
OTLP) to your configured observability backend.

<!-- TODO: link to a reference section on other ways to start the Distro once we have those docs. -->


# Elastic Observability setup

You'll need somewhere to send the gathered OpenTelemetry data, so it can be
viewed and analyzed. The `@elastic/opentelemetry-node` package supports sending
to any OTLP endpoint (e.g. an OpenTelemetry collector instance). This section
shows how to create an [Elastic Observability](https://www.elastic.co/observability)
cloud deployment and get the data you need to configure the Distro to send
data to it.

1. Register at [cloud.elastic.co](https://cloud.elastic.co/registration), if you haven't already. This supports starting **free trial**.

2. After registering, click "Create deployment" at <https://cloud.elastic.co/home>. Once that is created, click "Open" to visit your Kibana home page, e.g. `https://{DEPLOYMENT_NAME}.kb.{REGION}.cloud.es.io/app/home#/getting_started`.

To configure the Distro you'll need the deployment's OTLP endpoint and
authorization data to set the appropriate `OTLP_*` environment variables. You
can find these in Kibana's APM tutorial.

![Kibana's APM tutorial showing OTel settings](./img/otlp-endpoint-settings.png)

3. In Kibana:

- search for "APM Tutorial",
- scroll down to the "APM Agents" section and select the "OpenTelemetry" tab,
- the appropriate values for `OTEL_EXPORTER_OTLP_ENDPOINT` and
`OTEL_EXPORTER_OTLP_HEADERS` are shown there.

For example:

```
export OTEL_EXPORTER_OTLP_ENDPOINT=https://my-deployment.apm.us-west1.gcp.cloud.es.io
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer P....l"
```

Use these environment variables in the [Configuration](#configuration) step
above to configure the Distro.

## Authenticate using an APM Agent key (API key)

It is also possible to authenticate to an Elastic Observability endpoint using
an "APM Agent key". These are revocable API keys. To create and manage
APM Agent keys, see the "Agent Keys" tab in "APM Settings" in Kibana.

![Kibana's APM Agent Keys section](./img/kibana-apm-agent-keys.png)

When using an APM Agent key, the `OTEL_EXPORTER_OTLP_HEADERS` is set using a
different auth schema (`ApiKey` rather than `Bearer`). For example:

```
export OTEL_EXPORTER_OTLP_ENDPOINT=https://my-deployment.apm.us-west1.gcp.cloud.es.io
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=ApiKey TkpXUkx...dVZGQQ=="
```
Binary file added packages/opentelemetry-node/docs/img/kibana-apm-agent-keys.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added packages/opentelemetry-node/docs/img/otlp-endpoint-settings.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Loading