Add reference to processor.{event,type} removal in changelog and update docs

[marclop]

In 8.10.0 onwards, we removed processor.event and processor.name from the model completely (elastic/apm-data#79). Now, these are set by Elasticsearch in ingest pipelines: #11028. This all works well for the majority of use cases.

However, it appears we overlooked or dismissed use-cases where we users use those fields in Logstash for routing purposes (https://www.elastic.co/guide/en/apm/guide/current/logstash-output.html#ls-config-pipeline)

Since those fields are no longer set within APM Server, users can't use those fields' values in Logstash to route events. However, they could use [data_stream][type] instead of [processor][event] to route traces, metrics and logs to a datastream if they wanted to.

[bmorelli25]

@colleenmcginnis, there might be some relation to the changes made in #11165. Not sure, but posting so I don't forget.

[bmorelli25]

Ah yes. @marclop, I was able to get the following config to work in 8.10 (which includes using [processor][event]. See this thread: #11165 (comment).

output {
  if [@metadata][beat] == "apm-server" {
    if [processor][event] == "metric" {
      elasticsearch {
        index => "%{[data_stream][type]}-custom-%{[data_stream][namespace]}"
        action => "create"
        cloud_id => "${CLOUD_ID}"
        cloud_auth => "${CLOUD_AUTH}"
      }
    } else {
      elasticsearch {
        data_stream => "true"
        cloud_id => "${CLOUD_ID}"
        cloud_auth => "${CLOUD_AUTH}"
      }
    }
  }
}

[marclop]

@bmorelli25 it seems we only merged the changes in APM Server on July 18th. The documentation PR was tested on July 14th. Which means that unless you've re-tested the configuration above with a released 8.10.x build, it's highly unlikely it'll work.

[bmorelli25]

Ahh. My apologies for not checking the merge dates. That makes sense. Thanks!