Skip to content

Latest commit

 

History

History
214 lines (159 loc) · 11.8 KB

CHANGELOG-developer.asciidoc

File metadata and controls

214 lines (159 loc) · 11.8 KB
Raw

This changelog is intended for community Beat developers. It covers the major breaking changes to the internal APIs in the official Beats and changes related to developing a Beat like code generators or fields.yml. Only the major changes will be covered in this changelog that are expected to affect community developers. Each breaking change added here should have an explanation on how other Beats should be migrated.

Note: This changelog was only started after the 6.3 release.

Beats version 8.0.0

Breaking changes

  • Replace custom Pins type for a slice of string for defining the ca_sha256 values.

Added

  • Added linter step to the CI pipeline based on golangci-lint linter and Mage

Beats version 7.5.1

Beats version 7.5.0

Breaking changes

  • Build docker and kubernetes features only on supported platforms. 13509

  • Need to register new processors to be used in the JS processor in their init functions. 13509

Added

  • Compare event by event in testadata framework to avoid sorting problems 13747

Beats version 7.4.1

Beats version 7.4.0

Breaking changes

  • For "metricbeat style" generated custom beats, the mage target GoTestIntegration has changed to GoIntegTest and GoTestUnit has changed to GoUnitTest. 13341

Added

  • Add ClientFactory to TCP input source to add SplitFunc/NetworkFuncs per client. 8543

  • Introduce beat.OutputChooses publisher mode. 12996

  • Ensure that beat.Processor, beat.ProcessorList, and processors.ProcessorList are compatible and can be composed more easily. 12996

  • Add support to close beat.Client via beat.CloseRef (a subset of context.Context). 13031

  • Add checks for types and formats used in fields definitions in fields.yml files. 13188

  • Makefile included in generator copies files from beats repository using git archive instead of cp. 13193

Beats version 7.3.2

Beats version 7.3.1

Beats version 7.3.0

Added

  • Add new option IgnoreAllErrors to libbeat.common.schema for skipping fields that failed while converting. 12089

Beats version 7.2.1

Beats version 7.2.0

Breaking changes

  • Move Fields from package libbeat/common to libbeat/mapping. 11198

Added

  • Metricset generator generates beta modules by default now. 10657

  • The beat.Event accessor methods now support @metadata keys. 10761

  • Assertion for documented fields in tests fails if any of the fields in the tested event is documented as an alias. 10921

  • Support for Logger in the Metricset base instance. 11106

  • Filebeat modules can now use ingest pipelines in YAML format. 11209

  • Prometheus helper for metricbeat contains now Namespace field for prometheus.MetricsMappings 11424

  • Update Jinja2 version to 2.10.1. 11817

  • Reduce idxmgmt.Supporter interface and rework export commands to reuse logic. 11777,https://github.com/elastic/beats/pull/12065[12065],https://github.com/elastic/beats/pull/12067[12067],https://github.com/elastic/beats/pull/12160[12160]

  • Update urllib3 version to 1.24.2 11930

  • Add libbeat/common/cleanup package. 12134

  • Only Load minimal template if no fields are provided. 12103

  • Add new option IgnoreAllErrors to libbeat.common.schema for skipping fields that failed while converting. 12089

  • Deprecate setup cmds for template and ilm-policy. Add new setup cmd for index-management. 12132

Beats version 7.1.1

Beats version 7.1.0

Beats version 7.0.1

Beats version 7.0.0-GA

The list below covers the major changes between 7.0.0-rc2 and 7.0 only.

Added

  • Added support for using PYTHON_EXE to control what Python interpreter is used by make and mage. Example: export PYTHON_EXE=python2.7. 11212

Beats version 7.0.0-rc2

Beats version 7.0.0-rc1

Breaking changes

  • Remove support for deprecated GenRootCmd methods. 10721

  • Remove SkipNormalization, SkipAgentMetadata, SkipAddHostName. 10801 10769

Bugfixes

  • Align default index between elasticsearch and logstash and kafka output. 10841

  • Fix duplication check for append_fields option. 10959

Added

  • Introduce processing.Support to instance.Setting. This allows Beats to fully modify the event processing. 10801

Beats version 7.0.0-beta1

Breaking changes

  • Outputs receive Index Manager as additional parameter. The index manager can be used to create an index selector. 10347

  • Remove support for loading dashboards to Elasticsearch 5. 10451

Added

  • Allow multiple object type configurations per field. 9772

  • Move agent metadata addition to a processor. 9952

  • Add (*common.Config).Has and (*common.Config).Remove. 10363

  • Introduce ILM and IndexManagement support to beat.Settings. 10347

  • Generating index pattern on demand instead of shipping them in the packages. 10478

Beats version 7.0.0-alpha2

The list below covers the major changes between 6.3.0 and 7.0.0-alpha2 only.

Breaking changes

  • The beat.Pipeline is now passed to cfgfile.RunnerFactory. Beats using libbeat for module reloading or autodiscovery need to be adapted. 7017

  • Moving of TLS helper functions and structs from output/tls to tlscommon. 7054

  • Port fields.yml collector to Golang 6911

  • Dashboards under _meta/kibana are expected to be decoded. See #7224 for a conversion script. 7265

  • Constructor (github.com/elastic/beats/libbeat/output/codec/json).New expects a new escapeHTML parameter. 7445

  • Packaging has been refactored and updates are required. See the PR for migration details. 7388

  • make fields has been modified to use Mage (https://magefile.org/) in an effort to make the building a Beat more cross-platform friendly (e.g. Windows). This requires that your Beat has a magefile.go with a fields target. The FIELDS_FILE_PATH make variable is no longer used because the value is specified in magefile.go. 7670

  • Outputs must implement String. 6404

  • Renamed -beat-name CLI option used in kibana_index_pattern.go to -beat for consistency with other scripts in dev-tools/cmd. 8615

  • Systemd unit file template used on Linux packaging now includes environment variables to ease flag overriding. One of them includes the -e flag, making beats log to stderr by default on systemd uses. 8942

  • Removed dashboards and index patterns generation for Kibana 5. 8927

  • Move generator packages of Filebeat from scripts/generator to generator. 9147

Bugfixes

  • Fix permissions of generated Filebeat filesets. 7140

  • Collect fields from _meta/fields.yml too. 8397

  • Fix issue on asset generation that could lead to different results in Windows. 8464

  • Remove default version qualifier, you can use VERSION_QUALIFIER environment variable to set it. 9148

Added

  • Libbeat provides a global registry for beats developer that allow to register and retrieve plugin. 7392

  • Added more options to control required and optional fields in schema.Apply(), error returned is a plain nil if no error happened 7335

  • Packaging on MacOS now produces a .dmg file containing an installer (.pkg) and uninstaller for the Beat. 7481

  • Added mage targets goTestUnit and goTestIntegration for executing 'go test'. This captures the log to a file, summarizes the result, produces a coverage profile (.cov), and produces an HTML coverage report. See mage -h goTestUnit. 7766

  • Beats packaging now build non-oss binaries from code located in the x-pack folder. 7783

  • New function AddTagsWithKey is added, so common.MapStr can be enriched with tags with an arbitrary key. 7991

  • Move filebeat/reader to libbeat/reader 8206

  • Libbeat provides a new function cmd.GenRootCmdWithSettings that should be preferred over deprecated functions cmd.GenRootCmd, cmd.GenRootCmdWithRunFlags, and cmd.GenRootCmdWithIndexPrefixWithRunFlags. 7850

  • Set current year in generator templates. 8396

  • You can now override default settings of libbeat by using instance.Settings. 8449

  • Add -space-id option to export_dashboards.go script to support Kibana Spaces 7942

  • Add -name option to asset.go script to explicitly name the asset rather than using its filename. 8693

  • Add -out option to kibana_index_pattern.go to control the output dir to make it possible to write the generated output to build/kibana instead of _meta/kibana.generated (but the output dir remains unchanged at this point). 8615

  • Add module_fields.go for generated fields.go files for modules. 8615

  • Add mage.GenerateModuleReferenceConfig for generating reference config files that include configuration sections from the module directory. 8615

  • Add mage.GenerateFieldsGo for generating fields.go files. 8615

  • Add mage.KibanaDashboards for collecting Kibana dashboards and generating index patterns. 8615

  • Allow to disable config resolver using the Settings.DisableConfigResolver field when initializing libbeat. 8769

  • Add mage.AddPlatforms to allow to specify dependent platforms when building a beat. 8889

  • Add cfgwarn.CheckRemoved6xSetting(s) to display a warning for options removed in 7.0. 8909

  • Add docker image building to mage.Package. 8898

  • Simplified exporting of dashboards. 7730

  • Update Beats to use go 1.11.2 8746

  • Allow/Merge fields.yml overrides 9188

  • Filesets can now define multiple ingest pipelines, with the first one considered as the entry point pipeline. 8914

  • Add group_measurements_by_instance option to windows perfmon metricset. 8688

  • Bump ECS version to 1.10.0. 25734