New OpenSearch API source implementation

[sb2k16]

Description

In order for DataPrepper to support all OpenSearch Document API(s), we need to build a new source similar to the existing http source. This pull request is intended to implement a new OpenSearch API source like opensearch_api similar to http source. This source should support the Document API Bulk.

This pull request includes the following:

1. Path and HTTP methods:
   • POST _bulk
   • POST <index>/_bulk
2. Optional URL parameters
   • pipeline and routing. (TODO: pipeline parameter handling on Sink side)
3. Multi-line JSON Bulk Request payload

Example pipeline configuration with opensearch_api source looks like:

simple-sample-pipeline:
  source:
    opensearch_api:
      path: "/opensearch"
      port: 9202
  sink:
   ...

Issues Resolved

Contributes to #248

Check List

• New functionality includes testing.
• New functionality has a documentation issue. Please link to it in this PR.
• New functionality has javadoc added
• Commits are signed with a real name per the DCO

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[dlvenable]

Does the health check return any data?

What I meant by my comment from the other PR is that we should return the same JSON format that the OpenSearch root endpoint returns.

e.g.

docker run -p 9200:9200 -p 9600:9600 -e "discovery.type=single-node" -d opensearchproject/opensearch:2.11.1

curl -k -u "admin:admin" 'https://localhost:9200/'
{
  "name" : "9c3406b376ef",
  "cluster_name" : "docker-cluster",
  "cluster_uuid" : "ud9QWcTcQruIwlEIjIpYdg",
  "version" : {
    "distribution" : "opensearch",
    "number" : "2.11.1",
    "build_type" : "tar",
    "build_hash" : "6b1986e964d440be9137eba1413015c31c5a7752",
    "build_date" : "2023-11-29T21:45:35.524809067Z",
    "build_snapshot" : false,
    "lucene_version" : "9.7.0",
    "minimum_wire_compatibility_version" : "7.10.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "The OpenSearch Project: https://opensearch.org/"
}

Maybe we don't return all of that. But, maybe we return {} to start. What I want to avoid is making this return a health response that will not match with the root endpoint going forward.

I'm ok with just removing the health endpoint for the sake of this PR if you want.

[sb2k16]

Thanks @dlvenable. It makes sense to not return a response which does not match the root endpoint response to opensearch. I have disabled it. If customers want to access, data prepper will respond with a 404.

[dlvenable]

Thank you @sb2k16. This is a great contribution and a great new feature for Data Prepper!