Combine values under the same key from a list of key-value pairs

[engechas]

Is your feature request related to a problem? Please describe.
As a user, I would like to have the ability to transform a map to a list, or flatten a list of maps to a list.

Example input:

{
  "mylist": [
    { "a": "b" },
    { "a": "c" },
    { "x": "y" },
  ]
}

Example output:

{
  "a": ["b", "c"],
  "x": ["y"]
}

Describe the solution you'd like
A new processor created to implement the above described functionality

[dlvenable]

@engechas , I'm a bit unsure how we have the first example to begin with.

"a": "b",
"a": "c",

There should be only one a in the incoming Event. How do we have multiple a keys?

[engechas]

You're right, the first one would not be possible. I've removed it

[oeyh]

The example given looks like what the ListToMap processor can handle with some changes, or a new processor that aggregates objects inside an array.

I created another issue for "MapToList" processor that performs approximately the reversed operation of the ListToMap processor.

[oeyh]

If I can simplify the request a bit so that the objects in mylist all have same keys:
input:

{
  "mylist": [
    { "a": "b" },
    { "a": "c" },
    { "a": "y" },
  ]
}

output:

{
  "a": ["b", "c", "y"]
}

this could already help with the original user request (from an internal ticket) as well as #3963 and #3964.

We can achieve this transformation with a small change to the existing list_to_map processor:

• Make key an optional config. When key is not specified, use value_key literally as the key in the result map.

The processor config for the above example would look like this:

  processor:
    - list_to_map:
        value_key: "a"
        source: "mylist"

[dlvenable]

@oeyh , The original ask in this issue is to get the key from the inner map and then combine the values together.

I do think this can be achieved by using the existing list_to_map processor. But, it seems we are looking for more of a dynamic target key. Maybe a boolean would be most appropriate?

- list_to_map:
    source: mylist
    use_source_key: true

[oeyh]

@dlvenable My only concern with the above approach is that the behavior for value_key would change. value_key specifies the key to extract value; if not specified, we take the entire object as value (see #2410). We would get result like this:

{
  "a": [{ "a": "b" }, { "a": "c" }]
  "x": [{ "x": "y" }]
}

But we can probably add another boolean flag extract_value to go with use_source_key?

- list_to_map:
    source: mylist
    use_source_key: true
    extract_value: true