plugin architecture: how to handle new fields added by plugins

[mmguero]

@mmguero cloned issue idaholab/Malcolm#583 on 2024-10-01:

This is a sub-issue of Malcolm "plugin architecture" #399

In the example of something like a custom zeek package, new fields are going to need to be defined. To be fully integrated, they have to be defined in a few places:

• opensearch templates

• in arkime as a custom field and/or custom view

• there may be custom logstash stuff that needs to be done, for example, with a new Zeek package:

  • this gets tricky as it puts its fingers in a few places here

  • perhaps just requiring JSON parsing for zeek would fix the issue, rather than having to define the fields?

  • or, maybe cleaner, we could split this entire parsing section into separate logstash .conf files and dynamically generate them, perhaps

See Adding new log fields for related documentation