Add exclude_options configuration to filter options for generate

[emcfarlane]

This adds a new plugin option exclude_options for use on generation. The excluded options are stripped from the Image creating a shallow clone of the FileDescriptors that make up the ImageFiles. Any imports that are no longer required due to options being removed are also removed. The option is applied for the plugin. Multiple filtered images that reference the original image may be created when exclude options are set.

For example to filter buf.validate types from the plugin protoc-gen-es you can now provide the following exclude_options types:

plugins:
  - local: protoc-gen-es
    opt: target=ts
    out: gen
    include_imports: true
    exclude_options: # exclude all of protovalidate
      - "buf.validate.oneof"
      - "buf.validate.message"
      - "buf.validate.field"

The image filter functions ImageFilteredByTypes and ImageFilteredByTypesWithOptions has been replaced by the single function FilterImage which takes the image and a set of options. Options include WithIncludeTypes, WithExcludeTypes, WithIncludeOptions and WithExcludeOptions. The previous behaviour is replicated by setting the types in WithIncludeTypes.

The filter image will by default do a shallow copy. This allows for sharing of the original image to multiple filters. To avoid the overhead of copying an option WithMutateInPlace can be set to mutate in place.

BenchmarkFilterImage_WithSourceCodeInfo-8 (current)      15538             77970 ns/op           42243 B/op        957 allocs/op
BenchmarkFilterImage_WithSourceCodeInfo-8 (copy)          7560            165533 ns/op          118865 B/op       2345 allocs/op
BenchmarkFilterImage_WithSourceCodeInfo-8 (mutate)       25021             47970 ns/op           41541 B/op        531 allocs/op

Fixes #645

[github-actions]

The latest Buf updates on your PR. Results from workflow Buf CI / buf (pull_request).

Build	Format	Lint	Breaking	Updated (UTC)
✅ passed	✅ passed	✅ passed	✅ passed	Mar 4, 2025, 9:15 PM

[jhump]

There is already a "types" config, allowing a filter of what types to include. We made it a struct with an "includes" field with the intention of perhaps adding "excludes" in the future.

So maybe we should that there and then allow the "types" to be specified per plugin, not just at the top-level. (May have to think about how the two interact, if types are specified at both the top-level and for a particular plugin: do they merge or does the plugin definition override the top-level one).

This would be more consistent and also be more expressive: one could strip any kind of element from the image, not just options. Also, there is already logic in the type filtering code to handle auto-pruning imports based on what's left. The main trick with excluding arbitrary elements is that, if an enum or a message is indicated, we may need to mutate other messages to remove fields that refer to that enum or message.

[emcfarlane]

I think the approaches could be merged to a general filter too. The difference was required to get the non mutating behaviour. We want this filter applied at the plugin level so to avoid mutating the Image for every plugin option it only does a shallow copy as needed. Which is similar to the approach taken for source retention options in protopluginutil. There was also the performance case of excluding vs including on small set of types. The current walks from the set of types, but that would be all the types in the image for the option exclusions.

Storing the deletes as a trie I think we can get a performant approach for both use cases. When walking the file descriptors we can mark nodes for inclusion in the image on entry, then on the exit visit if any child node is required we can mark those excluded nodes as enclosing, updating the trie on the exit.

Edit: actually the mark on exit won't work. Will have to add nodes using the transitive closure. Will try merge the solutions.

[emcfarlane]

@jhump I reconfigured the image filtering to use the same code paths for types and options. For the configuration: I've kept them as separate things for now as the behaviour of the two parameters are slightly different. Types filters must be present in the image else a not found error is returned. Options filters may not exist, they don't directly exclude/include a type, and options may specify both extension and non-extension fields. We could expand the config to include both a types include and exclude parameter at the input and plugin level, as well as the parameters include_options and exclude_options, but for now only exclude_options has been added. These are merged when used together