How to create analyzer combining hf model and deny list?

[nogaeps]

Hey,

Im trying to create an analyzer that that uses both deny list and hf model:

HF model:

conf_file = "hf_recognizer.yaml"
provider = NlpEngineProvider(conf_file=conf_file)
nlp_engine = provider.create_engine()

Deny lists:

yaml_file = "recognizers.yaml"
registry = RecognizerRegistry()
registry.add_recognizers_from_yaml(yaml_file)

Combine into single analyzer:

analyzer = AnalyzerEngine(
    nlp_engine=nlp_engine,
    supported_languages=["en"],
    registry=registry
)

When trying to use both registry and nlp_engine to create an AnalyzerEngine, the analyzer uses only the Deny-List.
I'd like to create a model that search for the deny-lists I've created but when no entity is found, it uses the nlp_engine of the hf-model.
Can it be done using a single analyzerEngine or only as multi-steps function that I should build?

Thanks!

[omri374]

Hi, when you create a RecognizerRegistry and add recognizers from file, only these recognizers would be used.
You can either call registry.load_predefined_recognizers() yourself (which would include the transformers recognizer), or add the new recognizer at a later stage:

analyzer = AnalyzerEngine(
    nlp_engine=nlp_engine,
    supported_languages=["en"]
)

analyzer.registry.add_recognizers_from_yaml(yaml_file)

[nogaeps]

Cool thanks!!
Is there a way to priorities entities so the output will contain the higher priority when there is a conflict?
For example, my analyzer should detect a company name as COMP but also as PORTFOLIO since the exact name is in the deny list.
I'd like the token to get be associated with PORTFOLIO only since its in higher priority.

[omri374]

One way to affect priorities is through the confidence scores provided by each recognizer. Alternatively, you can take the output of the analyze method, and do some post processing on the results, for example filter out duplicates in a specific way.