-
Notifications
You must be signed in to change notification settings - Fork 0
Open
Labels
enhancementNew feature or requestNew feature or request
Milestone
Description
Add an internal verification step to cross-check the LLM's claims (gene names, locus tags) against the input annotation file.
Tasks
- Create a new helper function that runs after the LLM call.
- This function will parse the
potential_featuresand check if thesupporting_genesandlocus_tagsexist in the original data. - Define the additions to the JSON scheme (how explicit to be in the tag)
- The function will add a new key to each feature in the JSON, (
"verification_status"). - If a gene or locus tag is not found or gets mismatched, it will tag it appropriately (
"verification_status": "possible hallucination"). - The helper function for the human-readable
.txtreport will add some notes on detected hallucinations in the output - Test thoroughly with known-good and known-bad data.
- Update documentation to explain the verification status tags.
- (Optional) look into more advanced hallucination detection techniques (RAG)
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request