add dynamic retrieval set filtering

[pickleerik]

results are very sensitive to the choice of the top k parameter (limit of the context search functionality). You could use something akin to a top-p parameter or a otsu's algorithm based filtering if there are clear groups of retrieval scores.