Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fixes and improvements to task based llm as judge #1366

Open
wants to merge 9 commits into
base: main
Choose a base branch
from
Open

Fixes and improvements to task based llm as judge #1366

wants to merge 9 commits into from
+107 −65

Commits on Nov 19, 2024

  1. Add info and functionality to GenericInferenceEngine 

    Signed-off-by: lilacheden <lilach.edel@gmail.com>
    @lilacheden
    lilacheden committed Nov 19, 2024
    Configuration menu
    Copy the full SHA
    a6d407a View commit details
    Browse the repository at this point in the history

  2. Set task-based llmaj format according to inference engine, unless sup… 

    …plied directly

Signed-off-by: lilacheden <lilach.edel@gmail.com>
    @lilacheden
    lilacheden committed Nov 19, 2024
    Configuration menu
    Copy the full SHA
    4002a01 View commit details
    Browse the repository at this point in the history

  3. set default format to None for binary juges 

    Signed-off-by: lilacheden <lilach.edel@gmail.com>
    @lilacheden
    lilacheden committed Nov 19, 2024
    Configuration menu
    Copy the full SHA
    0fcee1f View commit details
    Browse the repository at this point in the history

  4. add data_classification_policy to example 

    Signed-off-by: lilacheden <lilach.edel@gmail.com>
    @lilacheden
    lilacheden committed Nov 19, 2024
    Configuration menu
    Copy the full SHA
    4d3ef47 View commit details
    Browse the repository at this point in the history

  5. make contexts optional for answer_correctness 

    Signed-off-by: lilacheden <lilach.edel@gmail.com>
    @lilacheden
    lilacheden committed Nov 19, 2024
    Configuration menu
    Copy the full SHA
    bd9a685 View commit details
    Browse the repository at this point in the history

  6. Adjust examples to new metrics paths 

    Signed-off-by: lilacheden <lilach.edel@gmail.com>
    @lilacheden
    lilacheden committed Nov 19, 2024
    Configuration menu
    Copy the full SHA
    c9d5b0b View commit details
    Browse the repository at this point in the history

  7. Use get_artifact in task based llmasjudge 

    Signed-off-by: lilacheden <lilach.edel@gmail.com>
    @lilacheden
    lilacheden committed Nov 19, 2024
    Configuration menu
    Copy the full SHA
    1e409fe View commit details
    Browse the repository at this point in the history

  8. git precommit format fix 

    Signed-off-by: lilacheden <lilach.edel@gmail.com>
    @lilacheden
    lilacheden committed Nov 19, 2024
    Configuration menu
    Copy the full SHA
    f3ea550 View commit details
    Browse the repository at this point in the history

  9. fix GenericEnferenceEngine error on mock_inference_mode 

    Signed-off-by: lilacheden <lilach.edel@gmail.com>
    @lilacheden
    lilacheden committed Nov 19, 2024
    Configuration menu
    Copy the full SHA
    f0f5dab View commit details
    Browse the repository at this point in the history