We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
There was an error while loading. Please reload this page.
1 parent 9914956 commit 32aad3eCopy full SHA for 32aad3e
mxeval/evaluation.py
@@ -91,7 +91,11 @@ def evaluate_functional_correctness(
91
results to f"{sample_file}_results.jsonl"
92
"""
93
94
- problems = read_problems(problem_file)
+ if type(problem_file) is not dict:
95
+ problems = read_problems(problem_file)
96
+ else:
97
+ print("Skip reading problems -- using problem_file (dict) as problems")
98
+ problems = problem_file
99
100
# see execution.py for details
101
# Check the generated samples against test suites.
0 commit comments