Skip to content

Commit 32aad3e

Browse files
authored
Allow passing problems (dict) directly
1 parent 9914956 commit 32aad3e

File tree

1 file changed

+5
-1
lines changed

1 file changed

+5
-1
lines changed

mxeval/evaluation.py

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -91,7 +91,11 @@ def evaluate_functional_correctness(
9191
results to f"{sample_file}_results.jsonl"
9292
"""
9393

94-
problems = read_problems(problem_file)
94+
if type(problem_file) is not dict:
95+
problems = read_problems(problem_file)
96+
else:
97+
print("Skip reading problems -- using problem_file (dict) as problems")
98+
problems = problem_file
9599

96100
# see execution.py for details
97101
# Check the generated samples against test suites.

0 commit comments

Comments
 (0)