Skip to content

Commit

Permalink
Merge pull request #129 from SeryioGonzalez/patch-1
Browse files Browse the repository at this point in the history
Update evaluate-sdk.md
  • Loading branch information
Court72 authored Dec 30, 2024
2 parents c18cb3f + 2872db0 commit deb7e87
Showing 1 changed file with 3 additions and 5 deletions.
8 changes: 3 additions & 5 deletions articles/ai-studio/how-to/develop/evaluate-sdk.md
Original file line number Diff line number Diff line change
Expand Up @@ -444,20 +444,18 @@ Sometimes a large language model isn't needed for certain evaluation metrics. Th
class AnswerLengthEvaluator:
def __init__(self):
pass

# A class is made a callable my implementing the special method __call__
def __call__(self, *, answer: str, **kwargs):
return {"answer_length": len(answer)}
```

Then run the evaluator on a row of data by importing a callable class:

```python
with open("answer_len/answer_length.py") as fin:
print(fin.read())

from answer_len.answer_length import AnswerLengthEvaluator

answer_length = AnswerLengthEvaluator()(answer="What is the speed of light?")
answer_length_evaluator = AnswerLengthEvaluator()
answer_length = answer_length_evaluator(answer="What is the speed of light?")

print(answer_length)
```
Expand Down

0 comments on commit deb7e87

Please sign in to comment.