v6.1.0

What's Changed

• Bump tonic-textual from 1.2.1 to 2.0.2 by @dependabot in #179
• Bump openai from 1.23.2 to 1.30.5 by @dependabot in #178
• Bump litellm from 1.35.20 to 1.37.19 by @dependabot in #176
• Bump pytest from 8.1.1 to 8.2.1 by @dependabot in #173
• Bump jinja2 from 3.1.3 to 3.1.4 by @dependabot in #160
• Bump requests from 2.31.0 to 2.32.0 by @dependabot in #174
• Bump urllib3 from 2.2.1 to 2.2.2 by @dependabot in #188
• [ENHANCEMENT] Adds openrouter support by @sky-2002 in #168
• Bump pytest from 8.1.1 to 8.2.2 by @dependabot in #181
• Bump typing-extensions from 4.11.0 to 4.12.2 by @dependabot in #185
• Bump pytest-asyncio from 0.23.6 to 0.23.7 by @dependabot in #183
• Bump pydantic from 2.7.1 to 2.7.4 by @dependabot in #187
• Bump litellm from 1.37.19 to 1.40.8 by @dependabot in #186
• Update dependencies & tiktoken version by @ethan-tonic in #189
• Added more LLM evaluators (Mistral, Cohere, Together AI, AWS Bedrock, AWS Sagemaker) by @Ryan-Rishi in #159
• Update pyproject.toml by @ethan-tonic in #190

New Contributors

• @sky-2002 made their first contribution in #168

Full Changelog: v6.0.0...v6.1.0

v6.0.0

What's Changed

• Added monitoring by @ethan-tonic in #167
• Bump tqdm from 4.66.2 to 4.66.4 by @dependabot in #165
• Bump ruff from 0.3.7 to 0.4.3 by @dependabot in #161
• fix: environ setting for azure openai by @gmuffiness in #157

New Contributors

• @gmuffiness made their first contribution in #157

Full Changelog: v5.0.0...v6.0.0

v5.0.0

Now supports Claude and Gemini for scorers

What's Changed

• Bump idna from 3.6 to 3.7 by @dependabot in #139
• Bump openai from 1.14.3 to 1.18.0 by @dependabot in #142
• Bump pydantic from 2.6.4 to 2.7.0 by @dependabot in #140
• Bump typing-extensions from 4.10.0 to 4.11.0 by @dependabot in #135
• Bump ruff from 0.3.4 to 0.3.7 by @dependabot in #141
• LiteLLM Integration by @Ryan-Rishi in #144
• Update README.md by @anuj-tonic in #156

New Contributors

• @anuj-tonic made their first contribution in #156

Full Changelog: v4.0.6...v5.0.0

v4.0.6

What's Changed

• Attempt to disable telemetry on workflows, again by @akamor in #134
• Fix run metadata llm evaluator check by @ethan-tonic in #137
• Fix max exponential backoff by @ethan-tonic in #138

Full Changelog: v4.0.5...v4.0.6

v4.0.5

What's Changed

• Add ChatBees vs OpenAI RAG benchmark to tonic validate examples by @ndb-rkang in #115
• Added field to store the llm used for the eval. by @Ryan-Rishi in #126
• Added parameters to expose LLM prompts from eval by @Ryan-Rishi in #128
• Disable tracking for github workflows by @akamor in #131
• Log runtime and sdk version in telemetry by @akamor in #133
• Track SDK backend in telemetry by @akamor in #132

New Contributors

• @ndb-rkang made their first contribution in #115
• @Ryan-Rishi made their first contribution in #126

Full Changelog: v4.0.4...v4.0.5

v4.0.4

What's Changed

• Swap off alru by @ethan-tonic in #120
• Fix Answer Contains PII by @ethan-tonic in #121
• Add sdk tests using pytest by @ethan-tonic in #122
• Move pytest into dev dependencies by @ethan-tonic in #124
• Added type validation by @ethan-tonic in #123

Full Changelog: v4.0.3...v4.0.4

v4.0.3

Important: This release fixes Validate not working on Python 3.8 and 3.9

What's Changed

• Add slack notification for package publish by @calexander3 in #111
• Add support for tags and loosen typing on Metadata by @akamor in #112
• Fix on Python 3.8 and 3.9 by @ethan-tonic in #119
• Bump openai from 1.14.2 to 1.14.3 by @dependabot in #117
• Bump ruff from 0.3.3 to 0.3.4 by @dependabot in #116
• Adding simple E2E test by @akamor in #118

New Contributors

• @calexander3 made their first contribution in #111

Full Changelog: v4.0.2...v4.0.3

v4.0.2

What's Changed

• Fix alru dependencies by @ethan-tonic in #110

Full Changelog: v4.0.1...v4.0.2

v4.0.1

What's Changed

• Fix TQDM progress bar counts by @ethan-tonic in #106
• Fix bullet point parsing by @ethan-tonic in #107
• V4.0.1 Bump by @ethan-tonic in #108

Full Changelog: v4.0.0...v4.0.1

v4.0.0

What's Changed

• Fix callback being called twice by @ethan-tonic in #99
• Create dependabot.yml by @ethan-tonic in #101
• Bump openai from 1.7.1 to 1.14.2 by @dependabot in #102
• Bump ruff from 0.1.15 to 0.3.3 by @dependabot in #103
• 🚀🚀 Add async to making evaluation 60 times faster by @ethan-tonic in #100
• Bump tiktoken from 0.5.2 to 0.6.0 by @dependabot in #104
• Bump sphinx-rtd-theme from 1.3.0 to 2.0.0 by @dependabot in #105

New Contributors

• @dependabot made their first contribution in #102

Full Changelog: v3.4.0...v4.0.0