These are the release notes for GPTMe version v0.11.0.

Contributors

Thanks to everyone who contributed to this release:

Changelog

Changes since v0.10.4:

feat: mirror working directory in shell and Python process (#49) (0b9c322)
feat: support paths/URLs in any prompt, refactored entrypoint to call a new public API with core logic (#37) (aaf60e5)

Click to expand

fix: exit with appropriate exitcode if evals pass/fail (3a0e4dc)
fix: fixed shell cd test (9932b27)
fix: fixed bugs in eval, upload/download binary files, cd to cwd before every shell command (cefbbe8)
fix: fixed shell output printing (no extra newlines) (cf91873)
fix: fixed a spelling error (8c1eada)
fix: import NotRequired from typing_extensions (2718eba)
fix: improved path detection in prompt (3f74635)
fix: add price_input and price_output to model metadata, refactored ModelDict TypedDict into ModelMeta dataclass (a0f1a73)
fix: switched to ipython for handling Python execution (#41) (b75182c)
fix: fixed regression in 78e5f52 (7160b97)
fix: output stdout/stderr directly for shell commands, to indicate progress (be40f10)
fix: fixed handling codeblocks that contain triple backticks (78e5f52)
fix: fix uncaught assert when patching missing file (fixes #39) (e34fb2c)

Click to expand

docs: improved entrypoints' docs for better cli docs (8950650)
docs: added demos page to docs (ab2687c)
test: run evals as tests, refactor evals, added python-xdist for parallel testing (14ca2df)
test: switch from gpt-3.5-turbo to gpt-4-1106-preview ("gpt-4-turbo") in cli tests (75e79bd)
refactor: made eval abstractions more general (#48) (bf64f20)
refactor: moved init code into init.py (0e1a0f5)
test: minor improvements to eval (e3aa336)
test: further eval improvements (bfc2f14)
test: added eval test that accepts stdin (28e3a3c)
test: fixed bugs and improved output in evals (953614f)
test: continued work on evals (0c070ec)
test: added basic eval code (be678f3)
docs: fixed link to demo (66e9e49)
docs: added more demos (7de6c74)
docs: added link to issue (0a0f7ca)
test: added complex plotting example to test-integration.sh (3c1524e)

Full Changelog: v0.10.4...v0.11.0