v2.1.0

2.1.0 (2023-12-21)

Features

• instantiate evaluators by criteria (#1983) (9c72616)
• support function calling for run_evals (#1978) (8be325c)
• traces: add v1/traces HTTP endpoint to handle ExportTraceServiceRequest (3c94dea)
• traces: add v1/traces HTTP endpoint to handle ExportTraceServiceRequest (#1968) (3c94dea)
• traces: add retrieval summary to header (#2006) (8af0582)
• traces: evaluation summary on the header (#2000) (965beb0)

Bug Fixes

• make alert icon for exceptions visible (#2001) (e7a6567)

v2.0.0

2.0.0 (2023-12-20)

⚠ BREAKING CHANGES

• Update llm_classify and llm_generate interfaces (#1974)

Features

• Add async submission to llm_generate (#1965) (5999133)
• add support for explanations to run_evals (#1975) (5143529)
• evaluation column selectors (#1932) (ed07809)
• openai streaming tool calls (#1936) (6dd14cf)
• support running multiple evals at once (#1742) (79d4473)
• Update llm_classify and llm_generate interfaces (#1974) (9fd35a1)

Bug Fixes

• Add lock failsafe (#1956) (9ddbd9c)
• llama-index extra (#1958) (d9b68eb)
• LlamaIndex compatibility fix (#1940) (052349d)
• Model stability enhancements (#1939) (dca42e0)
• traces: span summary root span filter (#1981) (d286f07)

Documentation

• Add anyscale tutorial (#1941) (e47c8d0)
• autogen link (#1946) (c3fb4ce)
• Clear anyscale tutorial outputs (#1942) (63580a6)
• RAG Evaluation (GITBOOK-378) (429f537)
• sync (#1947) (c72bbac)
• traces: autogen tracing tutorial (#1945) (0fd02ff)
• update rag eval notebook (#1950) (d06b8b7)
• update rag evals docs (#1954) (aa6f36a)
• Using phoenix with HuggingFace LLMs- getting started (#1916) (b446972)

v1.9.0

1.9.0 (2023-12-11)

Features

• Add retries to Bedrock (#1927) (2728c3e)

Documentation

• Add LLM Tracing+Evals notebook with keyless example (#1928) (4c4aac6)

v1.8.0

1.8.0 (2023-12-10)

Features

• embeddings: audio support (#1920) (61cc550)
• openai streaming function call message support (#1914) (25279ca)

v1.7.0

1.7.0 (2023-12-09)

Features

• Instrument LlamaIndex streaming responses (#1901) (f46396e)
• openai async streaming instrumentation (#1900) (06d643b)
• traces: query spans into dataframes (#1910) (6b51435)

Bug Fixes

• traces: span evaluations missing from the header (#1908) (5ace81e)

v1.6.0

1.6.0 (2023-12-08)

Features

• openai streaming spans show up in the ui (#1888) (ffa1d41)
• support instrumentation for openai synchronous streaming (#1879) (b6e8c73)
• traces: display document retrieval metrics on trace details (#1902) (0c35229)
• traces: filterable span and document evaluation summaries (#1880) (f90919c)
• traces: graphql query for document evaluation summary (#1874) (8a6a063)

Documentation

• llm ops overview notebook (#1882) (5d15c3c)

v1.5.1

1.5.1 (2023-12-06)

Bug Fixes

• Rate limiter will sometimes tank request rate with many concurrent requests (#1855) (2530569)

v1.5.0

1.5.0 (2023-12-06)

Features

• evals: Human vs AI Evals (#1850) (e96bd27)
• semantic conventions for tool_calls array in OpenAI ChatCompletion messages (#1837) (c079f00)
• support asynchronous chat completions for openai instrumentation (#1849) (f066e10)
• traces: document retrieval metrics based on document evaluation scores (#1826) (3dfb7bd)
• traces: document retrieval metrics on trace / span tables (#1873) (733d233)
• traces: evaluation annotations on traces for associating spans with eval metrics (#1693) (a218a65)
• traces: server-side span filter by evaluation result values (#1858) (6b05f96)
• traces: span evaluation summary (aggregation metrics of scores and labels) (#1846) (5c5c3d6)

Bug Fixes

• allow streaming response to be iterated by user (#1862) (76a2443)
• trace dataset to disc (#1798) (278d344)

Documentation

• RAG evaluation notebook using traces (#1857) (4b67805)
• Retrieval Chunks (GITBOOK-372) (39976d3)

v1.4.0

1.4.0 (2023-11-30)

Features

• propagate error status codes to parent spans for improved visibility into trace exceptions (#1824) (1a234e9)

v1.3.0

1.3.0 (2023-11-30)

Features

• Add OpenAI Rate limiting (#1805) (115e044)
• evals: show span evaluations in trace details slideout (#1810) (4f0e4dc)
• evaluation ingestion (no user-facing feature is added) (#1764) (7c4039b)
• feature flags context (#1802) (a2732cd)
• Implement asynchronous submission for OpenAI evals (#1754) (30c011d)
• reference link correctness evaluation prompt template (#1771) (bf731df)
• traces: configurable endpoint for the exporter (#1795) (8515763)
• traces: display document evaluations alongside the document (#1823) (2ca3613)
• traces: server-side sort of spans by evaluation result (score or label) (#1812) (d139693)
• traces: show all evaluations in the table" (#1819) (2b27333)
• traces: Trace page header with latency, status, and evaluations (#1831) (1d88efd)

Bug Fixes

• enhance llama-index callback support for exception events (#1814) (8db01df)
• pin llama-index temporarily (#1806) (d6aa76e)
• remove sklearn metrics not available in sagemaker (#1791) (20ab6e5)
• traces: convert (non-list) iterables to lists during protobuf construction due to potential presence of ndarray when reading from parquet files (#1801) (ca72747)
• traces: make column selector sync'd between tabs (#1816) (125431a)

Documentation

• Environment documentation (GITBOOK-370) (dbbb0a7)
• Explanations (GITBOOK-371) (5f33da3)
• No subject (GITBOOK-369) (656b5c0)
• sync for 1.3 (#1833) (4d01e83)
• update default value of variable in run_relevance_eval (GITBOOK-368) (d5bcaf8)