Merge branch 'main' into isaac/threadhowtoupdate

langchain-ai · Feb 6, 2025 · f3bf431 · f3bf431
2 parents 8c7715c + 37ca436
commit f3bf431
Show file tree

Hide file tree

Showing 7 changed files with 43 additions and 4 deletions.
diff --git a/Makefile b/Makefile
@@ -15,7 +15,7 @@ build-api-ref:
 	$(PYTHON) langsmith-sdk/python/docs/create_api_rst.py
 	LC_ALL=C $(PYTHON) -m sphinx -T -E -b html -d langsmith-sdk/python/docs/_build/doctrees -c langsmith-sdk/python/docs langsmith-sdk/python/docs langsmith-sdk/python/docs/_build/html -j auto
 	$(PYTHON) langsmith-sdk/python/docs/scripts/custom_formatter.py langsmith-sdk/docs/_build/html/
-	cd langsmith-sdk/js && yarn && yarn run build:typedoc --useHostedBaseUrlForAbsoluteLinks true --hostedBaseUrl "https://$${VERCEL_URL:-docs.smith.langchain.com}/reference/js/"
+	cd langsmith-sdk/js && yarn && yarn run build:typedoc --useHostedBaseUrlForAbsoluteLinks true --hostedBaseUrl "https://docs.smith.langchain.com/reference/js/"
 
 vercel-build: install-vercel-deps build-api-ref 
 	mkdir -p static/reference/python

diff --git a/docs/evaluation/concepts/index.mdx b/docs/evaluation/concepts/index.mdx
@@ -139,11 +139,15 @@ Learn [how run pairwise evaluations](/evaluation/how_to_guides/evaluate_pairwise
 
 Each time we evaluate an application on a dataset, we are conducting an experiment.
 An experiment contains the results of running a specific version of your application on the dataset.
+To understand how to use the LangSmith experiment view, see [how to analyze experiment results](/evaluation/how_to_guides/analyze_single_experiment).
+
+![Experiment view](./static/experiment_view.png)
+
 Typically, we will run multiple experiments on a given dataset, testing different configurations of our application (e.g., different prompts or LLMs).
 In LangSmith, you can easily view all the experiments associated with your dataset.
 Additionally, you can [compare multiple experiments in a comparison view](/evaluation/how_to_guides/compare_experiment_results).
 
-![Example](./static/comparing_multiple_experiments.png)
+![Comparison view](./static/comparison_view.png)
 
 ## Annotation queues
 
@@ -191,7 +195,7 @@ Often these are triggered when you are making app updates (e.g. updating models
 LangSmith's comparison view has native support for regression testing, allowing you to quickly see examples that have changed relative to the baseline.
 Regressions are highlighted red, improvements green.
 
-![Regression](./static/regression.png)
+![Comparison view](./static/comparison_view.png)
 
 ### Backtesting
 

diff --git a/docs/evaluation/concepts/static/comparison_view.png b/docs/evaluation/concepts/static/comparison_view.png
diff --git a/docs/evaluation/concepts/static/experiment_view.png b/docs/evaluation/concepts/static/experiment_view.png
diff --git a/docs/evaluation/concepts/static/regression.png b/docs/evaluation/concepts/static/regression.png
diff --git a/docs/evaluation/how_to_guides/vitest_jest.mdx b/docs/evaluation/how_to_guides/vitest_jest.mdx
@@ -576,6 +576,41 @@ ls.describe("generate sql demo", () => {
 });
 ```
 
+## Configuring test suites
+
+You can configure test suites with values like metadata or a custom client by passing an extra argument to
+`ls.describe()` for the full suite or by passing a `config` field into `ls.test()` for individual tests:
+
+```ts
+ls.describe("test suite name", () => {
+  ls.test(
+    "test name",
+    {
+    inputs: { ... },
+    referenceOutputs: { ... },
+    // Extra config for the test run
+    config: { tags: [...], metadata: { ... } }
+  },
+  {
+    name: "test name",
+    tags: ["tag1", "tag2"],
+    skip: true,
+    only: true,
+    }
+  );
+}, {
+  testSuiteName: "overridden value",
+  metadata: { ... },
+  // Custom client
+  client: new Client(),
+});
+```
+
+The test suite will also automatically extract environment variables from `process.env.ENVIRONMENT`, `process.env.NODE_ENV` and
+`process.env.LANGSMITH_ENVIRONMENT` and set them as metadata on created experiments. You can then filter experiments by metadata in LangSmith's UI.
+
+See [the API refs](https://docs.smith.langchain.com/reference/js/functions/vitest.describe) for a full list of configuration options.
+
 ## Dry-run mode
 
 If you want to run the tests without syncing the results to LangSmith, you can set omit your LangSmith tracing environment variables or set

diff --git a/docs/self_hosting/organization_charts.mdx b/docs/self_hosting/organization_charts.mdx
@@ -7,7 +7,7 @@ table_of_contents: true
 # Viewing trace counts across your organization
 
 :::note
-This feature is available on Helm chart versions 0.8.21 and later.
+This feature is available on Helm chart versions 0.9.5 and later.
 :::
 
 LangSmith automatically generates and syncs organization usage charts for self-hosted installations.