File tree Expand file tree Collapse file tree 2 files changed +36
-1
lines changed
docs/evaluation/how_to_guides Expand file tree Collapse file tree 2 files changed +36
-1
lines changed Original file line number Diff line number Diff line change @@ -15,7 +15,7 @@ build-api-ref:
15
15
$(PYTHON ) langsmith-sdk/python/docs/create_api_rst.py
16
16
LC_ALL=C $(PYTHON ) -m sphinx -T -E -b html -d langsmith-sdk/python/docs/_build/doctrees -c langsmith-sdk/python/docs langsmith-sdk/python/docs langsmith-sdk/python/docs/_build/html -j auto
17
17
$(PYTHON ) langsmith-sdk/python/docs/scripts/custom_formatter.py langsmith-sdk/docs/_build/html/
18
- cd langsmith-sdk/js && yarn && yarn run build:typedoc --useHostedBaseUrlForAbsoluteLinks true --hostedBaseUrl " https://$$ {VERCEL_URL:- docs.smith.langchain.com} /reference/js/"
18
+ cd langsmith-sdk/js && yarn && yarn run build:typedoc --useHostedBaseUrlForAbsoluteLinks true --hostedBaseUrl " https://docs.smith.langchain.com/reference/js/"
19
19
20
20
vercel-build : install-vercel-deps build-api-ref
21
21
mkdir -p static/reference/python
Original file line number Diff line number Diff line change @@ -576,6 +576,41 @@ ls.describe("generate sql demo", () => {
576
576
});
577
577
```
578
578
579
+ ## Configuring test suites
580
+
581
+ You can configure test suites with values like metadata or a custom client by passing an extra argument to
582
+ ` ls.describe() ` for the full suite or by passing a ` config ` field into ` ls.test() ` for individual tests:
583
+
584
+ ``` ts
585
+ ls .describe (" test suite name" , () => {
586
+ ls .test (
587
+ " test name" ,
588
+ {
589
+ inputs: { ... },
590
+ referenceOutputs: { ... },
591
+ // Extra config for the test run
592
+ config: { tags: [... ], metadata: { ... } }
593
+ },
594
+ {
595
+ name: " test name" ,
596
+ tags: [" tag1" , " tag2" ],
597
+ skip: true ,
598
+ only: true ,
599
+ }
600
+ );
601
+ }, {
602
+ testSuiteName: " overridden value" ,
603
+ metadata: { ... },
604
+ // Custom client
605
+ client: new Client (),
606
+ });
607
+ ```
608
+
609
+ The test suite will also automatically extract environment variables from ` process.env.ENVIRONMENT ` , ` process.env.NODE_ENV ` and
610
+ ` process.env.LANGSMITH_ENVIRONMENT ` and set them as metadata on created experiments. You can then filter experiments by metadata in LangSmith's UI.
611
+
612
+ See [ the API refs] ( https://docs.smith.langchain.com/reference/js/functions/vitest.describe ) for a full list of configuration options.
613
+
579
614
## Dry-run mode
580
615
581
616
If you want to run the tests without syncing the results to LangSmith, you can set omit your LangSmith tracing environment variables or set
You can’t perform that action at this time.
0 commit comments