Prompt management guide

src/docs.json

@@ -911,6 +911,7 @@
                 "pages": [
                   "langsmith/prompt-engineering",
                   "langsmith/prompt-engineering-concepts",
+                  "langsmith/prompt-management-best-practices",
                   {
                     "group": "Create and update prompts",
                     "pages": [

src/langsmith/manage-prompts-programmatically.mdx

@@ -5,9 +5,7 @@ sidebarTitle: Manage prompts programmatically
 
 You can use the LangSmith Python and TypeScript SDK to manage prompts programmatically.
 
-<Note>
-Previously this functionality lived in the `langchainhub` package which is now deprecated. All functionality going forward will live in the `langsmith` package.
-</Note>
+<Warning>We do not recommend pulling prompts from the Prompt Hub in your production code on every request. Please see our guide on [best practices for using the Prompt Hub](/langsmith/prompt-management-best-practices) for more information.</Warning>
 
 ## Install packages
 

src/langsmith/manage-prompts.mdx

@@ -65,15 +65,20 @@ Some common use cases of this include:
 
 Navigate to the **Prompts** section in the left-hand sidebar or from the application homepage. In the top right corner, click on the `+ Webhook` button.
 
-Add a webhook URL and any required headers.
+1. Add a webhook URL.
+2. Select the event(s) you want to trigger the webhook for:
+    - On each commit to a prompt.
+    - When a new tag is created for a prompt.
+    - When a tag is moved for a prompt.
+3. Add any required headers.
+4. Test out your webhook by clicking **Send test notification**. This will send a test notification to the webhook URL you provided with a sample payload.
+4. Click **Create Webhook**.
 
 <Note>
 You can only configure one webhook per workspace. If you want to configure multiple per workspace or set up a different webhook for each prompt, let us know in the [LangChain Forum](https://forum.langchain.com/).
 </Note>
 
-To test out your webhook, click the **Send test notification** button. This will send a test notification to the webhook URL you provided with a sample payload.
-
-The sample payload is a JSON object with the following fields:
+### Webhook Payload
 
 - `prompt_id`: The ID of the prompt that was committed.
 - `prompt_name`: The name of the prompt that was committed.
@@ -82,13 +87,12 @@ The sample payload is a JSON object with the following fields:
 - `created_by`: The author of the commit.
 - `manifest`: The manifest of the prompt.
 
-### Trigger the webhook
-
-Commit to a prompt to trigger the webhook you've configured.
 
-#### Use the Playground
+### Trigger the webhook
+Webhooks are triggered on prompt changes in the UI or via the API.
 
-If you do this in the Playground, you'll be prompted to deselect the webhooks you'd like to avoid triggering.
+#### In the UI
+If you have a webhook configured, when you create a prompt commit in the Playground, you will see the Trigger Webhook options. You can deselect the webhook if you don't want to trigger it on that specific commit.
 
 ![](/langsmith/images/commit-prompt-playground.png)
 
@@ -111,4 +115,3 @@ Here you'll find all of the publicly listed prompts in the LangChain Hub. You ca
 To view prompts tied to your workspace, visit the **Prompts** tab in the sidebar.
 
 ![](/langsmith/images/prompts-tab.png)
-

src/langsmith/prompt-management-best-practices.mdx

@@ -0,0 +1,58 @@
+---
+title: Prompt Hub best practices
+sidebarTitle: Prompt Hub best practices
+---
+
+Prompts are a fundamental component of your LLM application's logic. A minor change can significantly change an LLM's response or tool selection, making lifecycle management important. LangSmith provides tools to manage prompts across development, testing, and production environments:
+
+- **Prompt Hub**: A central store for managing prompts. This is a especially useful source of truth for your prompts when you are collaborating with a team to iterate on them.
+- **Playground**: A sandbox for testing and experimenting with prompts and models, helping you refine their behavior before deploying.
+
+This guide shows you how to build a workflow for developing, testing, and deploying prompts with LangSmith.
+
+## Develop and iterate on prompts in the playground
+
+Prompt development is an iterative, experimental process. Using the [Playground](/prompt_engineering/how_to_guides#playground), you can:
+- **Test with different inputs** and **compare the results** of different prompts and LLMs side-by-side.
+- **Refine your prompt** using the embedded [Prompt Canvas](/prompt_engineering/how_to_guides/prompt_canvas) to have an LLM improve your prompt.
+- **Prototype tools, set structured outputs** and test the interaction within the Playground.
+- **[Run evaluations](/langsmith/run-evaluation-from-prompt-playground)** over your prompt in the Playground. Share experiments with teammates to get feedback and collaboratively optimize performance.
+
+Once ready, you can save a prompt to the Prompt Hub. A new commit is created each time you save a prompt, establishing a complete commit history enables you to track changes and revert to previous versions if needed.
+
+## Manage prompts across environments
+
+When managing prompts across your dev/staging/production environments, you want to balance speed of iteration across a team with the need for stability. We recommend using the Prompt Hub along with [commit tags](/langsmith/manage-prompts#commit-tags) to manage prompts across dev/testing/staging environments and a seperate workflow for managing prompts in production.
+
+### Update prompts in dev/testing/staging environments
+
+After testing in the Playground, you'll want to see how the prompt interacts within the context of your application. A recommend workflow is:
+
+1. Set up your dev/testing/staging environments to [pull prompts from the Prompt Hub](/langsmith/manage-prompts-programmatically#pull-a-prompt) using defined commit tags (e.g., `dev`, `staging`) for each environment. This creates a dynamic reference to the prompt version you want to use in a particular environment.
+2. After saving a new version of your prompt, [move the commit tag](/langsmith/manage-prompts#move-a-tag) for the desired environment to the new version of the prompt. Your application will automatically pull the new version on its next request to the Prompt Hub.
+3. Test the prompt across environments.
+
+<Tip> Integrate prompt evaluations into CI/CD. [Prompt webhooks](/langsmith/manage-prompts#configure-a-webhook) can be used in order to trigger CI when a prompt has new commit, or when a commit tag is moved. Use [pytest](/langsmith/pytest) or [jest/vitest](/langsmith/vitest-jest) to run LangSmith evaluations as part of your test suite, blocking promotion if quality thresholds fail. </Tip>
+
+### Update prompts in production
+
+<Warning> It is not recommended to put an API call to the Prompt Hub in the hot path of your application on every request. Instead, we recommend either pulling the prompt into your code directly or fetching the prompt once and reusing it. </Warning>
+
+[Prompt webhooks](/langsmith/manage-prompts#configure-a-webhook) can be used in order to keep prompts in sync with your production environment. For example a notification can be sent when a prompt gets the `prod` tag added to it, which can then be used as a trigger to pull the latest version of the prompt into your production environment.
+
+#### Pull prompts into your code
+
+A common approach is to [sync prompts from the Prompt Hub with your source code](/langsmith/prompt-commit), so they can be **version-controlled and deployed alongside your application code**.
+
+The workflow would look like:
+1. A LangSmith user moves the `prod` tag to a new prompt commit.
+2. The webhook triggers a CI/CD pipeline (e.g., GitHub Actions, GitLab CI) that pulls the commit with the `prod` tag into your repository.
+3. The pipeline commits the prompt change to your repo.
+
+
+#### Store prompts in a database or cache prompts in your application
+
+If you want your prompts **versioned independenty of your application code**, you can sync them to a database or cache them in your application. This allows you to update, A/B test prompts without redeploying your application.
+
+- **Database**: Store prompts in a database as a form of dynamic configuration, and update them with a CI/CD process when a webhook is triggered.
+- **Cache**: Cache prompts in the application in memory, and update them by invalidating the cache on webhook receipt.