-
Notifications
You must be signed in to change notification settings - Fork 1.2k
Bits AI Dev Agent - Docs Updates #31828
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
domalessi
wants to merge
4
commits into
master
Choose a base branch
from
domalessi/bits-ai-dev-agent
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file was deleted.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,110 @@ | ||
--- | ||
title: Bits AI Dev Agent | ||
further_reading: | ||
- link: "https://www.datadoghq.com/blog/bits-ai-dev-agent/" | ||
tag: "Blog" | ||
text: "Automatically identify issues and generate fixes with the Bits AI Dev Agent" | ||
--- | ||
|
||
{{< callout url="http://datadoghq.com/product-preview/bits-ai-dev-agent" >}} | ||
Bits AI Dev Agent is in Preview. To sign up, click <strong>Request Access</strong> and complete the form. | ||
{{< /callout >}} | ||
|
||
## Overview | ||
|
||
{{< img src="bits_ai/dev_agent/error_tracking_assistant.png" alt="Bits AI Dev Agent suggesting a fix for an IndexError in a Django app" style="width:100%;">}} | ||
|
||
Bits AI Dev Agent is a generative AI coding assistant that uses observability data from Datadog to automatically detect, diagnose, and fix high-impact issues in your code. It integrates with GitHub to create production-ready pull requests, iterates on fixes using CI logs and developer feedback, and draws on multiple Datadog products to generate contextual fixes. | ||
|
||
## Supported Datadog products | ||
|
||
Bits AI Dev Agent is available for the following Datadog products: | ||
|
||
| Product | Availability | Issues Detected | | ||
|---------------------------|----------------------|--------------------------------------------------------------------| | ||
| [Error Tracking][1] | Preview | Crashes, panics, exceptions, unhandled errors | | ||
| [Code Security][2] | Preview | Security issues in first-party code and open source dependencies | | ||
| [Continuous Profiler][3] | Preview | Code-level performance issues | | ||
| [Test Optimization][4] | Preview | Flaky tests | | ||
|
||
## Key capabilities | ||
|
||
Bits AI Dev Agent provides comprehensive code remediation across multiple areas of your development workflow: | ||
|
||
### Error tracking | ||
|
||
Bits AI Dev Agent diagnoses and remediates code issues with automated context and unit-tested fixes: | ||
- Determines whether an error can be fixed through code and generates unit tests. | ||
- Provides links to relevant files and methods for streamlined navigation. | ||
- Analyzes errors asynchronously as they arrive. | ||
- Marks errors with a **Fix available** status and enables filtering to surface those issues. | ||
|
||
### Flaky test remediation | ||
|
||
Bits AI Dev Agent fixes unreliable tests in CI/CD pipelines and verifies that the tests remain stable. | ||
|
||
### Trace investigation | ||
|
||
Bits AI Dev Agent debugs errors and latency directly from traces using natural language queries: | ||
- Parses and summarizes large traces. | ||
- Diagnoses likely root causes. | ||
- Generates code fixes for errors or latency issues within a trace. | ||
|
||
### Code fix generation | ||
|
||
Bits AI Dev Agent applies automated code changes based on Datadog insights such as CCM Recommendations, APM Recommendations, and Profiling Insights. | ||
|
||
### Code security | ||
|
||
Bits AI Dev Agent remediates vulnerabilities at scale, from single issues to large backlogs. You can: | ||
- Create PR batches to fix multiple vulnerabilities at once. | ||
- Use the Campaign tool to push PRs incrementally and manage review workload across teams. | ||
|
||
### Pull request assistance | ||
|
||
Bits AI Dev Agent works in GitHub to request changes, fix CI failures, and respond to review comments: | ||
|
||
- Generates PR titles and descriptions based on your PR template. | ||
- Opens PRs as drafts, iterates using CI logs, and marks them ready for review when checks pass. | ||
- Continues iterating in response to chat messages and review feedback. | ||
**Note**: Comment `@Datadog` to ask for updates to the PR. The Dev Agent never auto-merges PRs. | ||
|
||
Go to **Bits AI** > **Dev Agent** > **[Code sessions][7]** to see all Dev Agent code sessions and generated PRs. You can search by service, source, and status. | ||
|
||
### Bits AI SRE integration | ||
|
||
Bits AI Dev Agent generates fixes from SRE-led investigations. Learn more about [Bits AI SRE][8]. | ||
|
||
## Get started | ||
|
||
To enable Bits AI Dev Agent: | ||
|
||
1. Install the [GitHub integration][5]. | ||
2. Grant the required permissions. | ||
3. Send CI logs to Datadog if you want the Dev Agent to iterate on PRs using build feedback. | ||
|
||
For full installation and configuration instructions, see [Setup][6]. | ||
|
||
## Limitations | ||
|
||
- Bits AI Dev Agent is an AI-based product, which means outputs may vary for the same input and sometimes require human review. | ||
- Bits AI Dev Agent does not support multi-repository investigations. | ||
|
||
## Security considerations | ||
|
||
Observability data (such as traces, logs, and metrics) can include content from untrusted sources. Attackers might attempt to inject this data to influence agent behavior (for example, by injecting prompts that cause unintended code changes). | ||
|
||
Datadog applies automated security and quality checks on agent output, but safeguards are limited and may not detect all malicious or unsafe code. Review and test all changes generated by the Dev Agent before merging them into production. | ||
|
||
## Further reading | ||
|
||
{{< partial name="whats-next/whats-next.html" >}} | ||
|
||
[1]: /error_tracking | ||
[2]: /security/code_security | ||
[3]: /profiler/ | ||
[4]: /tests/ | ||
[5]: https://app.datadoghq.com/integrations/github | ||
[6]: /bits_ai_dev_agent/setup | ||
[7]: https://app.datadoghq.com/code?tab=my-sessions | ||
[8]: /bits_ai/bits_ai_sre/ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,92 @@ | ||
--- | ||
title: Bits AI Dev Agent Setup | ||
disable_toc: false | ||
--- | ||
|
||
## Overview | ||
|
||
Bits AI Dev Agent integrates with GitHub to open, update, and iterate on pull requests based on issues detected in Datadog. | ||
|
||
<div class="alert alert-info">Bits AI Dev Agent supports GitHub only.</div> | ||
|
||
## Setup | ||
|
||
### Step 1: Enable the GitHub integration | ||
|
||
Install the [GitHub integration][5]. For full installation and configuration steps, see the [GitHub integration guide][6]. | ||
|
||
### Step 2: Configure GitHub permissions | ||
|
||
The GitHub integration must be granted the following permissions to enable basic Dev Agent functionality: | ||
|
||
- **Repository Permissions** | ||
- `Contents: Read & Write` | ||
- `Pull Requests: Read & Write` | ||
- **Subscribe to Events** | ||
- `Push` | ||
|
||
#### Additional permissions for CI integration | ||
|
||
To allow the Dev Agent to use CI logs when iterating on pull requests, you must send CI logs to Datadog and enable the Auto-commit feature. This requires additional permissions: | ||
|
||
- **Repository permissions** | ||
- `Checks: Read` | ||
- `Commit statuses: Read only` | ||
- **Subscribe to events** | ||
- `Check run` | ||
- `Check suite` | ||
- `Issue comment` | ||
- `Status` | ||
|
||
### Step 3: Configure telemetry tagging | ||
|
||
Bits AI Dev Agent uses the `service` and `version` telemetry tags to match detected issues (such as errors or vulnerabilities) to the version of code that was running at the time. | ||
|
||
To configure telemetry tagging, see [Tag your telemetry with Git information][7]. You can also configure service-to-repository mapping manually in **Settings**. | ||
|
||
### Step 4: (Optional) Enable Auto-push | ||
|
||
Enabling Auto-Push allows the Dev Agent to push commits directly to a branch. With Auto-Push, the Dev Agent can: | ||
- Open merge-ready pull requests that fix errors, vulnerabilities, and other issues. | ||
- Update pull requests in response to your comments in GitHub. | ||
- Resolve CI failures automatically. | ||
|
||
If Auto-Push is disabled, you must review and approve code in Datadog before it is pushed. | ||
|
||
### Step 5: Configure custom instructions | ||
|
||
The Dev Agent ingests custom instruction files from your repository, including: | ||
|
||
- `.cursorrules` | ||
- `.windsurfrules` | ||
- `copilot-instructions.md` | ||
- `claude.md` | ||
- `agents.md` | ||
- `agent.md` | ||
|
||
You can also define global custom instructions in **Settings**, which apply to all Dev Agent sessions. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @leachface Same Q here: I brought this over from the Google Doc but don't know what Settings are being referred to. Would like to give more specific instructions if possible. |
||
|
||
## Troubleshooting | ||
|
||
If you encounter issues during setup: | ||
|
||
- **GitHub integration not connecting**: Verify your GitHub account has the necessary permissions and that the integration is properly configured. | ||
- **Missing telemetry data**: Ensure your applications are properly instrumented and sending data to Datadog with the required tags. | ||
- **Dev Agent not creating pull requests**: Check that Auto-Push is enabled (if desired) and that the GitHub integration has the required repository permissions. | ||
|
||
For additional help, see the [GitHub integration troubleshooting guide][9] or contact [Datadog support][10]. | ||
|
||
## Further reading | ||
|
||
{{< partial name="whats-next/whats-next.html" >}} | ||
|
||
[1]: /error_tracking | ||
[2]: /security/code_security | ||
[3]: /profiler/ | ||
[4]: /tests/ | ||
[5]: https://app.datadoghq.com/integrations/github | ||
[6]: /integrations/github/ | ||
[7]: /integrations/guide/source-code-integration/?tab=go#tag-your-telemetry-with-git-information | ||
[8]: https://app.datadoghq.com/metric/summary | ||
[9]: /integrations/github/#troubleshooting | ||
[10]: /help/ |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@leachface Are you able to provide more specific instructions on the manual route here?