Conversation
- Introduced a new skill for detecting sensitive information leakage in AI agent responses, including API keys, credentials, and PII. - Added comprehensive documentation for the skill, including usage examples and API reference. - Implemented a data leakage scanning tool with support for static and dynamic test case generation. - Enhanced the requirements with the addition of PyYAML for YAML file handling. - Created various prompt sets for testing, including basic and advanced attack vectors. - Established a modular architecture for security scanning tools, facilitating future enhancements and integrations.
- Introduced a new agent security reviewer skill to aggregate findings from various security detection modules and generate risk reports based on OWASP ASI classification. - Added detailed documentation for the agent security reviewer, including its purpose, workflow, and integration with existing tools. - Created a new report generator for producing structured XML reports of vulnerabilities, enhancing the reporting process for agent-based applications. - Updated data leakage detection documentation to reference the new security reviewer for classification and report aggregation. - Enhanced the models to include OWASP ASI categories for better organization and clarity in security assessments.
- Updated severity levels in agent security reviewer, data leakage detection, and related documentation to standardize categories as High, Medium, and Low, removing the Critical classification. - Adjusted output formats and reporting mechanisms to reflect the new severity structure. - Enhanced models and evaluation criteria to align with the updated severity framework, ensuring consistency across all security scanning tools.
V4 add agent adapter tools & config-scanner agent
- Updated the agent security reviewer documentation to clarify its purpose and workflow, emphasizing the use of OWASP Top 10 for Agentic Applications 2026 classification. - Enhanced the data leakage detection module with improved integration capabilities, including context validation and LLM-based evaluations. - Revised output formats for vulnerability reports, ensuring consistency and clarity in XML structure. - Removed deprecated system prompt references and streamlined the review workflow for better usability.
update tool dialogue & configuration scan
|
|
||
| // 创建用户专属目录 | ||
| userDir := getAgentUserDir(username) | ||
| if err := os.MkdirAll(userDir, 0755); err != nil { |
Check failure
Code scanning / CodeQL
Uncontrolled data used in path expression High
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 27 days ago
In general, to fix uncontrolled path usage, you must ensure that any path built from user-controlled data is (1) syntactically safe (no traversal/meta characters) and (2) resolved to an absolute path that is verified to remain under a known safe root directory. This should be done as close as possible to where the path is used with filesystem APIs (os.MkdirAll, os.Open, etc.), or by centralizing path construction in helper functions that enforce these rules.
For this handler, the best fix with minimal functional change is:
- Harden
getAgentUserDirso that it returns an absolute, normalized path under a fixed safe directory derived fromAgentConfigRoot, and refuses (by returning a safe error value) if resolution fails or escapes that directory. - Update
HandleSaveAgentConfigto use the (now safer)getAgentUserDirand handle possible errors before callingos.MkdirAll. - Keep the existing
validateUsernamelogic but do not rely on it as the only defense. Instead of silently substitutingPublicUserwhen validation fails, explicitly reject the request, which both eliminates surprising behavior and ensuresusernameis always of a safe form.
Concretely, within common/websocket/knowledge2_api.go:
- Change
getAgentUserDirto return(string, error)and internally:- Resolve
AgentConfigRootto an absolute path. - Join it with
username. - Call
filepath.Cleanandfilepath.Abson the result. - Verify the final
absPathhas the safe-root prefix (usingstrings.HasPrefixon a normalized root with trailing separator). - On any failure, return an error.
- Resolve
- Update
HandleSaveAgentConfigso that:- It rejects invalid usernames (using
validateUsername) with a 400 instead of forcingPublicUser. - It calls the new
getAgentUserDirand, if it returns an error, responds with a 400/500 and does not callos.MkdirAll. - Use the returned safe directory path for
os.MkdirAll.
- It rejects invalid usernames (using
These changes only touch the shown code and reuse existing imports (filepath, strings, fmt, errors), so no new imports are necessary.
| @@ -326,9 +326,28 @@ | ||
| const AgentConfigRoot = "data/agents" | ||
| const PublicUser = "public_user" | ||
|
|
||
| // getAgentUserDir 获取用户的 agent 配置目录 | ||
| func getAgentUserDir(username string) string { | ||
| return filepath.Join(AgentConfigRoot, username) | ||
| // getAgentUserDir 获取用户的 agent 配置目录(确保路径在 AgentConfigRoot 之内) | ||
| func getAgentUserDir(username string) (string, error) { | ||
| // 计算安全的根目录绝对路径 | ||
| rootAbs, err := filepath.Abs(AgentConfigRoot) | ||
| if err != nil { | ||
| return "", fmt.Errorf("failed to resolve agent config root: %w", err) | ||
| } | ||
| // 组合并规范化用户目录 | ||
| userPath := filepath.Join(rootAbs, username) | ||
| userAbs, err := filepath.Abs(userPath) | ||
| if err != nil { | ||
| return "", fmt.Errorf("failed to resolve user path: %w", err) | ||
| } | ||
| // 确保最终路径仍然在 AgentConfigRoot 之下,防止路径穿越 | ||
| rootWithSep := rootAbs | ||
| if !strings.HasSuffix(rootWithSep, string(os.PathSeparator)) { | ||
| rootWithSep += string(os.PathSeparator) | ||
| } | ||
| if userAbs != rootAbs && !strings.HasPrefix(userAbs, rootWithSep) { | ||
| return "", errors.New("invalid username path") | ||
| } | ||
| return userAbs, nil | ||
| } | ||
|
|
||
| // validateUsername 验证用户名安全性(防止路径穿越) | ||
| @@ -448,7 +467,11 @@ | ||
| func HandleSaveAgentConfig(c *gin.Context) { | ||
| username := c.GetString("username") | ||
| if !validateUsername(username) { | ||
| username = PublicUser | ||
| c.JSON(http.StatusBadRequest, gin.H{ | ||
| "status": 1, | ||
| "message": "用户名非法", | ||
| }) | ||
| return | ||
| } | ||
|
|
||
| name := strings.TrimSpace(c.Param("name")) | ||
| @@ -497,7 +520,14 @@ | ||
| } | ||
|
|
||
| // 创建用户专属目录 | ||
| userDir := getAgentUserDir(username) | ||
| userDir, err := getAgentUserDir(username) | ||
| if err != nil { | ||
| c.JSON(http.StatusBadRequest, gin.H{ | ||
| "status": 1, | ||
| "message": "用户名目录非法: " + err.Error(), | ||
| }) | ||
| return | ||
| } | ||
| if err := os.MkdirAll(userDir, 0755); err != nil { | ||
| c.JSON(http.StatusInternalServerError, gin.H{ | ||
| "status": 1, |
| return | ||
| } | ||
|
|
||
| if err := os.WriteFile(targetPath, []byte(content), 0644); err != nil { |
Check failure
Code scanning / CodeQL
Uncontrolled data used in path expression High
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 27 days ago
In general, the fix is to ensure that any user-controlled component used to build a filesystem path is validated or constrained so it cannot escape an intended root directory. That means: (1) validating that username is safe before using it to construct userDir, and/or (2) resolving the final candidate path(s) to absolute paths and checking that they are contained within the intended base directory (derived from AgentConfigRoot).
The most targeted and robust change, without altering higher-level behavior, is to harden resolveAgentConfigPathForWrite. This function is where a user-influenced username is converted into a directory and where the final path is computed before being returned to the caller and passed into os.WriteFile. We can:
- Ensure
usernameis valid by reusing the existingvalidateUsernamecheck and falling back toPublicUserif it fails (mirroring whatHandleSaveAgentConfigalready does). - Normalize and constrain the generated paths so they always reside under the intended base directory. To do this, we compute a base directory as
baseDir := filepath.Join(AgentConfigRoot)(optionallyAbsif desired), then for each candidate file (userDir/name.yamland.yml) we compute the absolute path and verify that it is still insidebaseDirusingstrings.HasPrefix(absPath, baseDir+string(os.PathSeparator))or an equivalent safe-prefix check. If the check fails, return an error instead of allowing writing outside the base directory.
Concretely, in common/websocket/knowledge2_api.go, you only need to modify the body of resolveAgentConfigPathForWrite. The function will now (a) sanitize username (fallback to PublicUser), (b) compute baseDir and ensure it exists (or at least use it in checks), (c) for each candidate file, Abs it and ensure it is a descendant of baseDir before using it. No new imports are required because os, filepath, strings, and errors are already imported. All callers can stay unchanged, preserving existing functionality except that malicious or malformed usernames/paths will now be rejected safely.
| @@ -665,20 +665,46 @@ | ||
|
|
||
| // resolveAgentConfigPathForWrite 解析写入路径(写入用户目录) | ||
| func resolveAgentConfigPathForWrite(username, name string) (string, error) { | ||
| // Ensure username is safe; fall back to public user if invalid | ||
| if !validateUsername(username) { | ||
| username = PublicUser | ||
| } | ||
|
|
||
| userDir := getAgentUserDir(username) | ||
|
|
||
| // Compute the absolute base directory for all agent configs | ||
| baseDir, err := filepath.Abs(AgentConfigRoot) | ||
| if err != nil { | ||
| return "", err | ||
| } | ||
|
|
||
| candidates := []string{ | ||
| filepath.Join(userDir, name+".yaml"), | ||
| filepath.Join(userDir, name+".yml"), | ||
| } | ||
| for _, path := range candidates { | ||
| _, statErr := os.Stat(path) | ||
|
|
||
| for _, p := range candidates { | ||
| absPath, err := filepath.Abs(p) | ||
| if err != nil { | ||
| return "", err | ||
| } | ||
| // Ensure the resolved path is within the baseDir to prevent directory traversal | ||
| if !strings.HasPrefix(absPath, baseDir+string(os.PathSeparator)) && absPath != baseDir { | ||
| return "", fmt.Errorf("invalid path resolved for agent config") | ||
| } | ||
|
|
||
| _, statErr := os.Stat(absPath) | ||
| if statErr == nil { | ||
| return path, nil | ||
| return absPath, nil | ||
| } | ||
| if statErr != nil && !errors.Is(statErr, os.ErrNotExist) { | ||
| return "", statErr | ||
| } | ||
|
|
||
| // Remember the first valid-in-baseDir candidate to use as default | ||
| candidates[0] = absPath | ||
| } | ||
|
|
||
| return candidates[0], nil | ||
| } | ||
|
|
|
|
||
| // listAgentConfigNamesFromDir 从指定目录读取配置名称列表 | ||
| func listAgentConfigNamesFromDir(dir string) ([]string, error) { | ||
| entries, err := os.ReadDir(dir) |
Check failure
Code scanning / CodeQL
Uncontrolled data used in path expression High
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 27 days ago
In general, to fix uncontrolled path usage, you should normalize and validate any user-derived path components before creating filesystem paths, and enforce that the final absolute path stays within a known, safe base directory. Even if individual call sites validate input, putting the core safety checks close to the actual path construction makes the code resilient to future misuse.
For this specific code, the safest non-breaking fix is:
- Make
AgentConfigRootan absolute, cleaned path at startup usingfilepath.Absandfilepath.Clean. This ensures consistent behavior for subsequent safety checks. - Change
getAgentUserDirso it:- Cleans the
usernamecomponent. - Rejects any username that would introduce path traversal (e.g.,
.., path separators), independent ofvalidateUsername. - Builds the path with
filepath.Join. - Resolves it to an absolute path and verifies that the result still lies under the cleaned
AgentConfigRootusingstrings.HasPrefix. - If validation fails for any reason, falls back to the directory for
PublicUser, which is safe.
- Cleans the
- This way, any future use of
getAgentUserDirautomatically benefits from these checks, and thedirargument passed tolistAgentConfigNamesFromDiris guaranteed to be withinAgentConfigRoot, directly addressing the CodeQL alert without changing the external behavior of the handlers.
Concretely, all changes will be in common/websocket/knowledge2_api.go:
- Add a new package-level variable
agentConfigRootAbsand aninitfunction that computes its absolute, cleaned value fromAgentConfigRoot. If that computation fails (unlikely), we conservatively leave it as the originalAgentConfigRoot. - Replace the implementation of
getAgentUserDirso it:- Uses
filepath.Cleanon the username. - Rejects unsafe usernames: empty, containing path separators (
/oros.PathSeparator), or"..". - Joins the cleaned username to
agentConfigRootAbs, obtains the absolute path, and checks it starts withagentConfigRootAbs + string(os.PathSeparator)(and also allows exact equality). - On any failure, falls back to returning the directory for
PublicUser.
- Uses
No new external packages are required: we reuse os, filepath, and strings which are already imported.
| filepath.Join(userDir, name+".yml"), | ||
| } | ||
| for _, path := range candidates { | ||
| _, statErr := os.Stat(path) |
Check failure
Code scanning / CodeQL
Uncontrolled data used in path expression High
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 27 days ago
In general, to fix uncontrolled path usage, you must ensure that any path derived from user input is properly constrained. This is typically done by (1) validating the input (disallowing separators, .., etc.) when it is meant to be a single component, and/or (2) resolving the final path against a known safe root and verifying that the resulting absolute path still lies under that root before any filesystem operations (read, write, stat, mkdir, etc.) occur.
For this code, the best fix with minimal functional change is:
- Keep
validateUsernameas-is, but add a helper that safely resolves a user’s agent config directory as an absolute path under a fixed safe root. - Use this helper in place of
getAgentUserDir(username)wherever we prepare to touch the filesystem, so that:- We always compute
absUserDir := filepath.Abs(filepath.Join(AgentConfigRoot, username)). - We reject any path that does not remain under
AgentConfigRootby checking withstrings.HasPrefix(absUserDir, absRoot+string(os.PathSeparator))(or an equivalent robust check).
- We always compute
- Use this safe directory both when creating directories and when enumerating candidate config file paths.
Within common/websocket/knowledge2_api.go, we will:
- Introduce a new function, e.g.
getSafeAgentUserDir(username string) (string, error), placed neargetAgentUserDir/validateUsername, that:- Resolves
AgentConfigRootto an absolute path once per call. - Builds
userDir := filepath.Join(AgentConfigRoot, username), getsabsUserDir := filepath.Abs(userDir), and verifiesabsUserDiris still inside the resolvedAgentConfigRoot(by prefix check). - Returns an error if the check fails.
- Resolves
- Update the creation of the user directory in
HandleSaveAgentConfigto usegetSafeAgentUserDirrather than the rawgetAgentUserDir, and use the resulting safe directory path foros.MkdirAll. - Update the code around line 669–674 (where
candidatesandpathare built andos.Stat(path)is called) to:- Use
getSafeAgentUserDir(username)to computeuserDir. - Ensure that candidate paths (
filepath.Join(userDir, name+".yaml" / ".yml")) are under the safe root (this will hold automatically ifuserDiris safe).
- Use
These changes stay within the shown file and reuse existing imports (filepath, strings, os, errors), so no new dependencies are required.
| @@ -331,6 +331,30 @@ | ||
| return filepath.Join(AgentConfigRoot, username) | ||
| } | ||
|
|
||
| // getSafeAgentUserDir 返回安全的 agent 配置目录(绝对路径并确保位于 AgentConfigRoot 下) | ||
| func getSafeAgentUserDir(username string) (string, error) { | ||
| // 解析根目录为绝对路径 | ||
| absRoot, err := filepath.Abs(AgentConfigRoot) | ||
| if err != nil { | ||
| return "", err | ||
| } | ||
|
|
||
| // 拼接用户目录并解析为绝对路径 | ||
| userDir := filepath.Join(AgentConfigRoot, username) | ||
| absUserDir, err := filepath.Abs(userDir) | ||
| if err != nil { | ||
| return "", err | ||
| } | ||
|
|
||
| // 确保用户目录仍然在根目录下,防止路径穿越 | ||
| absRootWithSep := absRoot + string(os.PathSeparator) | ||
| if absUserDir != absRoot && !strings.HasPrefix(absUserDir, absRootWithSep) { | ||
| return "", fmt.Errorf("invalid user directory") | ||
| } | ||
|
|
||
| return absUserDir, nil | ||
| } | ||
|
|
||
| // validateUsername 验证用户名安全性(防止路径穿越) | ||
| func validateUsername(username string) bool { | ||
| if username == "" { | ||
| @@ -496,8 +520,15 @@ | ||
| return | ||
| } | ||
|
|
||
| // 创建用户专属目录 | ||
| userDir := getAgentUserDir(username) | ||
| // 创建用户专属目录(使用安全路径) | ||
| userDir, err := getSafeAgentUserDir(username) | ||
| if err != nil { | ||
| c.JSON(http.StatusInternalServerError, gin.H{ | ||
| "status": 1, | ||
| "message": "创建目录失败: " + err.Error(), | ||
| }) | ||
| return | ||
| } | ||
| if err := os.MkdirAll(userDir, 0755); err != nil { | ||
| c.JSON(http.StatusInternalServerError, gin.H{ | ||
| "status": 1, |
| userDir := getAgentUserDir(username) | ||
| for _, ext := range []string{".yaml", ".yml"} { | ||
| path := filepath.Join(userDir, name+ext) | ||
| err := os.Remove(path) |
Check failure
Code scanning / CodeQL
Uncontrolled data used in path expression High
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 27 days ago
In general, to fix uncontrolled path usage, you should ensure that any path built from user input is constrained to a specific base directory and does not escape it. This is typically done by combining a trusted base directory with the (sanitized) user input, resolving the result to an absolute path, and then verifying that this absolute path still lies within the intended base directory. Only then should you perform file operations like os.Remove.
For this specific code, we already sanitize username with validateUsername and fall back to PublicUser when invalid. However, CodeQL still sees tainted data reaching os.Remove at line 690 through userDir := getAgentUserDir(username) and path := filepath.Join(userDir, name+ext). The minimal, functionality‑preserving hardening is to add a safety check after building userDir and before deleting files: (1) compute the absolute path of userDir, (2) compute the absolute path of each candidate file to delete, and (3) verify that each absolute candidate path has the absolute user directory as its prefix. If any resolution fails or the prefix check fails, we abort deletion and return an error. This prevents any path traversal abuse even if username validation fails elsewhere or changes in the future, and it also addresses the static analysis concern by demonstrating that the sink is reached only with a path within the allowed tree.
Concretely, in deleteAgentConfig (lines 686–699 in common/websocket/knowledge2_api.go), we will:
- Call
filepath.Abs(userDir)to getabsUserDir. - For each
ext, buildcandidate := filepath.Join(userDir, name+ext), then computeabsCandidate := filepath.Abs(candidate). - Check that
absCandidatestarts withabsUserDir + string(os.PathSeparator)or equalsabsUserDir; if not, return an error likeerrors.New("invalid path")without attempting deletion. - Only call
os.Remove(absCandidate)after this check.
We can use existing imports (os, filepath, errors, strings are already imported), so no new imports are needed. All changes are confined to the deleteAgentConfig function in this file.
| @@ -685,9 +685,24 @@ | ||
| // deleteAgentConfig 删除配置(只删除用户目录的配置) | ||
| func deleteAgentConfig(username, name string) (bool, error) { | ||
| userDir := getAgentUserDir(username) | ||
| absUserDir, err := filepath.Abs(userDir) | ||
| if err != nil { | ||
| return false, err | ||
| } | ||
|
|
||
| for _, ext := range []string{".yaml", ".yml"} { | ||
| path := filepath.Join(userDir, name+ext) | ||
| err := os.Remove(path) | ||
| candidate := filepath.Join(userDir, name+ext) | ||
| absCandidate, err := filepath.Abs(candidate) | ||
| if err != nil { | ||
| return false, err | ||
| } | ||
|
|
||
| // 防止路径穿越:确保目标文件在用户目录下 | ||
| if absCandidate != absUserDir && !strings.HasPrefix(absCandidate, absUserDir+string(os.PathSeparator)) { | ||
| return false, errors.New("invalid agent config path") | ||
| } | ||
|
|
||
| err = os.Remove(absCandidate) | ||
| if err == nil { | ||
| return true, nil | ||
| } |
…ool usage statistics - Updated `execute_stage` method to return dialogue statistics for each scan stage. - Modified `scan` method to accumulate total dialogue counts across stages. - Introduced tool usage statistics in `BaseAgent` to track tool invocation counts. - Adjusted report generation to include total tests conducted during the scan. - Removed optional test count reporting from the security reviewer prompt.
- Updated the agent class to extract both agent_type and agent_name from the provider configuration.
…s found" findings
- Changed default risk type from "low" to "safe" when no vulnerabilities are found. - Enhanced risk type determination logic to account for various vulnerability levels.
…ration - Improved user message formatting in `run_agent` to support English and Chinese based on the specified language. - Expanded documentation in `agent_security_reviewer.md` regarding system prompt disclosure assessment notes for clarity and robustness.
…zation - Reduced `max_iter` from 80 to 40 for improved performance. - Added language-specific instructions in `run_agent` to ensure outputs are generated in the correct language (English or Chinese).
…tion - Improved user message localization in `run_agent` to handle English and Chinese based on the specified language. - Updated sensitive information detection in `AgentScanner` to utilize regex patterns for better accuracy and severity classification. - Refactored skill search functionality to fallback to the full skill list when no matches are found for a query. - Adjusted task execution to use the context's language setting for improved consistency.
…ection - Introduced asynchronous handling in `BaseAgent` and `dialogue` functions to improve performance during skill execution. - Expanded the `ScanPipeline` to run detection skills concurrently, enhancing the overall scanning efficiency. - Updated the `data-leakage-detection` skill documentation for clarity and improved probing strategies. - Added a new `skill-runner` prompt to streamline single-skill execution and reporting of vulnerabilities. - Enhanced error handling in the dialogue tool to manage transient failures more effectively.
…gue processing - Updated `handle_response` in `BaseAgent` to support single tool invocation parsing, improving clarity and efficiency. - Enhanced JSON escaping in `AIProviderClient` to ensure proper formatting of prompts in API requests. - Revised documentation in `agent_security_reviewer.md` and `project_summary.md` to clarify processing strategies and optimize information gathering. - Added strict rules for dialogue calls to prevent unnecessary iterations during report generation.
No description provided.