Ai 2161 update search tool to search in configurations by mariankrotil · Pull Request #348 · keboola/mcp-server

mariankrotil · 2026-01-12T18:09:35Z

Description

Extended the search tool beyond table/bucket usage references to robust configuration-aware discovery across items (configuration, transformation, flow, data-app, etc.).

Key updates in this branch:

Added config-based search path reporting via match_scopes in SearchHit.
Reworked config matching to use JSONPath traversal (including scoped search) and return exact matched paths.
config-based search now always collects all matched paths per item (simpler agent behavior, more complete results).
Removed return_all_matches and case_sensitive from the search tool API to reduce complexity and ambiguity.
Improved scope handling for scalar paths (e.g. direct scope like parameters.api.baseUrl) while preserving descendant matching.
Kept output practical by exposing only most-specific paths in match_scopes (while internal matches keep full detail).
Updated search tool docs/examples and project system prompt guidance to explicitly cover config-based search usage.

Linear: AI-2161-buckets-tables-references

Change Type

Major (breaking changes, significant new features)
Minor (new features, enhancements, backward compatible)
Patch (bug fixes, small improvements, no new features)

Summary

Update search tool

Testing

[x ] Tested with Cursor AI desktop (Streamable-HTTP transports)

Optional testing

Tested with Cursor AI desktop (all transports)
Tested with claude.ai web and canary-orion MCP (SSE and Streamable-HTTP)
Tested with In Platform Agent on canary-orion
Tested with RO chat on canary-orion

Checklist

Self-review completed
Unit tests added/updated (if applicable)
Integration tests added/updated (if applicable)
Project version bumped according to the change type (if applicable)
Documentation updated (if applicable)

…de-links-in-get-tables-exp

linear · 2026-01-12T18:09:38Z

AI-2161 Include upstream links in table lists/detail

…de-links-in-get-tables-exp

…ne filtering Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

src/keboola_mcp_server/tools/search.py

Co-authored-by: Codex <codex@openai.com>

mariankrotil · 2026-02-18T07:23:57Z

@vita-stejskal heads-up: I switched config search to JSONPath-based matching in configuration payloads (including scoped search). I also updated the examples in the search tool description, and config-based results now return matched JSONPaths (match_scopes) so agent can see exactly where the value was found in the config, as we decided.

vita-stejskal · 2026-02-18T13:46:04Z

tests/tools/test_search.py

+                    }
+                ],
+                [('test-config', ['parameters.query', 'storage.input[0].source'])],
+            ),


You should add at least one test with the scopes that proves that the searching really is constrained by the scopes. For example by having the same value of alpha under two different paths, but searching only at one path and checking that the other path is not hit.

vita-stejskal · 2026-02-18T14:10:02Z

src/keboola_mcp_server/tools/search.py

@@ -148,6 +155,15 @@ def check_id_fields(self) -> 'SearchHit':
    def with_matches(self, matches: list['PatternMatch']) -> 'SearchHit':


A nitpick -- I'd remove this to set_matches to point out that this function mutates the existing SearchHit instance rather than creating a new one with the matches field replaced. The docstring already says this clearly, but the with_* prefix slightly contradicts this, because we typically use it for immutable setters that create a copy of the object with one particular field replaced.

vita-stejskal · 2026-02-18T14:23:42Z

src/keboola_mcp_server/tools/search.py

+            scope
+            for scope in unique_scopes
+            if not any(
+                other != scope and other.startswith(scope) and other[len(scope) : len(scope) + 1] in {'.', '['}


It would probably be more efficient to use:

other.startswith(scope) and len(other) > len(scope) and other[len(scope)] in ('.', '[')

fetching a single character is faster than creating a single-char sub-string

creating a tuple is more efficient than creating a set

vita-stejskal · 2026-02-18T14:30:27Z

src/keboola_mcp_server/tools/search.py

+        self._all_nodes_expr = jsonpath_ng.parse('$..*')
+        self._scope_exprs = []
+        for scope in self.search_scopes:
+            normalized = scope if scope.startswith('$') else f'$.{scope}'


I think that you should use the new _normalize_jsonpath() function from PR#394 to handle the #-prefixed keys properly. Without that the jsonpath_ng.parse() is likely to fail for paths such as authorization.#apiKey.

vita-stejskal · 2026-02-18T14:33:16Z

src/keboola_mcp_server/tools/search.py

    _compiled_patterns: list[re.Pattern] = PrivateAttr(default_factory=list)
    _clean_patterns: list[str] = PrivateAttr(default_factory=list)
+    _all_nodes_expr: JSONPath | None = PrivateAttr(default=None)
+    _scope_exprs: list[tuple[str, JSONPath, JSONPath]] = PrivateAttr(default_factory=list)


It would be useful to mention what the tuple elements are.

vita-stejskal · 2026-02-18T14:37:09Z

src/keboola_mcp_server/tools/search.py

-            return [PatternMatch(scope=None, patterns=matched)]
-        return []
+        # No scope provided – search all descendants and return exact match paths.
+        if self._all_nodes_expr is None:


Can this really be None if it is set in the after-validator?

vita-stejskal · 2026-02-18T14:39:12Z

src/keboola_mcp_server/tools/search.py

-        if matched := self.match_patterns(configuration):
-            return [PatternMatch(scope=None, patterns=matched)]
-        return []
+        # No scope provided – search all descendants and return exact match paths.


It'd be easier to read the code, if this code-path were moved to the else-branch of the if self.search_scopes statement.

vita-stejskal · 2026-02-18T14:58:57Z

src/keboola_mcp_server/tools/search.py

+    - user_input: "Find components/transformations using my_bucket in input or output mappings"
+        -> patterns=["my_bucket"], item_types=["configuration", "transformation"], search_type="config-based",
+        scopes=["storage.input", "storage.output"]
+        -> Returns matches with paths like `storage.input[0].source` or `storage.output[0].target`


Are those paths accurate? Do they not look more like storage.input.tables[0].source and similar? In general, the input/output mappings can contain both tables and files.

vita-stejskal · 2026-02-18T15:01:36Z

src/keboola_mcp_server/tools/search.py

+
+    - user_input: "Find transformations using this table / column / specific code in its script"
+        -> patterns=["element"], item_types=["transformation"], search_type="config-based",
+        scopes=["parameters"]


This example is not accurate, I think. The table could be also mentioned under the storage scope (i.e. in the input/output mappings).

vita-stejskal · 2026-02-18T15:08:07Z

src/keboola_mcp_server/tools/search.py

+                    return matches
+        return matches
+
+    def _find_scalar_matches_for_expr(self, configuration: JsonDict, parsed_expr: JSONPath) -> list[PatternMatch]:


This function is pretty much the same as _find_matches_for_expr() function. They could easily be collapsed to a single function with an extra "search_subtree" parameter (or "scalars_only" parameter or something like that).

vita-stejskal · 2026-02-18T15:09:59Z

This looks much better now. I think that we are nearly done. Thanks!

mariankrotil added 12 commits January 6, 2026 14:28

AI-2161 feat: add complex component config search tool

25c94e5

AI-2161 feat: add instruction examples

7478604

Merge branch 'AI-2161-include-links-in-get-tables' into AI-2161-inclu…

5f3449d

…de-links-in-get-tables-exp

Merge AI-2161

82ee62e

Merge AI-2161

2eeacb4

AI-2161 fix: ignore private matches in SearchHit equality

c8a43cb

AI-2161 test: use regex mode in search regex test

88d6615

AI-2161 refactor: remove usage tool registration

ff2f68b

AI-2161 fix: return extractor configs as configuration items

dc1da3e

AI-2161 fix: include configurations in table usage search

4ef683b

AI-2161 test: add config-based search integration test

da22a09

AI-2161 style: apply tox

cbe58d7

mariankrotil requested a review from vita-stejskal January 12, 2026 18:09

mariankrotil changed the base branch from main to AI-2161-include-links-in-get-tables January 12, 2026 18:13

mariankrotil changed the title ~~Ai 2161 include references in tables and buckets~~ Ai 2161 update search tool to search in configurations Jan 16, 2026

mariankrotil self-assigned this Jan 20, 2026

mariankrotil added 6 commits January 21, 2026 15:13

Merge AI-2161-storage-ref into AI-2161-search-tool-update

fbb3aca

AI-2161 chore: update version

e10c074

Merge branch 'AI-2161-include-links-in-get-tables' into AI-2161-inclu…

e8998f1

…de-links-in-get-tables-exp

AI-2161 refactor: add component to search type

e421ed8

AI-2161 fix: update import

5ca04b5

Merge branch 'AI-2161-include-links-in-get-tables' into AI-2161-inclu…

81610a9

…de-links-in-get-tables-exp

Base automatically changed from AI-2161-include-links-in-get-tables to main January 23, 2026 08:46

mariankrotil and others added 6 commits January 23, 2026 12:07

Merge branch 'main' into AI-2161-include-links-in-get-tables-exp

3e6c3e4

Merge branch 'main' into AI-2161-include-links-in-get-tables-exp

d05fe02

AI-2161 chore: update version

c1fc287

AI-2161 fix: missing return in validator, doc defaults, typos, and No…

05b868e

…ne filtering Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Merge branch 'main' into AI-2161-include-links-in-get-tables-exp

dcdad7c

AI-2161: fix search tool's docstring

6de7ea4

vita-stejskal reviewed Feb 8, 2026

View reviewed changes

src/keboola_mcp_server/tools/search.py Outdated Show resolved Hide resolved

mariankrotil and others added 8 commits February 18, 2026 06:31

AI-2161 feat: simplify search API and improve config scope matching

0e6686e

AI-2161 test: add config-based search scope coverage

ac4e665

AI-2161 docs: refresh search tool documentation

b390a73

AI-2161 docs: add draft search description

19ba658

AI-2161 chore: update lockfile package version

f2a489e

AI-2161 chore: remove draft search description file

6f4e672

Merge main into AI-2161

5cc8b08

AI-2161 docs: mention config-based search in project system prompt

d6240cb

Co-authored-by: Codex <codex@openai.com>

mariankrotil requested a review from vita-stejskal February 18, 2026 07:15

AI-2161 docs: rename project prompt section to finding items

52c32ea

vita-stejskal reviewed Feb 18, 2026

View reviewed changes

		@@ -148,6 +155,15 @@ def check_id_fields(self) -> 'SearchHit':
		def with_matches(self, matches: list['PatternMatch']) -> 'SearchHit':

Comments

Conversation

mariankrotil commented Jan 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Change Type

Summary

Testing

Optional testing

Checklist

Uh oh!

linear bot commented Jan 12, 2026

Uh oh!

Uh oh!

mariankrotil commented Feb 18, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

vita-stejskal commented Feb 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

mariankrotil commented Jan 12, 2026 •

edited

Loading