Benchmark simpleqa #43
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#40) * Add Confluence tools integration with search and page retrieval capabilities * fix: address PR comments - Replace redmadrobot.com with yourcompany.com - Remove CORS middleware - Remove SmartPlatform from examples - Remove search_duration from output - Remove hardcoded space keys - Change page_id type to int * refactor: remove redundant page_id validation Since page_id is now an int with gt=0 validation in Pydantic model, we don't need to check isdigit() anymore * deps: add beautifulsoup4 for Confluence HTML parsing * style: fix pre-commit issues * Remove CORS middleware comment from endpoints.py Removed commented-out CORS middleware section. --------- Co-authored-by: Valerii Kovalskii <vakovalskii@users.noreply.github.com> Co-authored-by: Lysenko Artem <34771157+virrius@users.noreply.github.com>
- Add ExtractPageContentTool for deep content extraction from URLs via Tavily Extract API - Simplify WebSearchTool to return only titles, links and 100-char snippets - Optimize prompt caching: static system prompt (97%+ cache hit rate) - Move dynamic data (date) into user messages per Anthropic best practices - Extract all prompt templates to separate files for easier management - Fix ConfluenceVectorSearchTool naming (remove hardcoded tool_name) - Fix source management: ExtractPageContentTool now updates existing sources instead of overwriting - Improve system prompt: list all tools with real class names - Add ISO date format for better model understanding Result: 90% cost reduction on input tokens + 80% latency improvement
- Make _log_reasoning accept any BaseTool, not just ReasoningTool - Add specific handling for ReasoningTool vs other tools - Fix _reasoning_phase to only provide ReasoningTool in tools list - Prevents AttributeError when model unexpectedly returns ClarificationTool Fixes: 'ClarificationTool' object has no attribute 'remaining_steps'
…/disable config - Remove SmartPlatformConfig and all Smart Platform related code - Remove ConfluenceVectorSearchTool and vector search functionality - Add 'enabled' field to ConfluenceConfig for conditional tool loading - Update research.py to check confluence.enabled before loading tools - Remove project-specific few-shot examples from system prompt - Update citation examples to be more generic - Clean up Confluence space references in prompts
This reverts commit fe539e2.
Added missing bs4 dependency
Feat/prompt caching 0.2.6
virrius
requested changes
Oct 16, 2025
…-deep-research into benchmark_simpleqa
virrius
approved these changes
Oct 20, 2025
MiXaiLL76
approved these changes
Oct 20, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Доработал ошибку, которая возникала при окончании работы агента
Добавил скрипт для прогонки бенчмарка simpleqa