Skip to content

Conversation

@j1z0
Copy link
Contributor

@j1z0 j1z0 commented Nov 14, 2025

This repository is public. Do not put here any private DataRobot or customer's data: code, datasets, model artifacts, .etc.

Summary

Parameterize MCP Integration Tests

Summary

This PR simplifies and parameterizes MCP (Model Context Protocol) integration tests across all agent frameworks (CrewAI, LangGraph, LlamaIndex).

Changes

1. MCP Test Parameterization

Consolidated duplicate MCP integration tests across all three frameworks by using pytest.mark.parametrize:

Before

  • Each framework had two separate test methods:
    • test_agent_loads_mcp_tools_from_external_url_in_invoke
    • test_agent_loads_mcp_tools_from_datarobot_deployment_in_invoke

After

  • Single parametrized test method per framework:
    • test_agent_loads_mcp_tools_in_invoke with two test cases (external URL and DataRobot deployment)

Files Modified

  • template/{{agent_app_name}}/tests/test_mcp_templates/test_mcp_crewai.j2
  • template/{{agent_app_name}}/tests/test_mcp_templates/test_mcp_langgraph.j2
  • template/{{agent_app_name}}/tests/test_mcp_templates/test_mcp_llamaindex.j2

3. Test Simplification

  • Removed agent_kwargs parameterization: Tests now hardcode api_key and api_base directly within the test function
  • Removed tools_factory complexity: All test cases now use default_tools from the common mocks fixture
  • Simplified environment setup: Environment variables are passed via env_patch dictionary in pytest.param definitions
  • Cleaner assertions: Tests only check for expected_adapter_url when provided (external URL case)

Benefits

  • Reduced code duplication: Eliminated duplicate test logic across external URL and DataRobot deployment scenarios
  • Easier maintenance: Adding new MCP connection scenarios only requires adding a new pytest.param entry
  • More reliable CLI tests: Fixed hanging and connection refused errors in smoke tests
  • Simpler test structure: Removed unnecessary parameterization complexity

Testing

  • All existing tests pass
  • CLI smoke tests complete successfully without hanging
  • MCP integration tests verify both external URL and DataRobot deployment scenarios

Rationale

@@ -1,3 +1,3 @@
"""
Tests for MCP CrewAI integration - verifying agents have MCP tools configured.
"""
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we change this header to:

# Copyright 2025 DataRobot, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#   http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When this merges into recipes we hit copyright errors. We should probably add a test for this to the repo, but its not quite as simple with the jinja

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

5 participants