Support offical MCP Gem as a Backend

[patvice]

Add Official MCP SDK Support with Adapter Pattern

Overview

This PR introduces a major architectural refactor that adds support for the official Ruby MCP SDK while maintaining full backward compatibility. The gem now uses an adapter pattern that allows users to choose between the native full-featured implementation or the official Anthropic-maintained SDK.

What's New

🎛️ Dual SDK Support (v0.8.0+)

Users can now choose their MCP implementation:

• :ruby_llm adapter (default) - Native full-featured implementation with all advanced MCP features
• :mcp_sdk adapter (new) - Official Anthropic-maintained SDK with core MCP features

🏗️ Adapter Architecture

Introduced a flexible adapter system that:

• Provides a common interface (BaseAdapter) for all MCP implementations
• Allows per-client adapter configuration
• Supports global default adapter settings
• Enables mixing different adapters in the same application

Key Changes

New Files & Architecture

• Adapter System:

  • lib/ruby_llm/mcp/adapters/base_adapter.rb - Base adapter interface
  • lib/ruby_llm/mcp/adapters/ruby_llm_adapter.rb - Native implementation adapter
  • lib/ruby_llm/mcp/adapters/mcp_sdk_adapter.rb - Official SDK adapter
  • Custom transport wrappers for MCP SDK (stdio, http, streamable_http, sse)

• Native Implementation Reorganization:

  • Moved all native implementation code into lib/ruby_llm/mcp/native/ namespace
  • Better separation of concerns between protocol, transport, and client logic
  • New native/client.rb, native/transport.rb, and native/protocol.rb

Enhanced Configuration

• Global adapter selection: config.default_adapter = :ruby_llm
• Per-client adapter override: adapter: :mcp_sdk
• YAML configuration support for adapters

Documentation

• New comprehensive Adapters Guide with:

  • Feature comparison table
  • Transport compatibility matrix
  • Usage examples for both adapters
  • Migration guide and best practices
  • Troubleshooting section

• Updated README with adapter selection guidance

• Enhanced configuration documentation

Feature Comparison

Feature	RubyLLM Adapter	MCP SDK Adapter
Core Features
Tools, Resources, Prompts	✅	✅
Resource Templates	✅	❌
Transports
Basic HTTP	❌	✅
Streamable	✅	Custom
stdio	✅	Custom
SSE	✅	Custom
Advanced Features
Completions, Logging	✅	❌
Sampling, Roots	✅	❌
Progress Tracking	✅	❌
Human-in-the-Loop	✅	❌
Elicitation	✅	❌

Testing

• Extensive test coverage: 31,255 insertions across 305 files
• New adapter-specific test suites:
  • spec/ruby_llm/mcp/adapters/mcp_sdk_sse_spec.rb
  • spec/ruby_llm/mcp/native/transport_spec.rb
  • spec/ruby_llm/mcp/native/transports/*.rb
• Updated all existing specs to work with adapter pattern
• Added adapter test helpers in spec/support/adapter_test_helpers.rb
• 200+ new VCR cassettes for comprehensive integration testing

Backward Compatibility

✅ Fully backward compatible - existing code continues to work without changes:

• Default adapter is :ruby_llm (maintains current behavior)
• All existing APIs remain unchanged
• No breaking changes to public interfaces

Usage Examples

Using the Native Adapter (Default)

client = RubyLLM::MCP.client(
  name: "filesystem",
  adapter: :ruby_llm,  # Optional, this is the default
  transport_type: :stdio,
  config: { command: "mcp-server-filesystem", args: ["/path"] }
)

Using the Official SDK Adapter

# Requires: gem 'mcp', '~> 0.4'
client = RubyLLM::MCP.client(
  name: "api_server",
  adapter: :mcp_sdk,
  transport_type: :http,
  config: { url: "https://api.example.com/mcp" }
)

Mixed Adapter Usage

RubyLLM::MCP.configure do |config|
  config.default_adapter = :ruby_llm
  
  config.mcp_configuration = [
    { name: "filesystem", adapter: :ruby_llm, ... },  # Needs roots
    { name: "weather", adapter: :mcp_sdk, ... }       # Simple API
  ]
end

Dependencies

• Adds optional dependency: mcp gem (>= 0.4) - only required when using :mcp_sdk adapter
• Updated gemspec to include optional dependencies

Migration Path

For users upgrading from v0.7.x:

1. No action required - default behavior unchanged
2. Optionally add gem 'mcp' to use official SDK adapter
3. See Adapters Guide for migration details

---

Testing Checklist

• All existing tests pass
• New adapter tests added
• Integration tests with both adapters
• Documentation updated
• Example code verified
• Backward compatibility verified

Review Focus Areas

• Adapter abstraction design
• Transport compatibility layer
• Feature parity between adapters
• Documentation completeness

[parruda]

@patvice will this break #88?

[patvice]

This will need to be updated when OAuth 2.1 support is merged but that's fine. This is still not quiet ready yet