Skip to content

Let Gradio apps also be MCP Servers #10984

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 143 commits into from
Apr 29, 2025
Merged

Let Gradio apps also be MCP Servers #10984

merged 143 commits into from
Apr 29, 2025

Conversation

abidlabs
Copy link
Member

@abidlabs abidlabs commented Apr 9, 2025

Gradio apps are also now MCP Servers. TLDR: to use, just add mcp_server=True in .launch()

This works for Gradio apps running locally as well as hosted Gradio apps on Spaces. Keep reading on how to test this PR...

Testing MCP Server Running on Spaces

To make this easy to test, I've launched two Gradio apps on Spaces using this PR that you can use to test:

Testing MCP Server Running Locally

To test this out locally, first install gradio from this PR following the instructions below. Also install mcp e.g. pip install mcp

Then, launch a Gradio app like this:

import gradio as gr

def letter_counter(word, letter):
    """
    Count the number of occurrences of a letter in a word or text.

    Args:
        word (str): The input text to search through
        letter (str): The letter to search for

    Returns:
        str: A message indicating how many times the letter appears
    """
    word = word.lower()
    letter = letter.lower()
    count = word.count(letter)    
    return count

# Create and launch the Gradio interface
demo = gr.Interface(
    fn=letter_counter,
    inputs=["textbox", "textbox"],
    outputs="number",
    title="Letter Counter",
    description="Enter text and a letter to count how many times the letter appears in the text."
)

if __name__ == "__main__":
    demo.launch(mcp_server=True) # Alternatively use the GRADIO_MCP_SERVER env variable

Note the .launch(mcp_server=True).

Once you launch the Gradio app, the "view API" page should include a 4th "language" option called MCP. There will also be a code box that shows that configuration that you need to paste inside your MCP Client (e.g. Cline).

image

Note that since this uses SSE protocol, this will work for any client that supports that such as Cline or Cursor. I've also added instructions for integrating with clients like Claude Desktop that do not

You can also test with the MCP Inspector, which is a fantastic debugging tool.

Once you post the config json in e.g. Cline, you should be able to ask a question like: "how many rs in strawberry" and the model should respond like this:

image

Closes: #10949

@gradio-pr-bot
Copy link
Collaborator

gradio-pr-bot commented Apr 9, 2025

🪼 branch checks and previews

Name Status URL
Spaces ready! Spaces preview
Website building...
Storybook ready! Storybook preview
🦄 Changes detected! Details

Install Gradio from this PR

pip install https://gradio-pypi-previews.s3.amazonaws.com/37eed343820421df566b8aaec4210c92b60e2087/gradio-5.27.1-py3-none-any.whl

Install Gradio Python Client from this PR

pip install "gradio-client @ git+https://github.com/gradio-app/gradio@37eed343820421df566b8aaec4210c92b60e2087#subdirectory=client/python"

Install Gradio JS Client from this PR

npm install https://gradio-npm-previews.s3.amazonaws.com/37eed343820421df566b8aaec4210c92b60e2087/gradio-client-1.14.2.tgz

Use Lite from this PR

<script type="module" src="https://gradio-lite-previews.s3.amazonaws.com/37eed343820421df566b8aaec4210c92b60e2087/dist/lite.js""></script>

@gradio-pr-bot
Copy link
Collaborator

gradio-pr-bot commented Apr 9, 2025

🦄 change detected

This Pull Request includes changes to the following packages.

Package Version
@gradio/core minor
gradio minor
gradio_client minor
  • Maintainers can select this checkbox to manually select packages to update.

With the following changelog entry.

Let Gradio apps also be MCP Servers

Maintainers or the PR author can modify the PR title to modify this entry.

Something isn't right?

  • Maintainers can change the version label to modify the version bump.
  • If the bot has failed to detect any changes, or if this pull request needs to update multiple packages to different versions or requires a more comprehensive changelog entry, maintainers can update the changelog file directly.

@abidlabs abidlabs changed the title Let any Gradio app also be an MCP Server Let Gradio apps also be MCP Servers Apr 9, 2025
@abidlabs abidlabs changed the title Let Gradio apps also be MCP Servers Let Gradio apps also be MCP Servers [PoC] Apr 10, 2025
@abidlabs abidlabs added v: patch A change that requires a patch release and removed v: patch A change that requires a patch release labels Apr 29, 2025
Copy link
Collaborator

@freddyaboulton freddyaboulton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

@@ -12,11 +12,14 @@
import InstallSnippet from "./InstallSnippet.svelte";
import CodeSnippet from "./CodeSnippet.svelte";
import RecordingSnippet from "./RecordingSnippet.svelte";
import CopyButton from "./CopyButton.svelte";
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tool description shows undefined if the parameter does not have a description in the docstring. We should be showing the same info in /schema, no?

image

code={JSON.stringify(
{
mcpServers: {
gradio: {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should use the space-id here if available so that mcp serves have unique names for spaces.

print("output", output)
return self.postprocess_output_data(output["data"])

@server.list_tools()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is an issue when using multiple gradio apps as mcp servers at the same time in cursor.

I had two active gradio mcp sessions, one was a dummy audio reversal interface, and the other was a dummy audio transcription server. I gave each server a unique name but each had one tool called predict.

When I told it to transcribe audio, it kept using the predict function from the dummy audio reversal interface instead of the audio transcription server. Even after I prompted it to use the correct mcp server! Only after disabling the other mcp server did it work.

This is an app issue instead of a server issue (we should file an issue with cursor?), but should we give our tools unique names (perhaps by prepending with the space id when available) to maximize the chance of gradio mcp servers "just working"?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here are screenshots of the interaction:

image image image

Here I manually disabled the other gradio mcp server
image

@abidlabs
Copy link
Member Author

Everything should be addressed now @freddyaboulton! MCP tool names now include a prefix if running on Spaces:

image

Copy link
Collaborator

@freddyaboulton freddyaboulton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome PR! Left some questions mainly on the code in the new guide. Functionality is great. let's ship ✅

stdio_transport = await self.exit_stack.enter_async_context(stdio_client(server_params))
self.stdio, self.write = stdio_transport

self.session = await self.exit_stack.enter_async_context(ClientSession(self.stdio, self.write))
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the benefit of using AsyncExitStack over just using ClientSession and stdio_client in an async with ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure, will leave it to @yvrjsharma to make the change if needed

self.anthropic = Anthropic()
self.tools = []

def connect(self, server_path: str) -> str:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think connect can be async if it's just just used in the connect button click event.

{"role": "assistant", "content": "Please connect to an MCP server first."}
], gr.Textbox(value="")

new_messages = loop.run_until_complete(self._process_query(message, history))
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be clearer if everything was async rather than mixing sync and async scopes via loop.run_until_complete. Anthropic has an async client so I think it's possible just to make everything async.

@abidlabs
Copy link
Member Author

Switched over the MCP Client Guide to use the Gradio Python Client. I don't have the full context for the other changes, so I think it's fine to leave as is until @yvrjsharma is back!

@abidlabs
Copy link
Member Author

Thanks so much for the reviews and contributions @freddyaboulton @yvrjsharma @evalstate @cocktailpeanut!

@abidlabs abidlabs enabled auto-merge (squash) April 29, 2025 21:23
@abidlabs abidlabs merged commit 8dab577 into main Apr 29, 2025
23 checks passed
@abidlabs abidlabs deleted the mcp branch April 29, 2025 22:00
freddyaboulton pushed a commit that referenced this pull request May 13, 2025
* changes

* testing

* add changeset

* changes

* changes

* changes

* add changeset

* changes

* changes

* revert demo

* changes

* fixes

* changes

* format

* changes

* changes

* format

* notebook

* lcean

* format

* format

* add changeset

* add changeset

* changes

* changes

* changes

* changes

* changes

* format

* changes

* changes

* changes

* changes

* add changeset

* Changes

* changes

* changes

* changes

* changes

* changes

* changes

* add changeset

* hygiene

* next

* changes

* changes

* changes

* changes

* changes

* changes

* changes

* add demo

* changes

* changes

* changes

* changes

* changes

* changes

* changes

* changes

* changes

* changes

* changes

* changes

* revert demos

* changes

* changes

* format

* fix utils

* changes

* format

* changes

* more tests

* changes

* changes

* changes

* changes

* pydantic

* changes

* change

* mcp

* calculator

* remove base64

* changes

* changes

* changes

* changes

* add changeset

* changes

* add changeset

* guide

* changes

* changes

* guides

* change

* changes

* changes

* guide

* changes

* changes

* changes

* changes

* tests

* reqs

* fix

* change

* changes

* changes

* changes

* changes

* changes

* changes

* fixes

* fixes

* changes

* format

* guide

---------

Co-authored-by: gradio-pr-bot <gradio-pr-bot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support Model Context Protocol (MCP) in Gradio
7 participants