Tools: sharepoint tool (#929)

* feat(backend): added sharepoint tool

* feat(web_assistant): added sharepoint tool

* chore(backend): removed session from args in tool call

* chore(community): cleanup community tool call

---------

Co-authored-by: EugeneP <eugene@lightsonsoftware.com>

README.md

@@ -16,6 +16,7 @@ Toolkit is a deployable all-in-one RAG application that enables users to quickly
   - [How to setup Gmail](/docs/custom_tool_guides/gmail.md)
   - [How to setup Slack Tool](/docs/custom_tool_guides/slack.md)
   - [How to setup Github Tool](/docs/custom_tool_guides/github.md)
+  - [How to setup Sharepoint](/docs/custom_tool_guides/sharepoint.md)
   - [How to setup Google Text-to-Speech](/docs/text_to_speech.md)
   - [How to add authentication](/docs/auth_guide.md)
   - [How to deploy toolkit services](/docs/service_deployments.md)
@@ -28,30 +29,39 @@ Toolkit is a deployable all-in-one RAG application that enables users to quickly
 ![](/docs/assets/toolkit.gif)
 
 ## Try Now:
-There are two main ways for quickly running Toolkit: local and cloud. See the specific instructions given below. 
+
+There are two main ways for quickly running Toolkit: local and cloud. See the specific instructions given below.
+
 ### Local
-*You will need to have [Docker](https://www.docker.com/products/docker-desktop/), [Docker-compose >= 2.22](https://docs.docker.com/compose/install/), and [Poetry](https://python-poetry.org/docs/#installation) installed. [Go here for a more detailed setup.](/docs/setup.md)*  
+
+_You will need to have [Docker](https://www.docker.com/products/docker-desktop/), [Docker-compose >= 2.22](https://docs.docker.com/compose/install/), and [Poetry](https://python-poetry.org/docs/#installation) installed. [Go here for a more detailed setup.](/docs/setup.md)_
 Note: to include community tools when building locally, set the `INSTALL_COMMUNITY_DEPS` build arg in the `docker-compose.yml` to `true`.
 
 Both options will serve the frontend at http://localhost:4000.
 
 #### Using `make`
+
 Use the provided Makefile to simplify and automate your development workflow with Cohere Toolkit, including Docker Compose management, testing, linting, and environment setup.
+
 ```bash
 git clone https://github.com/cohere-ai/cohere-toolkit.git
 cd cohere-toolkit
 make first-run
 ```
 
 #### Docker Compose only
+
 Use Docker Compose directly if you want to quickly spin up and manage your container environment without the additional automation provided by the Makefile.
+
 ```bash
 git clone https://github.com/cohere-ai/cohere-toolkit.git
 cd cohere-toolkit
 docker compose up
 docker compose run --build backend alembic -c src/backend/alembic.ini upgrade head
 ```
+
 ### Cloud
+
 #### GitHub Codespaces
 
 To run this project using GitHub Codespaces, please refer to our [Codespaces Setup Guide](/docs/github_codespaces.md).
@@ -63,7 +73,7 @@ To run this project using GitHub Codespaces, please refer to our [Codespaces Set
 - **Interfaces** - any client-side UI, currently contains two web apps, one agentic and one basic, and a Slack bot implementation.
   - Defaults to Cohere's Web UI at `src/interfaces/assistants_web` - A web app built in Next.js. Includes a simple SQL database out of the box to store conversation history in the app.
   - You can change the Web UI using the docker compose file.
-- **Backend API** - in `src/backend` this follows a similar structure to the [Cohere Chat API](https://docs.cohere.com/reference/chat) but also include customizable elements: 
+- **Backend API** - in `src/backend` this follows a similar structure to the [Cohere Chat API](https://docs.cohere.com/reference/chat) but also include customizable elements:
   - **Model** - you can customize with which provider you access Cohere's Command models. By default included in the toolkit is Cohere's Platform, Sagemaker, Azure, Bedrock, HuggingFace, local models. [More details here.](/docs/command_model_providers.md)
   - **Retrieval**- you can customize tools and data sources that the application is run with.
 - **Service Deployment Guides** - we also include guides for how to deploy the toolkit services in production including with AWS, GCP and Azure. [More details here.](/docs/service_deployments.md)

docs/custom_tool_guides/sharepoint.md

@@ -0,0 +1,39 @@
+# Sharepoint Tool Setup
+
+To setup the Sharepoint tool you need to configure API access via the following steps
+
+## 1. Configure Tenant ID and Client ID
+
+Your Microsoft Tenant ID and Client ID can be found my navigating to the [Micorsoft Entra Admin Center](https://entra.microsoft.com/) and then going to the `Overview` Page under the `Identity Section`. There the Tenant ID is listed as Tenant ID, and the Client ID is listed as the Application ID.
+
+Copy your Tenant ID into the `configuration.yaml` file in the config directory of the backend, and your Client ID into the `secrets.yaml` file in the config directory of the backend.
+
+## 2. Register New Application
+
+Navigate to the `App registration` page under `Applications` on the same [Micorsoft Entra Admin Center](https://entra.microsoft.com/) website.
+
+Click `New registration` to register a new application. Enter a name and select the proper account type. Single tenant is the norm unless you know of otherwise.
+
+Under redirect URI select Web as the path should be `/v1/tool/auth`. For example:
+
+```bash
+    https://<your_backend_url>/v1/tool/auth
+```
+
+Click `Register` to Complete the Application Registration
+
+## 3. Configure Permissions
+
+Under the newly registered application navigate to the `API permissions` page. There you need to Click `Add a permission`, select `Microsoft Graph`, then `delegated permissions`. Next search `files.read.all` and check the box, then search `sites.read.all` and check the box. Then Click `Add permissions`.
+
+## 3. Configure Client Secret
+
+Under the newly registered application navigate to the `Certificates & secrets` page. Click `New client secret`, enter a description and an expiry then click `Add`. Your new Client Secret is only available to copy under the `value` column of the table right now. Copy it into the `secrets.yaml` file in the config directory of the backend.
+
+## 5. Run the Backend and Frontend
+
+run next command to start the backend and frontend:
+
+```bash
+make dev
+```

src/backend/config/configuration.template.yaml

@@ -42,7 +42,8 @@ tools:
       - public_repo
     default_repos:
       - cohere-ai/cohere-toolkit
-      - EugeneLightsOn/cohere-toolkit
+  sharepoint:
+    tenant_id:
   # To disable the use of the tools preamble, set it to false
   use_tools_preamble: true
 feature_flags:

src/backend/config/secrets.template.yaml

@@ -40,6 +40,9 @@ tools:
   github:
     client_id:
     client_secret:
+  sharepoint:
+    client_id:
+    client_secret:
 auth:
   secret_key:
   google_oauth:
@@ -53,4 +56,4 @@ auth:
     client_secret:
     well_known_endpoint:
 google_cloud:
-  api_key:
+  api_key:

src/backend/config/settings.py

@@ -271,6 +271,22 @@ class HybridWebSearchSettings(BaseSettings, BaseModel):
     site_filters: Optional[List[str]] = []
 
 
+class SharepointSettings(BaseSettings, BaseModel):
+    model_config = SETTINGS_CONFIG
+    tenant_id: Optional[str] = Field(
+        default=None,
+        validation_alias=AliasChoices("SHAREPOINT_TENANT_ID", "tenant_id"),
+    )
+    client_id: Optional[str] = Field(
+        default=None,
+        validation_alias=AliasChoices("SHAREPOINT_CLIENT_ID", "client_id"),
+    )
+    client_secret: Optional[str] = Field(
+        default=None,
+        validation_alias=AliasChoices("SHAREPOINT_CLIENT_SECRET", "client_secret"),
+    )
+
+
 class ToolSettings(BaseSettings, BaseModel):
     model_config = SETTINGS_CONFIG
 
@@ -302,6 +318,9 @@ class ToolSettings(BaseSettings, BaseModel):
     gmail: Optional[GmailSettings] = Field(
         default=GmailSettings()
     )
+    sharepoint: Optional[SharepointSettings] = Field(
+        default=SharepointSettings()
+    )
     use_tools_preamble: Optional[bool] = Field(
         default=False,
         validation_alias=AliasChoices("USE_TOOLS_PREAMBLE", "use_tools_preamble")

src/backend/config/tools.py

@@ -14,6 +14,7 @@
     PythonInterpreter,
     ReadFileTool,
     SearchFileTool,
+    SharepointTool,
     SlackTool,
     TavilyWebSearch,
     WebScrapeTool,
@@ -40,6 +41,7 @@ class Tool(Enum):
     Slack = SlackTool
     Gmail = GmailTool
     Github = GithubTool
+    Sharepoint = SharepointTool
 
 
 def get_available_tools() -> dict[str, ToolDefinition]:

src/backend/tests/unit/tools/test_lang_chain.py

@@ -72,7 +72,13 @@ async def test_wiki_retriever_no_docs() -> None:
     ):
         result = await retriever.call({"query": query}, ctx)
 
-    assert result == [ToolError(type=ToolErrorCode.OTHER, success=False, text='No results found.', details='No results found for the given params.').model_dump()]
+    expected_error = ToolError(
+        type=ToolErrorCode.OTHER,
+        success=False,
+        text='No results found.',
+        details='No results found for the given params.'
+    ).model_dump()
+    assert result == [expected_error]
 
 
 
@@ -156,4 +162,10 @@ async def test_vector_db_retriever_no_docs() -> None:
         mock_db.as_retriever().get_relevant_documents.return_value = mock_docs
         result = await retriever.call({"query": query}, ctx)
 
-    assert result == [ToolError(type=ToolErrorCode.OTHER, success=False, text='No results found.', details='No results found for the given params.').model_dump()]
+    expected_error = ToolError(
+        type=ToolErrorCode.OTHER,
+        success=False,
+        text='No results found.',
+        details='No results found for the given params.'
+    ).model_dump()
+    assert result == [expected_error]

src/backend/tools/__init__.py

@@ -1,12 +1,14 @@
 from backend.tools.brave_search import BraveWebSearch
 from backend.tools.calculator import Calculator
 from backend.tools.files import ReadFileTool, SearchFileTool
+from backend.tools.github import GithubAuth, GithubTool
 from backend.tools.gmail import GmailAuth, GmailTool
 from backend.tools.google_drive import GoogleDrive, GoogleDriveAuth
 from backend.tools.google_search import GoogleWebSearch
 from backend.tools.hybrid_search import HybridWebSearch
 from backend.tools.lang_chain import LangChainVectorDBRetriever, LangChainWikiRetriever
 from backend.tools.python_interpreter import PythonInterpreter
+from backend.tools.sharepoint import SharepointAuth, SharepointTool
 from backend.tools.slack import SlackAuth, SlackTool
 from backend.tools.tavily_search import TavilyWebSearch
 from backend.tools.web_scrape import WebScrapeTool
@@ -29,4 +31,8 @@
     "SlackAuth",
     "GmailTool",
     "GmailAuth",
+    "SharepointTool",
+    "SharepointAuth",
+    "GithubTool",
+    "GithubAuth",
 ]

src/backend/tools/base.py

@@ -1,7 +1,7 @@
 import datetime
 from abc import ABC, abstractmethod
 from enum import StrEnum
-from typing import Any, Dict, List
+from typing import Any
 
 import requests
 from fastapi import Request
@@ -11,6 +11,7 @@
 from backend.crud import tool_auth as tool_auth_crud
 from backend.database_models.database import DBSessionDep
 from backend.database_models.tool_auth import ToolAuth
+from backend.schemas.context import Context
 from backend.schemas.tool import ToolDefinition
 from backend.services.logger.utils import LoggerFactory
 from backend.tools.utils.tools_checkers import check_tool_parameters
@@ -157,20 +158,22 @@ def _handle_tool_specific_errors(cls, error: Exception, **kwargs: Any) -> None:
         ...
 
     @classmethod
-    def get_tool_error(cls, details: str, text: str = "Error calling tool", error_type: ToolErrorCode = ToolErrorCode.OTHER):
+    def get_tool_error(
+        cls, details: str, text: str = "Error calling tool", error_type: ToolErrorCode = ToolErrorCode.OTHER,
+    ) -> list[dict[str, str]]:
         tool_error = ToolError(text=f"{text} {cls.ID}.", details=details, type=error_type).model_dump()
         logger.error(event=f"Error calling tool {cls.ID}", error=tool_error)
         return [tool_error]
 
     @classmethod
-    def get_no_results_error(cls):
+    def get_no_results_error(cls) -> list[dict[str, str]]:
         tool_error = ToolError(text="No results found.", details="No results found for the given params.").model_dump()
         return [tool_error]
 
     @abstractmethod
     async def call(
-            self, parameters: dict, ctx: Any, **kwargs: Any
-    ) -> List[Dict[str, Any]]:
+        self, parameters: dict, ctx: Context, **kwargs: Any,
+    ) -> list[dict[str, Any]]:
         ...
 
     @classmethod
@@ -248,7 +251,7 @@ def try_refresh_token(
     @abstractmethod
     def retrieve_auth_token(
             self, request: Request, session: DBSessionDep, user_id: str
-    ) -> str:
+    ) -> str|None:
         ...
 
     def get_token(self, session: DBSessionDep, user_id: str) -> str:

src/backend/tools/brave_search/tool.py

@@ -1,7 +1,7 @@
-from typing import Any, Dict, List
+from typing import Any
 
 from backend.config.settings import Settings
-from backend.database_models.database import DBSessionDep
+from backend.schemas.context import Context
 from backend.schemas.tool import ToolCategory, ToolDefinition
 from backend.tools.base import BaseTool, ToolArgument
 from backend.tools.brave_search.client import BraveClient
@@ -40,11 +40,11 @@ def get_tool_definition(cls) -> ToolDefinition:
                 "Returns a list of relevant document snippets for a textual query retrieved "
                 "from the internet using Brave Search."
             ),
-        )
+        ) # type: ignore
 
     async def call(
-        self, parameters: dict, ctx: Any, session: DBSessionDep, **kwargs: Any
-    ) -> List[Dict[str, Any]]:
+        self, parameters: dict, ctx: Context, **kwargs: Any
+    ) -> list[dict[str, Any]]:
         query = parameters.get("query", "")
 
         # Get domain filtering from kwargs

src/backend/tools/calculator.py

@@ -1,7 +1,8 @@
-from typing import Any, Dict, List
+from typing import Any
 
 from py_expression_eval import Parser
 
+from backend.schemas.context import Context
 from backend.schemas.tool import ToolCategory, ToolDefinition
 from backend.tools.base import BaseTool
 
@@ -35,11 +36,11 @@ def get_tool_definition(cls) -> ToolDefinition:
             category=ToolCategory.Function,
             error_message=cls.generate_error_message(),
             description="A powerful multi-purpose calculator capable of a wide array of math calculations.",
-        )
+        ) # type: ignore
 
     async def call(
-        self, parameters: dict, ctx: Any, **kwargs: Any
-    ) -> List[Dict[str, Any]]:
+        self, parameters: dict, ctx: Context, **kwargs: Any
+    ) -> list[dict[str, Any]]:
         logger = ctx.get_logger()
 
         math_parser = Parser()

src/backend/tools/files.py

@@ -1,7 +1,8 @@
 from enum import StrEnum
-from typing import Any, Dict, List
+from typing import Any
 
 import backend.crud.file as file_crud
+from backend.schemas.context import Context
 from backend.schemas.tool import ToolCategory, ToolDefinition
 from backend.tools.base import BaseTool
 
@@ -43,16 +44,19 @@ def get_tool_definition(cls) -> ToolDefinition:
             error_message=cls.generate_error_message(),
             category=ToolCategory.FileLoader,
             description="Returns the chunked textual contents of an uploaded file.",
-        )
+        ) # type: ignore
 
+    @classmethod
     def get_info(cls) -> ToolDefinition:
         return ToolDefinition(
             display_name="Calculator",
             description="A powerful multi-purpose calculator capable of a wide array of math calculations.",
             error_message=cls.generate_error_message(),
-        )
+        ) # type: ignore
 
-    async def call(self, parameters: dict, **kwargs: Any) -> List[Dict[str, Any]]:
+    async def call(
+        self, parameters: dict, ctx: Context, **kwargs: Any,
+    ) -> list[dict[str, Any]]:
         file = parameters.get("file")
 
         session = kwargs.get("session")
@@ -113,16 +117,16 @@ def get_tool_definition(cls) -> ToolDefinition:
             error_message=cls.generate_error_message(),
             category=ToolCategory.FileLoader,
             description="Searches across one or more attached files based on a textual search query.",
-        )
+        ) # type: ignore
 
     async def call(
-        self, parameters: dict, ctx: Any, **kwargs: Any
-    ) -> List[Dict[str, Any]]:
+        self, parameters: dict, ctx: Context, **kwargs: Any,
+    ) -> list[dict[str, Any]]:
         query = parameters.get("search_query")
         files = parameters.get("files")
 
         session = kwargs.get("session")
-        user_id = kwargs.get("user_id")
+        user_id = kwargs.get("user_id", "")
 
         if not query or not files:
             return self.get_tool_error(

src/backend/tools/github/__init__.py

@@ -0,0 +1,11 @@
+from backend.tools.github.auth import GithubAuth
+from backend.tools.github.constants import (
+    GITHUB_TOOL_ID,
+)
+from backend.tools.github.tool import GithubTool
+
+__all__ = [
+    "GithubAuth",
+    "GithubTool",
+    "GITHUB_TOOL_ID",
+]