Add lookup_tweet_by_id to X Toolkit (#165)

This PR introduces the `lookup_tweet_by_id` tool to the X toolkit,
enabling users to retrieve tweet details by tweet ID. This enhancement
extends the toolkit's capabilities, allowing for more comprehensive
interactions with the X (Twitter) API.

**Key Changes:**

- **Added `lookup_tweet_by_id` Tool:**
- Implemented the `lookup_tweet_by_id` function in `tools/tweets.py`,
which allows users to fetch tweet information using a tweet ID.
- Included error handling for API response codes and expanded URLs in
tweets to assist language models in avoiding hallucinations due to
shortened URLs.

- **Enhanced Toolkit Structure:**
- Added several configuration files to the X toolkit to establish a
standardized project structure, which in the future will be generated by
`arcade new`. These include:
- `.pre-commit-config.yaml`: Defines pre-commit hooks for code quality
checks.
    - `.ruff.toml`: Configuration for the Ruff linter.
    - `LICENSE`: MIT License file for the toolkit.
- `Makefile`: Contains common commands for building, testing, and
linting the toolkit.

- **Updated Makefile:**
- Added `make check-toolkits` command to the top-level `Makefile`. This
command runs code quality tools for each toolkit that contains a
`Makefile`.

**Additional Notes:**

- **Tests:**
- Added unit tests for the new `lookup_tweet_by_id` tool in
`tests/test_tweets.py`.
- Included tests for the user lookup functionality in
`tests/test_users.py`.

- **Linting and Code Quality:**
- Configured pre-commit hooks and Ruff linter to enforce code standards.
- Updated the `pyproject.toml` file with development dependencies for
testing and linting.

-

---------

Co-authored-by: Eric Gustin <eric@arcade-ai.com>

Makefile

@@ -25,6 +25,19 @@ check: ## Run code quality tools.
 	@echo "🚀 Static type checking: Running mypy"
 	@cd arcade && poetry run mypy $(git ls-files '*.py')
 
+
+.PHONY: check-toolkits
+check-toolkits: ## Run code quality tools for each toolkit that has a Makefile
+	@echo "🚀 Running 'make check' in each toolkit with a Makefile"
+	@for dir in toolkits/*/ ; do \
+		if [ -f "$$dir/Makefile" ]; then \
+			echo "🛠️ Checking toolkit $$dir"; \
+			(cd "$$dir" && make check); \
+		else \
+			echo "🛠️ Skipping toolkit $$dir (no Makefile found)"; \
+		fi; \
+	done
+
 .PHONY: test
 test: ## Test the code with pytest
 	@echo "🚀 Testing code: Running pytest"
@@ -144,4 +157,6 @@ help:
 	@echo "🛠️ Arcade AI Dev Commands:\n"
 	@grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}'
 
+
+
 .DEFAULT_GOAL := help

toolkits/x/.pre-commit-config.yaml

@@ -0,0 +1,18 @@
+files: ^./
+repos:
+  - repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: "v4.4.0"
+    hooks:
+      - id: check-case-conflict
+      - id: check-merge-conflict
+      - id: check-toml
+      - id: check-yaml
+      - id: end-of-file-fixer
+      - id: trailing-whitespace
+
+  - repo: https://github.com/astral-sh/ruff-pre-commit
+    rev: v0.6.7
+    hooks:
+      - id: ruff
+        args: [--fix]
+      - id: ruff-format

toolkits/x/.ruff.toml

@@ -0,0 +1,44 @@
+target-version = "py39"
+line-length = 100
+fix = true
+
+[lint]
+select = [
+    # flake8-2020
+    "YTT",
+    # flake8-bandit
+    "S",
+    # flake8-bugbear
+    "B",
+    # flake8-builtins
+    "A",
+    # flake8-comprehensions
+    "C4",
+    # flake8-debugger
+    "T10",
+    # flake8-simplify
+    "SIM",
+    # isort
+    "I",
+    # mccabe
+    "C90",
+    # pycodestyle
+    "E", "W",
+    # pyflakes
+    "F",
+    # pygrep-hooks
+    "PGH",
+    # pyupgrade
+    "UP",
+    # ruff
+    "RUF",
+    # tryceratops
+    "TRY",
+]
+
+[lint.per-file-ignores]
+"**/tests/*" = ["S101"]
+
+[format]
+preview = true
+skip-magic-trailing-comma = false

toolkits/x/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024, arcadeai
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

toolkits/x/Makefile

@@ -0,0 +1,53 @@
+.PHONY: help
+
+help:
+	@echo "🛠️ x Commands:\n"
+	@grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}'
+
+.PHONY: install
+install: ## Install the poetry environment and install the pre-commit hooks
+	@echo "📦 Checking if Poetry is installed"
+	@if ! command -v poetry &> /dev/null; then \
+		echo "📦 Installing Poetry with pip"; \
+		pip install poetry; \
+	else \
+		echo "📦 Poetry is already installed"; \
+	fi
+	@echo "🚀 Installing package in development mode with all extras"
+	poetry install --all-extras
+
+.PHONY: build
+build: clean-build ## Build wheel file using poetry
+	@echo "🚀 Creating wheel file"
+	poetry build
+
+.PHONY: clean-build
+clean-build: ## clean build artifacts
+	@echo "🗑️ Cleaning dist directory"
+	rm -rf dist
+
+.PHONY: test
+test: ## Test the code with pytest
+	@echo "🚀 Testing code: Running pytest"
+	@poetry run pytest -W ignore -v --cov --cov-config=pyproject.toml --cov-report=xml
+
+.PHONY: coverage
+coverage: ## Generate coverage report
+	@echo "coverage report"
+	coverage report
+	@echo "Generating coverage report"
+	coverage html
+
+.PHONY: bump-version
+bump-version: ## Bump the version in the pyproject.toml file
+	@echo "🚀 Bumping version in pyproject.toml"
+	poetry version patch
+
+.PHONY: check
+check: ## Run code quality tools.
+	@echo "🚀 Checking Poetry lock file consistency with 'pyproject.toml': Running poetry check --lock"
+	@poetry check --lock
+	@echo "🚀 Linting code: Running pre-commit"
+	@poetry run pre-commit run -a
+	@echo "🚀 Static type checking: Running mypy"
+	@poetry run mypy --config-file=pyproject.toml

toolkits/x/arcade_x/tools/tweets.py

@@ -1,20 +1,24 @@
-from typing import Annotated
+from typing import Annotated, Any
 
 import httpx
-
 from arcade.sdk import ToolContext, tool
 from arcade.sdk.auth import X
-from arcade.sdk.errors import ToolExecutionError
+from arcade.sdk.errors import RetryableToolError
+
 from arcade_x.tools.utils import (
     expand_urls_in_tweets,
+    get_headers_with_token,
     get_tweet_url,
     parse_search_recent_tweets_response,
 )
 
 TWEETS_URL = "https://api.x.com/2/tweets"
 
 
-# Manage Tweets Tools. See developer docs for additional available parameters: https://developer.x.com/en/docs/x-api/tweets/manage-tweets/api-reference
+# Manage Tweets Tools. See developer docs for additional available parameters:
+# https://developer.x.com/en/docs/x-api/tweets/manage-tweets/api-reference
+
+
 @tool(
     requires_auth=X(
         scopes=["tweet.read", "tweet.write", "users.read"],
@@ -26,19 +30,12 @@ async def post_tweet(
 ) -> Annotated[str, "Success string and the URL of the tweet"]:
     """Post a tweet to X (Twitter)."""
 
-    headers = {
-        "Authorization": f"Bearer {context.authorization.token}",
-        "Content-Type": "application/json",
-    }
+    headers = get_headers_with_token(context)
     payload = {"text": tweet_text}
 
     async with httpx.AsyncClient() as client:
         response = await client.post(TWEETS_URL, headers=headers, json=payload, timeout=10)
-
-    if response.status_code != 201:
-        raise ToolExecutionError(
-            f"Failed to post a tweet during execution of '{post_tweet.__name__}' tool. Request returned an error: {response.status_code} {response.text}"
-        )
+        response.raise_for_status()
 
     tweet_id = response.json()["data"]["id"]
     return f"Tweet with id {tweet_id} posted successfully. URL: {get_tweet_url(tweet_id)}"
@@ -51,16 +48,12 @@ async def delete_tweet_by_id(
 ) -> Annotated[str, "Success string confirming the tweet deletion"]:
     """Delete a tweet on X (Twitter)."""
 
-    headers = {"Authorization": f"Bearer {context.authorization.token}"}
+    headers = get_headers_with_token(context)
     url = f"{TWEETS_URL}/{tweet_id}"
 
     async with httpx.AsyncClient() as client:
         response = await client.delete(url, headers=headers, timeout=10)
-
-    if response.status_code != 200:
-        raise ToolExecutionError(
-            f"Failed to delete the tweet during execution of '{delete_tweet_by_id.__name__}' tool. Request returned an error: {response.status_code} {response.text}"
-        )
+        response.raise_for_status()
 
     return f"Tweet with id {tweet_id} deleted successfully."
 
@@ -72,33 +65,33 @@ async def search_recent_tweets_by_username(
     max_results: Annotated[
         int, "The maximum number of results to return. Cannot be less than 10"
     ] = 10,
-) -> Annotated[dict, "Dictionary containing the search results"]:
-    """Search for recent tweets (last 7 days) on X (Twitter) by username. Includes replies and reposts."""
+) -> Annotated[dict[str, Any], "Dictionary containing the search results"]:
+    """Search for recent tweets (last 7 days) on X (Twitter) by username.
+    Includes replies and reposts."""
 
-    headers = {
-        "Authorization": f"Bearer {context.authorization.token}",
-        "Content-Type": "application/json",
-    }
-    params = {
+    headers = get_headers_with_token(context)
+    params: dict[str, int | str] = {
         "query": f"from:{username}",
         "max_results": max(max_results, 10),  # X API does not allow 'max_results' less than 10
     }
-    url = "https://api.x.com/2/tweets/search/recent?expansions=author_id&user.fields=id,name,username,entities&tweet.fields=entities"
+    url = (
+        "https://api.x.com/2/tweets/search/recent?"
+        "expansions=author_id&user.fields=id,name,username,entities&tweet.fields=entities"
+    )
 
     async with httpx.AsyncClient() as client:
         response = await client.get(url, headers=headers, params=params, timeout=10)
+        response.raise_for_status()
 
-    if response.status_code != 200:
-        raise ToolExecutionError(
-            f"Failed to search recent tweets during execution of '{search_recent_tweets_by_username.__name__}' tool. Request returned an error: {response.status_code} {response.text}"
-        )
-
-    response_data = response.json()
+    response_data: dict[str, Any] = response.json()
 
-    # Expand the urls that are in the tweets
-    expand_urls_in_tweets(response_data.get("data", []), delete_entities=True)
+    # Expand the URLs that are in the tweets
+    response_data["data"] = expand_urls_in_tweets(
+        response_data.get("data", []), delete_entities=True
+    )
 
-    parse_search_recent_tweets_response(response_data)
+    # Parse the response data
+    response_data = parse_search_recent_tweets_response(response_data)
 
     return response_data
 
@@ -115,44 +108,81 @@ async def search_recent_tweets_by_keywords(
     max_results: Annotated[
         int, "The maximum number of results to return. Cannot be less than 10"
     ] = 10,
-) -> Annotated[dict, "Dictionary containing the search results"]:
+) -> Annotated[dict[str, Any], "Dictionary containing the search results"]:
     """
-    Search for recent tweets (last 7 days) on X (Twitter) by required keywords and phrases. Includes replies and reposts
-    One of the following input parametersMUST be provided: keywords, phrases
+    Search for recent tweets (last 7 days) on X (Twitter) by required keywords and phrases.
+    Includes replies and reposts.
+    One of the following input parameters MUST be provided: keywords, phrases
     """
 
     if not any([keywords, phrases]):
-        raise ValueError(
-            "At least one of keywords or phrases must be provided to the '{search_recent_tweets_by_keywords.__name__}' tool."
+        raise RetryableToolError(  # noqa: TRY003
+            "No keywords or phrases provided",
+            developer_message="Predicted inputs didn't contain any keywords or phrases",
+            additional_prompt_content="Please provide at least one keyword or phrase for search",
+            retry_after_ms=500,  # Play nice with X API rate limits
         )
 
-    headers = {
-        "Authorization": f"Bearer {context.authorization.token}",
-        "Content-Type": "application/json",
-    }
+    headers = get_headers_with_token(context)
+
     query = "".join([f'"{phrase}" ' for phrase in (phrases or [])])
     if keywords:
         query += " ".join(keywords or [])
 
-    params = {
-        "query": query,
+    params: dict[str, int | str] = {
+        "query": query.strip(),
         "max_results": max(max_results, 10),  # X API does not allow 'max_results' less than 10
     }
-    url = "https://api.x.com/2/tweets/search/recent?expansions=author_id&user.fields=id,name,username,entities&tweet.fields=entities"
+    url = (
+        "https://api.x.com/2/tweets/search/recent?"
+        "expansions=author_id&user.fields=id,name,username,entities&tweet.fields=entities"
+    )
 
     async with httpx.AsyncClient() as client:
         response = await client.get(url, headers=headers, params=params, timeout=10)
+        response.raise_for_status()
 
-    if response.status_code != 200:
-        raise ToolExecutionError(
-            f"Failed to search recent tweets during execution of '{search_recent_tweets_by_keywords.__name__}' tool. Request returned an error: {response.status_code} {response.text}"
-        )
+    response_data: dict[str, Any] = response.json()
+
+    # Expand the URLs that are in the tweets
+    response_data["data"] = expand_urls_in_tweets(
+        response_data.get("data", []), delete_entities=True
+    )
+
+    # Parse the response data
+    response_data = parse_search_recent_tweets_response(response_data)
+
+    return response_data
 
-    response_data = response.json()
 
-    # Expand the urls that are in the tweets
-    expand_urls_in_tweets(response_data.get("data", []), delete_entities=True)
+@tool(requires_auth=X(scopes=["tweet.read", "users.read"]))
+async def lookup_tweet_by_id(
+    context: ToolContext,
+    tweet_id: Annotated[str, "The ID of the tweet you want to look up"],
+) -> Annotated[dict[str, Any], "Dictionary containing the tweet data"]:
+    """Look up a tweet on X (Twitter) by tweet ID."""
+
+    headers = get_headers_with_token(context)
+    params = {
+        "expansions": "author_id",
+        "user.fields": "id,name,username,entities",
+        "tweet.fields": "entities",
+    }
+    url = f"{TWEETS_URL}/{tweet_id}"
 
-    parse_search_recent_tweets_response(response_data)
+    async with httpx.AsyncClient() as client:
+        response = await client.get(url, headers=headers, params=params, timeout=10)
+        response.raise_for_status()
+
+    response_data: dict[str, Any] = response.json()
+
+    # Get the tweet data
+    tweet_data = response_data.get("data")
+    if tweet_data:
+        # Expand the URLs that are in the tweet
+        expanded_tweet_list = expand_urls_in_tweets([tweet_data], delete_entities=True)
+        response_data["data"] = expanded_tweet_list[0]
+    else:
+        response_data["data"] = {}
 
     return response_data