feat: implementation #1

MQ37 · 2025-01-31T13:53:03Z

This is a new integration package for LangChain that integrates Apify. This is the new recommended way instead of using the LangChain community.

This project was bootstrapped using the langchain-cli template for integration packages.

Implemented LangChain tool to call Apify Actors
Ported ApifyDatasetLoader from langchain community
Ported ApifyWrapper from langchain community
Wrote basic unit and integration tests - see DEVELOPMENT.md file

…c unit and integration tests

… override

.gitignore

docs/examples/tools_example.py

jirimoravcik

Hey Kuba @MQ37,
first, thanks for the pull request, it's a lot of work and it looks great.
I have some comments to improve the code:

At Apify, we write the word "Actor" capitalized in any places where it's text (and doesn't need to follow things like snake_case)
I've noticed you like to use msg = 'error message' and then raise ValueError(msg). I personally feel like it's not necessary to separate the statements like that.
I see that you use double quotes in your code. I believe single quotes would be better, e.g. see configuration from our Python SDK https://github.com/apify/apify-sdk-python/blob/master/pyproject.toml#L115
I noticed some inconsistencies between how you formatted function arguments, some have the :param notation, some not. It would be nice to unify.
It would be nice to have a README file that gives users more insight into what the package and Apify can do for them

Thank you

docs/examples/tools_example.py

README.md

.github/workflows/run_code_checks.yml

langchain_apify/wrappers.py

langchain_apify/document_loaders.py

langchain_apify/utils.py

langchain_apify/tools.py

Co-authored-by: Jiří Moravčík <jiri.moravcik@gmail.com>

MQ37 · 2025-02-04T13:06:56Z

Hey Kuba @MQ37, first, thanks for the pull request, it's a lot of work and it looks great. I have some comments to improve the code:

* At Apify, we write the word "Actor" capitalized in any places where it's text (and doesn't need to follow things like `snake_case`)

* I've noticed you like to use `msg = 'error message'` and then `raise ValueError(msg)`. I personally feel like it's not necessary to separate the statements like that.

* I see that you use double quotes in your code. I believe single quotes would be better, e.g. see configuration from our Python SDK https://github.com/apify/apify-sdk-python/blob/master/pyproject.toml#L115

* I noticed some inconsistencies between how you formatted function arguments, some have the `:param` notation, some not. It would be nice to unify.

* It would be nice to have a README file that gives users more insight into what the package and Apify can do for them

Thank you

Hey Jirka @jirimoravcik, thank you for your valuable review 👍

capitalized the word Actor, it should not appear in lowercase form anywhere in the text, thank you for noticing this
kept msg and raise separate instead of using literal, thank you for the personal suggestion but I think ruff docs explanation makes sense - https://docs.astral.sh/ruff/rules/raw-string-in-exception/
changed the ruff config to use single quotes and added additional rules
fixed the inconsistencies and set google pydoc style convention, so all docstrings should follow this convention
added README header and improved based on your suggestions, thank you (also changed the repo description)

I'd even go further and reach out to marketing to help write a punchy paragraph here, I think it's worth it

Do you know who I can reach to help me with improving the README header?

Thank you for your time and effort 👍

jirimoravcik · 2025-02-04T13:31:33Z

Do you know who I can reach to help me with improving the README header?

Try #apify-content-tasks Slack channel

jirimoravcik · 2025-02-04T14:52:23Z

langchain_apify/utils.py

+        msg = f'Failed to get the Actor object ID for {actor_id}.'
+        raise ValueError(msg)
+
+    url = f'https://api.apify.com/v2/acts/{actor_obj_id}/builds/default'


@danpoletaev will you be adding this to the Python client please?

No, it will not be added to the python/js client unless really needed. Because it will be breaking change.

Currently for BuildClients we use /actor-builds namespace instead of /acts namespace

So it won't be accessible at all via cilents? That doesn't feel right. Is there some discussion I could read? Thank you

Here is the link to zenhub task: https://app.zenhub.com/workspaces/platform-team-5f6454160d9f82000fa6733f/issues/gh/apify/apify-core/18314

"adding the /default subroute to the API client would require some major restructuring, since currently ActorClient.build() builds the Actor, it does not retrieve a specific build"

jirimoravcik

Thank you

langchain_apify/tools.py

Co-authored-by: Jiří Moravčík <jiri.moravcik@gmail.com>

jirispilka

I like it, thanks!

A couple of notes from me:

Github repo about: please provide text there, this is searchable field (not this Apify + 🦜🔗 = 🤖🔧🌐)
use are missing line-length in pyproject.toml. I would suggest to use 120

[tool.ruff]
line-length = 120

Again, some of the issues I saw should be reported by IDE. Like that you are using a variable in doc that is not in a function definition

Otherwise, good job 👍🏻

README.md

langchain_apify/utils.py

jirispilka · 2025-02-04T21:34:41Z

langchain_apify/utils.py

+T = TypeVar('T', ApifyClient, ApifyClientAsync)
+
+
+def create_apify_client(t: type[T], token: str) -> T:


Please do not use just variable t. Never.
Also. why to create typevar, what is benefit over

def create_apify_client(apify_client: ApifyClient | ApifyClientAsync, token: str) -> ApifyClient | ApifyClientAsync:

I know, it is a bit longer, but readable

Please do not use just variable t. Never.

makes sense, this is unreadable for someone who did not write the code. Renamed to client_cls

Also. why to create typevar, what is benefit over

TypeVar is needed when you need to work with a single type in the scope. If simple union was used then you would have to always check the client type using isinstanceof and after that you could work with the client, otherwise type checker would be angry - this way you can just pass the client type class that you want to create (sync or async) and since it is typevar the type checker knows that if you passed ApifyClient it has to return ApifyClient and cannot return ApifyClientAsync and vice versa

langchain_apify/utils.py

langchain_apify/wrappers.py

Co-authored-by: Jiří Spilka <jiri.spilka@apify.com>

…refactor, added line length, ruff rules

MQ37 · 2025-02-05T10:34:11Z

I like it, thanks!

A couple of notes from me:

* Github repo about: please provide text there, this is searchable field (not this Apify + 🦜🔗 = 🤖🔧🌐)

* use are missing line-length in pyproject.toml. I would suggest to use 120

[tool.ruff]
line-length = 120

* Again, some of the issues I saw should be reported by IDE. Like that you are using a variable in doc that is not in a function definition

Otherwise, good job 👍🏻

Thank you for your review and effort 👍

refactored the code
changed the repo description to "Apify integration for LangChain 🦜🔗 "
set the ruff line length to 120
found out when you run ruff in preview mode it reports these issues (for example the docstring issues and method could be static) - will check the code also in preview mode

jirispilka

Thanks! Looks good

README.md

MQ37 added 3 commits January 31, 2025 14:29

initial version with actors tool, dataset loader, apify wrapper, basi…

03a61d5

…c unit and integration tests

update readme

6ff48be

add basic gh workflows

8c286eb

MQ37 self-assigned this Jan 31, 2025

MQ37 added 11 commits January 31, 2025 14:53

fix workflow

662cac1

gh workflow all groups fix

71c604a

fix workflow

f6bd4cf

add all current python version to code check, supress mypy error with…

8a864ff

… override

fill in license

b601072

use optional for python 3.9 compatibility

bcdcb88

fix union, fix workflow trigger

9b41188

add pre/release workflows

7256bc6

remove import error try blocks

b45da20

update readme, gitignore dist

34da126

add dev.md

480f49c

MQ37 requested a review from jirimoravcik February 2, 2025 19:41

MQ37 added 2 commits February 3, 2025 10:31

update readme add openai key env var

d930e21

fix dev.md

64c073e

jirimoravcik reviewed Feb 3, 2025

View reviewed changes

.gitignore Outdated Show resolved Hide resolved

jirimoravcik reviewed Feb 3, 2025

View reviewed changes

docs/examples/tools_example.py Outdated Show resolved Hide resolved

MQ37 requested a review from jirispilka February 3, 2025 10:21

extend .gitignore

3960dd7

MQ37 requested a review from jirimoravcik February 3, 2025 13:47

jirimoravcik suggested changes Feb 3, 2025

View reviewed changes

MQ37 and others added 6 commits February 4, 2025 09:32

Update README.md

fe55350

Co-authored-by: Jiří Moravčík <jiri.moravcik@gmail.com>

Update README.md

4c4dab9

Co-authored-by: Jiří Moravčík <jiri.moravcik@gmail.com>

update and fix readme

4c71532

improve readme, clean up gh workflows

a3a14c7

monero -> apify, add test comments

138e942

add return types description, refactor create_apify_client

122efc7

MQ37 added 3 commits February 4, 2025 13:29

add eval type backport to fix type annotation for py 3.9

cd43beb

add readme header

fc04a1a

Poetry Project CI -> Code checks

30b9e34

MQ37 requested a review from jirimoravcik February 4, 2025 13:13

jirimoravcik reviewed Feb 4, 2025

View reviewed changes

jirimoravcik approved these changes Feb 4, 2025

View reviewed changes

langchain_apify/tools.py Outdated Show resolved Hide resolved

langchain_apify/tools.py Outdated Show resolved Hide resolved

MQ37 and others added 2 commits February 4, 2025 19:17

Update langchain_apify/tools.py

57692c2

Co-authored-by: Jiří Moravčík <jiri.moravcik@gmail.com>

fix typos and grammar

10977bd

jirispilka suggested changes Feb 4, 2025

View reviewed changes

MQ37 and others added 12 commits February 5, 2025 09:25

Update README.md

2a1064f

Co-authored-by: Jiří Spilka <jiri.spilka@apify.com>

Update README.md

6b54c00

Co-authored-by: Jiří Spilka <jiri.spilka@apify.com>

Update README.md

b126a7c

Co-authored-by: Jiří Spilka <jiri.spilka@apify.com>

Update README.md

194ff6b

Co-authored-by: Jiří Spilka <jiri.spilka@apify.com>

Update Makefile

6422b20

Co-authored-by: Jiří Spilka <jiri.spilka@apify.com>

Update docs/examples/tools_example.py

e099f46

Co-authored-by: Jiří Spilka <jiri.spilka@apify.com>

Update langchain_apify/utils.py

30aebaa

Co-authored-by: Jiří Spilka <jiri.spilka@apify.com>

Update langchain_apify/tools.py

c103aa4

Co-authored-by: Jiří Spilka <jiri.spilka@apify.com>

Update langchain_apify/wrappers.py

872cb1d

Co-authored-by: Jiří Spilka <jiri.spilka@apify.com>

Update langchain_apify/tools.py

35ff80a

Co-authored-by: Jiří Spilka <jiri.spilka@apify.com>

apply PR review suggestions, improve readme and development MD, code …

ba619dd

…refactor, added line length, ruff rules

fix docstrings, sort __all__

18195f1

MQ37 requested a review from jirispilka February 5, 2025 10:39

jirispilka approved these changes Feb 5, 2025

View reviewed changes

MQ37 merged commit 0163fce into main Feb 5, 2025
6 checks passed

MQ37 deleted the feat/implementation branch February 5, 2025 13:45

BraniganLee reviewed Feb 7, 2025

View reviewed changes

README.md Show resolved Hide resolved

BraniganLee reviewed Feb 7, 2025

View reviewed changes

README.md Show resolved Hide resolved

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: implementation #1

feat: implementation #1

MQ37 commented Jan 31, 2025 •

edited

Loading

jirimoravcik left a comment

MQ37 commented Feb 4, 2025

jirimoravcik commented Feb 4, 2025

jirimoravcik Feb 4, 2025

danpoletaev Feb 5, 2025

jirimoravcik Feb 5, 2025

danpoletaev Feb 5, 2025

jirimoravcik left a comment

jirispilka left a comment

jirispilka Feb 4, 2025

MQ37 Feb 5, 2025 •

edited

Loading

MQ37 commented Feb 5, 2025

jirispilka left a comment

		T = TypeVar('T', ApifyClient, ApifyClientAsync)


		def create_apify_client(t: type[T], token: str) -> T:

feat: implementation #1

feat: implementation #1

Conversation

MQ37 commented Jan 31, 2025 • edited Loading

jirimoravcik left a comment

Choose a reason for hiding this comment

MQ37 commented Feb 4, 2025

jirimoravcik commented Feb 4, 2025

jirimoravcik Feb 4, 2025

Choose a reason for hiding this comment

danpoletaev Feb 5, 2025

Choose a reason for hiding this comment

jirimoravcik Feb 5, 2025

Choose a reason for hiding this comment

danpoletaev Feb 5, 2025

Choose a reason for hiding this comment

jirimoravcik left a comment

Choose a reason for hiding this comment

jirispilka left a comment

Choose a reason for hiding this comment

jirispilka Feb 4, 2025

Choose a reason for hiding this comment

MQ37 Feb 5, 2025 • edited Loading

Choose a reason for hiding this comment

MQ37 commented Feb 5, 2025

jirispilka left a comment

Choose a reason for hiding this comment

MQ37 commented Jan 31, 2025 •

edited

Loading

MQ37 Feb 5, 2025 •

edited

Loading