Initial DynamicDeps support. #201
Conversation
|
|
||
|
|
||
| def parse_book(self, response, book_page: BookPage, dynamic: DynamicDeps): | ||
| other_dep = dynamic[OtherDep] |
There was a problem hiding this comment.
Is there a way to get a list of the created dependencies in the DynamicDeps (i.e. the available keys)? It seems we should document it. Currently only getting by type is documented, if I'm not mistaken.
There was a problem hiding this comment.
I assumed the user already knows which deps are there because it's they who set the meta key, but in other uses cases (which?) they can look at dynamic.keys(), do you want to document this or are you thinking about something else?
There was a problem hiding this comment.
There could be a single callback which gets requests from different places, with different "inject" meta; this "inject" meta can be dynamic as well. For example, it can be a list of item classes to extract from a particular page, configured at spider start. Isn't it a point of DynamicDeps that the callback doesn't know what are the dependencies? Otherwise they can be specified in a signature.
As for the documentation, I think documenting DynamicDeps as a dict subclass could be enough.
|
Looks great overall 👍 |
No description provided.