Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inquiry about Pub.dev API Rate Limit #8008

Open
barabobBOB opened this issue Sep 3, 2024 · 8 comments
Open

Inquiry about Pub.dev API Rate Limit #8008

barabobBOB opened this issue Sep 3, 2024 · 8 comments

Comments

@barabobBOB
Copy link

I am using the Pub.dev API to retrieve package information.
However, I couldn't find any documentation regarding the API rate limits.
I would like to know if there is a rate limit for the Pub.dev API, and if so, what are the specifics (e.g., requests per minute, per hour, etc.).
Understanding this would help us avoid unnecessary throttling and ensure optimal usage of the API.

Using API Path: https://pub.dev/api/packages/{package_name}

@isoos
Copy link
Collaborator

isoos commented Sep 3, 2024

@barabobBOB: Could you please describe what's your use and access pattern of the API(s)?

We currently rate limit actions that change state (e.g. uploading a new package version) or harder to cache (e.g. search).

@barabobBOB
Copy link
Author

@isoos I use the API endpoint https://pub.dev/api/package-names to retrieve package names. After obtaining the package names, I send numerous requests to collect metadata for all packages by combining the package names with https://pub.dev/api/packages/{package_name}. I'm curious if there is a rate limiting policy for this usage.

@barabobBOB
Copy link
Author

@isoos Additionally, I understand that the response from https://pub.dev/api/packages/{package_name} does not include the license information. I am curious if there is a way to retrieve the license information for a pub.dev library via an API, rather than through GitHub or the official website.

@isoos
Copy link
Collaborator

isoos commented Sep 3, 2024

I'm guessing here: are you polling and storing the package data periodically? We are looking for ways to make such use cases better, so I'm curious about the frequency you are looking for, or whether we could provide further API features that could potentially reduce the load (on both sides).

We have no current plans to expose the license information on this API endpoint (many reasons for that, e.g. it depends on the actual analysis of the package content, which may change, and may not run on all versions).

@barabobBOB
Copy link
Author

I plan to send requests to the package-specific API endpoint (e.g., https://pub.dev/api/packages/http) only once per day. Additionally, I plan to send requests to the https://pub.dev/api/packages/{package_name} API at a rate of 20 requests per second.

@isoos
Copy link
Collaborator

isoos commented Sep 3, 2024

Would it help if you would get a list of package names that have changed in the past 1/ 3 / 7 days? The content for the rest on /api/packages/<package> would not change in such cases.

@barabobBOB
Copy link
Author

That sounds like a great idea! It would be a very effective way to optimize the process and reduce unnecessary requests. Having access to a list of package names that have changed in the past 1, 3, or 7 days would be extremely helpful.

@ChillarAnand
Copy link

@isoos Do you plan to provide any data dumps that can used for initial bootstrapping?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants