exempt service call from using json #4
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The
fastly service list --jsoncommand returns a massive amount of unnecessary data, especially when your account has nearly 100 services.More specifically, every service includes an array with details on every version its ever had. We have nearly 100 services, some of which have 700+ versions. Our
fastly service list --jsoncurrently returns over 1.15 MB of text, or roughly 448,310 tokens, and that number can only grow.This immediately overloads the LLM context for most models and causes the call to fail.
I've found this fix to work well, as it allows the LLM to truly obtain a simple list of services, then query what's necessary after that.
Perhaps you could instead intercept the json to remove the excess data before passing it to the LLM, but this seemed like the most straightforward approach.