Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bugfix/self route toggle #93

Open
wants to merge 12 commits into
base: dev
Choose a base branch
from
Open

Bugfix/self route toggle #93

wants to merge 12 commits into from

Conversation

saisakul
Copy link
Contributor

@saisakul saisakul commented Feb 17, 2025

Context

In Redbox graph, when a user selects a document, there are 3 possible scenarios:
exceed limit (>1M) → return document is too large
exceed context (>128K) → self_route
pass (<=128K) go to summarise
In the current setting, self_route is off, which means summarise is always called, and search graph is never called.
This affects Redbox performance when users ask questions on specific parts of the document.

Before:
Screenshot 2025-02-17 at 13 00 29

Changes proposed in this pull request

  • create ENV var for setting self_route and change the code to get the setting from ENV
  • Change the graph so that self_route is called when request token less than limit

After:
Screenshot 2025-02-18 at 09 42 54

Guidance to review

Relevant links

Things to check

  • I have added any new ENV vars in all deployed environments
  • I have tested any code added or changed
  • I have run integration tests

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant