Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix(ai-proxy): Fix Cohere breaks with model parameter in body; Fix OpenAI token counting for function requests; Fix user sending own-model parameter #13000

Merged
merged 12 commits into from
Jun 20, 2024
Merged

fix(ai-proxy): Fix Cohere breaks with model parameter in body; Fix OpenAI token counting for function requests; Fix user sending own-model parameter #13000

merged 12 commits into from
Jun 20, 2024

Conversation

tysoekong
Copy link
Contributor

@tysoekong tysoekong commented May 8, 2024 •
edited
Loading

Summary

Minor fix rollup from RC2, for function calling fixes and other broken AI config checks.

Checklist

  • The Pull Request has tests
  • A changelog file has been created under changelog/unreleased/kong or skip-changelog label added on PR if changelog is unnecessary. README.md
  • There is a user-facing docs PR against https://github.com/Kong/docs.konghq.com - PUT DOCS PR HERE

Issue reference

KAG-4596

@tysoekong
Copy link
Contributor Author

Happy 13,000th pull request!!

@tysoekong tysoekong force-pushed the fix/ai_proxy_rc2_feedback branch 3 times, most recently from 3cd4e1a to 7bef31f Compare June 3, 2024 10:00
@fffonion
Copy link
Contributor

fffonion commented Jun 4, 2024

changes look fine, the commit message as well the linked ticket description should be more descriptive, let's describe what's actually being fixed instead of RC2 feedback.

@tysoekong tysoekong changed the title fix(ai-proxy): RC2 feedback fix(ai-proxy): Fix Cohere breaks with model parameter in body; Fix OpenAI token counting for function requests; Fix user sending own-model parameter Jun 5, 2024
@Kong Kong deleted a comment from ttyS0e Jun 5, 2024
@fffonion
Copy link
Contributor

fffonion commented Jun 7, 2024 •
edited
Loading

@tysoekong please resolve conflict. would be ideal if you keep 4 commits as there're four changelogs, as it may become headache if we want to revert one of them. not a hard blocker though.

@ttyS0e
Copy link
Contributor

ttyS0e commented Jun 7, 2024

I just have to also investigate the streaming issue, it would make no sense to leave unpatched.

@pull-request-size pull-request-size bot added size/L and removed size/M labels Jun 10, 2024
@tysoekong tysoekong force-pushed the fix/ai_proxy_rc2_feedback branch 4 times, most recently from 7d54c33 to 082be0c Compare June 10, 2024 21:31
fffonion
fffonion approved these changes Jun 11, 2024
View reviewed changes
Copy link
Contributor

@fffonion fffonion left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

merge on CI pass.
reviewers please review each commits individually

vm-001
vm-001 reviewed Jun 12, 2024
View reviewed changes
kong/llm/drivers/anthropic.lua Outdated Show resolved Hide resolved
kong/llm/drivers/cohere.lua Show resolved Hide resolved
tysoekong and others added 12 commits June 18, 2024 11:30
@tysoekong
fix(ai-proxy): check response tokens always string
e090846
@tysoekong
fix(ai-proxy): prevent user choosing own model, if one is specified i…
38a8e95 
…n plugin conf
@tysoekong
fix(ai-proxy): prefer model tuning parameters from plugin config, ove…
a645ca1 
…r user requests
@tysoekong
fix(ai-proxy): cohere and anthropic now accept non-empty model param
a08bf7b
@tysoekong
fix(ai-proxy): streaming when chunks contains truncated SSE messages
a1a9961
@tysoekong
fix(ai-proxy): analytics missing when model is cjson.null
b73af7e
@tysoekong
fix(ai-proxy): invalid precedence on model tuning params
fd6907b
@tysoekong
fix(ai-proxy): userdata token text returned when function calling
b2c0d89
@tysoekong
fix(ai-proxy): changelog tense
9801ca6
@tysoekong
fix(ai-proxy): token streaming truncation
6bdb7a9
@tysoekong
fix(ai-proxy): remove nil checks on model and tuning parameters
ece09eb
@tysoekong
fix(ai-proxy): test-mode for streaming tests
ace23f1
@tysoekong
Copy link
Contributor Author

READY TO GO

fffonion
fffonion approved these changes Jun 18, 2024
View reviewed changes
Copy link
Contributor

@fffonion fffonion left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

moving forward, we might want to setup a SSE server to catch more more corner cases

@fffonion fffonion merged commit 116f017 into master Jun 20, 2024
25 checks passed
@fffonion fffonion deleted the fix/ai_proxy_rc2_feedback branch June 20, 2024 01:48
@github-actions github-actions bot added the incomplete-cherry-pick A cherry-pick was incomplete and needs manual intervention label Jun 20, 2024
@team-gateway-bot team-gateway-bot mentioned this pull request Jun 20, 2024
Merged
3 tasks
@team-gateway-bot
Copy link
Collaborator

Successfully created backport PR for release/3.7.x:

@Kong Kong deleted a comment from team-gateway-bot Jun 25, 2024
@kikito kikito removed the incomplete-cherry-pick A cherry-pick was incomplete and needs manual intervention label Jun 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@catbro666 catbro666 catbro666 left review comments

@liverpool8056 liverpool8056 liverpool8056 approved these changes

@vm-001 vm-001 vm-001 approved these changes

@fffonion fffonion fffonion approved these changes

@hanshuebner hanshuebner Awaiting requested review from hanshuebner

@locao locao Awaiting requested review from locao

@jschmid1 jschmid1 Awaiting requested review from jschmid1

@gruceo gruceo Awaiting requested review from gruceo

Assignees

@tysoekong tysoekong

Labels
backport release/3.7.x cherry-pick kong-ee schedule this PR for cherry-picking to kong/kong-ee plugins/ai-proxy plugins/ai-request-transformer plugins/ai-response-transformer size/L
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
8 participants
@tysoekong @fffonion @ttyS0e @team-gateway-bot @vm-001 @catbro666 @liverpool8056 @kikito