Skip to content

Commit

Permalink
Refactor OpenAI Plugin (#57)
Browse files Browse the repository at this point in the history
* fix: type annotate RequestResult

* feat: refactor streaming requests for OpenAI AI plugin

* fix: return if the server response is blank

* fix: flatten usage conditional

* feat: switch to config var to determine api type

* fix: finish making non-streaming api robust against bad responses

* fix: make RequestResult.calculate_result type safe

* fix: off by one error in output_tokens_before_timeout count

* fix: check all tokens for output_tokens_before_timeout

* fix: organize misc pieces

* feat: clean up the Plugin ABC

* feat: dont record blank tokens that come before first non-blank

* fix: move deepget back to plugin

This avoids a circular dependency

* Revert "feat: clean up the Plugin ABC"

This reverts commit 3e5ff9b.

We can't do this without changing how the plugins handle request_func
since @abstractmethods must be implemented not assigned.

* fix: don't rely on usage data for token count

* feat: set sampling defaults for all request types

* fix: implement feedback

* fix: standardize max_output accross all request types

* feat: count tokens in each response

* fix: log missing special responses

* fix: track first token by usage

* fix: omit non-tokens as they break latency tracking

* fix: improve documentation and logging
  • Loading branch information
sjmonson authored Oct 17, 2024
1 parent 044694b commit 5d25a82
Show file tree
Hide file tree
Showing 2 changed files with 251 additions and 161 deletions.
Loading

0 comments on commit 5d25a82

Please sign in to comment.