Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* fix: type annotate RequestResult * feat: refactor streaming requests for OpenAI AI plugin * fix: return if the server response is blank * fix: flatten usage conditional * feat: switch to config var to determine api type * fix: finish making non-streaming api robust against bad responses * fix: make RequestResult.calculate_result type safe * fix: off by one error in output_tokens_before_timeout count * fix: check all tokens for output_tokens_before_timeout * fix: organize misc pieces * feat: clean up the Plugin ABC * feat: dont record blank tokens that come before first non-blank * fix: move deepget back to plugin This avoids a circular dependency * Revert "feat: clean up the Plugin ABC" This reverts commit 3e5ff9b. We can't do this without changing how the plugins handle request_func since @abstractmethods must be implemented not assigned. * fix: don't rely on usage data for token count * feat: set sampling defaults for all request types * fix: implement feedback * fix: standardize max_output accross all request types * feat: count tokens in each response * fix: log missing special responses * fix: track first token by usage * fix: omit non-tokens as they break latency tracking * fix: improve documentation and logging
- Loading branch information