Avoid hitting the database for every chunk

[duarme]

Hi @alexrudall,
Your tutorial is helping me develop a similar feature.
One problem I found is that the current code hits the database for every chunk it receives from the OpenAI stream.
Here is my solution:

class GetAiResponseJob < ActiveJob::Base
  # ...

  private

  def call_openai(chat:)
    OpenAI::Client.new.chat(
      parameters: {
        model: 'gpt-3.5-turbo',
        messages: Message.for_openai(chat.messages),
        temperature: 0.7,
        stream: stream_proc(chat:),
        n: 1 # Are you sure you need that `RESPONSES_PER_MESSAGE` complication in a tutorial?
      }
    )
    @message.save! # This way, we hit the DB only twice: when @message is created, and when it's updated here.
  end

  def create_message(chat:)
    message = chat.messages.create(role: 'assistant', content: '', response_number: 0)
    message.broadcast_created
    message
  end

  def stream_proc(chat:)
    @message = create_message(chat:)
    buffer = ''
    proc do |chunk, _bytesize|
      new_content = chunk.dig('choices', 0, 'delta', 'content')
      if new_content
        buffer += new_content
        @message.content = buffer # This way we don't hit the database on every chunk
        @message.broadcast_updated # but we can still call `broadcast_updated`
      end
    end
  end
end

If you like this idea, I can prepare a PR 🙂