Skip to content

Chatbot: Keep track of original LLM answer #477

@AdamFipke

Description

@AdamFipke

Description

After a user edits a chatbot question, we don't keep track of what the original LLM response was. Keeping track of the differences between these two could be valuable research data.

I would just add a userModifiedAnswer column somewhere, default to null. You could also show the differences on the frontend too, like if you wanted to show the professor how the TAs have been modifying the AI answers.

Note that this is harder than it looks due to the mess of how there are two chatbot entities, one on the HelpMe repo and one on the Chatbot repo (which is also a vectorStore rather than relational data so adding extra columns is a little messy).

Also, Anytime Questions keep track of the original AI answer (aiAnswerText). Depending on when this issue is solved, there may be a hard link between asyncQuestions and chatbot questions, allowing this aiAnswerText and userModifiedAnswer fields to be merged conceptually (note that asyncQuestion.answerText is originally a copy of aiAnswerText, meaning you would only want to copy over asyncQuestion.answerText if it's not the same as aiAnswerText).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions