Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: bump WIP llama2 image to 13B model (from 7B) #267

Merged
merged 1 commit into from
Aug 22, 2023
Merged

Conversation

mikix
Copy link
Contributor

@mikix mikix commented Aug 22, 2023

This also fixes a few bugs:

  • bad default internal docker url
  • not long enough of a timeout waiting for a llama2 answer
  • but most importantly, increase max_new_tokens value so that answers are a reasonable length

The ultimate end goal of this llama2 support is likely the 70B model. But this is a further march in that direction, while still being able to run on reasonably available hardware.

For an example synthea clinical note, it generates reasonable sounding (and accurate to the note) summaries like:

Sure! Here is a summary of the clinical note:                                   
                                                                                
The patient is a 44-year-old non-Hispanic white female who presents with no     
chief complaint. She has a history of part-time employment, not being in the    
labor force, full-time employment, stress, and social isolation. She is married,
has never smoked, and identifies as heterosexual. She comes from a middle       
socioeconomic background and has completed some college courses. Her current    
medications include simvastatin, etonogestrel, kyleena, seasonique, and mirena. 
The patient's allergies are unknown.                                            
                                                                                
The assessment and plan include the insertion of an intrauterine contraceptive  
device.                                                                         

Checklist

  • Consider if documentation (like in docs/) needs to be updated
  • Consider if tests should be added

@mikix mikix merged commit 9723ef2 into main Aug 22, 2023
2 checks passed
@mikix mikix deleted the mikix/llama2-13b branch August 22, 2023 19:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants