Run Google Nano Locally on your desktop

Step 1: Get Chrome Canary (nightly build for devs) and configure it with instructions below. Many thanks to https://kharms.ai/nano for the instructions:

To try it out, you need to:

be on Mac or Windows

be on Chrome dev / canary (version >= 127)

enable chrome://flags/#optimization-guide-on-device-model (set to Enabled BypassPerfRequirement)

enable chrome://flags/#prompt-api-for-gemini-nano (set to Enabled)

re-launch Chrome and revisit page

if this fails, try chrome://components and look for On Device Model

this should have a version which is not 0.0.0.0 -- you can bang on the update button if you'd like (don't bang too hard)

eventually this will have the right value (after downloading the model), then you'll need to relaunch chrome and try again

Step 2: Down the code (index.html) from this repo and open it in Chrome Canary directly (not your regular Chrome)

Step 3: Enter a prompt and test it out. It should work without wifi/locally. Note that the reponses might be slow (local) and not be reliable (as expected with any LLM)

Demo:

demo.mov

Note: This is just a demo and not meant for regular or commercial uses.

Also, huge shoutout to the folks behind windowai

Edit: 27 Jun

Some udpates today from Google team. Please refer to it in case of any issues -- https://docs.google.com/document/d/1VG8HIyz361zGduWgNG7R_R8Xkv0OOJ8b5C9QKeCjU0c/edit#heading=h.5s2qlonhpm36