[JS] Add GenAI Node.js bindings #1193
Conversation
github-actions
bot
added
category: sampling
github-actions
bot
added
the
category: tokenizers
github-actions
bot
added
category: llm_bench
vishniakov-nikolai
force-pushed
the
feature/nodejs-bindings
branch
from
7784bba to
222940d
Compare
Co-authored-by: Vladimir Zlobin <vladimir.zlobin@intel.com>
Co-authored-by: Vladimir Zlobin <vladimir.zlobin@intel.com>
Co-authored-by: Vladimir Zlobin <vladimir.zlobin@intel.com>
Co-authored-by: Vladimir Zlobin <vladimir.zlobin@intel.com>
|
#1193 (comment) and #1193 (comment) are left |
Wovchena
requested review from
ilya-lavrenov
and removed request for
ilya-lavrenov
|
|
||
| - name: Install build dependencies | ||
| run: | | ||
| sudo -E ${OPENVINO_REPO}/install_build_dependencies.sh |
There was a problem hiding this comment.
for other jobs in this file we re-use docker images from OpenVINO repo.
| restore-keys: | | ||
| ${{ runner.os }}-${{ runner.arch }}-ccache-ov-and-genai-${{ matrix.build-type }} | ||
|
|
||
| - name: Build openvino + genai |
There was a problem hiding this comment.
why do we need to build them together?
As you can see, we use OpenVINO provider github actions above. So, we don't build OpenVINO anymore here.
| path: ${{ env.BUILD_DIR }}/genai_nodejs_bindings.tar.gz | ||
| if-no-files-found: 'error' | ||
|
|
||
| genai_nodejs_samples_tests: |
There was a problem hiding this comment.
can we perform samples tests within a single JS related job which builds GenAI JS and performs all the tests ?
| - name: Run tests | ||
| run: npm test | ||
| env: | ||
| MODEL_PATH: ${{ env.JS_SRC_DIR }}/tests/models/Llama-3.2-3B-Instruct-openvino-8bit |
There was a problem hiding this comment.
it's very heavy model for tests. Please, use something like katuni4ka/tiny-random-baichuan2
There was a problem hiding this comment.
please, rename chat_sample folder to text_generation.
See #1411 where we move all samples from the same category to a single folder.
| INCLUDES DESTINATION runtime/include) | ||
|
|
||
| # samples do not need to be built for NPM package |
There was a problem hiding this comment.
samples => development files
| @@ -133,13 +133,29 @@ if(MSVC OR APPLE) | |||
| set(ARCH_DIR ${ARCH_DIR}/${CMAKE_BUILD_TYPE}) | |||
| endif() | |||
|
|
|||
| # Put binaries at the top level for NPM package | |||
| if(CPACK_GENERATOR STREQUAL "NPM") | |||
There was a problem hiding this comment.
looks like if we follow @Wovchena we will not break anything for GenAI specifically
Adding Node.js bindings for GenAI pipelines.
Limitations
Current version it's primary backbone of future development. Supports bindings of
LLMPipelineonly.TODO