Skip to content

Commit

Permalink
[JS] Add sources for nodejs package of tokenizers (openvinotoolkit#312)
Browse files Browse the repository at this point in the history
* Add sources for nodejs package of tokenizers

* Update version in package.json

* Download binaries script refactoring

* Add .npmignore

* Replace path to linux archive at the storage to centos7

* Update binary version in package.json to 2024.5.0

* Add openvino-node as dependency and use binary manager from it

* Update openvino-node package to 2024.5.0

* Specify vesion of openvino-tokenizers-node package as preview 2024.5.0-preview

---------

Co-authored-by: Mikhail Ryzhov <[email protected]>
  • Loading branch information
vishniakov-nikolai and mryzhov authored Nov 25, 2024
1 parent b961c74 commit 63d6ff4
Show file tree
Hide file tree
Showing 8 changed files with 754 additions and 0 deletions.
2 changes: 2 additions & 0 deletions js/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
bin
node_modules
5 changes: 5 additions & 0 deletions js/.npmignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
bin
tests
thirdparty

*.tgz
30 changes: 30 additions & 0 deletions js/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# OpenVINO Tokenizers

This package contains binaries extension for
[openvino-node](https://www.npmjs.com/package/openvino-node) package

## Installation

```bash
npm install openvino-tokenizers-node
```
After package installation script that downloads and extracts binaries will be executed. Binaries will be placed in the `node_modules/openvino-tokenizers-node/bin` directory.

This package is a part of [openvino-node](https://www.npmjs.com/package/openvino-node) package and should be used with it. Version of this package should be the same as version of openvino-node package.

## Usage

```javascript
const { addon: ov } = require('openvino-node');
const openvinoTokenizers = require('openvino-tokenizers-node');

const core = new ov.Core();
core.addExtension(openvinoTokenizers.path); // Add tokenizers extension

// Load tokenizers model
// ...
```

[License](https://github.com/openvinotoolkit/openvino/blob/master/LICENSE)

Copyright © 2018-2024 Intel Corporation
60 changes: 60 additions & 0 deletions js/openvino-tokenizers.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
const os = require('node:os');
const path = require('node:path');

module.exports = {
path: getPathToBinary({
platform: os.platform(),
arch: os.arch(),
}),
getPathToBinary,
};

function getPathToBinary(osProps) {
const { arch, platform } = osProps;

if (platform === 'win32' && arch !== 'x64')
throw new Error(`Version for windows and '${arch}' is not supported.`);

return path.join(
__dirname,
'bin/runtime',
libOrBin(platform),
getDirnameByArch(arch),
platform === 'linux' ? '' : 'Release',
getBinaryFilename(platform),
);
}

function getDirnameByArch(arch) {
switch (arch) {
case 'x64':
return 'intel64';
case 'arm64':
case 'armhf':
return 'arm64';
default:
throw new Error(`Unsupported architecture: ${arch}`);
}
}

function libOrBin(platform) {
switch (platform) {
case 'win32':
return 'bin';
default:
return 'lib';
}
}

function getBinaryFilename(platform) {
switch (platform) {
case 'win32':
return 'openvino_tokenizers.dll';
case 'linux':
return 'libopenvino_tokenizers.so';
case 'darwin':
return 'libopenvino_tokenizers.dylib';
default:
throw new Error(`Unsupported platform: ${platform}`);
}
}
Loading

0 comments on commit 63d6ff4

Please sign in to comment.