Skip to content

Latest commit

 

History

History

javascript

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
 
 
 
 
 
 

Javascript Usage

I wrote the Javascript version for the browser, and then I made some modifications so it should work unchanged with Node.js. However, I haven't tested it on Node.js, so if you do use it and it doesn't work, create an issue.

<script src="tokenmonster.js"></script>
const vocab = new TokenMonster();
vocab.load(vocab_URL);
// in the browser vocab_URL must be a URL, but in Node.js it can be either a URL or a local filepath

let tokens = vocab.tokenize(inputText);

const decoder = vocab.Decoder()
const tokenStrDecoded = decoder.detokenize(tokens);

The entirety of capcode.js is also included within the tokenmonster.js file. It uses only native libraries that are available in both browsers and Node.js, and it has no other dependencies. Currently capcode level 1 (deleteToken only) is not supported.