Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: cboard-org/cboard-ai-engine
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: 1.2.0
Choose a base ref
...
head repository: cboard-org/cboard-ai-engine
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: main
Choose a head ref
Loading
Showing with 651 additions and 308 deletions.
  1. +2 −0 .gitignore
  2. +20 −103 README.md
  3. +216 −5 package-lock.json
  4. +6 −2 package.json
  5. +64 −0 run.ts
  6. +7 −2 src/constants.ts
  7. +129 −192 src/engine.ts
  8. +1 −2 src/index.ts
  9. +158 −0 src/lib/symbolSets.ts
  10. +28 −0 src/types/arasaac.d.ts
  11. +2 −2 src/types/global-symbols.d.ts
  12. +18 −0 src/utils/language.ts
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
.idea
node_modules
.env
.env.local
.env.development.local

# Ignore test-related files
/coverage.data
123 changes: 20 additions & 103 deletions README.md
Original file line number Diff line number Diff line change
@@ -4,7 +4,7 @@ This engine powers the Cboard AI builder, designed to generate content suggestio

With a simple prompt, it will generate a list of suggestions that can be used to create an AAC board. Each suggestion will be associated with a text description and a list of images.

The images are retrieved from the [Global Symbols](https://www.globalsymbols.com/) website, and the text descriptions are generated using the [OpenAI Node API Library](https://www.npmjs.com/package/openai). If the image is not found, the engine will create a new pictogram using the Cboard Pictonizer.
The images can be retrieved from [ARASAAC](https://arasaac.org/index.html) or [Global Symbols](https://www.globalsymbols.com/), and the text descriptions are generated using the [OpenAI Node API Library](https://www.npmjs.com/package/openai).

## Table of Contents

@@ -13,9 +13,8 @@ The images are retrieved from the [Global Symbols](https://www.globalsymbols.com
- [Initialization](#initialization)
- [Types](#types)
- [Methods](#methods)
- [getSuggestionsAndProcessPictograms](#getSuggestionsAndProcessPictograms)
- [getSuggestions](#getSuggestions)
- [pictonizer](#pictonizer)
- [isContentSafe](#isContentSafe)
- [Error Handling](#error-handling)
- [License](#license)

@@ -41,14 +40,13 @@ import { initEngine } from "cboard-ai-engine";
const engineInstance = initEngine({
openAIConfiguration,
globalSymbolsApiURL,
pictonizerApiURL,
});

const suggestions = await engineInstance.getSuggestionsAndProcessPictograms({
prompt: "Brazilian food",
maxSuggestions: 5,
symbolSet: "arasaac",
language: "eng",
language: "en", // Use two letter language code or locale (en.US for example)
symbolSet: "arasaac"
});
```

@@ -59,6 +57,7 @@ const engineInstance = initEngine({
openAIConfiguration,
globalSymbolsApiURL,
pictonizerConfiguration,
arasaacURL
});
```

@@ -79,29 +78,15 @@ const openAIConfiguration = {

- `globalSymbolsApiURL`: The Global Symbols API URL. Default is `https://www.globalsymbols.com/api/v1/labels/search/`. Optional.

- `pictonizerConfiguration`: Object with the Pictonizer configuration. Optional. If not passed, the `pictonizer()` method will return a "empty" image. For more info see [pictonizer](#pictonizer).

```typescript
import { type PictonizerConfiguration } from "cboard-ai-engine";

const pictonizerConfiguration = {
URL: process.env.PICTONIZER_URL,
token: process.env.PICTONIZER_AUTH_TOKEN,
keyWords: 'arasaac pictograms',
} as PictonizerConfiguration;
```
- `arasaacURL`: The ARASAAC API URL. Default is `https://api.arasaac.org/api/pictograms`. Optional.

Return:

It returns an instance of the engine with the following methods:

- `getSuggestionsAndProcessPictograms`: This method is used to get the suggestions and process the pictograms. It will first search for the images in Global Symbols, and if no image is found, it will generate a new one with the Cboard Pictonizer.

It returns a list of suggestions that can be used to create an AAC board. Each one is associated with a text description and a pictogram.

- `getSuggestions`: This method is used to get suggestions with images solely from Global Symbols.

- `pictonizer`: This method is used to generate a new pictogram using the Cboard Pictonizer.
- `isContenSafe`: This method is used to check if the content is safe.

## Types

@@ -113,62 +98,18 @@ export type Suggestion = {
label: string; // The text description of the suggestion
locale: string; // The language of the suggestion
pictogram: {
isAIGenerated: boolean; // If the pictogram was generated by the AI
images:
| {
id: string; // Indentifier for the image from Global Symbols
symbolSet: string; // The symbol set of the image
url: string; // The URL of the image
}[]
| AIImage[];
};
};

export type AIImage = {
blob: Blob | null; // The image blob
ok: boolean; // If the image was generated successfully
error?: string; // The error message if the image was not generated
prompt: string; // The prompt used to generate the image
};
```

## Methods

### getSuggestionsAndProcessPictograms

```typescript
async function getSuggestionsAndProcessPictograms({
prompt,
maxSuggestions,
symbolSet,
language = DEFAULT_LANGUAGE,
}: {
prompt: string;
maxSuggestions: number;
symbolSet?: string;
language: string;
}): Promise<Suggestion[]>;
```

This method is used to get the suggestions and process the pictograms. It will first search for the images in Global Symbols, and if no image is found, it will generate a new one with the Cboard Pictonizer.

It returns a list of suggestions that can be used to create an AAC board. Each one is associated with a text description and a pictogram.

Parameters:

- `prompt` : The prompt to be used to generate the suggestions. Required. Type: string.

- `maxSuggestions`: The maximum number of suggestions to be returned. Default is 10. Optional. Type: number.

- `symbolSet`: The symbol set to be used. If `undefined`, images will be searched across all Global Symbol image banks. Optional. Type: string.

- `language`: The language to be used. Default is `eng`. Optional. Type: string.

Return:

It returns an array of Suggestion.


### getSuggestions

```typescript
@@ -195,64 +136,40 @@ Parameters:

- `symbolSet`: The symbol set to be used. If `undefined`, images will be searched across all Global Symbol image banks. Optional. Type: string.

- `language`: The language to be used. Default is `eng`. Optional. Type: string.
- `language`: The language to be used. Default is `en`. Use two-letters code or locale Optional. Type: string.

Return:

It returns an array of Suggestion.

### pictonizer

```javascript
const pictogram = await engineInstance.pictonizer(imagePrompt: string) => Promise<AIImage>;
### isContentSafe

```typescript
async function isContentSafe({
text,
}: {
text: string;
}): Promise<boolean>;
```

This method is used to generate a new pictogram using the Cboard Pictonizer.
This method is used to check if the provided text is safe for use.

Parameters:

- `imagePrompt` : The prompt to be used to generate the pictogram. Required. Type: string.

- `text`: The text to be checked. Required. Type: string.

Return:

It returns an `AIImage` object.

```javascript
const AIImage = {
blob: Blob;
ok: true;
error: undefined;
prompt: "Brazilian food arasaac pictograms";
}

// NOTE: The prompt is a concatenation of the prompt
// and the keyWords from the PictonizerConfiguration.
```

If no URL or Token is passed on the `PictonizerConfiguration` parameter, or occurs an error while generating. It will return:

```javascript
const AIImage = {
blob: null;
ok: false;
error: "ERROR: Can't generate image";
prompt: "Brazilian food arasaac pictograms";
}

// NOTE: The prompt is a concatenation of the prompt
// and the keyWords from the PictonizerConfiguration.
```

And no error will be thrown.
It returns a boolean indicating whether the content is safe.

## Error Handling

When an error occurs, an error will be thrown. It is recommended to use a try/catch block to handle it.

```javascript
try {
const suggestions = await engineInstance.getSuggestionsAndProcessPictograms({
const suggestions = await engineInstance.getSuggestions({
prompt,
maxSuggestions,
symbolSet,
Loading