Add support for ESM models

[xenova]

Usage

Example: Protein sequence classification w/ Xenova/esm2_t6_8M_UR50D_sequence_classifier_v1.
(adapted from this example):

import { pipeline } from '@xenova/transformers';

// Create text classification pipeline
const classifier = await pipeline('text-classification', 'Xenova/esm2_t6_8M_UR50D_sequence_classifier_v1');

// Suppose these are your new sequences that you want to classify
// Additional Family 0: Enzymes
const new_sequences_0 = [
    'ACGYLKTPKLADPPVLRGDSSVTKAICKPDPVLEK',
    'GVALDECKALDYLPGKPLPMDGKVCQCGSKTPLRP',
    'VLPGYTCGELDCKPGKPLPKCGADKTQVATPFLRG',
    'TCGALVQYPSCADPPVLRGSDSSVKACKKLDPQDK',
    'GALCEECKLCPGADYKPMDGDRLPAAATSKTRPVG',
    'PAVDCKKALVYLPKPLPMDGKVCRGSKTPKTRPYG',
    'VLGYTCGALDCKPGKPLPKCGADKTQVATPFLRGA',
    'CGALVQYPSCADPPVLRGSDSSVKACKKLDPQDKT',
    'ALCEECKLCPGADYKPMDGDRLPAAATSKTRPVGK',
    'AVDCKKALVYLPKPLPMDGKVCRGSKTPKTRPYGR',
]

// Additional Family 1: Receptor Proteins
const new_sequences_1 = [
    'VGQRFYGGRQKNRHCELSPLPSACRGSVQGALYTD',
    'KDQVLTVPTYACRCCPKMDSKGRVPSTLRVKSARS',
    'PLAGVACGRGLDYRCPRKMVPGDLQVTPATQRPYG',
    'CGVRLGYPGCADVPLRGRSSFAPRACMKKDPRVTR',
    'RKGVAYLYECRKLRCRADYKPRGMDGRRLPKASTT',
    'RPTGAVNCKQAKVYRGLPLPMMGKVPRVCRSRRPY',
    'RLDGGYTCGQALDCKPGRKPPKMGCADLKSTVATP',
    'LGTCRKLVRYPQCADPPVMGRSSFRPKACCRQDPV',
    'RVGYAMCSPKLCSCRADYKPPMGDGDRLPKAATSK',
    'QPKAVNCRKAMVYRPKPLPMDKGVPVCRSKRPRPY',
]

// Additional Family 2: Structural Proteins
const new_sequences_2 = [
    'VGKGFRYGSSQKRYLHCQKSALPPSCRRGKGQGSAT',
    'KDPTVMTVGTYSCQCPKQDSRGSVQPTSRVKTSRSK',
    'PLVGKACGRSSDYKCPGQMVSGGSKQTPASQRPSYD',
    'CGKKLVGYPSSKADVPLQGRSSFSPKACKKDPQMTS',
    'RKGVASLYCSSKLSCKAQYSKGMSDGRSPKASSTTS',
    'RPKSAASCEQAKSYRSLSLPSMKGKVPSKCSRSKRP',
    'RSDVSYTSCSQSKDCKPSKPPKMSGSKDSSTVATPS',
    'LSTCSKKVAYPSSKADPPSSGRSSFSMKACKKQDPPV',
    'RVGSASSEPKSSCSVQSYSKPSMSGDSSPKASSTSK',
    'QPSASNCEKMSSYRPSLPSMSKGVPSSRSKSSPPYQ',
]

// Merge all sequences
const new_sequences = [...new_sequences_0, ...new_sequences_1, ...new_sequences_2];

// Get the predicted class for each sequence
const predictions = await classifier(new_sequences);

// Output the predicted class for each sequence
for (let i = 0; i < predictions.length; ++i) {
    console.log(`Sequence: ${new_sequences[i]}, Predicted class: '${predictions[i].label}'`)
}
// Sequence: ACGYLKTPKLADPPVLRGDSSVTKAICKPDPVLEK, Predicted class: 'Enzymes'
// ... (truncated)
// Sequence: AVDCKKALVYLPKPLPMDGKVCRGSKTPKTRPYGR, Predicted class: 'Enzymes'
// Sequence: VGQRFYGGRQKNRHCELSPLPSACRGSVQGALYTD, Predicted class: 'Receptor Proteins'
// ... (truncated)
// Sequence: QPKAVNCRKAMVYRPKPLPMDKGVPVCRSKRPRPY, Predicted class: 'Receptor Proteins'
// Sequence: VGKGFRYGSSQKRYLHCQKSALPPSCRRGKGQGSAT, Predicted class: 'Structural Proteins'
// ... (truncated)
// Sequence: QPSASNCEKMSSYRPSLPSMSKGVPSSRSKSSPPYQ, Predicted class: 'Structural Proteins'

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[xenova]

cc @Amelie-Schreiber, the creator of many of the most popular ESM models on the HF hub! I am using one of your models for the above tutorial, showing how it can be run completely locally in a web browser with transformers.js! 🥳

PS: the reason for the separate repo is that it requires ONNX model weights as well as a tokenizer.json file, which the original repo does not contain. If you're interested in making your original repo web-compatible, you can add the missing files and add the transformers.js tag to the README.md for discoverability. 🤗