Skip to content

Latest commit

 

History

History
 
 

220-cross-lingual-books-alignment

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
 
 
 
 

Cross-lingual Books Alignment With Transformers and OpenVINO™

Binder Colab

Cross-lingual text alignment is the task of matching sentences in a pair of texts that are translations of each other. In this notebook, you'll learn how to use a deep learning model to create a parallel book in English and German

This method helps you learn languages but also provides parallel texts that can be used to train machine translation models. This is particularly useful if one of the languages is low-resource or you don't have enough data to train a full-fledged translation model.

The notebook shows how to accelerate the most computationally expensive part of the pipeline - getting vectors from sentences - using the OpenVINO™ framework.

Pipeline The notebook guides you through the entire process of creating a parallel book: from obtaining raw texts to building a visualization of aligned sentences. Here is the pipeline diagram:

By visualizing the result, you can evaluate which steps in the pipeline can be improved, which is also indicated in the diagram.

Visualization

Benchmark