Add QELM to list. #850
Add QELM to list. #850
Conversation
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
|
|
| @@ -0,0 +1,10 @@ | |||
| name = "QELM" | |||
| url = "https://github.com/R-D-BioTech-Alaska/Qelm" | |||
| description = "An innovative project that merges the power of quantum computing with natural language processing to create a next-generation language model. Leveraging Qiskit and Qiskit Aer, QELM explores the potential of quantum circuits in enhancing language understanding and generation capabilities. Utilizing Qubits, Qelm can train a model that would normally take up gigabytes worth of data in llm files all the way down to miniature sizes. With this size, llm's can run instantly with no loss of capabilities or intelligence and on small computers instead requiring data centers to run single models. The goal is to create llm's that are instant, smarter and can be utilized anywhere and by anyone." | |||
There was a problem hiding this comment.
This description needs to be <135 characters, preferably focussing on what users can do with this package. Take a look at the other ecosystem projects for examples.
Also:
Utilizing Qubits, Qelm can train a model that would normally take up gigabytes worth of data in llm files all the way down to miniature sizes. With this size, llm's can run instantly with no loss of capabilities or intelligence and on small computers instead requiring data centers to run single models.
Can you explain what you mean by "minature sizes"? It would also be good for ecosystem users to show benchmarks for the size / performance claims.
There was a problem hiding this comment.
The best way to describe "miniature sizes" in this context would be as ultra-compact models. So far, we have trained two basic quantum-enhanced language models (QELMs) using quantum-inspired parameters (working on the third). For testing purposes, we utilized CPU threading with 12 threads, ran 2 epochs, and used a learning rate of 0.05. Each training session averaged about 8 hours. Whereas we can leverage gpu, we are utilizing cpu to sort of "bottleneck" it.
Currently, the models produce nonsensical outputs because they were trained on small, basic datasets and have not yet been properly tokenized or configured for inference. Our primary goal was to test the training process and ensure it functioned as expected. We've recently started training with a real dataset based on Global Health Statistics, and our next step is to move on to the RedPajama 1T dataset for more comprehensive testing.
When comparing model sizes, we observed significant differences between a traditional LLM and our QELM. For example, training a small, fabricated dataset with a traditional LLM yielded a model size of approximately 125 MB, whereas the same dataset trained on QELM resulted in a model size of just 238 KB. Despite this compact size, both models shared similar training times and produced equally nonsensical outputs, highlighting the early stage of development for QELM. However the Qelm was able to respond instantly without any cpu throttling, whereas the llm did slightly.
This project is still in its infancy, and as it’s being handled primarily by myself with assistance from a friend, we haven’t reached the bench-marking phase yet. However, we hope to conduct performance benchmarks soon. We are also working on creating a dedicated website for the project.
Additionally, the 135-character input limitation issue has been resolved. If you need further details or have any questions, feel free to reach out. Thank you!
github-actions
bot
force-pushed
the
submission-849
branch
from
4a9b872 to
92f027a
Compare
Add QELM to list.
Closes #849