Loki: An Open-source Tool for Fact Verification

Overview

Loki is our open-source solution designed to automate the process of verifying factuality. It provides a comprehensive pipeline for dissecting long texts into individual claims, assessing their worthiness for verification, generating queries for evidence search, crawling for evidence, and ultimately verifying the claims. This tool is especially useful for journalists, researchers, and anyone interested in the factuality of information. To stay updated, please subscribe to our newsletter at our website or join us on Discord!

Quick Start

Clone the repository and navigate to the project directory

git clone https://github.com/Libr-AI/OpenFactVerification.git
cd OpenFactVerification

Installation with poetry (option 1)

1. Install Poetry by following it installation guideline.
2. Install all dependencies by running:

poetry install

Installation with pip (option 2)

1. Create a Python environment at version 3.9 or newer and activate it.

2. Navigate to the project directory and install the required packages:

pip install -r requirements.txt

Configure API keys

You can choose to export essential api key to the environment

• Example: Export essential api key to the environment

export SERPER_API_KEY=... # this is required in evidence retrieval if serper being used
export OPENAI_API_KEY=... # this is required in all tasks

Alternatively, you configure API keys via a YAML file, see user guide for more details.

A sample test case:

Usage

The main interface of Loki fact-checker located in factcheck/__init__.py, which contains the check_response method. This method integrates the complete fact verification pipeline, where each functionality is encapsulated in its class as described in the Features section.

Used as a Library

from factcheck import FactCheck

factcheck_instance = FactCheck()

# Example text
text = "Your text here"

# Run the fact-check pipeline
results = factcheck_instance.check_response(text)
print(results)

Used as a Web App

python webapp.py --api_config demo_data/api_config.yaml

Multimodal Usage

# String
python -m factcheck --modal string --input "MBZUAI is the first AI university in the world"
# Text
python -m factcheck --modal text --input demo_data/text.txt
# Speech
python -m factcheck --modal speech --input demo_data/speech.mp3
# Image
python -m factcheck --modal image --input demo_data/image.webp
# Video
python -m factcheck --modal video --input demo_data/video.m4v

Customize Your Experience

For advanced usage, please see our user guide.

Try Our Online Service

As we continue to evolve and enhance our fact-checking solution, we're excited to invite you to become an integral part of our journey. By registering for our Supporter Edition, you're not just unlocking a suite of advanced features and benefits; you're also fueling the future of trustworthy information.

Below is a screenshot of our online service. Click here to try it now!

Contributing to Loki project

Welcome and thank you for your interest in the Loki project! We welcome contributions and feedback from the community. To get started, please refer to our Contribution Guidelines.

Acknowledgments

• Special thanks to all contributors who have helped in shaping this project.

Stay Connected and Informed

Don’t miss out on the latest updates, feature releases, and community insights! We invite you to subscribe to our newsletter and become a part of our growing community.

💌 Subscribe now at our website!

Star History

Cite as

@misc{li2024lokiopensourcetoolfact,
      title={Loki: An Open-Source Tool for Fact Verification}, 
      author={Haonan Li and Xudong Han and Hao Wang and Yuxia Wang and Minghan Wang and Rui Xing and Yilin Geng and Zenan Zhai and Preslav Nakov and Timothy Baldwin},
      year={2024},
      eprint={2410.01794},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2410.01794}, 
}