AlphaPharm: a reinforcement-active learning framework to streamline lead optimization in drug discovery

Official PyTorch implementation of paper "AlphaPharm: a reinforcement-active learning framework to streamline lead optimization in drug discovery".

Contents

• 1. Overview
• 2. Installation guide
• 3. Run AlphaPharm with a single command
• 4. Reproduce the results
  • 4.1 Datasets
  • 4.2 Training
  • 4.3 Testing
  • 4.4 Reproduce the selection process of identifying AA-35
• 5. Running on your own data
• 6. Supplementary data
• 7. Contact

1. Overview

AlphaPharm is a reinforcement-active learning framework that generates effective plans to progressively identify promising drug candidates in trial and error. Starting with no property information of derivatives, AlphaPharm emulates a medicinal chemist to establish an iterative, progressive process of hypothesizing molecules, learning causality from successes and failures, and fine-tuning the planning strategy to ascertain candidate molecules. Specifically, AlphaPharm formalizes this iterative progression as a Markov decision process, where a policy network learns to measure the molecule diversity and property improvement iteratively. In AlphaPharm, we introduce a causal reward system that encourages both the discovery of the molecule with a new best property and the selection of counterfactual samples. To maximize the returned rewards, AlphaPharm is encouraged to find the optimal compound for a specific property as soon as possible, thus promoting property enhancement and experimental trial minimization.

Thanks to the above design, AlphaPharm significantly outperforms all the current active learning and few-shot learning methods. It should be noted that AlphaPharm astoundingly singled out a drug candidate, AA-35, with the most potent analgesic activity, from a pool of 51 compounds in just six trials. These findings validate proof-of-concept and highlight the promise of AlphaPharm as a powerful AI-driven computational tool for drug discovery.

2. Installation guide

Dependency

The code has been implemented in the following environment:

Package	Version
Python	3.7
PyTorch	1.8.0
CUDA	11.1
RDKit	2022.9.1
DeepChem	2.7.1
py-xgboost-gpu	1.5.1

The code should work with Python >= 3.7. You could change the package version according to your needs.

Install via Conda and Pip

conda create -n AlphaPharm python=3.7
conda activate AlphaPharm

pip install torch==1.8.0+cu111 torchvision==0.9.0+cu111 torchaudio==0.8.0 -f https://download.pytorch.org/whl/torch_stable.html
conda install cudatoolkit=11.1 cudnn

pip install rdkit
pip install deepchem
pip install openpyxl
conda install -c conda-forge py-xgboost-gpu

3. Run AlphaPharm with a single command

As soon as you execute bash run_example.sh, the testing process will be started, performing the molecule identification process for certain properties with the model. You will get the performance at the bottom of a log file with the following formats:

Task:  molnet__thermosol
Begin molecule identification process...
**********Step 0:**********
Random initialization:  [SMILES of a molecule]
Human: Conducts experiment and obtains the property value:  [Property value]
**********Step 1:**********
Random initialization:  [SMILES of a molecule]
Human: Conducts experiment and obtains the property value:  [Property value]
...

If you want to test AlphaPharm on the other properties, you can edit the run_example.sh file by revising the value of the --task_id argument. (Domain of this argument: [0, 1, 2 ... 145])

4. Reproduce the results

4.1 Datasets

The dataset that AlphaPharm used in the training, validation, and testing is located at data. Please download and move it to the folder data/dataset by following steps:

1. Download the dataset archive dataset.rar from data.

2. Unzip the dataset by running the command:

sudo apt-get install rar
rar x dataset.rar

3. Run the command: mv dataset ./data/dataset.

4.2 Training

Training from scratch

You could train your own AlphaPharm from scratch with the following bash order.

python run.py --mode train --checkpoint_path [saved_model_path] --train_path [training path] --va_path [validation path]

For example:

python -u run.py --mode train  --checkpoint_path ./results --train_path data/dataset/training-set --val_path data/dataset/validation-set

Then you will see the training process with the following formats:

[2023-10-24 15:30:52,632:: train::INFO] Building model...

[2023-10-24 15:30:52,640:: train::INFO] Training model...

[2023-10-24 15:31:52,644:: train::INFO] [Train] Iter 200 | reward [a floating point number]

[2023-10-24 15:32:53,329:: train::INFO] [Train] Iter 400 | reward [a floating point number]

...

[2023-10-24 15:32:53,329:: train::INFO] [Validation] Iter 5000 | reward [a floating point number]

[Time it takes]: The training process will take several days to converge on a single GPU. More time is required for the CPU setting.

Trained model checkpoint

We uploaded the trained model to the checkpoints folder.

4.3 Testing

Testing on the dataset

You could test the model on the test dataset.

python -u run.py --mode test --searchtimes 1 --test_times [times of testing]  --checkpoint_path [saved_model_path] --pri true --task_id [task_id] --num_iter [max number of steps]

For example:

python -u run.py --searchtimes 1 --mode test  --checkpoint_path checkpoints/almodel.pt  --pri true --task_id 0 --test_path ./data/dataset/testing-set  --num_iter 40

[Output]:

********Statistic Performance********
Task:  molnet__thermosol
Average success rate: [a floating point number] %
Average search steps: [a floating point number]
time cost [time] s

[Time it takes]: The testing process will take several minutes to obtain the performance on a single GPU. More time is required for the CPU setting.

4.4 Reproduce the selection process of identifying AA-35

python -u interact.py   --mode test   --checkpoint_path checkpoints/almodel.pt   --custom data/data-LD

5. Running on your own data

To applied the trained AlphaPharm on your own data, you should first build the dataset and then run the AlphaPharm program. (1) Build the custom dataset.

 python databuild.py --src_path [file_name].xlsx --dest_path [folder to save]  --column [Property name in the excel file]

For example, we take the lidocaine data to build the dataset.

 python databuild.py --src_path data/lidocaine_data.xlsx --dest_path data_test  --column Objective

(2) Run the AlphaPharm program.

python -u interact.py   --mode test   --checkpoint_path checkpoints/almodel.pt   --custom data_test

[Time it takes]: The process will take several minutes to obtain the performance on a single GPU.

6. Supplementary data

supplementary/ directory

• statistics_of_the_dataset.xlsx: The list of the 1412 properties, which were divided into the training, validation, and test set with the ratio of 8:1:1. This list reports the property name, the property type (e.g., EC50, IC50, Ki, Kd, etc.), and the sample number of the property.
• lidocaine_data.xlsx: The 51 derivatives of lidocaine and the corresponding objective function values.
• Wetlab-1. ALL Compound-CFA.xlsx: The CFA data of all the compounds used in our work.
• Wetlab-2. AUC.xlsx: The AUC data of all the compounds used in our work.
• Wetlab-3. Writhing Latency (min).xlsx: The in vivo data of the writhing latency.
• ...

7. Contact

If you encounter any problems during the setup of environment or the execution of AlphaPharm, do not hesitate to contact [email protected] or [email protected]. You could also create an issue under the repository: https://github.com/To-phoenix-zhw/AlphaPharm.