ProteinGAN.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "view-in-github"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/https-deeplearning-ai/GANs-Public/blob/master/ProteinGAN.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ViSpF4I70O4b"
      },
      "source": [
        "# ProteinGAN: Generative Adversarial Network for Functional Protein Generation\n",
        "*Please note that this is an optional notebook that is meant to introduce more advanced concepts, if you're up for a challenge. So, don't worry if you don't completely follow every step! We provide external resources for extra base knowledge required to grasp some components of the advanced material.*\n",
        "\n",
        "[ProteinGAN](https://www.biorxiv.org/content/10.1101/789719v2) was developed by [Biomatters Designs](https://www.biomatterdesigns.com/) and [Zelezniak lab at Chalmers University of Technology](https://twitter.com/AZelezniak).\n",
        "\n",
        "## Goal\n",
        "The goal of this notebook is to demonstrate that core GAN ideas can be applied outside of the image domain. In this notebook, you will be able to play around with a pre-trained ProteinGAN model to see how it can be used in bioinformatics to generate functional molecules.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "q04P9icA8xIK"
      },
      "source": [
        "## Background\n",
        "\n",
        "\n",
        "### Proteins\n",
        "\n",
        "Proteins are large, complex molecules that play many critical roles in living organisms, including humans. You can think of them as very tiny, programmable robots used by nature to perform various functions, e.g. building, modifying or breaking down other molecules, aiding in cell replication and division, and transporting other proteins inside of cells. Apart from the crucial cellular functions, proteins are used virtually everywhere in our daily life, starting from animal nutrition and washing powders down to costly drugs and therapeutic antibodies. Using synthetic biology, protein engineering, adaptive evolutions experimental techniques, researchers enhance proteins' properties, making them more active or \"sticky\" towards a particular drug target or resistant to harsh environemental conditions. However, it is challenging to randomly modify proteins in a \"biochemically meaningful\" way such that protein would remain functional leading in a very costly time-consuming experiments. Thus generating natural-like diverse proteins that remain functional is of outstanding importance for biotechnology and biomedical applications. \n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 621
        },
        "id": "6BRDjqgjZgkg",
        "outputId": "5a8d46cb-bdd1-4d79-86a0-5af3523822f1"
      },
      "outputs": [],
      "source": [
        "from IPython.display import YouTubeVideo\n",
        "YouTubeVideo('wJyUtbn0O5Y', start=75, end=80, autoplay=1, controls=0, loop=1, width=800, height=600)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rLPKgWGCZdL7"
      },
      "source": [
        "*Above, animation of motor protein responsible for transporting objects in cells*\n",
        "\n",
        "Source: https://www.youtube.com/watch?v=wJyUtbn0O5Y"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "l2PDRSD4kDTR"
      },
      "source": [
        "\n",
        "Proteins, like images, can be represented in various ways on the computer. Images are represented as integers from 0 to 256 that indicate the intensity of red, green, or blue (RGB) color. Proteins, similarly, use letters to represent 20 unique amino acids, like the one below: \n",
        "\n",
        "> MKYATLLEYAFQALKNSYAPYSRFRVGAALLSDDGEVVTGCNVENASYGLSMCAERTAVFRAVAQGVKKFDAIAVVSGKVNPVYPCGACRQVLREFNPRLTVVVAGPGKKPLTTSLDKLLPKSFGKESLRRR\n",
        "\n",
        "Raw pixel RGB values are easy for computers to work with, though they are not very meaningful to the human eye, which is why they are displayed as images on the screen. Similarly, the sequence of amino acids is a compact, convenient representation of the actual molecule, while the more meaningful view of the protein molecule is its 3D structure. For an example, see [Cytidine deaminase](https://colab.research.google.com/drive/1O0_wyl3i-9F-5mDTlShaMfR2uOWHKwwE#scrollTo=Q277ab8R9WEU).\n",
        " \n",
        "For you to appreciate and reason about the outputs, you want your models (GANs) to ultimately produce meaningful structures. There are two important common features that make images and proteins both suitable candidates for GANs:\n",
        "\n",
        "* A random combination of building blocks, whether amino acids or pixels, will not produce a realistic outcomes. This means the GAN cannot simply guess! There are meaningful, realistic patterns of pixels and amino acids that it must model and generate.\n",
        "* The mathematical formula for how to evaluate the correctness of the generated item is unknown. For images, correctness is  \"realism\" -- how realistic does a generated image of a dog look? There's no math formula for that, so instead you have another model (the discriminator!) learn to assess that. The same goes for proteins.\n",
        "\n",
        "\n",
        "|         | Image      | Protein  |\n",
        "| ------- |:----------:| --------:|\n",
        "| Data type | integers from 0 to 256 | vocab of 20 amino acids |\n",
        "| Dimension| 2D | 1D|\n",
        "| Number of possible variants | $3*256^{size}$ |  $20^{length}$  |\n",
        "\n",
        " \n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8huQi0yQ8qla"
      },
      "source": [
        "### ProteinGAN\n",
        " \n",
        "ProteinGAN is a generative adversarial network adapted to generate functional protein sequences. At its core, it consists of common building blocks: a discriminator and generator, spectral normalization (as in the [SN-GAN optional notebook](https://www.coursera.org/learn/build-basic-generative-adversarial-networks-gans/ungradedLab/c2FPs/optional-sn-gan)), and a loss function based on earth mover's distance (as in the [WGAN-GP assignment](https://www.coursera.org/learn/build-basic-generative-adversarial-networks-gans/programming/mTm3U/wgan)), etc. \n",
        "\n",
        "To make the GAN concept work in the field of synthetic biology, the generator and discriminator architectures have been modified to handle sequences of categorical values, capture long-distance relationships, as well as discriminate between various areas in the sequences. This is a major difference from pixel values in images and helps specifically with this type of long, categorical, and sequential data. One question to mull over: could this data processing and understanding help with generating text? \n",
        "\n",
        "\n",
        "**Data pre-processing.** The explored protein space is very unevenly distributed. Some proteins and their close variants are widely studied while others are just recorded in public databases. Without the balancing, the neural network mainly focuses on big clusters of similar well-studied sequences while treating unrepresented cluster members as anomalies. ProteinGAN has in-built upsampling capability to balance the dataset based on the size of the cluster in order to preserve the diversity of sequences.\n",
        "\n",
        "**Discrete values.** One of the biggest differences between images and proteins is the data type: while images consist of continuous values, proteins are built from discrete building blocks. To address this challenge for backpropagation, ProteinGAN employs the [Gumbel-Softmax trick with temperature](https://arxiv.org/abs/1611.01144), which serves as a differentiable approximation to sampling discrete data. This allows to end-to-end training of the discriminator and generator while operating in discrete input space. \n",
        "\n",
        "**Convergence.** GANs are known to be difficult to train due to stability issues. The discrete nature of the input further aggravates this problem. Despite the implementation of spectral normalization and WGAN loss, the optimization of ProteinGAN did not lead to convergence. However, as demonstrated in [this paper](https://arxiv.org/abs/1801.04406), training with zero-centered gradient penalties leads to improved training and guarantees local convergence even if data and generator distributions are not continuous. Adapting the implementation of [non-saturating loss with R1 regularization](https://arxiv.org/abs/1801.04406) greatly improves the performance of the GAN as demonstrated in the figure below.\n",
        "\n",
        "\n",
        "![Loss performance](https://drive.google.com/uc?export=view&id=1GBwiEm328DeLV29F7gUzUHP-o1uIYqSK)\n",
        "\n",
        "> *GAN performance in the first 35k steps using different losses. Model performances were measured using [BLOSUM45 scores](https://en.wikipedia.org/wiki/BLOSUM) (in the nutshell, similarity score which takes into account substitution probabilities of amino acids in known seuqences) against training sequences for the first 35,000 steps (average of 3 runs with different random seeds).*\n",
        "\n",
        "\n",
        "For more information please refer [ProteinGAN paper](https://www.biorxiv.org/content/10.1101/789719v2) "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "scLQLcKcIeSS"
      },
      "source": [
        "## Setup"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ogtiZDjSjiOJ",
        "outputId": "ffe7ea1c-91c8-49ec-98c2-71a58d81202b"
      },
      "outputs": [],
      "source": [
        "# Installing dependencies\n",
        "! pip install biopython\n",
        "! pip install py3Dmol\n",
        "! apt-get install -y clustalo"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "kEbt5Aq9YSyL",
        "outputId": "df25a09e-b727-4c6e-af68-89fa2e4e7778"
      },
      "outputs": [],
      "source": [
        "# Downloading pre-trained ProteinGAN model\n",
        "!gdown https://drive.google.com/uc?id=1vuxHqsRHc-K-gqXK7AJqaC54f7PtKmOX\n",
        "!unzip pre_trained_protein_gan.zip"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "7F9vYRgXV2uf"
      },
      "outputs": [],
      "source": [
        "# Helper methods\n",
        "import shutil\n",
        "\n",
        "from Bio.Blast import NCBIWWW\n",
        "from Bio.Blast import NCBIXML\n",
        "\n",
        "import numpy as np\n",
        "import pandas as pd\n",
        "\n",
        "import py3Dmol\n",
        "\n",
        "# A mapping between amino acids ids and their corresponding letters\n",
        "ID_TO_AMINO_ACID = {0: '0', 1: 'A', 2: 'C', 3: 'D', 4: 'E', 5: 'F', 6: 'G', 7: 'H', 8: 'I', 9: 'K', 10: 'L', 11: 'M', 12: 'N', 13: 'P', 14: 'Q', 15: 'R', 16: 'S', 17: 'T', 18: 'V', 19: 'W', 20: 'Y'}\n",
        "\n",
        "def to_seqs(model_output):\n",
        "  \"\"\"Takes ProteinGAN output and returns list of generated protein sequences\"\"\"\n",
        "  human_readable_seqs = []\n",
        "  seqs = model_output[\"prediction\"]\n",
        "  for i in range(len(seqs)):\n",
        "    human_readable_seq =\"\".join([ID_TO_AMINO_ACID[a] for a in seqs[i].numpy()])\n",
        "    human_readable_seq = human_readable_seq.replace(\"0\", \"\")\n",
        "    human_readable_seqs.append(human_readable_seq)\n",
        "  return human_readable_seqs\n",
        "\n",
        "def get_blast_results(seq):\n",
        "  \"\"\"Takes a protein sequence, calls BLAST server and returns parsed results\"\"\"\n",
        "  print(\"Calling BLAST server. This might take a while\")\n",
        "  r = NCBIWWW.qblast(\"blastp\", \"nr\", seq, hitlist_size = 5, expect=0.5, \n",
        "                     word_size=6, matrix_name=\"BLOSUM62\")\n",
        "  blast_record = NCBIXML.read(r)\n",
        "\n",
        "  to_df = []\n",
        "\n",
        "  for a in blast_record.alignments:\n",
        "    to_df.append({\"name\": a.hit_def,\"identity\": a.hsps[0].identities,\n",
        "                  \"subject\": a.hsps[0].sbjct})\n",
        "\n",
        "  return pd.DataFrame(to_df)\n",
        "\n",
        "def append_to_fasta(path, seqs, prefix):\n",
        "  \"\"\"Appends new sequences to existing file in FASTA format.\"\"\"\n",
        "  fasta = \"\"\n",
        "  for i, seq in enumerate(seqs):\n",
        "    fasta += f\">{prefix}_{i}\\n{seq}\\n\"\n",
        "  print(fasta, file=open(path, 'a'))\n",
        "\n",
        "def interpolate(starting, ending, steps):\n",
        "  \"\"\"\n",
        "  Interpolates between starting and end points. Steps parameter determines \n",
        "  how many interpolated points will be returned.\n",
        "  \"\"\"\n",
        "  points = [starting]\n",
        "  step = (ending-starting)/steps\n",
        "  for i in range(steps):\n",
        "    starting = starting + step\n",
        "    points.append(starting)\n",
        "  return np.asanyarray(points)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Q277ab8R9WEU"
      },
      "source": [
        "## Cytidine deaminase\n",
        "This demonstration will use a relatively small protein called *cytidine deaminase* for simplicity. Its function in organisms is essential to DNA and RNA degradation. **Our aim is to be able to create variants of this protein that exhibit different properties.**\n",
        " \n",
        "Below is an example of cytidine deaminase 3D structure.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 531
        },
        "id": "J7EFUwbbWy9X",
        "outputId": "f60ba6c1-5c64-4c73-e223-4388029743b4"
      },
      "outputs": [],
      "source": [
        "view = py3Dmol.view(query='pdb:1UX1')\n",
        "view.setStyle({'cartoon':{'color':'spectrum'}})\n",
        "print(\"Cytidine deaminase\")\n",
        "view"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hy4cQXYSqe7X"
      },
      "source": [
        "## Random amino acid sequence\n",
        "\n",
        "Let's consider a very naive approach first: choosing amino acids at random. As mentioned before, only a very tiny portion of amino acids can make up a protein with a desired function. So... what are the odds?\n",
        " \n",
        "There are around 17k annotated sequences that are categorized as cytidine deaminase: [see here](https://www.uniprot.org/uniprot/?query=ec%3A3.5.4.5+taxonomy%3A%22Bacteria+%5B2%5D%22+length%3A%5B64+TO+256%5D&sort=score)\n",
        " \n",
        "The protein length varies depending on the organism, but let's say you want to generate 131 length cytidine deaminase. So there are:  $20^{131}$ possible combinations (just for comparison: there are ~ $10^{80}$ atoms in the observable universe!) \n",
        " \n",
        "It's safe to say that random sequences are unlikely to work. Even brute forcing all combinations is not an option. Nevertheless, let's try to generate a sequence to see what happens. :)\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "id": "5X5tLlgIucAR",
        "outputId": "34bac72f-2d9a-455c-fda6-99e17def84f4"
      },
      "outputs": [],
      "source": [
        "np.random.seed(42)\n",
        "random_seq = \"\".join(np.random.choice(list(ID_TO_AMINO_ACID.values())[1:], 131))\n",
        "random_seq"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iwpDnHa7vLCv"
      },
      "source": [
        "Here, you see a 131 letter long amino acid sequence. It is hard to tell anything about this sequence only by looking. So instead, you can use a bioinformatics tool called Blast [(Basic Local Alignment Search Tool)](https://blast.ncbi.nlm.nih.gov/Blast.cgi) that searches a large database of known proteins to find the most similar matches. In most cases, a random sequence should not return any high-similarity results. \n",
        "\n",
        "If you do get anything returned, it should have a small _identity value_, which is the percentage of the sequence that matches. When the identity value is small, this means that only a small fragment of the sequence could be identified as a part of some random protein."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 49
        },
        "id": "nBoKiboYvG73",
        "outputId": "04aca09f-bac8-43e9-be8f-8d0ddc0174ff"
      },
      "outputs": [],
      "source": [
        "get_blast_results(random_seq)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_P-wnHQYjvqZ"
      },
      "source": [
        "## ProteinGAN sequences\n",
        "\n",
        "What if, instead, you train a GAN to generate desirable (realistic, reasonable, non-random) protein sequences? \n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "o6G2mA7vPVlc"
      },
      "outputs": [],
      "source": [
        "import tensorflow as tf\n",
        "tf.random.set_seed(42)\n",
        "from absl import logging\n",
        "logging.set_verbosity(\"ERROR\")\n",
        "tf.get_logger().setLevel(\"ERROR\")\n",
        "\n",
        "# Loading pre-trained model.\n",
        "model = tf.saved_model.load(\"pre_trained_protein_gan/\").signatures[\"serving_default\"]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "id": "EbC5FmhhD1vI",
        "outputId": "1daf4fc2-f98a-4bcc-b289-e8bcbf6d78f4"
      },
      "outputs": [],
      "source": [
        "# Choosing random points from latent space.\n",
        "noise = tf.random.truncated_normal([64, 128], stddev=0.5, dtype=tf.float32)\n",
        "\n",
        "# Feeding noise to generator to get an output.\n",
        "model_output = model(noise)\n",
        "\n",
        "# Model returns indices of amino acids. Here we convert them to actual letters.\n",
        "seqs = to_seqs(model_output)\n",
        "seqs[0]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fk_9Nn7FxnG0"
      },
      "source": [
        "Again, not much can be said about the sequence just by looking at it (unless you're a protein savant). Time to run BLAST again!"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 221
        },
        "id": "TLjZJMXmpaZx",
        "outputId": "805df8ea-cf9f-4070-f5b6-7e520d44529f"
      },
      "outputs": [],
      "source": [
        "get_blast_results(seqs[0])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fUKadK0Vx05y"
      },
      "source": [
        "Nice! This time, you got some matches that are either cytidine deaminase or other types of deaminase with a high indentity. This is a good indication that the GAN works well in generating realistic protein sequences."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JpAlWkRgvSJs"
      },
      "source": [
        "## Latent space\n",
        "\n",
        "As you already know, GANs learn to map points in the latent space to generated items. You can explore this latent space and perform a meaningful modifications to a generated item by moving in different directions. On generated faces, that might be changing hair color or adding sunglasses. Here, it's also to change something semantically meaningful, but for protein sequences.\n",
        "\n",
        "To start off, you can play with the diversity of generated sequences by changing how widely you sample the latent space. This can be achieved by modifying the standard deviation of the distribution. Let's try 0.1 and 1.0 to start!"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Ls7MUljzvRwG"
      },
      "outputs": [],
      "source": [
        "# Generating sequences from points which are close to each other\n",
        "model_output = model(tf.random.truncated_normal([64, 128], stddev=0.1, dtype=tf.float32))\n",
        "small_var_seqs = to_seqs(model_output)\n",
        "\n",
        "# Generating sequences more distrbuted points\n",
        "model_output = model(tf.random.truncated_normal([64, 128], stddev=1.0, dtype=tf.float32))\n",
        "large_var_seqs = to_seqs(model_output)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "J-Ns4dAgRYXh"
      },
      "outputs": [],
      "source": [
        "# Creating fasta files which will be used for clustalo to calculate distances\n",
        "#pre_trained_protein_gan/train_rep.fasta - contains some representative sequences of training dataset\n",
        "shutil.copy(\"pre_trained_protein_gan/train_rep.fasta\",\"sequences.fasta\")\n",
        "#Appending generated sequences to training sequences\n",
        "append_to_fasta(\"sequences.fasta\", small_var_seqs, \"small_var\")\n",
        "append_to_fasta(\"sequences.fasta\", large_var_seqs, \"large_var\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xiykPRgYh4OC"
      },
      "source": [
        "[Clustalo](http://www.clustal.org/omega/) is a bioinformatics tool for biological sequence alignment and comparison that calculates the edit distances between multiple strings, taking into account that some letters are more similar than others biologically. You can use it to calculate all-to-all distances from different protein sequence sets - training representatives, sequences generated using low and high standard deviation."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "7D-4DqB_FcV1",
        "outputId": "8e1ecd11-2459-41fa-f7d4-9744bb603cc4"
      },
      "outputs": [],
      "source": [
        "! clustalo -i sequences.fasta -o fasta.aln --threads=2 -v --full --distmat-out=dist_out.dist --force"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "NIm7_vnPcKe4"
      },
      "outputs": [],
      "source": [
        "from sklearn.manifold import TSNE\n",
        "\n",
        "#Loading calculated distances\n",
        "distance_matrix = pd.read_csv(\"dist_out.dist\", delimiter='\\s+', skiprows=[0],header=None,index_col=0)\n",
        "distance_matrix.columns = distance_matrix.index.values\n",
        "\n",
        "#Using TSNE to compress all pair wise distances between sequences into two components which then could be plotted.\n",
        "tsne = TSNE(n_components=2, metric='precomputed', init='random')\n",
        "coordinates_2d = tsne.fit_transform(distance_matrix.values)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 483
        },
        "id": "a9gIgVbNJrBW",
        "outputId": "fabbe86e-324b-4881-83d6-e645587820fe"
      },
      "outputs": [],
      "source": [
        "from matplotlib import pyplot as plt\n",
        "\n",
        "\n",
        "# Plotting train representatives and generated sequences with different diversity\n",
        "plt.figure(figsize=(12, 8))\n",
        "plt.scatter(coordinates_2d[:-128,0], coordinates_2d[:-128,1], c=\"green\", label=\"Train representative sequences\", alpha=0.5, s=30)\n",
        "small_var_el = distance_matrix.index.str.contains(\"small_var\")\n",
        "plt.scatter(coordinates_2d[small_var_el,0], coordinates_2d[small_var_el,1], c=\"orange\", label=\"Generated sequences with 0.1 standard deviation\")\n",
        "large_var_el = distance_matrix.index.str.contains(\"large_var\")\n",
        "plt.scatter(coordinates_2d[large_var_el,0], coordinates_2d[large_var_el,1], c=\"red\", label=\"Generated sequences with 1.0 standard deviation \")\n",
        "plt.legend()\n",
        "plt.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nSFhBtY3NhQR"
      },
      "source": [
        "As expected, oranges sequences are more similar to each other than the red ones. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Y62ZHV1SUqYA"
      },
      "source": [
        "### Controlling biological properties\n",
        "\n",
        "After generating realistic sequences, you want to be able to control their properties. As with images, it's possible to find a direction in the latent space that will change a specific property of the generated outcome. Here, you can vary values of the 100th dimension and measure the molecular weight of generated sequences. You'll use the [biopython](https://biopython.org/) library to calculate the molecule's weight. "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "i3osQT59e-JT"
      },
      "outputs": [],
      "source": [
        "from scipy.stats import pearsonr\n",
        "from Bio.SeqUtils.ProtParam import ProteinAnalysis\n",
        "\n",
        "# Changing the values of 100th dimension from -1.0 to 1.0\n",
        "d = 99 \n",
        "starting = np.zeros([128])\n",
        "starting[d] = -1.0\n",
        "ending = np.zeros([128])\n",
        "ending[d] = 1.0\n",
        "points = interpolate(starting, ending, 1023)\n",
        "\n",
        "seqs = []\n",
        "for i in range(0, 1024, 64):\n",
        "  model_output = model(tf.constant(points[i:i+64], tf.float32))\n",
        "  seqs.extend(to_seqs(model_output))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dNvRNj2vVTFh"
      },
      "source": [
        "Then, you can calculate the molecular weight of each sequence and calculate the correlation with latent space direction."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "mmxAP-WwUZ8q",
        "outputId": "e7803654-673c-44c6-d40b-fb8a2f3cdf9e"
      },
      "outputs": [],
      "source": [
        "w = [ProteinAnalysis(s).molecular_weight() for s in seqs] \n",
        "pearsonr(w, points[:,d])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 502
        },
        "id": "JHeiy1y9hFfp",
        "outputId": "d898af5d-2f2d-4c79-d5ac-fb4e6686d5ae"
      },
      "outputs": [],
      "source": [
        "plt.figure(figsize=(16, 8))\n",
        "plt.scatter(points[:,d], w, c = 'b', s = 20, label = 'Molecule weight')\n",
        "plt.xlabel(\"Latent dimension value\", fontsize = 15)\n",
        "plt.ylabel(\"Molecular weight\", fontsize = 15)\n",
        "plt.legend(fontsize = 14)\n",
        "plt.grid(True)\n",
        "plt.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "937OnQVxVjKh"
      },
      "source": [
        "Of course, this is a very simplistic example; it is a good illustrative example of how latent space can be explored.\n",
        "\n",
        "\n",
        "## Summary\n",
        "\n",
        "In summary, you have learned about:\n",
        "\n",
        "* Proteins as non-random sequences of 20 amino acids (aa) that nature has tweaked over billions of years of evolution to drive essential life processes;\n",
        "\n",
        "*  ProteinGAN and its technical features outlining the challenges of learning long-biological sequences such as proteins;\n",
        "\n",
        "* Generating random protein sequences from a family of cytidine deaminases using a generator from a pre-trained ProteinGAN model;\n",
        "\n",
        "* Visualizing biological sequences using sequence alignments and dimensionality reduction;\n",
        "\n",
        "* Exploring latent space dimensions and connecting it with physicochemical properties of generated proteins.\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_kH5E53bgFGC"
      },
      "source": [
        "## Author's Contribution\n",
        "\n",
        "[Donatas Repečka](https://www.linkedin.com/in/donatasrep/) (Biomatter Designs) was responsible for the notebook's content and design; \n",
        "\n",
        "[Aleksej Zelezniak](https://twitter.com/AZelezniak)  (Zelezniak lab at Chalmers University of Technology) gave input into summarizing and editing the text.\n",
        "\n",
        "## Acknowledgment\n",
        "The authors would like to thank [Biomatter Designs](https://www.biomatterdesigns.com/) and [DeepLearning.AI](https://www.deeplearning.ai/) teams for their comments and insightful suggestions:\n",
        "\n",
        "* [Vykintas Jauniškis](https://www.linkedin.com/in/vykintas-jauniskis/) (Biomatter Designs);\n",
        "* [Laurynas Karpus](https://www.linkedin.com/in/laurynaskarpus/) (Biomatter Designs);\n",
        "* [Audrius Laurynėnas](https://www.linkedin.com/in/audrius-lauryn%C4%97nas-307687b2/) (Biomatter Designs);\n",
        "* [Aurimas Repečka](https://www.linkedin.com/in/aurimas-repe%C4%8Dka-23064ab2/) (Biomatter Designs);\n",
        "* [Irmantas Rokaitis](https://www.linkedin.com/in/irmantas-rokaitis-52336b18b/) (Biomatter Designs);\n",
        "* [Audronė Valančiūtė](https://www.linkedin.com/in/audron%C4%97-valan%C4%8Di%C5%ABt%C4%97-730785158/) (Biomatter Designs);\n",
        "* [Antanas Žilakauskis](https://www.linkedin.com/in/zilakauskis95/) (Biomatter Designs).\n"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "collapsed_sections": [],
      "include_colab_link": true,
      "name": "C1W3: ProteinGAN (Optional)",
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.10.13"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}