test.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Multiple Questions Generation on the given text using NLP"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Importing all the necessary libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[nltk_data] Downloading package punkt to\n",
      "[nltk_data]     C:\\Users\\smsha\\AppData\\Roaming\\nltk_data...\n",
      "[nltk_data]   Package punkt is already up-to-date!\n",
      "[nltk_data] Downloading package brown to\n",
      "[nltk_data]     C:\\Users\\smsha\\AppData\\Roaming\\nltk_data...\n",
      "[nltk_data]   Package brown is already up-to-date!\n",
      "[nltk_data] Downloading package wordnet to\n",
      "[nltk_data]     C:\\Users\\smsha\\AppData\\Roaming\\nltk_data...\n",
      "[nltk_data]   Package wordnet is already up-to-date!\n",
      "[nltk_data] Downloading package stopwords to\n",
      "[nltk_data]     C:\\Users\\smsha\\AppData\\Roaming\\nltk_data...\n",
      "[nltk_data]   Package stopwords is already up-to-date!\n",
      "[nltk_data] Downloading package omw-1.4 to\n",
      "[nltk_data]     C:\\Users\\smsha\\AppData\\Roaming\\nltk_data...\n",
      "[nltk_data]   Package omw-1.4 is already up-to-date!\n"
     ]
    }
   ],
   "source": [
    "#import all the neccessary libraries\n",
    "import warnings\n",
    "warnings.filterwarnings(\"ignore\")\n",
    "import torch\n",
    "from transformers import T5ForConditionalGeneration,T5Tokenizer\n",
    "from sense2vec import Sense2Vec\n",
    "from sentence_transformers import SentenceTransformer\n",
    "from textwrap3 import wrap\n",
    "import random\n",
    "import numpy as np\n",
    "import nltk\n",
    "nltk.download('punkt')\n",
    "nltk.download('brown')\n",
    "nltk.download('wordnet')\n",
    "from nltk.corpus import wordnet as wn\n",
    "from nltk.tokenize import sent_tokenize\n",
    "nltk.download('stopwords')\n",
    "from nltk.corpus import stopwords\n",
    "import string\n",
    "import pke\n",
    "import traceback\n",
    "from flashtext import KeywordProcessor\n",
    "from collections import OrderedDict\n",
    "from sklearn.metrics.pairwise import cosine_similarity\n",
    "nltk.download('omw-1.4')\n",
    "from strsimpy.normalized_levenshtein import NormalizedLevenshtein\n",
    "import pickle\n",
    "import time\n",
    "import os "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Checking for GPU availability"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cpu\n"
     ]
    }
   ],
   "source": [
    "import torch\n",
    "device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n",
    "print(device)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Getting the pretrained models and tokenizers from disc:\n",
    "1. Sense2vec model trained on reddit 2019 data.\n",
    "2. T5-base pretrained summarizer model.\n",
    "3. Pretrained T5 question generator model trained on Squad-V1 dataset.\n",
    "4. Msmarco-distilbert-base-v2 sentence transformer pretrained model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Summary model does not exists in the path specified, downloading the model from web....\n",
      "downloaded the summary model in  0.0471060315767924  min , now saving it to disc...\n",
      "Done. Saved the model to disc.\n",
      "Summary tokenizer does not exists in the path specified, downloading the model from web....\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "You are using the default legacy behaviour of the <class 'transformers.models.t5.tokenization_t5.T5Tokenizer'>. This is expected, and simply means that the `legacy` (previous) behavior will be used so nothing changes for you. If you want to use the new behaviour, set `legacy=False`. This should only be set if you understand what it means, and thoroughly read the reason why this was added as explained in https://github.com/huggingface/transformers/pull/24565\n",
      "Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "downloaded the summary tokenizer in  0.017959733804066975  min , now saving it to disc...\n",
      "Done. Saved the tokenizer to disc.\n",
      "Question model does not exists in the path specified, downloading the model from web....\n",
      "downloaded the question model in  0.03083808422088623  min , now saving it to disc...\n",
      "Done. Saved the model to disc.\n",
      "Question tokenizer does not exists in the path specified, downloading the model from web....\n",
      "downloaded the question tokenizer in  0.006409756342569987  min , now saving it to disc...\n",
      "Done. Saved the tokenizer to disc.\n",
      "Sentence transformer model does not exists in the path specified, downloading the model from web....\n",
      "downloaded the sentence transformer in  0.047541892528533934  min , now saving it to disc...\n",
      "Done saving to disc.\n"
     ]
    }
   ],
   "source": [
    "#we need to download the 2015 trained on reddit sense2vec model as it is shown to give better results than the 2019 one.\n",
    "s2v = Sense2Vec().from_disk('s2v_old')\n",
    "\n",
    "#getitng the summary model and its tokenizer\n",
    "if os.path.exists(\"./models/t5_summary_model.pkl\"):\n",
    "    with open('./models/t5_summary_model.pkl', 'rb') as f:\n",
    "        summary_model = pickle.load(f)\n",
    "    print(\"Summary model found in the disc, model is loaded successfully.\")\n",
    "\n",
    "else:\n",
    "    print(\"Summary model does not exists in the path specified, downloading the model from web....\")\n",
    "    start_time = time.time()\n",
    "    summary_model = T5ForConditionalGeneration.from_pretrained('t5-base')\n",
    "    end_time = time.time()\n",
    "\n",
    "    print(\"downloaded the summary model in \",(end_time-start_time)/60,\" min , now saving it to disc...\")\n",
    "\n",
    "    with open(\"t5_summary_model.pkl\", 'wb') as f:\n",
    "        pickle.dump(summary_model,f)\n",
    "    \n",
    "    print(\"Done. Saved the model to disc.\")\n",
    "\n",
    "if os.path.exists(\"./models/t5_summary_tokenizer.pkl\"):\n",
    "    with open('./models/t5_summary_tokenizer.pkl', 'rb') as f:\n",
    "        summary_tokenizer = pickle.load(f)\n",
    "    print(\"Summary tokenizer found in the disc and is loaded successfully.\")\n",
    "else: \n",
    "    print(\"Summary tokenizer does not exists in the path specified, downloading the model from web....\")\n",
    "\n",
    "    start_time = time.time()\n",
    "    summary_tokenizer = T5Tokenizer.from_pretrained('t5-base')\n",
    "    end_time = time.time()\n",
    "\n",
    "    print(\"downloaded the summary tokenizer in \",(end_time-start_time)/60,\" min , now saving it to disc...\")\n",
    "\n",
    "    with open(\"t5_summary_tokenizer.pkl\",'wb') as f:\n",
    "        pickle.dump(summary_tokenizer,f)\n",
    "\n",
    "    print(\"Done. Saved the tokenizer to disc.\")\n",
    "\n",
    "\n",
    "#Getting question model and tokenizer\n",
    "if os.path.exists(\"./models/t5_question_model.pkl\"):\n",
    "    with open('./models/t5_question_model.pkl', 'rb') as f:\n",
    "        question_model = pickle.load(f)\n",
    "    print(\"Question model found in the disc, model is loaded successfully.\")\n",
    "else:\n",
    "    print(\"Question model does not exists in the path specified, downloading the model from web....\")\n",
    "    start_time= time.time()\n",
    "    question_model = T5ForConditionalGeneration.from_pretrained('ramsrigouthamg/t5_squad_v1')\n",
    "    end_time = time.time()\n",
    "\n",
    "    print(\"downloaded the question model in \",(end_time-start_time)/60,\" min , now saving it to disc...\")\n",
    "\n",
    "    with open(\"t5_question_model.pkl\", 'wb') as f:\n",
    "        pickle.dump(question_model,f)\n",
    "    \n",
    "    print(\"Done. Saved the model to disc.\")\n",
    "\n",
    "if os.path.exists(\"./models/t5_question_tokenizer.pkl\"):\n",
    "    with open('./models/t5_question_tokenizer.pkl', 'rb') as f:\n",
    "        question_tokenizer = pickle.load(f)\n",
    "    print(\"Question tokenizer found in the disc, model is loaded successfully.\")\n",
    "else:\n",
    "    print(\"Question tokenizer does not exists in the path specified, downloading the model from web....\")\n",
    "\n",
    "    start_time = time.time()\n",
    "    question_tokenizer = T5Tokenizer.from_pretrained('ramsrigouthamg/t5_squad_v1')\n",
    "    end_time=time.time()\n",
    "\n",
    "    print(\"downloaded the question tokenizer in \",(end_time-start_time)/60,\" min , now saving it to disc...\")\n",
    "\n",
    "    with open(\"t5_question_tokenizer.pkl\",'wb') as f:\n",
    "        pickle.dump(question_tokenizer,f)\n",
    "\n",
    "    print(\"Done. Saved the tokenizer to disc.\")\n",
    "\n",
    "#Loading the models in to GPU if available\n",
    "summary_model = summary_model.to(device)\n",
    "question_model = question_model.to(device)\n",
    "\n",
    "#Getting the sentence transformer model and its tokenizer\n",
    "# paraphrase-distilroberta-base-v1\n",
    "if os.path.exists(\"./models/sentence_transformer_model.pkl\"):\n",
    "    with open(\"./models/sentence_transformer_model.pkl\",'rb') as f:\n",
    "        sentence_transformer_model = pickle.load(f)\n",
    "    print(\"Sentence transformer model found in the disc, model is loaded successfully.\")\n",
    "else:\n",
    "    print(\"Sentence transformer model does not exists in the path specified, downloading the model from web....\")\n",
    "    start_time=time.time()\n",
    "    sentence_transformer_model = SentenceTransformer(\"sentence-transformers/msmarco-distilbert-base-v2\")\n",
    "    end_time=time.time()\n",
    "\n",
    "    print(\"downloaded the sentence transformer in \",(end_time-start_time)/60,\" min , now saving it to disc...\")\n",
    "\n",
    "    with open(\"sentence_transformer_model.pkl\",'wb') as f:\n",
    "        pickle.dump(sentence_transformer_model,f)\n",
    "\n",
    "    print(\"Done saving to disc.\")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<img src=\"https://miro.medium.com/v2/resize:fit:1100/format:webp/1*iJcUH1F0TmCQE5p2wQt9og.png\"/>"
      ],
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from IPython.display import Image \n",
    "  \n",
    "# get the image \n",
    "Image(url=\"https://miro.medium.com/v2/resize:fit:1100/format:webp/1*iJcUH1F0TmCQE5p2wQt9og.png\") "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Defining all the utility functions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "def set_seed(seed: int):\n",
    "    random.seed(seed)\n",
    "    np.random.seed(seed)\n",
    "    torch.manual_seed(seed)\n",
    "    torch.cuda.manual_seed_all(seed)\n",
    "\n",
    "def postprocesstext (content):\n",
    "  \"\"\"\n",
    "  this function takes a piece of text (content), tokenizes it into sentences, capitalizes the first letter of each sentence, and then concatenates the processed sentences into a single string, which is returned as the final result. The purpose of this function could be to format the input content by ensuring that each sentence starts with an uppercase letter.\n",
    "  \"\"\"\n",
    "  final=\"\"\n",
    "  for sent in sent_tokenize(content):\n",
    "    sent = sent.capitalize()\n",
    "    final = final +\" \"+sent\n",
    "  return final\n",
    "\n",
    "def summarizer(text,model,tokenizer):\n",
    "  \"\"\"\n",
    "  This function takes the given text along with the model and tokenizer, which summarize the large text into useful information\n",
    "  \"\"\"\n",
    "  text = text.strip().replace(\"\\n\",\" \")\n",
    "  text = \"summarize: \"+text\n",
    "  # print (text)\n",
    "  max_len = 512\n",
    "  encoding = tokenizer.encode_plus(text,max_length=max_len, pad_to_max_length=False,truncation=True, return_tensors=\"pt\").to(device)\n",
    "\n",
    "  input_ids, attention_mask = encoding[\"input_ids\"], encoding[\"attention_mask\"]\n",
    "\n",
    "  outs = model.generate(input_ids=input_ids,\n",
    "                                  attention_mask=attention_mask,\n",
    "                                  early_stopping=True,\n",
    "                                  num_beams=3,\n",
    "                                  num_return_sequences=1,\n",
    "                                  no_repeat_ngram_size=2,\n",
    "                                  min_length = 75,\n",
    "                                  max_length=300)\n",
    "\n",
    "  dec = [tokenizer.decode(ids,skip_special_tokens=True) for ids in outs]\n",
    "  summary = dec[0]\n",
    "  summary = postprocesstext(summary)\n",
    "  summary= summary.strip()\n",
    "\n",
    "  return summary\n",
    "\n",
    "def get_nouns_multipartite(content):\n",
    "    \"\"\"\n",
    "    This function takes the content text given and then outputs the phrases which are build around the nouns , so that we can use them for context based distractors\n",
    "    \"\"\"\n",
    "    out=[]\n",
    "    try:\n",
    "        extractor = pke.unsupervised.MultipartiteRank()\n",
    "        extractor.load_document(input=content,language='en')\n",
    "        #    not contain punctuation marks or stopwords as candidates.\n",
    "        #pos = {'PROPN','NOUN',}\n",
    "        pos = {'PROPN', 'NOUN', 'ADJ', 'VERB', 'ADP', 'ADV', 'DET', 'CONJ', 'NUM', 'PRON', 'X'}\n",
    "\n",
    "        #pos = {'PROPN','NOUN'}\n",
    "        stoplist = list(string.punctuation)\n",
    "        stoplist += ['-lrb-', '-rrb-', '-lcb-', '-rcb-', '-lsb-', '-rsb-']\n",
    "        stoplist += stopwords.words('english')\n",
    "        # extractor.candidate_selection(pos=pos, stoplist=stoplist)\n",
    "        extractor.candidate_selection( pos=pos)\n",
    "        # 4. build the Multipartite graph and rank candidates using random walk,\n",
    "        #    alpha controls the weight adjustment mechanism, see TopicRank for\n",
    "        #    threshold/method parameters.\n",
    "        extractor.candidate_weighting(alpha=1.1,\n",
    "                                      threshold=0.75,\n",
    "                                      method='average')\n",
    "        keyphrases = extractor.get_n_best(n=15)\n",
    "        \n",
    "\n",
    "        for val in keyphrases:\n",
    "            out.append(val[0])\n",
    "    except:\n",
    "        out = []\n",
    "        #traceback.print_exc()\n",
    "\n",
    "    return out\n",
    "\n",
    "def get_keywords(originaltext):\n",
    "  \"\"\"\n",
    "  This function takes the original text and the summary text and generates keywords from both which ever are more relevant\n",
    "  This is done by checking the keywords generated from the original text to those generated from the summary, so that we get important ones \n",
    "  \"\"\"\n",
    "  keywords = get_nouns_multipartite(originaltext)\n",
    "  #print (\"keywords unsummarized: \",keywords)\n",
    "  #keyword_processor = KeywordProcessor()\n",
    "  #for keyword in keywords:\n",
    "    #keyword_processor.add_keyword(keyword)\n",
    "\n",
    "  #keywords_found = keyword_processor.extract_keywords(summarytext)\n",
    "  #keywords_found = list(set(keywords_found))\n",
    "  #print (\"keywords_found in summarized: \",keywords_found)\n",
    "\n",
    "  #important_keywords =[]\n",
    "  #for keyword in keywords:\n",
    "    #if keyword in keywords_found:\n",
    "      #important_keywords.append(keyword)\n",
    "\n",
    "  #return important_keywords\n",
    "  return keywords\n",
    "\n",
    "def get_question(context,answer,model,tokenizer):\n",
    "  \"\"\"\n",
    "  This function takes the input context text, pretrained model along with the tokenizer and the keyword and the answer and then generates the question from the large paragraph\n",
    "  \"\"\"\n",
    "  text = \"context: {} answer: {}\".format(context,answer)\n",
    "  encoding = tokenizer.encode_plus(text,max_length=384, pad_to_max_length=False,truncation=True, return_tensors=\"pt\").to(device)\n",
    "  input_ids, attention_mask = encoding[\"input_ids\"], encoding[\"attention_mask\"]\n",
    "\n",
    "  outs = model.generate(input_ids=input_ids,\n",
    "                                  attention_mask=attention_mask,\n",
    "                                  early_stopping=True,\n",
    "                                  num_beams=5,\n",
    "                                  num_return_sequences=1,\n",
    "                                  no_repeat_ngram_size=2,\n",
    "                                  max_length=72)\n",
    "\n",
    "\n",
    "  dec = [tokenizer.decode(ids,skip_special_tokens=True) for ids in outs]\n",
    "\n",
    "\n",
    "  Question = dec[0].replace(\"question:\",\"\")\n",
    "  Question= Question.strip()\n",
    "  return Question\n",
    "\n",
    "def filter_same_sense_words(original,wordlist):\n",
    "  \n",
    "  \"\"\"\n",
    "  This is used to filter the words which are of same sense, where it takes the wordlist which has the sense of the word attached as the string along with the word itself.\n",
    "  \"\"\"\n",
    "  filtered_words=[]\n",
    "  base_sense =original.split('|')[1] \n",
    "  #print (base_sense)\n",
    "  for eachword in wordlist:\n",
    "    if eachword[0].split('|')[1] == base_sense:\n",
    "      filtered_words.append(eachword[0].split('|')[0].replace(\"_\", \" \").title().strip())\n",
    "  return filtered_words\n",
    "\n",
    "def get_highest_similarity_score(wordlist,wrd):\n",
    "  \"\"\"\n",
    "  This function takes the given word along with the wordlist and then gives out the max-score which is the levenshtein distance for the wrong answers\n",
    "  because we need the options which are very different from one another but relating to the same context.\n",
    "  \"\"\"\n",
    "  score=[]\n",
    "  normalized_levenshtein = NormalizedLevenshtein()\n",
    "  for each in wordlist:\n",
    "    score.append(normalized_levenshtein.similarity(each.lower(),wrd.lower()))\n",
    "  return max(score)\n",
    "\n",
    "def sense2vec_get_words(word,s2v,topn,question):\n",
    "    \"\"\"\n",
    "    This function takes the input word, sentence to vector model and top similar words and also the question\n",
    "    Then it computes the sense of the given word\n",
    "    then it gets the words which are of same sense but are most similar to the given word\n",
    "    after that we we return the list of words which satisfy the above mentioned criteria\n",
    "    \"\"\"\n",
    "    output = []\n",
    "    #print (\"word \",word)\n",
    "    try:\n",
    "      sense = s2v.get_best_sense(word, senses= [\"NOUN\", \"PERSON\",\"PRODUCT\",\"LOC\",\"ORG\",\"EVENT\",\"NORP\",\"WORK OF ART\",\"FAC\",\"GPE\",\"NUM\",\"FACILITY\"])\n",
    "      most_similar = s2v.most_similar(sense, n=topn)\n",
    "      # print (most_similar)\n",
    "      output = filter_same_sense_words(sense,most_similar)\n",
    "      #print (\"Similar \",output)\n",
    "    except:\n",
    "      output =[]\n",
    "\n",
    "    threshold = 0.6\n",
    "    final=[word]\n",
    "    checklist =question.split()\n",
    "    for x in output:\n",
    "      if get_highest_similarity_score(final,x)<threshold and x not in final and x not in checklist:\n",
    "        final.append(x)\n",
    "    \n",
    "    return final[1:]\n",
    "\n",
    "def mmr(doc_embedding, word_embeddings, words, top_n, lambda_param):\n",
    "    \"\"\"\n",
    "    The mmr function takes document and word embeddings, along with other parameters, and uses the Maximal Marginal Relevance (MMR) algorithm to extract a specified number of keywords/keyphrases from the document. The MMR algorithm balances the relevance of keywords with their diversity, helping to select keywords that are both informative and distinct from each other.\n",
    "    \"\"\"\n",
    "\n",
    "    # Extract similarity within words, and between words and the document\n",
    "    word_doc_similarity = cosine_similarity(word_embeddings, doc_embedding)\n",
    "    word_similarity = cosine_similarity(word_embeddings)\n",
    "\n",
    "    # Initialize candidates and already choose best keyword/keyphrase\n",
    "    keywords_idx = [np.argmax(word_doc_similarity)]\n",
    "    candidates_idx = [i for i in range(len(words)) if i != keywords_idx[0]]\n",
    "\n",
    "    for _ in range(top_n - 1):\n",
    "        # Extract similarities within candidates and\n",
    "        # between candidates and selected keywords/phrases\n",
    "        candidate_similarities = word_doc_similarity[candidates_idx, :]\n",
    "        target_similarities = np.max(word_similarity[candidates_idx][:, keywords_idx], axis=1)\n",
    "\n",
    "        # Calculate MMR\n",
    "        mmr = (lambda_param) * candidate_similarities - (1-lambda_param) * target_similarities.reshape(-1, 1)\n",
    "        mmr_idx = candidates_idx[np.argmax(mmr)]\n",
    "\n",
    "        # Update keywords & candidates\n",
    "        keywords_idx.append(mmr_idx)\n",
    "        candidates_idx.remove(mmr_idx)\n",
    "\n",
    "    return [words[idx] for idx in keywords_idx]\n",
    "\n",
    "def get_distractors_wordnet(word):\n",
    "    \"\"\"\n",
    "    the get_distractors_wordnet function uses WordNet to find a relevant synset for the input word and then generates distractor words by looking at hyponyms of the hypernym associated with the input word. These distractors are alternative words related to the input word and can be used, for example, in educational or language-related applications to provide choices for a given word.\n",
    "    \"\"\"\n",
    "    distractors=[]\n",
    "    try:\n",
    "      syn = wn.synsets(word,'n')[0]\n",
    "      \n",
    "      word= word.lower()\n",
    "      orig_word = word\n",
    "      if len(word.split())>0:\n",
    "          word = word.replace(\" \",\"_\")\n",
    "      hypernym = syn.hypernyms()\n",
    "      if len(hypernym) == 0: \n",
    "          return distractors\n",
    "      for item in hypernym[0].hyponyms():\n",
    "          name = item.lemmas()[0].name()\n",
    "          #print (\"name \",name, \" word\",orig_word)\n",
    "          if name == orig_word:\n",
    "              continue\n",
    "          name = name.replace(\"_\",\" \")\n",
    "          name = \" \".join(w.capitalize() for w in name.split())\n",
    "          if name is not None and name not in distractors:\n",
    "              distractors.append(name)\n",
    "    except:\n",
    "      print (\"Wordnet distractors not found\")\n",
    "    return distractors\n",
    "\n",
    "def get_distractors (word,origsentence,sense2vecmodel,sentencemodel,top_n,lambdaval):\n",
    "  \"\"\"\n",
    "  this function generates distractor words (answer choices) for a given target word in the context of a provided sentence. It selects distractors based on their similarity to the target word's context and ensures that the target word itself is not included among the distractors. This function is useful for creating multiple-choice questions or answer options in natural language processing tasks.\n",
    "  \"\"\"\n",
    "  distractors = sense2vec_get_words(word,sense2vecmodel,top_n,origsentence)\n",
    "  #print (\"distractors \",distractors)\n",
    "  if len(distractors) ==0:\n",
    "    return distractors\n",
    "  distractors_new = [word.capitalize()]\n",
    "  distractors_new.extend(distractors)\n",
    "  # print (\"distractors_new .. \",distractors_new)\n",
    "\n",
    "  embedding_sentence = origsentence+ \" \"+word.capitalize()\n",
    "  # embedding_sentence = word\n",
    "  keyword_embedding = sentencemodel.encode([embedding_sentence])\n",
    "  distractor_embeddings = sentencemodel.encode(distractors_new)\n",
    "\n",
    "  # filtered_keywords = mmr(keyword_embedding, distractor_embeddings,distractors,4,0.7)\n",
    "  max_keywords = min(len(distractors_new),5)\n",
    "  filtered_keywords = mmr(keyword_embedding, distractor_embeddings,distractors_new,max_keywords,lambdaval)\n",
    "  # filtered_keywords = filtered_keywords[1:]\n",
    "  final = [word.capitalize()]\n",
    "  for wrd in filtered_keywords:\n",
    "    if wrd.lower() !=word.lower():\n",
    "      final.append(wrd.capitalize())\n",
    "  final = final[1:]\n",
    "  return final\n",
    "\n",
    "def get_mca_questions(context: str):\n",
    "    \"\"\"\n",
    "    this function generates multiple-choice questions based on a given context. It summarizes the context, extracts important keywords, generates questions related to those keywords, and provides randomized answer choices, including the correct answer, for each question.\n",
    "    \"\"\"\n",
    "    summarized_text = summarizer(context,summary_model,summary_tokenizer)\n",
    "\n",
    "    #imp_keywords = get_keywords(context ,summarized_text)\n",
    "    imp_keywords = get_keywords(context)\n",
    "    output_list=[]\n",
    "    for answer in imp_keywords:\n",
    "      output=\"\"\n",
    "      ques = get_question(summarized_text,answer,question_model,question_tokenizer)\n",
    "\n",
    "      distractors = get_distractors(answer.capitalize(),ques,s2v,sentence_transformer_model,40,0.2)\n",
    "\n",
    "      output = output + ques + \"\\n\"\n",
    "      if len(distractors) == 0:\n",
    "         distractors=imp_keywords\n",
    "\n",
    "      if len(distractors)>0:\n",
    "        random_integer = random.randint(0, 3)\n",
    "        alpha_list = ['(a)','(b)','(c)','(d)']\n",
    "        for d,distractor in enumerate(distractors[:4]):\n",
    "            if d == random_integer:\n",
    "               output = output + alpha_list[d] + answer + \"\\n\"\n",
    "            else:\n",
    "              output = output + alpha_list[d] + distractor + \"\\n\"\n",
    "        output = output + \"Correct answer is : \" + alpha_list[random_integer] + \"\\n\\n\"\n",
    "      \n",
    "      output_list.append(output)\n",
    "\n",
    "    mca_questions = output_list\n",
    "    return mca_questions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Testing the solution on an example text"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Example: 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "What is the name of the illustrious italian manufacturer of luxury sports cars?\n",
      "(a)automobili lamborghini\n",
      "(b)automobili lamborghini\n",
      "(c)lamborghini\n",
      "(d)groundbreaking innovations\n",
      "Correct answer is : (b)\n",
      "\n",
      "\n",
      "Lamborghini manufactures luxury sports cars and what else?\n",
      "(a)Bikes\n",
      "(b)suvs\n",
      "(c)Station wagons\n",
      "(d)Hatchbacks\n",
      "Correct answer is : (b)\n",
      "\n",
      "\n",
      "What is the illustrious italian manufacturer of luxury sports cars and suvs?\n",
      "(a)Lexus\n",
      "(b)Bmw\n",
      "(c)lamborghini\n",
      "(d)Maserati\n",
      "Correct answer is : (c)\n",
      "\n",
      "\n",
      "What has led to the evolution of Lamborghini?\n",
      "(a)automobili lamborghini\n",
      "(b)groundbreaking innovations\n",
      "(c)lamborghini\n",
      "(d)groundbreaking innovations\n",
      "Correct answer is : (b)\n",
      "\n",
      "\n",
      "What is the illustrious italian manufacturer of luxury sports cars and suvs?\n",
      "(a)automobili lamborghini\n",
      "(b)ferruccio lamborghini\n",
      "(c)lamborghini\n",
      "(d)groundbreaking innovations\n",
      "Correct answer is : (b)\n",
      "\n",
      "\n",
      "Along with ownership changes, innovations and groundbreaking innovations, what has influenced the evolution of Lamborghini?\n",
      "(a)economic downturns\n",
      "(b)Gfc\n",
      "(c)Capital flight\n",
      "(d)Increased unemployment\n",
      "Correct answer is : (a)\n",
      "\n",
      "\n",
      "What type of performance is lamborghini known for?\n",
      "(a)automobili lamborghini\n",
      "(b)suvs\n",
      "(c)lamborghini\n",
      "(d)formidable performance\n",
      "Correct answer is : (d)\n",
      "\n",
      "\n",
      "What were lamborghini's first models called?\n",
      "(a)inaugural models\n",
      "(b)suvs\n",
      "(c)lamborghini\n",
      "(d)groundbreaking innovations\n",
      "Correct answer is : (a)\n",
      "\n",
      "\n",
      "What is the style of Lamborghini?\n",
      "(a)audacious\n",
      "(b)suvs\n",
      "(c)lamborghini\n",
      "(d)groundbreaking innovations\n",
      "Correct answer is : (a)\n",
      "\n",
      "\n",
      "What type of investment did mycom setdco and indonesian group acquire Lamborghini?\n",
      "(a)Ministry\n",
      "(b)Expert advice\n",
      "(c)guidance\n",
      "(d)Understanding\n",
      "Correct answer is : (c)\n",
      "\n",
      "\n",
      "How long has Lamborghini been in business?\n",
      "(a)Blowout\n",
      "(b)Listening\n",
      "(c)Router\n",
      "(d)established\n",
      "Correct answer is : (d)\n",
      "\n",
      "\n",
      "In what years did Lamborghini's history begin?\n",
      "(a)Stint\n",
      "(b)College days\n",
      "(c)Mid-90s\n",
      "(d)early years\n",
      "Correct answer is : (d)\n",
      "\n",
      "\n",
      "What type of driving position does lamborghini have?\n",
      "(a)automobili lamborghini\n",
      "(b)suvs\n",
      "(c)lamborghini\n",
      "(d)rear-wheel-drive layout\n",
      "Correct answer is : (d)\n",
      "\n",
      "\n",
      "What is the hallmark of Lamborghini?\n",
      "(a)uncompromising quality\n",
      "(b)suvs\n",
      "(c)lamborghini\n",
      "(d)groundbreaking innovations\n",
      "Correct answer is : (a)\n",
      "\n",
      "\n",
      "How is Lamborghini headquartered in sant'agata bolognese?\n",
      "(a)automobili lamborghini\n",
      "(b)suvs\n",
      "(c)lamborghini\n",
      "(d)simultaneously\n",
      "Correct answer is : (d)\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "text_1 = \"Automobili Lamborghini, the illustrious Italian manufacturer of luxury sports cars and SUVs, is headquartered in the picturesque Sant'Agata Bolognese. This renowned automotive institution boasts a storied legacy, and its contemporary success is firmly underpinned by a fascinating history that has seen it evolve through ownership changes, economic downturns, and groundbreaking innovations.\\\n",
    "Ferruccio Lamborghini, a prominent Italian industrialist with a passion for automobiles, laid the foundation for this iconic marque in 1963. His vision was audacious - to challenge the supremacy of Ferrari, the undisputed titan of Italian sports cars. Under Ferruccio's guidance, Automobili Ferruccio Lamborghini S.p.A. was established, and it immediately began making waves in the automotive world.\\\n",
    "One of the hallmarks of Lamborghini's early years was its distinctive rear mid-engine, rear-wheel-drive layout. This design philosophy became synonymous with Lamborghini's commitment to creating high-performance vehicles. The company's inaugural models, such as the 350 GT, arrived in the mid-1960s and showcased Lamborghini's dedication to precision engineering and uncompromising quality.\\\n",
    "Lamborghini's ascendancy was nothing short of meteoric during its formative decade. It consistently pushed the boundaries of automotive technology and design. However, the heady days of growth were met with a sudden downturn when the world faced the harsh realities of the 1973 global financial crisis and the subsequent oil embargo. Lamborghini, like many other automakers, grappled with plummeting sales and financial instability.\\\n",
    "Ownership of Lamborghini underwent multiple transitions in the wake of these challenges. The company faced bankruptcy in 1978, marking a turbulent chapter in its history. The ownership baton changed hands several times, with different entities attempting to steer the storied brand to calmer waters.\\\n",
    "In 1987, American automaker Chrysler Corporation took the helm at Lamborghini. The Chrysler era saw Lamborghini continue to produce remarkable vehicles like the Diablo while operating under the umbrella of a global conglomerate. However, it was not a permanent arrangement.\\\n",
    "In 1994, Malaysian investment group Mycom Setdco and Indonesian group V'Power Corporation acquired Lamborghini, signaling another phase of transformation for the company. These new custodians brought fresh perspectives and investment to the brand, fueling its resurgence.\\\n",
    "A significant turning point occurred in 1998 when Mycom Setdco and V'Power sold Lamborghini to the Volkswagen Group, which placed the Italian marque under the stewardship of its Audi division. This move brought newfound stability and resources, ensuring Lamborghini's enduring presence in the luxury sports car arena.\\\n",
    "Over the ensuing years, Lamborghini witnessed remarkable expansions in its product portfolio. The V10-powered Huracán captured the hearts of sports car enthusiasts with its exquisite design and formidable performance. Simultaneously, Lamborghini ventured into the SUV market with the Urus, a groundbreaking vehicle powered by a potent twin-turbo V8 engine. This diversification allowed Lamborghini to cater to a broader range of customers without compromising on its commitment to luxury and performance.\\\n",
    "While these successes were noteworthy, Lamborghini was not immune to the challenges posed by global economic fluctuations. In the late 2000s, during the worldwide financial crisis and the subsequent economic downturn, Lamborghini's sales experienced a significant decline, illustrating the brand's vulnerability to external economic factors.\\\n",
    "Despite these challenges, Lamborghini maintained its relentless pursuit of automotive excellence. The company's flagship model, the V12-powered Aventador, reached the pinnacle of automotive engineering and design before concluding its production run in 2022. However, the story does not end here. Lamborghini is set to introduce the Revuelto, a V12/electric hybrid model, in 2024, exemplifying its commitment to embracing cutting-edge technologies and pushing the boundaries of performance.\\\n",
    "In addition to its road car production, Lamborghini has made notable contributions to other industries. The company manufactures potent V12 engines for offshore powerboat racing, further underscoring its prowess in high-performance engineering.\\\n",
    "Interestingly, Lamborghini's legacy extends beyond the realm of automobiles. Ferruccio Lamborghini founded Lamborghini Trattori in 1948, a separate entity from the automobile manufacturer, which continues to produce tractors to this day.\\\n",
    "Lamborghini's rich history is also intertwined with the world of motorsport. In a stark contrast to his rival Enzo Ferrari, Ferruccio Lamborghini decided early on not to engage in factory-supported racing, considering it too expensive and resource-intensive. Nonetheless, Lamborghini's engineers, many of whom were passionate about racing, embarked on ambitious projects, including the development of the iconic Miura sports coupe, which possessed racing potential while being road-friendly. This project marked a pivotal moment in Lamborghini's history, showcasing its ability to create vehicles that could excel on both the track and the road.Despite Ferruccio's reluctance, Lamborghini did make some forays into motorsport. In the mid-1970s, while under the management of Georges-Henri Rossetti, Lamborghini collaborated with BMW to develop and manufacture 400 cars for BMW, a venture intended to meet Group 4 homologation requirements. However, due to financial instability and delays in development, BMW eventually took control of the project, finishing it without Lamborghini's involvement.\\\n",
    "Lamborghini also briefly supplied engines to Formula One teams from 1989 to 1993. Teams like Larrousse, Lotus, Ligier, Minardi, and Modena utilized Lamborghini power units during this period. Lamborghini's best result in Formula One was achieved when Aguri Suzuki finished third at the 1990 Japanese Grand Prix.\\\n",
    "In addition to Formula One, Lamborghini was involved in other racing series. Notably, racing versions of the Diablo were developed for the Diablo Supertrophy, a single-model racing series that ran from 1996 to 1999. The Murciélago R-GT, a production racing car, was created to compete in events like the FIA GT Championship and the American Le Mans Series in 2004, achieving notable results in its racing endeavors.\\\n",
    "Lamborghini's connection with motorsport reflects the brand's commitment to engineering excellence, even though it shied away from factory-backed racing for much of its history.\\\n",
    "Beyond the realms of automotive engineering, Lamborghini has carved a distinct niche in the world of branding. The company licenses its prestigious brand to manufacturers who produce a wide array of Lamborghini-branded consumer goods, including scale models, clothing, accessories, bags, electronics, and even laptop computers. This strategic approach has enabled Lamborghini to extend its brand reach beyond the confines of the automotive industry.\\\n",
    "One fascinating aspect of Lamborghini's identity is its deep connection with the world of bullfighting. In 1962, Ferruccio Lamborghini visited the ranch of Don Eduardo Miura, a renowned breeder of Spanish fighting bulls. Impressed by the majestic Miura animals, Ferruccio decided to adopt a raging bull as the emblem for his burgeoning automaker. This emblem, now iconic, symbolizes Lamborghini's passion for performance, power, and the thrill of the chase.\\\n",
    "Lamborghini's vehicle nomenclature also reflects this bullfighting heritage, with many models bearing the names of famous fighting bulls or bull-related themes. The Miura, named after the Miura bulls, set the precedent, and subsequent models like the Murciélago, Gallardo, and Aventador continued this tradition.\\\n",
    "Furthermore, Lamborghini has enthusiastically embraced emerging automotive technologies, responding to environmental concerns and changing consumer preferences. The Sian, introduced as the company's first hybrid model, showcases Lamborghini's commitment to sustainable performance. With its innovative hybrid powertrain, the Sian combines electric propulsion with a naturally aspirated V12 engine to deliver breathtaking performance while minimizing emissions.\\\n",
    "Looking ahead, Lamborghini has ambitious plans to produce an all-electric vehicle, aligning with the broader industry trend towards electrification. While traditionalists may lament the absence of roaring V12 engines, Lamborghini recognizes the importance of evolving with the times, ensuring that future generations of enthusiasts can experience the thrill of a Lamborghini while contributing to a more sustainable future.\\\n",
    "In summary, Automobili Lamborghini stands as a testament to the enduring allure of Italian craftsmanship and automotive excellence. From its audacious beginnings as a challenger to Ferrari, Lamborghini has weathered storms, embraced innovation, and left an indelible mark on the world of sports cars. Its legacy is one of design brilliance, relentless pursuit of power, and a commitment to pushing the boundaries of what's possible in the realm of high-performance automobiles. Whether through its iconic V12-powered supercars, groundbreaking hybrids, or electrifying visions of the future, Lamborghini continues to captivate the hearts of automotive enthusiasts worldwide, cementing its status as a legendary and iconic brand.\"\n",
    "\n",
    "final_questions = get_mca_questions(text_1)\n",
    "for q in final_questions:\n",
    "    print(q)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Example: 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "What is the name of Musk's venture?\n",
      "(a)Tesla\n",
      "(b)elon musk\n",
      "(c)Richard branson\n",
      "(d)Jeff bezos\n",
      "Correct answer is : (b)\n",
      "\n",
      "\n",
      "The narrative explores the multifaceted relationship between elon musk and what?\n",
      "(a)Btc\n",
      "(b)Fiat\n",
      "(c)bitcoin\n",
      "(d)Cryptocurrency\n",
      "Correct answer is : (c)\n",
      "\n",
      "\n",
      "What is the name of elon musk?\n",
      "(a)musk\n",
      "(b)Gates\n",
      "(c)Bezos\n",
      "(d)Spacex\n",
      "Correct answer is : (a)\n",
      "\n",
      "\n",
      "Along with spacex, neuralink and boring company, what is a notable venture of Elon Musk?\n",
      "(a)tesla\n",
      "(b)Musk\n",
      "(c)Prius\n",
      "(d)Nissan\n",
      "Correct answer is : (a)\n",
      "\n",
      "\n",
      "What is elon musk known for?\n",
      "(a)Other tweets\n",
      "(b)tweets\n",
      "(c)Reddit posts\n",
      "(d)Facebook posts\n",
      "Correct answer is : (b)\n",
      "\n",
      "\n",
      "What aspect of bitcoin is elon musk known for?\n",
      "(a)Pricing\n",
      "(b)Ridiculous price\n",
      "(c)Market\n",
      "(d)price\n",
      "Correct answer is : (d)\n",
      "\n",
      "\n",
      "What type of investment did elon musk make?\n",
      "(a)elon musk\n",
      "(b)bitcoin investment\n",
      "(c)musk\n",
      "(d)tesla\n",
      "Correct answer is : (b)\n",
      "\n",
      "\n",
      "Along with tesla and neuralink, what is a notable venture of Elon Musk?\n",
      "(a)Ula\n",
      "(b)Reusability\n",
      "(c)spacex\n",
      "(d)Virgin galactic\n",
      "Correct answer is : (c)\n",
      "\n",
      "\n",
      "What does the narrative explore about bitcoin and elon musk?\n",
      "(a)Market volatility\n",
      "(b)volatility\n",
      "(c)Merchant adoption\n",
      "(d)Hedging\n",
      "Correct answer is : (b)\n",
      "\n",
      "\n",
      "What does elon musk have about bitcoin?\n",
      "(a)Ulterior motives\n",
      "(b)intentions\n",
      "(c)True thoughts\n",
      "(d)Own view\n",
      "Correct answer is : (b)\n",
      "\n",
      "\n",
      "What type of organizations does elon musk work for?\n",
      "(a)Govts\n",
      "(b)governments\n",
      "(c)Multinational corporations\n",
      "(d)Private interests\n",
      "Correct answer is : (b)\n",
      "\n",
      "\n",
      "What does the narrative explore about elon musk and bitcoin?\n",
      "(a)Other threads\n",
      "(b)comments\n",
      "(c)Downvoted posts\n",
      "(d)Negative votes\n",
      "Correct answer is : (b)\n",
      "\n",
      "\n",
      "What type of vehicle is elon musk known for?\n",
      "(a)Battery storage\n",
      "(b)electric vehicles\n",
      "(c)Internal combustion engines\n",
      "(d)Clean energy\n",
      "Correct answer is : (b)\n",
      "\n",
      "\n",
      "What does elon musk focus on?\n",
      "(a)energy consumption\n",
      "(b)Co2 output\n",
      "(c)Solar and wind\n",
      "(d)Carbon footprint\n",
      "Correct answer is : (a)\n",
      "\n",
      "\n",
      "What did critics of elon musk say about bitcoin?\n",
      "(a)elon musk\n",
      "(b)bitcoin\n",
      "(c)critics argued\n",
      "(d)tesla\n",
      "Correct answer is : (c)\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "text_2 = \"nts, controversies, and the broader implications for the cryptocurrency landscape.Elon Musk: A Brief Overview.Before delving into Musk's connection with Bitcoin, it's essential to understand who Elon Musk is and his role in the tech and automotive industries. Born in South Africa in 1971, Musk displayed a prodigious talent for technology from a young age. He moved to the United States to attend the University of Pennsylvania, where he earned dual bachelor's degrees in physics and economics.Musk's entrepreneurial journey began with the creation of Zip2, a software company he co-founded in 1995, which provided business directories and maps for newspapers. In 1999, Compaq acquired Zip2 for nearly $300 million, providing Musk with his first significant windfall.With his newfound wealth, Musk co-founded X.com, an online payment company, in 1999. X.com later evolved into PayPal and was sold to eBay in 2002 for $1.5 billion. Musk, however, did not rest on his laurels. Instead, he turned his attention to two ambitious and groundbreaking industries: electric vehicles and space exploration.Tesla, SpaceX, and Beyond.In 2004, Musk founded Tesla Motors (now Tesla, Inc.), with the goal of accelerating the world's transition to sustainable energy. Tesla's electric vehicles, beginning with the Roadster and followed by models like the Model S, Model 3, Model X, and Model Y, have transformed the automotive industry. Tesla's innovations in battery technology and electric drivetrains have played a crucial role in popularizing electric cars.Musk's SpaceX, founded in 2002, has achieved remarkable milestones in the field of space exploration. SpaceX developed the Falcon 1, the first privately developed liquid-fueled rocket to reach orbit, and later the Falcon 9 and Falcon Heavy, which have significantly reduced the cost of launching payloads into space. SpaceX also aims to establish a human presence on Mars through its Starship spacecraft.Beyond Tesla and SpaceX, Musk has pursued other ventures. Neuralink focuses on developing brain-computer interface technology, while The Boring Company is dedicated to creating underground transportation tunnels to alleviate urban congestion.Musk's achievements and ambitions have made him one of the most influential and scrutinized figures in the tech and business worlds. His social media presence, especially on Twitter, where he often shares updates on his companies and personal thoughts, has further amplified his reach and impact.Bitcoin: A Digital Revolution.Bitcoin, conceived in a 2008 whitepaper by the pseudonymous Satoshi Nakamoto, is a decentralized digital currency that operates on a peer-to-peer network. It represents a fundamental departure from traditional financial systems, as it doesn't rely on centralized authorities like banks or governments to validate and record transactions. Instead, Bitcoin transactions are confirmed by a network of computers, secured by cryptography, and recorded on a public ledger called the blockchain.Bitcoin's value proposition includes security, transparency, and the potential to serve as a store of value, similar to gold. It has gained traction as a means of transferring funds across borders, as a hedge against inflation, and as a speculative investment.As Bitcoin's popularity grew, it attracted attention from a diverse range of individuals and institutions, including tech entrepreneurs like Elon Musk.Elon Musk's Early Interest in Bitcoin.Musk's interest in Bitcoin became apparent through his tweets and comments on various occasions. While he didn't dive headfirst into the cryptocurrency, he showed a fascination with its technology and potential.In 2019, during a podcast interview with Ark Invest's Cathie Wood, Musk stated that Bitcoin is \\\"a far better way to transfer value than pieces of paper.\\\" He acknowledged its utility as a means of circumventing traditional financial intermediaries, highlighting its appeal to those who want greater control over their funds.Musk's acknowledgment of Bitcoin as a valuable innovation added to its credibility and mainstream acceptance. However, it was in the subsequent years that Musk's relationship with Bitcoin would become more complex, leading to significant fluctuations in the cryptocurrency's price and public perception.Tesla's Bitcoin Investment: A Game-Changer.The most pivotal moment in Elon Musk's involvement with Bitcoin came in early 2021 when Tesla, Inc. announced a groundbreaking move. In a filing with the U.S. Securities and Exchange Commission (SEC), Tesla revealed that it had purchased $1.5 billion worth of Bitcoin and intended to accept Bitcoin as a form of payment for its electric vehicles.This announcement sent shockwaves through the financial world, as it marked one of the most substantial endorsements of Bitcoin by a major corporation to date. Tesla's decision to allocate a portion of its corporate treasury to Bitcoin was seen as a bold move, signaling confidence in the cryptocurrency's long-term value.The rationale behind Tesla's Bitcoin investment, as outlined in its SEC filing, revolved around diversifying and maximizing returns on its cash reserves. Traditional avenues for cash management, like holding government bonds or placing funds in interest-bearing accounts, were yielding minimal returns. By allocating some of its capital to Bitcoin, Tesla sought to benefit from the cryptocurrency's potential for appreciation.This move had immediate repercussions on the price of Bitcoin, which surged to new all-time highs, surpassing $60,000 per BTC in early 2021. It also ignited a broader conversation about whether more corporations would follow Tesla's lead and allocate some of their treasuries to Bitcoin.Elon Musk's Tweets and Bitcoin's Volatility.Elon Musk's impact on Bitcoin extended beyond Tesla's investment. His prolific and often whimsical tweeting habits started influencing the cryptocurrency's price in ways that few could have predicted.Musk's tweets and public statements about Bitcoin and other cryptocurrencies were a double-edged sword. On one hand, his endorsement and public discussion of Bitcoin brought mainstream attention to the cryptocurrency, fueling interest and investment. On the other hand, his tweets could also lead to massive price swings, causing concern and frustration among investors and regulators.One of the most notable episodes occurred in May 2021 when Musk announced via Twitter that Tesla would no longer accept Bitcoin as payment for its vehicles, citing environmental concerns related to Bitcoin mining's energy consumption. This single tweet triggered a sharp decline in the price of Bitcoin, erasing billions of dollars in market value in a matter of hours.The environmental debate surrounding Bitcoin's energy use intensified following Musk's tweet. Critics argued that Bitcoin's energy consumption, driven by the proof-of-work consensus mechanism, was unsustainable and environmentally damaging. Proponents of Bitcoin countered that its energy use was comparable to, or even more efficient than, traditional banking and gold mining.Musk's influence over Bitcoin's price volatility didn't stop there. He continued to tweet about cryptocurrencies, often in cryptic and playful ways. For example, he tweeted about \\\"Baby Doge Coin\\\" and \\\"Shiba Inu,\\\" two meme cryptocurrencies, which led to speculative buying frenzies.These episodes prompted calls for greater regulation of Musk's Twitter activity and its potential impact on financial markets. Regulators expressed concerns about market manipulation and the need for greater transparency in Musk's communication regarding Tesla's Bitcoin holdings and intentions.Elon Musk's Impact on Altcoins.In addition to his influence on Bitcoin, Musk's tweets and comments have also affected various altcoins, which are cryptocurrencies other than Bitcoin. Dogecoin, in particular, emerged as a notable example of Musk's ability to move markets.Dogecoin, originally created as a meme cryptocurrency in 2013, gained a cult following on the internet due to its Shiba Inu dog logo and lighthearted community. However, its value remained relatively low and stable for years.In early 2021, Musk began tweeting about Dogecoin, referring to it as\\\"the people's crypto.\\\" His tweets, often accompanied by playful memes and comments, caused massive surges in Dogecoin\\'s price. At one point, it reached an all-time high of over $0.60 per DOGE, a substantial increase from its previous fractions-of-a-penny valuation.Musk's involvement with Dogecoin extended to his hosting of \\\"Saturday Night Live\\\" in May 2021. During the show, he referred to Dogecoin as a \\\"hustle,\\\" which led to a temporary price drop. Nevertheless, his overall engagement with Dogecoin contributed to its growing popularity and market capitalization.The phenomenon of \\\"Musk tweets\\\" driving the prices of cryptocurrencies sparked debates about the role of celebrity endorsements and social media influencers in the cryptocurrency space. It also raised questions about the fundamental value of assets like Dogecoin and the risks of investing based on internet trends and social media hype.The Impact of Elon Musk\\'s Environmental Concerns.Musk\\'s environmental concerns regarding Bitcoin mining were not limited to his May 2021 tweet. He continued to advocate for more sustainable practices within the cryptocurrency industry. He called on Bitcoin miners to transition to renewable energy sources and even engaged in discussions with prominent figures in the cryptocurrency community about potential solutions.These concerns about Bitcoin\\'s carbon footprint drew attention to a longstanding issue within the cryptocurrency space. Bitcoin's proof-of-work consensus mechanism requires significant computational power, which, in turn, demands substantial energy consumption. The majority of Bitcoin mining operations relied on fossil fuels, particularly coal, which contributed to concerns about its environmental impact.In response to Musk\\'s comments and mounting environmental criticism, some Bitcoin miners began exploring cleaner energy sources and sustainable practices. Several Bitcoin mining companies announced commitments to using renewable energy, and discussions about transitioning to a more eco-friendly consensus mechanism gained momentum.Musk's advocacy for sustainability in the cryptocurrency sector highlighted the growing importance of environmental, social, and governance (ESG) considerations for investors and corporations. It also led to broader conversations about the environmental impact of blockchain technologies and the need for responsible innovation in the crypto space.Elon Musk\\'s Evolving Stance on Bitcoin.Musk\\'s relationship with Bitcoin and his stance on the cryptocurrency have evolved over time. While he initially praised Bitcoin for its technological advantages and supported its adoption within Tesla, he later raised concerns about its environmental impact and price volatility.In July 2021, during a B Word conference, Musk revealed that SpaceX held Bitcoin and that he personally held Bitcoin, Ethereum, and Dogecoin. This disclosure indicated that, despite his reservations, Musk maintained a personal interest in cryptocurrencies.As the year progressed, Musk\\'s tweets about Bitcoin became less frequent, and his public statements about the cryptocurrency became more balanced. He acknowledged that he wanted to see Bitcoin succeed and that Tesla would likely accept Bitcoin again once its mining operations became more environmentally friendly.By late 2021, Musk indicated that he was working with Dogecoin developers to improve the cryptocurrency's efficiency, signaling his continued involvement in the crypto space.Elon Musk, Bitcoin, and Regulatory Scrutiny.The volatility in Bitcoin\\'s price caused by Musk\\'s tweets and statements raised concerns among regulators and lawmakers. The potential for market manipulation and the need for investor protection came into sharp focus.The U.S. Securities and Exchange Commission (SEC) and other regulatory bodies began examining Musk\\'s social media activity and its impact on cryptocurrency and stock markets. The SEC had previously clashed with Musk over his tweets regarding Tesla's stock, resulting in legal settlements and restrictions on his communication.While Musk\\'s influence over cryptocurrency markets is undoubtedly significant, the regulatory landscape for cryptocurrencies remains relatively nascent and complex. Regulators are grappling with how to address the unique challenges posed by the digital asset space, including the influence of high-profile individuals like Musk.Elon Musk's Vision for the Future of Cryptocurrency.Despite the complexities and controversies surrounding Elon Musk\\'s involvement with cryptocurrency, his vision for the future of digital assets and blockchain technology remains of interest.Musk has expressed support for the concept of decentralized finance (DeFi), which leverages blockchain technology to create open and permissionless financial systems. DeFi platforms enable activities like lending, borrowing, trading, and earning interest without the need for traditional financial intermediaries.Furthermore, Musk's involvement with SpaceX and his aspirations for Mars colonization have raised questions about the role of cryptocurrencies in space exploration. Some have speculated that cryptocurrencies could become the primary means of conducting financial transactions and economic activities on future Martian colonies.Additionally, Musk\\'s advocacy for renewable energy solutions aligns with the broader trend of sustainable blockchain technologies. Several cryptocurrencies, like Ethereum, are transitioning from proof-of-work to proof-of-stake consensus mechanisms, which are more energy-efficient and environmentally friendly. Musk's influence and financial resources could potentially contribute to the development and adoption of greener blockchain solutions.Conclusion: The Ongoing Saga of Elon Musk and Bitcoin.The relationship between Elon Musk and Bitcoin has been a rollercoaster ride, characterized by enthusiasm, controversy, and unpredictability. Musk\\'s tweets have demonstrated the power of celebrity influence in the world of cryptocurrency, shaping market sentiment and investor behavior.While Musk\\'s statements and actions have generated significant volatility in cryptocurrency markets, they have also drawn attention to important issues within the industry, such as environmental sustainability and responsible innovation. His involvement has forced a broader discussion about the role of cryptocurrencies in the global economy and their potential to reshape traditional financial systems.As cryptocurrency continues to evolve and mature, it remains to be seen how Musk\\'s influence and vision will intersect with this rapidly changing landscape. Whether through his investments, innovations, or advocacy for sustainability, Elon Musk will likely continue to be a central figure in the ongoing narrative of cryptocurrencies and their place in the digital age.\"\n",
    "\n",
    "final_questions = get_mca_questions(text_2)\n",
    "for q in final_questions:\n",
    "    print(q)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "A bad algorithm can lead to longer time to do what?\n",
      "(a)Burst damage\n",
      "(b)execute\n",
      "(c)Aoe\n",
      "(d)Squishy target\n",
      "Correct answer is : (b)\n",
      "\n",
      "\n",
      "What is the topic of this lecture?\n",
      "(a)Divisors\n",
      "(b)Iff\n",
      "(c)K=0\n",
      "(d)sum\n",
      "Correct answer is : (d)\n",
      "\n",
      "\n",
      "A bad algorithm can lead to longer time to do what?\n",
      "(a)execute\n",
      "(b)execute 3004 billion instructions\n",
      "(c)execute 3004 billion instructions\n",
      "(d)time\n",
      "Correct answer is : (b)\n",
      "\n",
      "\n",
      "A bad algorithm can lead to longer execution what?\n",
      "(a)Really long time\n",
      "(b)time\n",
      "(c)Rest\n",
      "(d)Very short amount\n",
      "Correct answer is : (b)\n",
      "\n",
      "\n",
      "When will a bad algorithm lead to longer execution time?\n",
      "(a)Ballerhorn\n",
      "(b)Moneys\n",
      "(c)eventually\n",
      "(d)Broken\n",
      "Correct answer is : (c)\n",
      "\n",
      "\n",
      "The topic of this lecture is significance of what?\n",
      "(a)Above steps\n",
      "(b)instructions\n",
      "(c)Correct procedure\n",
      "(d)Basic directions\n",
      "Correct answer is : (b)\n",
      "\n",
      "\n",
      "What is important to do about the importance of an algorithm?\n",
      "(a)Inkling\n",
      "(b)Thst\n",
      "(c)know\n",
      "(d)Straight answer\n",
      "Correct answer is : (c)\n",
      "\n",
      "\n",
      "A bad algorithm can lead to longer time to do what?\n",
      "(a)Only run\n",
      "(b)Next play\n",
      "(c)Pass\n",
      "(d)run\n",
      "Correct answer is : (d)\n",
      "\n",
      "\n",
      "How long is the execution time of an algorithm?\n",
      "(a)70 nanoseconds\n",
      "(b)sum\n",
      "(c)execute 3004 billion instructions\n",
      "(d)time\n",
      "Correct answer is : (a)\n",
      "\n",
      "\n",
      "When did dr. david mcdonald s start writing an algorithm?\n",
      "(a)execute\n",
      "(b)sum\n",
      "(c)execute 3004 billion instructions\n",
      "(d)initially\n",
      "Correct answer is : (d)\n",
      "\n",
      "\n",
      "When did dr. david mcdonald s teach about the importance of algorithms?\n",
      "(a)execute\n",
      "(b)sum\n",
      "(c)execute 3004 billion instructions\n",
      "(d)previously\n",
      "Correct answer is : (d)\n",
      "\n",
      "\n",
      "What type of algorithm can lead to longer execution time?\n",
      "(a)bad\n",
      "(b)Rectify\n",
      "(c)Fargo\n",
      "(d)Breaking\n",
      "Correct answer is : (a)\n",
      "\n",
      "\n",
      "How many billion instructions does the lecture cover?\n",
      "(a)execute\n",
      "(b)sum\n",
      "(c)run 3004 billion instructions\n",
      "(d)time\n",
      "Correct answer is : (c)\n",
      "\n",
      "\n",
      "When does the lecture conclude with an explanation of why we write good algorithms?\n",
      "(a)Sbisa\n",
      "(b)Pumped.\n",
      "(c)My birthday\n",
      "(d)finally\n",
      "Correct answer is : (d)\n",
      "\n",
      "\n",
      "What does the lecture conclude with an explanation of why?\n",
      "(a)execute\n",
      "(b)sum\n",
      "(c)write good algorithms\n",
      "(d)time\n",
      "Correct answer is : (c)\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "text_3 = \"\"\"In our previous lectures, we understood what is an algorithm and what are the characteristics of an algorithm. I hope those concepts are clear. Now, let's proceed with the significance of algorithms. In this lecture, we will understand what is the significance of algorithms. Why do we write good algorithms? What is the difference between a good algorithm and a bad algorithm? All these concepts will be clear in this lecture. So, let's get started and let's see what are the topics. The topic of this lecture is significance of algorithms. We will understand what is the significance of good algorithms. Let's start with this lecture. A bad algorithm can lead to longer execution time. Remember this, that a bad algorithm can lead to longer execution time. What does this mean? \n",
    "This means that if we convert a bad algorithm to its equivalent program and if we execute that program on our system, that program will take a lot of time. It will take a lot of time to execute and give us the output. And we don't want that. Why do we want to wait for so long? So, this is the meaning of a bad algorithm. When we convert a bad algorithm to its equivalent program, that specific program will take a lot of time to execute. So, that is why I have written a bad algorithm can lead to longer execution time. Now, let's see an example to understand this properly. Let's take a simple example of calculating the sum of first 10 to the power of 12 natural numbers. \n",
    "Let's write the algorithm for the same. First, we need to start the algorithm. This is step number 1. Step number 2 is to initialize the variable sum to 0. We want to calculate the sum of first 10 to the power of 12 natural numbers. For this, we need variable sum and we are initializing it to 0. So, that's the initial sum. Later, after the completion of the algorithm, we will get the sum of first 10 to the power of 12 natural numbers. That's what we will take care of. In order to calculate the sum of first 10 to the power of 12 natural numbers, we can use loops. We can iterate from 1 to 10 to the power of 12 and with each iteration, we can add the current iteration value to the sum variable. \n",
    "In this way, we will get the sum of first 10 to the power of 12 natural numbers. Let's realize this in our algorithm. Let's write step number 3. Enter the loop that iterates from 1 to 10 to the power of 12. Now we are entering the loop that iterates from 1 to 10 to the power of 12. This means our loop will run 10 to the power of 12 times. Step number 4 is to add the current iteration value to sum. And step number 5 is to repeat steps 4 and 5 so that our loop will run 10 to the power of 12 times. In each iteration, we will add the current iteration value to sum. Initially, we know sum is 0. When we enter our loop, we will get 1 as the current iteration value and that value will be added to sum. \n",
    "We will get 1 in sum because initial sum is 0 And now the new sum is 0 plus 1 which is equal to 1. In the second iteration, we will get value 2 and that value will be added to the previous sum which is 1. The new sum will be 1 plus 2 which is equal to 3. After this, we will get 3 as the value from the current iteration. This 3 will be added to the sum variable which means that 3 is added to the previous sum which was 3. Now we will get the new sum as 6 because 3 plus 3 is 6. In this way, we will get the sum of first 10 to the power of 12 natural numbers using the loop. \n",
    "Now finally, we must return the value of sum and then we can stop the algorithm. This one is a bad algorithm. Now why am I saying this algorithm is bad? You will get this in a moment. I told you already that a bad algorithm can lead to longer execution time. This means if we convert a bad algorithm to its equivalent program, that specific program will take a lot of time to execute. In order to see this, first let's convert this specific algorithm to its equivalent program. This is how the equivalent program looks like. Here, we are writing this function calculateSum. I am assuming that you already know some programming language. I have written this specific code in C programming language. If you know some programming language, then understanding this program will not be difficult at all. \n",
    "Although, I'll try to explain what I have written here. If you don't know any programming language, it's okay but it is important to understand at least the basics of some programming language. It would be easier for you to follow along this course if you already know one programming language. If you know the basics, it would be enough. Now, let's understand this calculateSum function. Within this function, I have defined variable sum and I have assigned value 0 to it. This variable is long int variable because this variable will hold a very large value at the end. That's why we are defining it as long int. Also, we are defining variable i and this variable i is also long int type. Now, after this, we have this for loop. \n",
    "This loop will run from 1 to 10 to the power of 12. Within this for loop, we have sum plus equal to i. This means we are adding the current iteration value to sum. This loop will run 10 to the power of 12 times. Here we have the increment step. In each iteration, this means what happens? First, i is initialized to 1. Then, this condition is checked. Is 1 less than this value? Yes, we will proceed. We will add the current iteration value to sum which means we will add 1 to 0. We will get 1 as the result. Eventually, sum will hold 1. After this, i is incremented. This means i becomes 2. Then this condition is again checked. In this way, the loop will run a total of 10 to the power of 12 times. \n",
    "And in this way, we will get the sum of first 10 to the power of 12 natural numbers. After this, we must return the value of sum. That's what I have written here in step number 6. These three steps are equivalent to this for loop and this line, this statement is equivalent to this step. Now, we know how the equivalent program of this algorithm looks like. When we execute this program, we will not get the output immediately. We will get the output after a long time. Now, why is that the case? In order to know this, we must know what are the number of instructions that will be executed by our computer. Let's find out the number of instructions here. How many instructions will be executed by our computer? \n",
    "Let's calculate the number of instructions. The number of instructions is equal to 2 plus 1 plus 10 to the power of 12 into 1 plus 1 plus 1 and then plus 1. First, let's observe this statement. Here we have two instructions. We are first defining variable sum and then we are defining variable i. So, these are the two instructions. That's why I have written 2 here. Then after this, within this for loop, we have the initialization statement. This is one instruction. Hence, I have added 1 here. And this will be executed only one time. But, these three 3 instructions will be executed 10 to the power of 12 times. This is how for loop works. These instructions will be executed 10 to the power of 12 times. \n",
    "That is why I have multiplied 1 plus 1 plus 1 by 10 to the power of 12. 1 for this instruction, 1 for this instruction and 1 for this instruction. Finally, I have added 1 for this instruction specific instruction. So how many instructions are there? There are three thousand and four billion instructions. This means our computer needs to execute three thousand and four billion instructions. So now we know we have three thousand and four billion instructions. These many instructions need to be executed by our computer. Let us assume that a computer takes 1 second to execute Y instructions. And let's say Y is equal to 100 million. This means that our computer takes 1 second to execute 100 million instructions. This one is a huge number. \n",
    "Within 1 second, our computer is capable of executing 100 million instructions. But how many instructions do we want to execute? We want to execute 3004 billion instructions. Now we want to know how much time our computer takes to execute these many instructions. Let's calculate this. We know time to run y instructions is equal to one second. We know this already from the statement that in order to execute y instructions, our computer take one second. So how much time our computer takes to execute one instruction? Let's see this. Time to run one instruction is equal to 1 by y seconds. From y, we want 1. How do we get 1? We need to divide this y by y in order to get 1. If we want to balance this equation, we need to divide this 1 by y. And that is why we are getting 1 by y here. \n",
    "So time to run one instruction is equal to 1 by y seconds. I hope this idea is clear. Now, how much time will x instructions take? Time to run x instructions will be x by y seconds. Why is that the case? We want x here. What we have here? 1. In order to get x, we need to multiply this by x. Then only we will get x here. Similarly, in order to balance this equation, we need to multiply this by x also. And that's why we are getting x by y here. So, in order to run x instructions, our computer will take x by y seconds. This x needs to be replaced by 3004 billion instructions. So, what we will get? Time to run 3004 billion instructions will be 3004 billion instructions divided by y instructions. \n",
    "What is y? y is equal to 100 million. So we need to divide 3004 billion by 100 million. Time to run 3004 billion instructions is equal to 3004 billion divided by 100 million. We will get nearly 30,000 seconds. And this is nearly equal to 500 minutes. And this is nearly equal to 8 hours. So, our computer program is taking nearly 8 hours to execute. To get the final output, this program is taking nearly 8 hours. This is the reason why when we execute this program, we will not get the output immediately. We will get the output nearly after 8 hours. That is why this algorithm is bad. Why this algorithm is bad? This algorithm is bad because when we convert this algorithm to its equivalent program, its program is going to take a lot of time. \n",
    "It is taking nearly 8 hours to execute. Can we write an algorithm which can lead to shorter execution time? Yes, we can do this. We want to know the sum of first 10 to the power of 12 natural numbers. For this purpose, we can write a better algorithm. You will see this in a moment. Please note this that a good algorithm will lead to shorter execution time. So, this is the difference between a good algorithm and a bad algorithm. A bad algorithm leads to longer execution time. On the other hand, a good algorithm leads to shorter execution time. Let's see this in action. Let's write a good algorithm to calculate the sum of first 10 to the power of 12 natural numbers. Let's do this. \n",
    "First, We are going to start our algorithm. Then we need to initialize n to 10 to the power of 12. We are going to initialize n to 10 to the power of 12. Now what is the next step? We will now calculate the sum of first 10 to the power of 12 natural numbers through this formula. We know this formula to calculate the sum of first n natural numbers. We know that n is initialized to 10 to the power of 12. So this formula will calculate the sum of first 10 to the power of 12 natural numbers. We just need to plug in the values here. So instead of using a loop, we can use this formula. Now what is the next step? \n",
    "The next step is to output the value of sum and finally, we can stop the algorithm. This is a good algorithm and you will see this in a moment why is that the case. Let's convert this to its equivalent program to see this in action. how many instructions do we need to execute and eventually how much time our computer will take. Let's convert this to C program like this. Again, we have the function calculateSum but this time the logic is different. Instead of using a loop, we are using this formula n into n plus 1 divide by 2. Here we are initializing n to 10 to the power of 12 and we are defining variable sum here. And here we are calculating the sum. \n",
    "Finally, we are returning sum. How many instructions do we have here? Let's calculate the number of instructions. Here we have two instructions. Then we have four instructions. This is not a single instruction. Here we have four instructions. First, we are adding n and 1. This is one instruction. Then we are multiplying n by the result of n plus 1. This is one instruction. Then we are dividing n into n plus 1 by 2. This is one instruction. So there are three instructions here. Then we are assigning the result of n into n plus 1 divided by 2 to sum. This assignment is also an instruction. So there are total four instructions. And then finally we have this instruction, return sum. So we have 2 plus 4 plus 1 instructions. \n",
    "This is equal to 7. So the number of instructions is 7. Now let's do the same calculation. Let's find out how much time our computer program takes to execute these many instructions. We know already that in order to execute 100 million instructions, our computer takes 1 second. In order to execute x instructions, our computer takes x by y seconds. This time, the value of x is 7. So, how much time it will take? Time to run 7 instructions is equal to 7 divided by 100 million. And this is equal to 0.00000007 seconds. This is equal to 70 nanoseconds. Can you imagine this change? 70 nanoseconds. Previously, our bad algorithm was taking nearly 8 hours to execute. But the good algorithm takes 70 nanoseconds, much, much lesser than 8 hours. \n",
    "Very, very fast. So, this specific algorithm, if we convert to its equivalent program, takes 70 nanoseconds. So, clearly good algorithm leads to shorter execution time. That's why we must always try to write good algorithms. And this is the reason why analysis of algorithms is needed. Why do we need to analyze our algorithms? There can be multiple algorithms to solve a problem. In order to know which algorithm is a good algorithm, we need to analyze those algorithms. We need to know how much time those algorithms will take. Now I hope it is clear what is the significance of good algorithm. So with this, we are done with this topic and this means we are done with this lecture. Okay friends, this is it for now. \n",
    "Thank you for watching this presentation. I will see you in the next one. Till then, take care and bye. \"\"\"\n",
    "final_questions = get_mca_questions(text_3)\n",
    "for q in final_questions:\n",
    "    print(q)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "base",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.7"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}