memOptix-analyst.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "1caf13eb",
   "metadata": {},
   "source": [
    "# memOptix\n",
    "**Purpose**\n",
    "\n",
    "The aim of this notebook is to run through a Windows memory dump in a structured manner to detect suspicious or malicious indicators and behaviours.\n",
    "\n",
    "Please note that this notebook was created based on the output generated from CrowdStrike's Supermem python script (https://github.com/CrowdStrike/SuperMem). SuperMem triage mode 2 or 3 can be run against the memory dump prior to running this notebook. You may decide to run volatility interactively if you wish to do so. Supermem dumps various other artefacts from the memory dump which can assist with triage.\n",
    "\n",
    "Following Open Source projects are used within this notebook:  \n",
    "https://github.com/microsoft/msticpy  \n",
    "https://github.com/volatilityfoundation/volatility3  \n",
    "https://github.com/CrowdStrike/SuperMem\n",
    "\n",
    "CSV data in the test data folder is from ACSC's Cyber Security Challenge (https://www.cyber.gov.au/acsc/view-all-content/news/acsc-cyber-security-challenge)\n",
    "\n",
    "Author : Janantha Marasinghe (@blueteam0ps_)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b8bba8ee",
   "metadata": {},
   "source": [
    "### <a href=#prerequisites>Pre-requisites</a>\n",
    "Let's first install the python packages that are pre-requisites for this script. Please note that msticpy only supports <b>python 3.8 or above</b>. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b3b34d27",
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install msticpy\\[azure] seaborn jellyfish pandas argparse colorama selenium pillow"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "40d7970a",
   "metadata": {},
   "source": [
    "### Importing all the libraries required"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ab75c90b",
   "metadata": {},
   "outputs": [],
   "source": [
    "from pickle import FALSE\n",
    "import jellyfish\n",
    "import pandas as pd\n",
    "import argparse\n",
    "import sys\n",
    "import re\n",
    "import os\n",
    "from colorama import init, Fore, Back, Style\n",
    "from IPython.display import display, HTML\n",
    "\n",
    "#Expand the width of the cells\n",
    "display(HTML(\"<style>.container { width:90% !important; }</style>\"))\n",
    "\n",
    "#Initialise msticpy\n",
    "import msticpy as mp\n",
    "from msticpy.transform.iocextract import IoCExtract\n",
    "mp.init_notebook(verbosity=0)\n",
    "\n",
    "#This will ensure the pandas output results do not get truncated\n",
    "pd.set_option('display.max_rows', None)\n",
    "pd.set_option('display.max_columns', None)\n",
    "pd.set_option('display.width', None)\n",
    "pd.set_option('display.max_colwidth', None)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b30789fc",
   "metadata": {},
   "source": [
    "### Setting up the MSTICPY CTI providers for IOC lookups\n",
    "<b>Make sure msticpyconfig.yaml exists in the same directory as this notebook and is populated with the API details</b>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "555ded4a",
   "metadata": {},
   "outputs": [],
   "source": [
    "intel = mp.TILookup()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6e5cef89",
   "metadata": {},
   "source": [
    "### <a href=#generatecsvs>Generate the relevant CSVs in preparation</a>\n",
    "<b>This is only required if you do not have the required CSVs</b> to run the notebook. Please note that this stage can take a while to complete.You will see a * within the square brackets next to the below function as long as it is processing the step. Do not proceed until you see a number within the square brackets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4c674782",
   "metadata": {},
   "outputs": [],
   "source": [
    "mem_inpath = \"PATH_TO_YOUR_MEMORY_IMAGE\"\n",
    "mem_outpath = \"PATH_TO_YOUR_CSV_OUTPUT\"\n",
    "volplug_lst = [\"windows.pslist.PsList\",\"windows.cmdline.CmdLine\",\" windows.netscan.NetScan\",\"windows.callbacks.Callbacks\",\"windows.driverscan.DriverScan\",\" windows.dlllist.DllList\",\"windows.svcscan.SvcScan\",\"windows.handles.Handles\",\"windows.malfind.Malfind\",\"windows.ssdt.SSDT\"]\n",
    "\n",
    "for plugin in volplug_lst:\n",
    "    vol_execute = \"bash -c 'vol -f \" + mem_inpath + \" -r csv \" + plugin + \" > \" + mem_outpath + plugin + \".csv'\"\n",
    "    os.system(vol_execute)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a3bb7a71",
   "metadata": {},
   "source": [
    "### YOU MAY CHOOSE TO \"RUN ALL BELOW\" CELL ACTION FROM THIS POINT "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2ad1fa14",
   "metadata": {},
   "source": [
    "### <a href=#csvfolderpath>Setting up variables for the notebook</a> \n",
    "Please paste the folder path containing the CSV output from Volatility. For e.g. /opt/memoptix-prod/testdata"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "eb3a4f59",
   "metadata": {},
   "outputs": [],
   "source": [
    "process_input = \"\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "192491ef",
   "metadata": {},
   "source": [
    "### <a href=#csv2df>Load CSV data into Dataframes</a>\n",
    "Load the following CSV files into a dataframes for processing. Analyst tasks are run against the pandas dataframes. Please ensure you go <a href=\"#csvfolderpath\">here</a> and put the path to your CSVs before running the following code block."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c93d58bd",
   "metadata": {},
   "outputs": [],
   "source": [
    "dfprocs = pd.read_csv(process_input + '/windows.pslist.PsList.csv')\n",
    "dfcmdline = pd.read_csv(process_input +'/windows.cmdline.CmdLine.csv')\n",
    "dfnetscan = pd.read_csv(process_input +'/windows.netscan.NetScan.csv')\n",
    "dfcallbacks = pd.read_csv(process_input +'/windows.callbacks.Callbacks.csv')\n",
    "dfdriverscan = pd.read_csv(process_input +'/windows.driverscan.DriverScan.csv')\n",
    "dfdlllist = pd.read_csv(process_input +'/windows.dlllist.DllList.csv')\n",
    "dfsvcscan = pd.read_csv(process_input +'/windows.svcscan.SvcScan.csv')\n",
    "dfhandles = pd.read_csv(process_input +'/windows.handles.Handles.csv')\n",
    "dfmalfind = pd.read_csv(process_input +'/windows.malfind.Malfind.csv')\n",
    "dfssdt = pd.read_csv(process_input +'/windows.ssdt.SSDT.csv')\n",
    "dfldrmod = pd.read_csv(process_input +'/windows.ldrmodules.LdrModules.csv')   "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "26afd496",
   "metadata": {},
   "source": [
    "### <a href=#ParentProcMap>Adding Parent Process Name to the process list data frame</a>\n",
    "Adding a parent process name \"ParentName\" column to make it easier to run searches. Please note that certain processes may not contain a parent process name. This is due to early exit of such parent processes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a0dd0a8c",
   "metadata": {},
   "outputs": [],
   "source": [
    "dfppidtxt = dfprocs[['PID', 'ImageFileName']].drop_duplicates()\n",
    "dfppidtxt.columns = ['PPID', 'ParentName']\n",
    "dfprocs = dfprocs.merge(dfppidtxt, on='PPID', how='left')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "af4f0254",
   "metadata": {},
   "source": [
    "<a href=#typesquat>Widely Targeted Windows processes</a>\n",
    "This array contains commonly targeted Windows process names which are often used by adversaries for blending in."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "882dde66",
   "metadata": {},
   "outputs": [],
   "source": [
    "actprocsbaselst =[\"svchost.exe\",\"wininit.exe\",\"lsass.exe\",\"explorer.exe\",\"iexplore.exe\",\"powershell.exe\",\"cmd.exe\",\"services.exe\",\"csrss.exe\",\"smss.exe\",\"silhost.exe\",\"taskhost.exe\",\"MpCmdRun.exe\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "32cb22e1",
   "metadata": {},
   "source": [
    "<a href=#singleprocs>Windows processes that should only have a single instance</a>\n",
    "This array contains a list of processes which should only have a single instance of itself during normal operation of Windows."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bd93ea85",
   "metadata": {},
   "outputs": [],
   "source": [
    "singleprocs = [\"system\",\"wininit.exe\",\"lsass.exe\",\"lsm.exe\",\"services.exe\",\"lsaiso.exe\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a4f25eac",
   "metadata": {},
   "source": [
    "### <a href=#pr1>PR1 - Identifying processes that should not have more than one instance</a>\n",
    "Checks to see if there are more than one process for processes listed in singleprocs array."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "83557058",
   "metadata": {},
   "outputs": [],
   "source": [
    "for procs in singleprocs:\n",
    "    p_count = dfprocs[dfprocs.ImageFileName.str.lower() == procs].shape[0]\n",
    "    if p_count == 1:\n",
    "        print(Fore.GREEN + \"[✓]No multiple instance(s) of \" + procs + \" found\")\n",
    "    elif p_count > 1:\n",
    "        print(Fore.RED + \"[!]Multiple instance(s) of \" + procs + \" found!\")\n",
    "        print(dfprocs[dfprocs.ImageFileName.str.lower() == procs][['ImageFileName','PID','PPID','ParentName']].to_string(index = False))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1ecba7a5",
   "metadata": {},
   "source": [
    "### <a href=#pr2>PR2 - Identifying suspicious parent-child process lineage</a>\n",
    "\n",
    "Parent processes of services.exe, iexplore.exe, svchost.exe, cmd.exe and powershell.exe are usually known. This search tries to identify when they are spawned from unusual parent processes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e0d4391e",
   "metadata": {},
   "outputs": [],
   "source": [
    "sus_services = dfprocs[(dfprocs.ImageFileName.str.lower() == \"services.exe\") & (dfprocs.ParentName.str.lower() != \"wininit.exe\")][['ImageFileName','PID','PPID','ParentName']]\n",
    "if(sus_services.empty):\n",
    "    print(Fore.GREEN + \"[✓]No suspicious instances of services.exe were found\")\n",
    "else:\n",
    "    print(Fore.RED + \"[!]Suspicious services.exe process(s) found\")\n",
    "    print(sus_services.to_string(index = False))\n",
    "    \n",
    "sus_iexplorer = dfprocs[(dfprocs.ImageFileName.str.lower() == \"iexplore.exe\") & (dfprocs.ParentName.str.lower() != \"explorer.exe\")][['ImageFileName','PID','PPID','ParentName']]\n",
    "if (sus_iexplorer.empty): \n",
    "    print(Fore.GREEN + \"[✓]No suspicious instances of iexplore.exe were found\")\n",
    "else:\n",
    "    print(Fore.RED + \"[!]iexplore.exe spawned from a suspicious parent\")\n",
    "    print(sus_iexplorer.to_string(index = False))\n",
    "sus_svchost= dfprocs[(dfprocs.ImageFileName.str.lower() == \"svchost.exe\") & (dfprocs.ParentName.str.lower() != \"services.exe\")][['ImageFileName','PID','PPID','ParentName']]\n",
    "if (sus_svchost.empty):\n",
    "    print(Fore.GREEN + \"[✓]No suspicious instances of svchost.exe were found\")\n",
    "else:\n",
    "    print(Fore.RED + \"[!]svchost.exe spawned from a suspicious parent\")\n",
    "    print(sus_svchost.to_string(index = False))\n",
    "sus_cli = dfprocs[((dfprocs.ImageFileName.str.lower() == \"cmd.exe\") | (dfprocs.ImageFileName.str.lower() == \"powershell.exe\")) & (dfprocs.ParentName.str.lower() != \"explorer.exe\")][['ImageFileName','PID','PPID','ParentName']]\n",
    "if (sus_cli.empty):\n",
    "    print(Fore.GREEN + \"[✓]No suspicious instances of cmd.exe or powershell.exe were found\")\n",
    "else:\n",
    "    print(Fore.RED + \"[!]cmd.exe or powershell.exe spawned by an unusual parent!\")\n",
    "    print(sus_cli.to_string(index = False))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b13c7008",
   "metadata": {},
   "source": [
    "### <a href=#pr3>PR3 - Identifying suspicious processes using Damerau-Levenshtein distance</a>\n",
    "This compares the Damerau-Levenshtein distance between a legitimate process (in actprocsbaselst array) and existing processes available in the memory image. The Damerau–Levenshtein distance works on the distance between two words, which is the least number of operations needed to change one word into another. Currently we are looking at 2 or fewer changes. Please note that the output of this task may need to be tweaked accordingly as it may result is false positives at times."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "eb291161",
   "metadata": {},
   "outputs": [],
   "source": [
    "#http://detect-respond.blogspot.com/2016/11/hunting-for-malware-critical-process.html\n",
    "#Following adds live processes to a list and deduplicates it\n",
    "liveprocs=[]\n",
    "levratedprocs = []\n",
    "for susproc in dfprocs[\"ImageFileName\"].str.lower():\n",
    "    if susproc not in liveprocs:\n",
    "        liveprocs.append(susproc)\n",
    "\n",
    "for x,y in [(x,y) for x in liveprocs for y in actprocsbaselst]:\n",
    "    levdistance = (jellyfish.damerau_levenshtein_distance(x,y))\n",
    "    if 0 < levdistance <= 2:\n",
    "        #print(x,y,levdistance)\n",
    "        levratedprocs.append([x, y,levdistance])\n",
    "\n",
    "#Convert list to a dataframe\n",
    "dflevprocs = pd.DataFrame(levratedprocs, columns = ['LiveProcess', 'BaseProcess','Damerau–Levenshtein distance'])\n",
    "\n",
    "#Reduce false positives by removing known process pairs\n",
    "procfpredux = dflevprocs[((dflevprocs.LiveProcess.str.lower().isin(['lsass.exe', 'csrss.exe','smss.exe','dllhost.exe','explorer.exe','svchost.exe','sihost.exe'])) & (dflevprocs.BaseProcess.str.lower().isin(['lsass.exe', 'csrss.exe','smss.exe','silhost.exe','iexplore.exe','svchost.exe']))) == False]\n",
    "\n",
    "if (procfpredux.empty):\n",
    "    print(Fore.GREEN + \"[✓]No processes were identified for Damerau-Levenshtein analysis\")\n",
    "else:\n",
    "    print(Fore.RED + \"[!]Suspicious processes were identified\")\n",
    "    display(procfpredux)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "10d5ee25",
   "metadata": {},
   "source": [
    "### <a href=#pr4>PR4 - Identifying svchost.exe or rundll32.exe without any command line arguments</a>\n",
    "svchost.exe or rundll32.exe should always contain command line arguments. If it doesn't, then it is highly suspicious. rundll32.exe without command line arguments is one of the behaviours associated with Cobalt Strike. svchost.exe without parameters is highly abnormal and often used by an adversary to blend in."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9ecaa837",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "nocmdargs_svc = (dfcmdline[dfcmdline[\"Args\"].str.match(r'^(.*(svchost|rundll32)\\.exe)$')])[['PID','Process','Args']]\n",
    "if (nocmdargs_svc.empty):\n",
    "    print(Fore.GREEN + \"[✓]No svchost.exe or rundll32.exe instance(s) identified without commandline arguments\")\n",
    "else:\n",
    "    print(Fore.RED + \"[!]svchost.exe or rundll32.exe without command line was detected!\")\n",
    "    print(nocmdargs_svc.to_string(index = False))\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cbe002ba",
   "metadata": {},
   "source": [
    "### <a href=#pr5>PR5 - Executables having paths in suspicious folder locations</a>\n",
    "Executables having a path in directories such as Desktop, Public, Documents, Downloads, Temp, ProgramData (root) are often suspicious."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e9badc25",
   "metadata": {},
   "outputs": [],
   "source": [
    "sus_procspawn = (dfcmdline[dfcmdline[\"Args\"].str.match(r'.*[A-Za-z]:\\\\.*(Desktop|Public|Documents|Downloads|Temp|ProgramData)\\\\\\w+\\.exe.*')])[['PID','Process','Args']]\n",
    "if (sus_procspawn.empty):\n",
    "    print(Fore.GREEN + \"[✓]No processes were found to have been spawned from suspicious locations\")\n",
    "else:\n",
    "    print(Fore.RED + \"[!]Suspicious processes identified!\")\n",
    "    print(sus_procspawn.to_string(index = False))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cb2eefad",
   "metadata": {},
   "source": [
    "### <a href=#pr6>PR6 - Process network graph</a>\n",
    "Graph visualisation can assist in identifying suspicious processes using parent-child relationships. Use the interactive options within the data graph to zoom in/out."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9ce332d6",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "dfprocs.mp_plot.network(\n",
    "    source_col=\"ImageFileName\",\n",
    "    target_col=\"ParentName\",\n",
    "    source_attrs=[\"PID\", \"ParentName\",\"CreateTime\",\"ExitTime\"],\n",
    "    edge_attrs=[\"CreateTime\"],\n",
    ")\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "159fd327",
   "metadata": {},
   "source": [
    "### <a href=#pr7>PR7 - Anomalous Process Detection using time duration</a>\n",
    "This analyst task involves visually observing the life time of a process at the time the memory dump was taken. Processes which has a shorter lifespan are suspicious and may require further analysis. In addition, you may also detect malicious or suspicious process behaviour just by visually observing the spawn time of different process as time goes by. First we create a new dataframe containing processes that has exit timestamps. This way we are reducing the noise. Then we create a \"Timeline Duration\" visualisation using the msticpy library."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5ffc5072",
   "metadata": {},
   "outputs": [],
   "source": [
    "dfprocs_exit = dfprocs[dfprocs.ExitTime.notnull()]\n",
    "mp.vis.timeline_duration.display_timeline_duration(dfprocs_exit, group_by=\"ImageFileName\", time_column=\"CreateTime\", end_time_column=\"ExitTime\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eb6643c1",
   "metadata": {},
   "source": [
    "### <a href=#PR8>PR8 - Suspicious commandline arguments</a>\n",
    "Checks to identify suspicious commandline arguments which are often observed in intrusions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "609abfd8",
   "metadata": {},
   "outputs": [],
   "source": [
    "sus_cmdargs = dfcmdline[dfcmdline[\"Args\"].str.match(r\".*(nop|NoLogo|noni|invoke-expression|WebClient|EncodedCommand|nop|reflection\\.assembly|-enc|IO\\.Compression|DownloadString|System\\.Net\\.WebClient|Hidden|iex|FromBase64|powershell\\.cpl|base64|MemoryStream)|COMSPEC|IABz|SQBFAFgA|aWV4I|aQBlAHgA|SUVYI|sekurlsa|(whoami|certutil|bitsadmin|mshta|StartW|MiniDump|comsvcs\\.dll).*\", case=False) == True][['PID','Process','Args']]\n",
    "if (sus_cmdargs.empty):\n",
    "    print(Fore.GREEN + \"[✓]No suspicious commandline arguments detected\")\n",
    "else:\n",
    "    print(Fore.RED + \"[!]Suspicious commandline arguments detected!\")\n",
    "    print(sus_cmdargs.to_string(index = False))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0f5b07b3",
   "metadata": {},
   "outputs": [],
   "source": [
    "ioc_extractor = IoCExtract()\n",
    "ioc_df = ioc_extractor.extract(data=dfcmdline, columns=['Args'])\n",
    "if len(ioc_df):\n",
    "    display(\"IoC patterns found in process tree\")\n",
    "    display(ioc_df)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0c8a8870",
   "metadata": {},
   "source": [
    "### <a href=#PR9>PR9 - rundll32.exe spawned by a suspicious parent process</a>\n",
    "Attempting to detect rundll32.exe spawning from suspicious parent processes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "10e0d90a",
   "metadata": {},
   "outputs": [],
   "source": [
    "susdllloc = dfprocs[((dfprocs.ImageFileName.str.lower() == \"rundll32.exe\")) & (dfprocs.ParentName.str.lower().isin(['winword.exe', 'excel.exe','msaccess.exe','lsass.exe','taskeng.exe','winlogon.exe','schtask.exe','regsvr32.exe','wmiprvse.exe','wsmprovhost.exe']))][['ImageFileName','PID','PPID','ParentName']]\n",
    "if (susdllloc.empty):\n",
    "    print(Fore.GREEN + \"[✓]No suspicious rundll32 parent detected\")\n",
    "else:\n",
    "    print(Fore.RED + \"[!]Suspicious rundll32.exe parent detected!\")\n",
    "    print(susdllloc.to_string(index = False))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2f5c1739",
   "metadata": {},
   "source": [
    "### <a href=#NT1>NT 1 - Visual network interactions</a>\n",
    "The following matrix plot shows confirmed intersects between source and destination IPs with descending order of connections."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b318b7c4",
   "metadata": {},
   "outputs": [],
   "source": [
    "from msticpy.vis.matrix_plot import plot_matrix\n",
    "plot_matrix(data=dfnetscan, x=\"ForeignAddr\", y=\"LocalAddr\", title=\"IP interactions\", sort=\"dsc\" ,  intersect=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e3340dde",
   "metadata": {},
   "source": [
    "### <a href=#NT2>NT2 - No of external network communications per public IP</a>\n",
    "The following bar chart shows external network connections with the number of occurrences within the memory image."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "01db90da",
   "metadata": {},
   "outputs": [],
   "source": [
    "dfnetscan[(dfnetscan[\"ForeignAddr\"].str.match(r'(?!^0\\.)(?!^10\\.)(?!^100\\.6[4-9]\\.)(?!^100\\.[7-9]\\d\\.)(?!^100\\.1[0-1]\\d\\.)(?!^100\\.12[0-7]\\.)(?!^127\\.)(?!^169\\.254\\.)(?!^172\\.1[6-9]\\.)(?!^172\\.2[0-9]\\.)(?!^172\\.3[0-1]\\.)(?!^192\\.0\\.0\\.)(?!^192\\.0\\.2\\.)(?!^192\\.88\\.99\\.)(?!^192\\.168\\.)(?!^198\\.1[8-9]\\.)(?!^198\\.51\\.100\\.)(?!^203.0\\.113\\.)(?!^22[4-9]\\.)(?!^23[0-9]\\.)(?!^24[0-9]\\.)(?!^25[0-5]\\.)(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\\.([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\\.([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\\.([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5]))$') == True)][\"ForeignAddr\"].value_counts().plot(kind='bar')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "68681335",
   "metadata": {},
   "source": [
    "### <a href=#NT3>NT3 - Intelligence matches for external network connections</a>\n",
    "Closed network connections that communicate with public IPs may be of interest."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c728f215",
   "metadata": {},
   "outputs": [],
   "source": [
    "ext_netclose = dfnetscan[(dfnetscan[\"ForeignAddr\"].str.match(r'(?!^0\\.)(?!^10\\.)(?!^100\\.6[4-9]\\.)(?!^100\\.[7-9]\\d\\.)(?!^100\\.1[0-1]\\d\\.)(?!^100\\.12[0-7]\\.)(?!^127\\.)(?!^169\\.254\\.)(?!^172\\.1[6-9]\\.)(?!^172\\.2[0-9]\\.)(?!^172\\.3[0-1]\\.)(?!^192\\.0\\.0\\.)(?!^192\\.0\\.2\\.)(?!^192\\.88\\.99\\.)(?!^192\\.168\\.)(?!^198\\.1[8-9]\\.)(?!^198\\.51\\.100\\.)(?!^203.0\\.113\\.)(?!^22[4-9]\\.)(?!^23[0-9]\\.)(?!^24[0-9]\\.)(?!^25[0-5]\\.)(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\\.([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\\.([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\\.([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5]))$') == True)][[\"ForeignAddr\",\"ForeignPort\",\"Owner\"]]\n",
    "if (ext_netclose.empty):\n",
    "    print(Fore.GREEN + \"[✓]No closed external connections were detected\")\n",
    "else:\n",
    "    print(Fore.RED + \"[!]Terminated external network connections detected\")\n",
    "    ext_netclose.to_string(index = False)\n",
    "    print(\"\\n\")\n",
    "    print(\"Positive Threat Intelligence Enrichment on the IPs\")\n",
    "    ext_netclose.drop_duplicates(subset =\"ForeignAddr\", keep = False, inplace = True)\n",
    "    resraw_intel = intel.lookup_iocs(data=ext_netclose, ioc_col=\"ForeignAddr\")\n",
    "    res_intel = resraw_intel[resraw_intel[\"Result\"]==True]\n",
    "    \n",
    "    #Dropping columns that are not required \n",
    "    res_intel_cut = res_intel.drop(['QuerySubtype','RawResult','Result'], axis=1)\n",
    "    display(res_intel_cut)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aaca50e2",
   "metadata": {},
   "source": [
    "### <a href=#NT4>NT4 - Suspicious web connections that are not related to browsers</a>\n",
    "Web browsers usually communicate over port 80,443 or 8080. This shows processors which are not common browsers that communicate over those ports."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f84f1c72",
   "metadata": {},
   "outputs": [],
   "source": [
    "sus_http = dfnetscan[(dfnetscan[\"ForeignPort\"] == 8080) | (dfnetscan[\"ForeignPort\"] == 80) | (dfnetscan[\"ForeignPort\"] == 443) & ((dfnetscan.Owner.str.lower() != \"msmpeng.exe\") & (dfnetscan.Owner.str.lower() != \"chrome.exe\") & (dfnetscan.Owner.str.lower() != \"msedge.exe\") & (dfnetscan.Owner.str.lower() != \"firefox.exe\") & (dfnetscan.Owner.str.lower() != \"iexplorer.exe\"))][[\"ForeignAddr\",\"ForeignPort\",\"Owner\"]]\n",
    "if (sus_http.empty):\n",
    "    print(Fore.GREEN + \"[✓]No suspicious web connections were detected\")\n",
    "else:\n",
    "    print(Fore.RED + \"[!]None browser based web connections were detected\")\n",
    "    print(sus_http.to_string(index = False))\n",
    "    print(\"\\n\")\n",
    "    print(\"Positive Threat Intelligence Enrichment on the IPs\")\n",
    "    resraw_intel2 = intel.lookup_iocs(data=sus_http, ioc_col=\"ForeignAddr\", providers=[\"GreyNoise\",\"OTX\"])\n",
    "    res_intel2 = resraw_intel2[resraw_intel2[\"Result\"]==True]\n",
    "    \n",
    "    #Dropping columns that are not required \n",
    "    res_intel2_cut = res_intel2.drop(['QuerySubtype','RawResult','Result'], axis=1)\n",
    "    display(res_intel2_cut)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "11e56e04",
   "metadata": {},
   "source": [
    "### <a href=#NT5>NT5 - IP address in commandline arguments</a>\n",
    "Detect explicit use of IP addresses in command line arguments."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "350db136",
   "metadata": {},
   "outputs": [],
   "source": [
    "ip_cmdline = dfcmdline[dfcmdline[\"Args\"].str.match(r'.*(?!^0\\.)(?!^10\\.)(?!^100\\.6[4-9]\\.)(?!^100\\.[7-9]\\d\\.)(?!^100\\.1[0-1]\\d\\.)(?!^100\\.12[0-7]\\.)(?!^127\\.)(?!^169\\.254\\.)(?!^172\\.1[6-9]\\.)(?!^172\\.2[0-9]\\.)(?!^172\\.3[0-1]\\.)(?!^192\\.0\\.0\\.)(?!^192\\.0\\.2\\.)(?!^192\\.88\\.99\\.)(?!^192\\.168\\.)(?!^198\\.1[8-9]\\.)(?!^198\\.51\\.100\\.)(?!^203.0\\.113\\.)(?!^22[4-9]\\.)(?!^23[0-9]\\.)(?!^24[0-9]\\.)(?!^25[0-5]\\.)(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\\.([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\\.([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\\.([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])).*') == True][['PID','Process','Args']]\n",
    "if (ip_cmdline.empty):\n",
    "    print(Fore.GREEN + \"[✓]No IPs in commandline arguments detected\")\n",
    "else:\n",
    "    print(Fore.RED + \"[!]IP address in commandline arguments detected!\")\n",
    "    print(ip_cmdline.to_string(index = False))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "875121cf",
   "metadata": {},
   "source": [
    "### <a href=#st1>ST1 - Check for malicious Kernel callbacks (indicator of rootkit activity)</a>\n",
    "Malicious kernel callbacks often don't include the name of the kernel module. This results it being named as UNKNOWN. If you see such an entry, then you should use the driverscan plugin to check files that are surrounding the memory region of the malicious kernel callback."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "517b4bd0",
   "metadata": {},
   "outputs": [],
   "source": [
    "rootkit_indicator = dfcallbacks[(dfcallbacks.Module.str.lower() == \"unknown\")][[\"Type\",\"Callback\",\"Module\",\"Detail\"]]\n",
    "if (rootkit_indicator.empty):\n",
    "    print(Fore.GREEN + \"[✓]No suspicious callbacks detected\")\n",
    "else:\n",
    "    print(Fore.RED + \"[!]Suspicious callbacks identified (behaviour of a rootkit)!\")\n",
    "    print(rootkit_indicator.to_string(index = False))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ef89e4f2",
   "metadata": {},
   "source": [
    "### <a href=#st2>ST2 - Suspicious SSDT hooking (indicator of rootkit activity)</a>\n",
    "Suspicious SSDT hooking by kernel modules that do not reside in the usual directory paths."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e96ae03e",
   "metadata": {},
   "outputs": [],
   "source": [
    "sus_kernelhook = dfssdt[(dfssdt[\"Module\"].str.match(r\".*(ntoskrnl1|win32k).*\",case=False) == False)][[\"Address\",\"Module\",\"Symbol\"]]\n",
    "if (sus_kernelhook.empty):\n",
    "    print(Fore.GREEN + \"[✓]No suspicious SSDT hooking detected\")\n",
    "else:\n",
    "    print(Fore.RED + \"[!]Suspicious SSDT hooking detected!\")\n",
    "    print(sus_kernelhook.to_string(index = False))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ac213548",
   "metadata": {},
   "source": [
    "### <a href=#ST3>ST3 - Check for drivers with the base address wiped out (indicator of rootkit activity)</a>\n",
    "Malware can wipe the base address of its kernel driver to hide itself. If you want to locate the actual malicious file associated with this activity, then use callback plugin to locate the address range associated with it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3e6ffb37",
   "metadata": {},
   "outputs": [],
   "source": [
    "sus_driveraddwipe = dfdriverscan[(dfdriverscan.Start.str.lower() == \"0x00000000\")][[\"Offset\",\"Start\",\"Driver Name\",\"Name\",\"Service Key\"]]\n",
    "if (sus_driveraddwipe.empty):\n",
    "    print(Fore.GREEN + \"[✓]No drivers found with zeroed out base address\")\n",
    "else:\n",
    "    print(Fore.RED + \"[!]Driver(s) found with base address zeroed out (Stealth)!\")\n",
    "    print(sus_driveraddwipe.to_string(index = False))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6ec69115",
   "metadata": {},
   "source": [
    "### <a href=#SVC1>SVC1 - Suspicious Windows service file names</a>\n",
    "Certain malicious tools leverage Windows services to carry out their actions. This check attempts to identify such Windows services by searching for the Service binary information."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e90c6f84",
   "metadata": {},
   "outputs": [],
   "source": [
    "sus_svcfnres = dfsvcscan[dfsvcscan[\"Binary\"].str.match(r\".*(cmd|comspec|powershell|echo|%SYSTEMROOT%).*\",case=False) == True][['Offset','PID','Start','State','Type','Name','Display','Binary']]\n",
    "if (sus_svcfnres.empty):\n",
    "    print(Fore.GREEN + \"[✓]No suspicious Windows service file name(s) identified\")\n",
    "else:\n",
    "    print(Fore.RED + \"[!]Suspicious Windows service file name(s) identified!\")\n",
    "    print(sus_svcfnres.to_string(index = False))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2a75fd1e",
   "metadata": {},
   "source": [
    "### <a href=#SVC2>SVC2 - Suspicious Windows Service Names</a>\n",
    "Certain offensive security tools often tend to use fixed character length and cases when registering Windows Services."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "acd5fcb7",
   "metadata": {},
   "outputs": [],
   "source": [
    "sussvcnameres = dfsvcscan[(dfsvcscan[\"Name\"].str.match(r\"^([a-zA-Z]{4}|[a-zA-Z]{8}|[a-zA-Z]{16}|[a-z0-9]{11}|BTOBTO|WCESERVICE|WCE SERVICE|winexesvc|DumpSvc|pwdump|gsecdump|cachedump)$\",case=False) == True) & (dfsvcscan[\"Start\"] == \"SERVICE_BOOT_START\" ) & (dfsvcscan[\"Binary\"].str.match(r'(?i)(C:\\\\Windows\\\\System32\\\\)|((C:\\\\ProgramData\\\\Microsoft)|(C:\\\\Program Files\\\\Common Files)|\\\\Driver.*)', na=False,case=False) == False)][['Offset','PID','Start','State','Type','Name','Display','Binary']]\n",
    "if (sussvcnameres.empty):\n",
    "    print(Fore.GREEN + \"[✓]No suspicious Windows service names identified\")\n",
    "else:\n",
    "    print(Fore.RED + \"[!]Suspicious Windows service name(s) identified!\")\n",
    "    print(sussvcnameres.to_string(index = False))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4e776d5d",
   "metadata": {},
   "source": [
    "### <a href=#NP1>NP1 - Suspicious named pipe(s)</a>\n",
    "Named pipes allow communication between processes within a host or with other hosts via network. They are often used by adversaries during lateral movement and privilege escalation. This check will look at the most commonly used ​​named pipes associated with malicious or suspicious tools. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "44636aa5",
   "metadata": {},
   "outputs": [],
   "source": [
    "susnamedpipe = dfhandles[(dfhandles[\"Type\"] == \"File\") & (dfhandles[\"Name\"].str.match(r\"\\\\Device\\\\NamedPipe\\\\(MSSE-[0-9a-f]{3,4}-server|status_[0-9a-f]{2}|msagent_[0-9a-f]{2}|postex_ssh_[0-9a-f]{4}|postex_[0-9a-f]{4}|mojo\\.5688\\.8052|wkssvc[0-9a-f]{2}|ntsvcs[0-9a-f]{2}|DserNamePipe[0-9a-f]{2}|SearchTextHarvester[0-9a-f]{2}|mypipe-f[0-9a-f]{2}|mypipe-h[0-9a-f]{2}|windows\\.update\\.manager[0-9a-f]{2}|windows\\.update\\.manager[0-9a-f]{3}|ntsvcs_[0-9a-f]{2}|scerpc_[0-9a-f]{2}|scerpc[0-9a-f]{2}|ntsvcs[0-9a-f]{2}|gruntsvc|psexesvc|remcom_comunication|PAExec|csexecsvc|svcctl|Posh|jaccdpqnvbrrxlaf|TestSVC|remcom).*\",case=False) == True)][['PID','Process','Offset','Type','GrantedAccess','Name']]\n",
    "if (susnamedpipe.empty):\n",
    "    print(Fore.GREEN + \"[✓]No suspicious named pipes were identified\")\n",
    "else:\n",
    "    print(Fore.RED + \"[!]Suspicious named pipe(s) detected!\")\n",
    "    print(susnamedpipe.to_string(index = False))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7b59b245",
   "metadata": {},
   "source": [
    "### <a href=#dl2>DL2- DLLs or Sys files located in unusual locations</a>\n",
    "Dlls or drivers files are rarely seen in Downloads, Temp, Documents, Public etc directories. The dlllist shows mapped DLLs for each process within the memory dump."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e6407d1c",
   "metadata": {},
   "outputs": [],
   "source": [
    "susdllloc = dfdlllist[(dfdlllist[\"Path\"].str.match(r\"^[A-Za-z]:\\\\.*(Temp|INetCache|Desktop|Downloads|Content|Documents|Public|ProgramData|Program Files|Program Files \\(x86\\))\\\\(\\w+).(dll|exe)$\",case=False) == True)][['PID','Process','Base','Name','Path','LoadTime']]\n",
    "if (susdllloc.empty):\n",
    "    print(Fore.GREEN + \"[✓]No DLL(s) located in unusual location(s)\")\n",
    "else:\n",
    "    print(Fore.RED + \"[!]Suspicious file identified in an unusual location!\")\n",
    "    print(susdllloc.to_string(index = False))\n",
    "    #Add other files associated with this finding for context\n",
    "    #Add potential mutexes created by the suspicious processes\n",
    "   "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f25c1da0",
   "metadata": {},
   "source": [
    "### <a href=#dl3>DL3- DLL(s) or Sys files with unusually long file names</a>\n",
    "Dlls or driver files with significantly long names are deemded suspicious."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8ade61e9",
   "metadata": {},
   "outputs": [],
   "source": [
    "susdllfname = dfdlllist[(dfdlllist[\"Name\"].str.match(r\"^[a-z0-9]{35,}\\.(dll|sys)\",case=False) == True)][['PID','Process','Base','Name','Path','LoadTime']]\n",
    "if (susdllfname.empty):\n",
    "    print(Fore.GREEN + \"[✓]No DLL(s) or Sys files with unusually long file names were detected\")\n",
    "else:\n",
    "    print(Fore.RED + \"[!]DLL(s) or Sys files with unusually long file names were detected!\")\n",
    "    print(susdllfname.to_string(index = False))\n",
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e89d72a5",
   "metadata": {},
   "source": [
    "### <a href=#mal1>MAL1 - MZ header detection within malfind preview</a>\n",
    "malfind plugin identifies hidden or injected code/DLLs in user mode memory. This often results in having the MZ header visible in the memory segment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "74a7d8b6",
   "metadata": {},
   "outputs": [],
   "source": [
    "procinject1=dfmalfind[dfmalfind[\"Hexdump\"].str.contains(\"MZ\") == True][['PID','Process','Start VPN','End VPN','Protection','Hexdump','Disasm']]\n",
    "if (procinject1.empty):\n",
    "    print(Fore.GREEN + \"[✓]No MZ header detected in malfind preview output\")\n",
    "else:\n",
    "    print(Fore.RED + \"[!]MZ header detected within malfind preview (Process Injection indicator)\")\n",
    "    print(procinject1.to_string(index = False))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7e858c72",
   "metadata": {},
   "source": [
    "### <a href=#mal2>MAL2 - Assembly code detection within malfind preview</a>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ffbfc98a",
   "metadata": {},
   "outputs": [],
   "source": [
    "procinject2=dfmalfind[dfmalfind[\"Hexdump\"].str.match(r\".*(mov\\s+epb,\\s?esp).*\") == True][['PID','Process','Start VPN','End VPN','Protection','Hexdump','Disasm']]\n",
    "if (procinject2.empty):\n",
    "    print(Fore.GREEN + \"[✓]No suspicious assembly code detected in malfind preview output\")\n",
    "else:\n",
    "    print(Fore.RED + \"[!]Suspicious assembly code detected in malfind preview output!\")\n",
    "    print(procinject2.to_string(index = False))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5b3b297a",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}