Skip to content

Latest commit

 

History

History
68 lines (59 loc) · 2.49 KB

README.md

File metadata and controls

68 lines (59 loc) · 2.49 KB
Raw

Preemptive Anomaly Detection of IoT Network on Knowledge Graph

Source File Location

  • GitHub code repository - link
  • code & binary files - link

Using Ampligraph

for non-temporal models

  1. Follow the instruction here to install Ampligraph
  2. Run one of the model code in /ampligraph/ampligraph_*.py

Example:

$: conda activate ampligraph
$: python3 ampligraph_TransE.py

Using Horovod

for both TA-based & DE-based models

  1. Follow the instructions here to install Horovod
  2. Follow the instructions here to install the TKGC codes
  3. Run one of the TKGC models

Example:

$: horovodrun -np 2 -H localhost:2 python -BW ignore main.py --dataset deNetwork3_1 --model TTransE --dropout 0.2 --embedding-size 8 --learning-rate 0.01 --epochs 100 --batch-size 32 --test-batch-size 30 --negative-samples 64 --filter --mode head --validation-frequency 2 --threads 2 --workers 1

Summary of Horovod & TKGC installation:

conda create -n horovod python=3.6
conda install -y openmpi
conda install -y -c cmake
conda install -y -c pytorch pytorch
conda install -y gxx_linux-64
HOROVOD_WITH_PYTORCH=1 pip install -v --no-cache-dir horovod[pytorch]
pip install -r requirements.txt

Converting raw network logs to CSV

  1. Amend the converter/main.py to use the appropriate dataset converter

Example:

from preprocessing.log2dataset_tkgc
#from preprocessing.log2dataset_tkgcTA
#from preprocessing.log2dataset
  1. Place the raw network log dataset conn.log.labeled that is to be converted
  2. Run the script converter/main.py to generate output.csv

Import into Neo4j

converting the immediate representation to knowledge graph

  1. Install Neo4j and create a graph database
  2. Place the output.csv in the database import folder
  3. Run the following cypher command:
LOAD CSV FROM "file:///output.csv" as row

CALL apoc.merge.node(['uid'], {id:row[0]}, {})
YIELD node as startNode
CALL apoc.merge.node([row[1]], {id:row[2]}, {})
YIELD node as endNode
CALL apoc.create.relationship(startNode, row[1], {id:row[1]}, endNode) yield rel
RETURN distinct 'done'

Experiment Results

The experiment results for both temporal (Horovod/TKGC) and non-temporal (Ampligraph) models are collated in the results folder