[DOC]: Ransomware Detection example, how to produce the input data?

[nyck33]

How would you describe the priority of this documentation request

Medium.

Describe the future/missing documentation

I asked ChatGPT which said,

The intended workflow for Morpheus likely involves automated data collection from various system components, which is then processed and analyzed for anomalies or patterns indicative of ransomware or other malicious activities. The data collection could be done through specialized software that monitors system activities and generates these JSON files. This software could be running on each client machine or could be part of a centralized monitoring system.

If you're looking to replicate this on a Linux system like Ubuntu, you could indeed write a series of Python scripts to collect similar data. However, a more scalable approach might be to use system monitoring tools that can output data in a format that Morpheus can ingest.

For example, you could use:

• ps for process listing
• netstat for network connections
• lsof for open files
• env for environment variables

You could then schedule these commands to run at regular intervals using cron and output their results to JSON files. These JSON files could then be sent to a centralized location for analysis by Morpheus.

Here's a simplified example using Python to collect process list and environment variables, similar to pslist and envars:

import json
import os
import subprocess

def get_pslist():
    ps_output = subprocess.getoutput("ps aux")
    ps_list = ps_output.strip().split("\n")[1:]
    return [{"user": x.split()[0], "pid": x.split()[1], "cmd": " ".join(x.split()[10:])} for x in ps_list]

def get_envars():
    return dict(os.environ)

def main():
    pslist_data = {
        "plugin": "pslist",
        "data": get_pslist()
    }

    envars_data = {
        "plugin": "envars",
        "data": get_envars()
    }

    with open("pslist.json", "w") as f:
        json.dump(pslist_data, f, indent=4)

    with open("envars.json", "w") as f:
        json.dump(envars_data, f, indent=4)

if __name__ == "__main__":
    main()

You can then schedule this script to run every 30 minutes using cron:

# Open the crontab editor
crontab -e

# Add the following line to run the script every 30 minutes
*/30 * * * * /usr/bin/python3 /path/to/your_script.py

This is a simplified example and may not capture all the details you need, but it should give you a starting point.

Although the example given in the repo seems to be Windows? Also, where can I learn more about the required inputs for this model? Ie. can I use it with fewer kinds of .json files in each snaphsot-x folder or more? I saw this: https://docs.nvidia.com/morpheus/models_and_datasets.html but there's no detailed explanations of the models. Ideally I'd like to read up on each model available on Morpheus with links to source code even if that's possible (or like detailed model cards? are they called like on Hugging Face that descrtibe what the model expects in terms of input and what it outputs (shape, types, etc.). That's two questions in one issue here with priority for the first one.

Where have you looked?

https://github.com/nyck33/Morpheus_nkim_fork/tree/branch-23.11/examples/ransomware_detection
https://docs.nvidia.com/morpheus/models_and_datasets.html
https://github.com/nyck33/Morpheus_nkim_fork/blob/branch-23.11/models/ransomware-models/ransomw-model-short-rf-20220126.sav

Code of Conduct

• I agree to follow this project's Code of Conduct
• I have searched the open documentation issues and have found no duplicates for this bug report

[jarmak-nv]

Hi @nyck33!

Thanks for submitting this issue - our team has been notified and we'll get back to you as soon as we can!
In the mean time, feel free to add any relevant information to this issue.

[nyck33]

I found some things like https://github.com/nv-morpheus/Morpheus/blob/branch-23.11/examples/ransomware_detection/common/feature_extractor.py