Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SOLVED] Analysis does not work #1967

Closed
4 of 6 tasks
eingel86 opened this issue Feb 15, 2024 · 18 comments
Closed
4 of 6 tasks

[SOLVED] Analysis does not work #1967

eingel86 opened this issue Feb 15, 2024 · 18 comments

Comments

@eingel86
Copy link

eingel86 commented Feb 15, 2024 •
edited
Loading

About accounts on capesandbox.com

  • Issues isn't the way to ask for account activation. Ping capesandbox in Twitter with your username

This is open source and you are getting free support so be friendly!

Prerequisites

Please answer the following questions for yourself before submitting an issue.

  • I am running the latest version
  • I did read the README!
  • I checked the documentation and found no answer
  • I checked to make sure that this issue has not already been filed
  • I'm reporting the issue to the correct repository (for multi-repository projects)
  • I have read and checked all configs (with all optional parts)

Expected Behavior

Please describe the behavior you are expecting. If your samples(x64) stuck in pending ensure that you set tags=x64 in hypervisor conf for x64 vms --> The analysis remains running and does not appear to be parsing the file. Yes I set the tag to x64.

Current Behavior

I am trying to run capev2 but I can't get it to work. From the web I import the file to be analyzed, select the vm where to test it and as soon as I select the analysis the process stays running without running.

Failure Information (for bugs)

I cannot tell whether it is a bug or an incorrect configuration of the environment

Failure Logs

feb 15 15:18:35 cape-sandbox systemd[1]: Started CAPE.
feb 15 15:18:37 cape-sandbox python3[16900]:
feb 15 15:18:37 cape-sandbox python3[16900]: .-----------------.
feb 15 15:18:37 cape-sandbox python3[16900]: | Cuckoo Sandbox? |
feb 15 15:18:37 cape-sandbox python3[16900]: | OH NOES! |\ '-.__.-'
feb 15 15:18:37 cape-sandbox python3[16900]: '-----------------' \ /oo |--.--,--,--.
feb 15 15:18:37 cape-sandbox python3[16900]: _.-'.i__i__i.'
feb 15 15:18:37 cape-sandbox python3[16900]: """""""""
feb 15 15:18:37 cape-sandbox python3[16900]: Cuckoo Sandbox 2.4-CAPE
feb 15 15:18:37 cape-sandbox python3[16900]: www.cuckoosandbox.org
feb 15 15:18:37 cape-sandbox python3[16900]: Copyright (c) 2010-2015
feb 15 15:18:37 cape-sandbox python3[16900]: CAPE: Config and Payload Extraction
feb 15 15:18:37 cape-sandbox python3[16900]: github.com/kevoreilly/CAPEv2
feb 15 15:18:38 cape-sandbox python3[16900]: XLMMacroDeobfuscator: pywin32 is not installed (only is required if you want to use MS Excel)
feb 15 15:18:38 cape-sandbox python3[16900]: FLARE_CAPA InvalidRuleSet
feb 15 15:18:38 cape-sandbox python3[16900]: Unable to import plugin "modules.processing.static": multiple exception types must be parenthesized (static.py, line 936)

feb 15 15:18:39 cape-sandbox python3[16958]: /usr/bin/tcpdump
feb 15 15:18:39 cape-sandbox python3[16900]: 2024-02-15 15:18:39,756 [lib.cuckoo.core.scheduler] INFO: Using "kvm" machine manager with max_analysis_count=1, max_machines_count=10, and max_vmstartup_count=5
feb 15 15:18:39 cape-sandbox python3[16900]: 2024-02-15 15:18:39,760 [lib.cuckoo.core.scheduler] INFO: Loaded 3 machine/s
feb 15 15:18:39 cape-sandbox python3[16900]: 2024-02-15 15:18:39,765 [lib.cuckoo.core.scheduler] INFO: Waiting for analysis tasks

feb 15 15:24:12 cape-sandbox python3[16900]: 2024-02-15 15:24:12,202 [lib.cuckoo.core.scheduler] INFO: Task #18: Starting analysis of FILE '/tmp/cuckoo-sflock/tmpowxzjs31/25c7d6530d1b7b4e8fe1.pdf'
feb 15 15:24:12 cape-sandbox python3[16900]: 2024-02-15 15:24:12,252 [lib.cuckoo.core.scheduler] INFO: Task #18: acquired machine Win10x64 (label=Win10x64, arch=x64, platform=windows)
feb 15 15:30:31 cape-sandbox python3[16900]: 2024-02-15 15:30:31,527 [lib.cuckoo.core.scheduler] ERROR: Timeout hit while for machine Win10x64 to change status
feb 15 15:30:31 cape-sandbox python3[16900]: Traceback (most recent call last):
feb 15 15:30:31 cape-sandbox python3[16900]: File "/opt/CAPEv2/lib/cuckoo/core/scheduler.py", line 491, in launch_analysis
feb 15 15:30:31 cape-sandbox python3[16900]: machinery.start(self.machine.label)
feb 15 15:30:31 cape-sandbox python3[16900]: File "/opt/CAPEv2/modules/machinery/kvm.py", line 35, in start
feb 15 15:30:31 cape-sandbox python3[16900]: super(KVM, self).start(label)
feb 15 15:30:31 cape-sandbox python3[16900]: File "/opt/CAPEv2/lib/cuckoo/common/abstracts.py", line 469, in start
feb 15 15:30:31 cape-sandbox python3[16900]: self._wait_status(label, self.RUNNING)
feb 15 15:30:31 cape-sandbox python3[16900]: File "/opt/CAPEv2/lib/cuckoo/common/abstracts.py", line 364, in _wait_status
feb 15 15:30:31 cape-sandbox python3[16900]: raise CuckooMachineError(f"Timeout hit while for machine {label} to change status")
feb 15 15:30:31 cape-sandbox python3[16900]: lib.cuckoo.common.exceptions.CuckooMachineError: Timeout hit while for machine Win10x64 to change status
feb 15 15:30:31 cape-sandbox python3[16900]: 2024-02-15 15:30:31,561 [lib.cuckoo.core.scheduler] WARNING: Task #18: Unable to stop machine Win10x64: Trying to stop an already stopped machine Win10x64
feb 15 15:30:31 cape-sandbox python3[16900]: 2024-02-15 15:30:31,593 [lib.cuckoo.core.resultserver] WARNING: ResultServer did not have a task with ID 18 and IP 192.168.122.126
feb 15 15:30:31 cape-sandbox python3[16900]: 2024-02-15 15:30:31,624 [lib.cuckoo.core.scheduler] ERROR:
feb 15 15:30:31 cape-sandbox python3[16900]: Traceback (most recent call last):
feb 15 15:30:31 cape-sandbox python3[16900]: File "/opt/CAPEv2/lib/cuckoo/core/scheduler.py", line 613, in run
feb 15 15:30:31 cape-sandbox python3[16900]: success = self.launch_analysis()
feb 15 15:30:31 cape-sandbox python3[16900]: File "/opt/CAPEv2/lib/cuckoo/core/scheduler.py", line 588, in launch_analysis
feb 15 15:30:31 cape-sandbox python3[16900]: raise CuckooDeadMachine()
feb 15 15:30:31 cape-sandbox python3[16900]: lib.cuckoo.core.scheduler.CuckooDeadMachine
feb 15 15:30:31 cape-sandbox python3[16900]: 2024-02-15 15:30:31,642 [lib.cuckoo.core.scheduler] INFO: Task #18: File already exists at '/opt/CAPEv2/storage/binaries/25c7d6530d1b7b4e8fe1b262234ee8691434ee9ca24e12b3f3419681f0de7208'
feb 15 15:30:31 cape-sandbox python3[16900]: 2024-02-15 15:30:31,643 [lib.cuckoo.core.scheduler] INFO: Task #18: Starting analysis of FILE '/tmp/cuckoo-sflock/tmpowxzjs31/25c7d6530d1b7b4e8fe1.pdf'

If instead I launch the command: journalctl -u cape-processor.service --follow , I get the following messages:

feb 15 15:18:35 cape-sandbox systemd[1]: Started CAPE report processor.
feb 15 15:18:38 cape-sandbox python3[16898]: XLMMacroDeobfuscator: pywin32 is not installed (only is required if you want to use MS Excel)
feb 15 15:18:38 cape-sandbox python3[16898]: FLARE_CAPA InvalidRuleSet
feb 15 15:18:38 cape-sandbox python3[16898]: Unable to import plugin "modules.processing.static": multiple exception types must be parenthesized (static.py, line 936)
feb 15 15:18:39 cape-sandbox python3[16898]: INFO:root:Processing analysis data

_via kvm web I see that the vms are not switched on and the file is not exploded.

what could be the reason? Thank you in advance for your help.

Regards
Engel_
@doomedraven
Copy link
Collaborator

first of all, why do you remove issues template? there is important info that we need to ask over and over, restore issue template, fill the date and then i will check your issue, otherwise i will just ignore

@doomedraven
Copy link
Collaborator

but it looks like your vm is in wrong state or bad permissions

[lib.cuckoo.core.scheduler] ERROR: Timeout hit while for machine Win10x64 to change status

i have feeling that you didn't read the whole docs https://capev2.readthedocs.io/en/latest/installation/guest/index.html

@eingel86
Copy link
Author

eingel86 commented Feb 15, 2024 •
edited
Loading

Dear @doomedraven thank you for your support and patience. What do you mean by "why do you remove issues template?"?

I stopped all services, then enabled the log view, then re-enabled all services and via web launched the file analysis. If more info is needed, can you tell me what files, commands, or anything else to provide?

I apologize if the information provided was not complete.

Are the VMs when I launch the analysis turned off, should they be turned on?

I created the VMs using this link: https://www.doomedraven.com/2020/04/how-to-create-virtual-machine-with-virt.html, but I believe I did not perform the steps instead described on this other page: https://www.doomedraven.com/2016/05/kvm.html#modifying-kvm-qemu-kvm-settings-for-malware-analysis.

This could be the problem.

Regards
Engel

@doomedraven
Copy link
Collaborator

issues ->

Having problem/bug/issue
Create a report to help us improve

all this info is critical for us and saves us a lot of time asking each user the same question over and over

## About accounts on [capesandbox.com](https://capesandbox.com/)
* Issues isn't the way to ask for account activation. Ping capesandbox in [Twitter](https://twitter.com/capesandbox) with your username

## This is open source and you are getting __free__ support so be friendly!

# Prerequisites

Please answer the following questions for yourself before submitting an issue.

- [ ] I am running the latest version
- [ ] I did read the README!
- [ ] I checked the documentation and found no answer
- [ ] I checked to make sure that this issue has not already been filed
- [ ] I'm reporting the issue to the correct repository (for multi-repository projects)
- [ ] I have read and checked all configs (with all optional parts)


# Expected Behavior

Please describe the behavior you are expecting. __If your samples(x64) stuck in pending ensure that you set tags=x64 in hypervisor conf for x64 vms__

# Current Behavior

What is the current behavior?

# Failure Information (for bugs)

Please help provide information about the failure if this is a bug. If it is not a bug, please remove the rest of this template.

## Steps to Reproduce

Please provide detailed steps for reproducing the issue.

1. step 1
2. step 2
3. you get it...

## Context

Please provide any relevant information about your setup. This is important in case the issue is not reproducible except for under certain conditions. Operating system version, bitness, installed software versions, test sample details/hash/binary (if applicable).

| Question         | Answer
|------------------|--------------------
| Git commit       | Type `$ git log \| head -n1` to find out
| OS version       | Ubuntu 16.04, Windows 10, macOS 10.12.3

## Failure Logs

Please include any relevant log snippets or files here.

about vm. my guide is how to create VM not related to CAPE. as CAPE docs says VM should be in running state when you take snapshot. please go and read WHOLE cape docs. you doing very basic mistakes that are explained in docs

@eingel86
Copy link
Author

Dear @doomedraven, Thank you for the clarification and information. I updated the first post by including the problem template, I apologize for not filling it out.

I will try to redo the VMs and related snapshots.

If it's not too much trouble, could you help me better understand how CAPE handles virtual machines before and after running a scan? Once the malicious object is launched in the VM and the analysis is provided, is the VM closed and deleted and then restored from the snapshot?

Regards
Engel

@doomedraven
Copy link
Collaborator

not exactly. You prepare a new clean VM, take snapshot in running state, cape restores that snapshot, submit malware, run it and turn off vm. on next sample run it restore clean running snapshot and so on in loop

@doomedraven
Copy link
Collaborator

please reread the documentation, if you have any question feel free to ask here

@eingel86
Copy link
Author

eingel86 commented Feb 22, 2024 •
edited
Loading

Dear @doomedraven,

I rebuilt the cape server according to the documentation. Now the VM starts, I can also see on the Gui side that it is switched on, but no action is performed. Afterwards the VM is switched off. On the gui side the system stays running and then after a while it goes to failure. Below is the log of the last analysis test performed:

feb 22 09:42:57 capev2sandbox systemd[1]: cape.service: State 'stop-sigterm' timed out. Killing.
feb 22 09:42:57 capev2sandbox systemd[1]: cape.service: Killing process 11186 (python) with signal SIGKILL.
feb 22 09:42:57 capev2sandbox systemd[1]: cape.service: Killing process 11197 (python) with signal SIGKILL.
feb 22 09:42:57 capev2sandbox systemd[1]: cape.service: Main process exited, code=killed, status=9/KILL
feb 22 09:42:57 capev2sandbox systemd[1]: cape.service: Failed with result 'timeout'.
feb 22 09:42:57 capev2sandbox systemd[1]: Stopped CAPE.
feb 22 09:42:57 capev2sandbox systemd[1]: cape.service: Consumed 5min 12.499s CPU time.

feb 22 09:59:48 capev2sandbox systemd[1]: Started CAPE.
feb 22 09:59:50 capev2sandbox python3[11542]:
feb 22 09:59:50 capev2sandbox python3[11542]: .-----------------.
feb 22 09:59:50 capev2sandbox python3[11542]: | Cuckoo Sandbox? |
feb 22 09:59:50 capev2sandbox python3[11542]: | OH NOES! |\ '-.__.-'
feb 22 09:59:50 capev2sandbox python3[11542]: '-----------------' \ /oo |--.--,--,--.
feb 22 09:59:50 capev2sandbox python3[11542]: _.-'.i__i__i.'
feb 22 09:59:50 capev2sandbox python3[11542]: """""""""
feb 22 09:59:50 capev2sandbox python3[11542]: Cuckoo Sandbox 2.4-CAPE
feb 22 09:59:50 capev2sandbox python3[11542]: www.cuckoosandbox.org
feb 22 09:59:50 capev2sandbox python3[11542]: Copyright (c) 2010-2015
feb 22 09:59:50 capev2sandbox python3[11542]: CAPE: Config and Payload Extraction
feb 22 09:59:50 capev2sandbox python3[11542]: github.com/kevoreilly/CAPEv2
feb 22 09:59:51 capev2sandbox python3[11542]: Unable to import plugin "modules.processing.sysmon": No module named 'xmltodict'
feb 22 09:59:51 capev2sandbox python3[11542]: No module named 'pydantic.functional_validators'
feb 22 09:59:51 capev2sandbox python3[11542]: FLARE-CAPA missed: poetry install
feb 22 09:59:51 capev2sandbox python3[11542]: 2024-02-22 09:59:51,851 [root] INFO: Updated running task ID 1 status to failed_analysis
feb 22 09:59:51 capev2sandbox python3[11569]: /usr/bin/tcpdump
feb 22 09:59:51 capev2sandbox python3[11542]: 2024-02-22 09:59:51,961 [lib.cuckoo.core.scheduler] INFO: Using "kvm" machine manager with max_analysis_count=0, max_machines_count=0, and max_vmstartup_count=0
feb 22 09:59:51 capev2sandbox python3[11542]: 2024-02-22 09:59:51,965 [lib.cuckoo.core.scheduler] INFO: Loaded 2 machine/s
feb 22 09:59:51 capev2sandbox python3[11542]: 2024-02-22 09:59:51,969 [lib.cuckoo.core.scheduler] INFO: Waiting for analysis tasks
feb 22 10:00:53 capev2sandbox python3[11542]: 2024-02-22 10:00:53,816 [lib.cuckoo.core.scheduler] INFO: Task #3: File already exists at '/opt/CAPEv2/storage/binaries/d60df902cea410c6cecc6c0852b1ee001cd89e298b2376288dde406e0ea2c59a'
feb 22 10:00:53 capev2sandbox python3[11542]: 2024-02-22 10:00:53,818 [lib.cuckoo.core.scheduler] INFO: Task #3: Starting analysis of FILE '/tmp/cuckoo-sflock/tmp6hunvt3q/d60df902cea410c6cecc.msi'
feb 22 10:00:54 capev2sandbox python3[11542]: 2024-02-22 10:00:54,057 [lib.cuckoo.core.scheduler] INFO: Task #3: acquired machine win10x64 (label=win10x64, arch=x64, platform=windows)
feb 22 10:01:12 capev2sandbox python3[11542]: 2024-02-22 10:01:12,031 [lib.cuckoo.common.integrations.parse_pe] ERROR: PE type not recognised: 'DOS Header magic not found.'
feb 22 10:01:12 capev2sandbox python3[11542]: 2024-02-22 10:01:12,089 [lib.cuckoo.core.scheduler] INFO: Enabled route 'none'.
feb 22 10:01:12 capev2sandbox python3[11542]: 2024-02-22 10:01:12,152 [lib.cuckoo.core.guest] INFO: Task #3: Starting analysis on guest (id=win10x64, ip=192.168.55.2)
feb 22 10:17:12 capev2sandbox python3[11542]: 2024-02-22 10:17:12,884 [lib.cuckoo.core.scheduler] ERROR: Machine win10x64: the guest initialization hit the critical timeout, analysis aborted
feb 22 10:17:12 capev2sandbox python3[11542]: Traceback (most recent call last):
feb 22 10:17:12 capev2sandbox python3[11542]: File "/opt/CAPEv2/lib/cuckoo/core/scheduler.py", line 526, in launch_analysis
feb 22 10:17:12 capev2sandbox python3[11542]: guest.start_analysis(options)
feb 22 10:17:12 capev2sandbox python3[11542]: File "/opt/CAPEv2/lib/cuckoo/core/guest.py", line 245, in start_analysis
feb 22 10:17:12 capev2sandbox python3[11542]: self.wait_available()
feb 22 10:17:12 capev2sandbox python3[11542]: File "/opt/CAPEv2/lib/cuckoo/core/guest.py", line 154, in wait_available
feb 22 10:17:12 capev2sandbox python3[11542]: raise CuckooGuestCriticalTimeout(
feb 22 10:17:12 capev2sandbox python3[11542]: lib.cuckoo.common.exceptions.CuckooGuestCriticalTimeout: Machine win10x64: the guest initialization hit the critical timeout, analysis aborted
feb 22 10:22:17 capev2sandbox python3[11542]: 2024-02-22 10:22:17,197 [lib.cuckoo.core.scheduler] ERROR:
feb 22 10:22:17 capev2sandbox python3[11542]: Traceback (most recent call last):
feb 22 10:22:17 capev2sandbox python3[11542]: File "/opt/CAPEv2/lib/cuckoo/core/scheduler.py", line 618, in run
feb 22 10:22:17 capev2sandbox python3[11542]: success = self.launch_analysis()
feb 22 10:22:17 capev2sandbox python3[11542]: File "/opt/CAPEv2/lib/cuckoo/core/scheduler.py", line 593, in launch_analysis
feb 22 10:22:17 capev2sandbox python3[11542]: raise CuckooDeadMachine()
feb 22 10:22:17 capev2sandbox python3[11542]: lib.cuckoo.core.scheduler.CuckooDeadMachine
feb 22 10:22:18 capev2sandbox python3[11542]: 2024-02-22 10:22:18,516 [lib.cuckoo.core.scheduler] INFO: Task #3: File already exists at '/opt/CAPEv2/storage/binaries/d60df902cea410c6cecc6c0852b1ee001cd89e298b2376288dde406e0ea2c59a'
feb 22 10:22:18 capev2sandbox python3[11542]: 2024-02-22 10:22:18,662 [lib.cuckoo.core.scheduler] INFO: Task #3: Starting analysis of FILE '/tmp/cuckoo-sflock/tmp6hunvt3q/d60df902cea410c6cecc.msi'

Let me know what other info, logs or captures you need so I can help. Thank you very much.

Note: as execution time I set 900.

Regards
Engel

@doomedraven
Copy link
Collaborator

is your VM is in running state when you take snapshot?

@eingel86
Copy link
Author

yes I confirm, they were up and running when I ran the snapshot.

@doomedraven
Copy link
Collaborator

did you test this? https://capev2.readthedocs.io/en/latest/installation/guest/agent.html#installing-the-agent

@eingel86
Copy link
Author

eingel86 commented Feb 22, 2024 •
edited
Loading

The test fails:
curl: (7) Failed to connect to 192.168.55.2 port 8000 after 0 ms: Connection refused

Going up to the VM, and running the netstat command, I see no open ports on 8000. The agent is saved in the folder "C:\ProgramData\Microsoft\Windows\Start Menu\Programs\StartUp" and I have also configured the "Task Scheduler" but it doesn't seem to start. Python 3.12 is installed and the file is saved with the extension pyw

Windows FIrewall: disable
Windows Defender: disable

@doomedraven
Copy link
Collaborator

well congrats you found the problem. now is your job to fix that, see docs how we suggest to run agent, and always verify that before taking snapshot

@eingel86
Copy link
Author

Yeah, too bad it doesn't run. When I launch it, cmd opens and closes again immediately, and it does this with both .pyw and .py extensions. I'll try to find out why.

Can you confirm that the file agent is: https://github.com/kevoreilly/CAPEv2/blob/master/agent/agent.py ?

Thanks :)

@doomedraven
Copy link
Collaborator

yes the file is correct, comment out those line to see the output https://github.com/kevoreilly/CAPEv2/blob/master/agent/agent.py#L58-L59, but later for production uncomment them

@eingel86
Copy link
Author

I uninstalled python 3.12 and installed the 32bit python 3.10.6 version, the 64bit version gives problems. Now running the command: curl VM_IP:8000 I get:

{"message": "CAPE Agent!", "version": "0.12", "features": ["execpy", "execute", "pinning", "logs", "largefile", "unicodepath"], "is_user_admin": false}admcape@cape

We should be there, right?

@eingel86
Copy link
Author

I'd say it's working now. I am doing some tests but I should have solved it. Thank you very much for your support and patience

@eingel86 eingel86 changed the title Analysis does not work [SOLVED] Analysis does not work Feb 22, 2024
@doomedraven
Copy link
Collaborator

dude you def need to start reading better everything https://github.com/kevoreilly/CAPEv2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@doomedraven @eingel86