No open port report in Deepexploit and scan show all ports open

[matcon]

everything was wonderful these days training with metasploitable, since before yesterday I scan but can not find open ports despite creating the report as an example: nmap_result_192.168.56.101
Starting Nmap 7.70 ( https://nmap.org ) at 2019-07-19 20:40 -04
Nmap scan report for 192.168.56.101
Host is up (0.0015s latency).
Not shown: 65506 closed ports
PORT STATE SERVICE VERSION
21/tcp open ftp vsftpd 2.3.4
22/tcp open ssh OpenSSH 4.7p1 Debian 8ubuntu1 (protocol 2.0)
23/tcp open telnet Linux telnetd
25/tcp open smtp Postfix smtpd
53/tcp open domain ISC BIND 9.4.2
80/tcp open http Apache httpd 2.2.8 ((Ubuntu) DAV/2)
111/tcp open rpcbind 2 (RPC #100000)
139/tcp open netbios-ssn Samba smbd 3.X - 4.X (workgroup: WORKGROUP)
445/tcp open netbios-ssn Samba smbd 3.X - 4.X (workgroup: WORKGROUP)
512/tcp open exec netkit-rsh rexecd
513/tcp open login
514/tcp open shell Netkit rshd
1099/tcp open rmiregistry GNU Classpath grmiregistry
1524/tcp open bindshell Metasploitable root shell
2049/tcp open nfs 2-4 (RPC #100003)
2121/tcp open ftp ProFTPD 1.3.1
3306/tcp open mysql MySQL 5.0.51a-3ubuntu5
3632/tcp open distccd distccd v1 ((GNU) 4.2.4 (Ubuntu 4.2.4-1ubuntu4))
5432/tcp open postgresql PostgreSQL DB 8.3.0 - 8.3.7
5900/tcp open vnc VNC (protocol 3.3)
6000/tcp open X11 (access denied)
6667/tcp open irc UnrealIRCd
6697/tcp open irc UnrealIRCd
8009/tcp open ajp13 Apache Jserv (Protocol v1.3)
8180/tcp open http Apache Tomcat/Coyote JSP engine 1.1
8787/tcp open drb Ruby DRb RMI (Ruby 1.8; path /usr/lib/ruby/1.8/drb)
35544/tcp open mountd 1-3 (RPC #100005)
41383/tcp open nlockmgr 1-4 (RPC #100021)
46105/tcp open status 1 (RPC #100024)
57343/tcp open rmiregistry GNU Classpath grmiregistry
Service Info: Hosts: metasploitable.localdomain, localhost, irc.Metasploitable.LAN; OSs: Unix, Linux; CPE: cpe:/o:linux:linux_kernel

Service detection performed. Please report any incorrect results at https://nmap.org/submit/ .
Nmap done: 1 IP address (1 host up) scanned in 125.48 seconds

I will copy this report from the msfconsole, after executing the script in training mode, and the script returns me:
[+] Execute Nmap against 192.168.56.101
[*] nmap -p0-65535 -T4 -Pn -sV -sT --min-rate 1000 -oX nmap_result_192.168.56.101.xml 192.168.56.101

[] Start time: 2019/07/19 20:40:24
[] Port scanning: 192.168.56.101 [Elapsed time: 0 s]
[] Executing keep_alive..
[] Port scanning: 192.168.56.101 [Elapsed time: 5 s]
[] Executing keep_alive..
[] Port scanning: 192.168.56.101 [Elapsed time: 10 s]
[] Executing keep_alive..
[] Port scanning: 192.168.56.101 [Elapsed time: 15 s]
[] Executing keep_alive..
[] Port scanning: 192.168.56.101 [Elapsed time: 20 s]
[] Executing keep_alive..
[] Port scanning: 192.168.56.101 [Elapsed time: 25 s]
[] Executing keep_alive..
[] Port scanning: 192.168.56.101 [Elapsed time: 30 s]
[] Executing keep_alive..
[] Port scanning: 192.168.56.101 [Elapsed time: 35 s]
[] Executing keep_alive..
[] Port scanning: 192.168.56.101 [Elapsed time: 40 s]
[] Executing keep_alive..
[] Port scanning: 192.168.56.101 [Elapsed time: 45 s]
[] Executing keep_alive..
[] Port scanning: 192.168.56.101 [Elapsed time: 50 s]
[] Executing keep_alive..
[] Port scanning: 192.168.56.101 [Elapsed time: 55 s]
[] Executing keep_alive..
[] Port scanning: 192.168.56.101 [Elapsed time: 60 s]
[] Executing keep_alive..
[] Port scanning: 192.168.56.101 [Elapsed time: 65 s]
[] Executing keep_alive..
[] Port scanning: 192.168.56.101 [Elapsed time: 70 s]
[] Executing keep_alive..
[] Port scanning: 192.168.56.101 [Elapsed time: 75 s]
[] Executing keep_alive..
[] Port scanning: 192.168.56.101 [Elapsed time: 80 s]
[] Executing keep_alive..
[] Port scanning: 192.168.56.101 [Elapsed time: 85 s]
[] Executing keep_alive..
[] Port scanning: 192.168.56.101 [Elapsed time: 90 s]
[] Executing keep_alive..
[] Port scanning: 192.168.56.101 [Elapsed time: 95 s]
[] Executing keep_alive..
[] Port scanning: 192.168.56.101 [Elapsed time: 100 s]
[] Executing keep_alive..
[] Port scanning: 192.168.56.101 [Elapsed time: 105 s]
[] Executing keep_alive..
[] End time : 2019/07/19 20:42:30
[+] Get port list from nmap_result_192.168.56.101.xml.
[!] No open port.
[!] Shutdown Deep Exploit...

reinstall the requirements of pip and remains the same, i used metasploit from another server using msgrpc and get the same result, some idea of how to fix it, I do not believe the json, there must be some dependency that will have been updated in my system that is not working correctly.

[cstayyab]

Facing exact same issue... I even tried to allow all the connections using Windows Firewall on Metasploitable 3. Nmap Scan shows all port open but when nmap is run using proxychains it shows all ports are closed. @matcon can you run nmap using proxychains? Please provide the output of nmap when run with proxychains.

[13o-bbr-bbq]

@matcon
Please show me the all content of the nmap_result_192.168.56.101.xml. .
If XML format of Nmap result has changed, DeepExploit can't extract open ports information.

[hamidb]

I had similar issue. You can modify the code to read nmap output directly from xml file.

[matcon]

this is xml i change to txt for upload here.
nmap_result_192.168.56.101.xml.txt

[cstayyab]

I had similar issue. You can modify the code to read nmap output directly from xml file.

@hamidb can you please send the modified code or tell where exactly to modify?

[qiwihui]

@cstayyab in get_port_list method, just replace nmap_result by nmap_result = open(nmap_result_file, 'rb').read() before using BeautifulSoup

[qiwihui]

@matcon This issue occurred because Msgrpc client failed to read whole nmap result. The cat command show the whole result while just first line was returned from the rpc client. Maybe it is due to there is '\n\n' between first line and the rest of lines.

machine_learning_security/DeepExploit/DeepExploit.py

Lines 888 to 895 in 2fc2558

	nmap_result = ''
	cat_cmd = 'cat ' + nmap_result_file + '\n'
	_ = self.client.call('console.write', [self.client.console_id, cat_cmd])
	time.sleep(3.0)
	time_count = 0
	while True:
	# Judgement of 'services' command finishing.
	ret = self.client.call('console.read', [self.client.console_id])

the ret should return:

{b'data': b'[*] exec: cat nmap_result_192.168.51.2.xml\n\n(and nmap result blabla...)', b'prompt': b'\x01\x02msf5\x01\x02 \x01\x02> ', b'busy': False}

it actually returned:

{b'data': b'[*] exec: cat nmap_result_192.168.51.2.xml\n\n', b'prompt': b'\x01\x02msf5\x01\x02 \x01\x02> ', b'busy': False}

[researchlab17]

Can the fixed portion of the code be uploaded into the main repo?

[cstayyab]

This error is because the Output of any bash command is not returned with the output. There should be a way to do that.

The line [*] exec: cat nmap_result_192.168.51.2.xml\n\n is output from MSFConsole and all the other output of nmap is printed in the bash child process that Msfconsole has open. The output of that child process (bash) is not being included in 'console.read'

[cstayyab]

@cstayyab in get_port_list method, just replace nmap_result by nmap_result = open(nmap_result_file, 'rb').read() before using BeautifulSoup

@qiwihui This solution does not work if Metasploit RPC is on another System in the network and the DeepExploit is running on some other System. Because the path to nmap file will be local but the file would actually exist on the Other System (which has Metasploit and MsgRPC)

[capce]

As mentioned in #49 there were two things to change to get it work for me.

First thing is here:
Replace Line 2226 with nmap_result = os.getcwd() + '/nmap_result_' + env.rhost + '.xml'

machine_learning_security/DeepExploit/DeepExploit.py

Lines 2226 to 2229 in 76a283d

	nmap_result = 'nmap_result_' + env.rhost + '.xml'
	nmap_command = env.nmap_command + ' ' + nmap_result + ' ' + env.rhost + '\n'
	env.execute_nmap(env.rhost, nmap_command, env.nmap_timeout)
	com_port_list, proto_list, info_list = env.get_port_list(nmap_result, env.rhost)

Second thing is:
Insert between line 914 and 915 nmap_result = open(nmap_result_file, 'rb').read() as @qiwihui suggested.

machine_learning_security/DeepExploit/DeepExploit.py

Lines 914 to 915 in 2fc2558

	info_list = []
	bs = BeautifulSoup(nmap_result, 'lxml')

[1939552724]

[!] 302/2006 linux/pop3/cyrus_pop3d_popsubfolders module is danger (rank: normal). Can't load.
[] 303/2006 Loaded exploit: linux/postgres/postgres_payload
[] 304/2006 Loaded exploit: linux/pptp/poptop_negative_read
[] 305/2006 Loaded exploit: linux/proxy/squid_ntlm_authenticate
[] 306/2006 Loaded exploit: linux/redis/redis_replication_cmd_exec
[] 307/2006 Loaded exploit: linux/samba/chain_reply
[] 308/2006 Loaded exploit: linux/samba/is_known_pipename
[] 309/2006 Loaded exploit: linux/samba/lsa_transnames_heap
[!] 310/2006 linux/samba/setinfopolicy_heap module is danger (rank: normal). Can't load.
[] 311/2006 Loaded exploit: linux/samba/trans2open
[!] 312/2006 linux/smtp/apache_james_exec module is danger (rank: normal). Can't load.
[] 313/2006 Loaded exploit: linux/smtp/exim4_dovecot_exec
[] 314/2006 Loaded exploit: linux/smtp/exim_gethostbyname_bof
[!] type:<class 'KeyError'>
[!] args:(b'rank',)
[!] b'rank'
[!] Failed: module.info

Hi, how can I solve the above problem?