Receiving XML exception from nmap.scan() results.

[PythonNmap]

Hi All,

I'm making use of the python-nmap library.

However, running a basic scan against 10.0.0.17/255.255.255.0 or more precisely 10.0.0.17/24, yields the following exception on the returned XML results:

Error:
Traceback (most recent call last): File "/usr/local/lib/python3.6/site-packages/nmap/nmap.py", line 322, in analyse_nmap_xml_scan dom = ET.fromstring(self._nmap_last_output) File "/usr/lib64/python3.6/xml/etree/ElementTree.py", line 1314, in XML parser.feed(text) File "<string>", line None xml.etree.ElementTree.ParseError: not well-formed (invalid token): line 4, column 1 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "./get_address", line 94, in nmapScan raw = nm.scan(hosts=vlan+'/'+str(cidr), arguments=' -v -sn -n -oG - ') File "/usr/local/lib/python3.6/site-packages/nmap/nmap.py", line 266, in scan nmap_warn_keep_trace=nmap_warn_keep_trace File "/usr/local/lib/python3.6/site-packages/nmap/nmap.py", line 327, in analyse_nmap_xml_scan raise PortScannerError(self._nmap_last_output)

The command is:

raw = nm.scan(hosts=vlan+'/'+str(cidr), arguments=' -v -sn -n -oG - ')

which translates to:

nmap -v -sn -n -oG - 10.0.0.117/24

Raw output would be similar to this:

# nmap -v -sn -n -oG - 10.0.0.117/24 # Nmap 6.40 scan initiated Sun Jan 19 23:22:18 2020 as: nmap -v -sn -n -oG - 10.0.0.117/24 # Ports scanned: TCP(0;) UDP(0;) SCTP(0;) PROTOCOLS(0;) Host: 10.0.0.0 () Status: Down Host: 10.0.0.1 () Status: Up Host: 10.0.0.2 () Status: Down Host: 10.0.0.3 () Status: Down . . . Host: 10.0.0.252 () Status: Down Host: 10.0.0.253 () Status: Down Host: 10.0.0.254 () Status: Down Host: 10.0.0.255 () Status: Down # Nmap done at Sun Jan 19 23:22:22 2020 -- 256 IP addresses (7 hosts up) scanned in 3.62 seconds

When running the resulting XML through some parsing, I get this (snippet):

Error:
[root@one01 ~]# cat text.txt | awk '{ gsub(/\\n/, "\n", $0); print; }' <?xml version="1.0"?> <?xml-stylesheet href="file:///usr/bin/../share/nmap/nmap.xsl" type="text/xsl"?> <!-- Nmap 6.40 scan initiated Sun Jan 19 23:38:18 2020 as: nmap -oX - -v -sn -n -oG - 10.0.0.117/24 --> # Nmap 6.40 scan initiated Sun Jan 19 23:38:18 2020 as: nmap -oX - -v -sn -n -oG - 10.0.0.117/24 <nmaprun scanner="nmap" args="nmap -oX - -v -sn -n -oG - 10.0.0.117/24" start="1579495098" startstr="Sun Jan 19 23:38:18 2020" version="6.40" xmloutputversion="1.04"> <verbose level="1"/> <debugging level="0"/> # Ports scanned: TCP(0;) UDP(0;) SCTP(0;) PROTOCOLS(0;) <taskbegin task="Ping Scan" time="1579495099"/> <taskend task="Ping Scan" time="1579495102" extrainfo="256 total hosts"/> <host><status state="down" reason="no-response" reason_ttl="0"/> <address addr="10.0.0.0" addrtype="ipv4"/> Host: 10.0.0.0 ()\tStatus: Down </host> <host><status state="up" reason="echo-reply" reason_ttl="255"/> <address addr="10.0.0.1" addrtype="ipv4"/> <hostnames> </hostnames> Host: 10.0.0.1 ()\tStatus: Up <times srtt="3292" rttvar="5000" to="100000"/>

The unaltered error:

Error:
ERROR:root:'<?xml version="1.0"?>\n<?xml-stylesheet href="file:///usr/bin/../share/nmap/nmap.xsl" type="text/xsl"?>\n<!-- Nmap 6.40 scan initiated Sun Jan 19 23:38:18 2020 as: nmap -oX - -v -sn -n -oG - 10.0.0.117/24 -->\n# Nmap 6.40 scan initiated Sun Jan 19 23:38:18 2020 as: nmap -oX - -v -sn -n -oG - 10.0.0.117/24\n<nmaprun scanner="nmap" args="nmap -oX - -v -sn -n -oG - 10.0.0.117/24" start="1579495098" startstr="Sun Jan 19 23:38:18 2020" version="6.40" xmloutputversion="1.04">\n<verbose level="1"/>\n<debugging level="0"/>\n# Ports scanned: TCP(0;) UDP(0;) SCTP(0;) PROTOCOLS(0;)\n<taskbegin task="Ping Scan" time="1579495099"/>\n<taskend task="Ping Scan" time="1579495102" extrainfo="256 total hosts"/>\n<host><status state="down" reason="no-response" reason_ttl="0"/>\n<address addr="10.0.0.0" addrtype="ipv4"/>\nHost: 10.0.0.0 ()\tStatus: Down\n</host>\n<host><status state="up" reason="echo-reply" reason_ttl="255"/>\n<address addr="10.0.0.1" addrtype="ipv4"/>\n<hostnames>\n</hostnames>\nHost: 10.0.0.1 ()\tStatus: Up\n<times srtt="3292" rttvar="5000" to="100000"/>\n</host>\n<host><status state="down" reason="no-response" reason_ttl="0"/>\n<address addr="10.0.0.2" addrtype="ipv4"/>\nHost: 10.0.0.2 ()\tStatus: Down\n</host>\n

Perhaps the -- in the above is causing the XML to throw an exception? Or the #'s ?

Thx,
TK

[buran]

I think it's the # - traceback is clear - line 4, column1.
And you can validate your snippet against xml validators and they also raise error on #.
Now - should it be there is another question.
According to their sample output - no, it shouldn't be there
https://nmap.org/book/output-formats-xml-output.html

Note that there is another one on line 8 - # Ports scanned: TCP(0;) UDP(0;) SCTP(0;) PROTOCOLS(0;)
Maybe ask nmap support why it's in their xml output?

[PythonNmap]

Yeah, after I posted I used a validator. Confirmed the same message.

However, the # is expected. There's a few formats nmap can output in, including XML. Meaning, I don't need to use python-nmap's XML generator, using instead an XML reader on nmap results captured from a system command. The option is -oX instead of -oG to nmap. I'll need to research this.

Wouldn't this make it more of a python-nmap coding question then? The #'s invalidate a ton of examples.

Thanks Buran!

---

Another clue. Looking closely though, line 3, that's the comment on line 4 encoded in HTML comments.

What this tells me is that the python-nmap logic did work converting a comment to HTML format but left the original comment in place as well. Why leave it twice? This suggests the #'s should have been removed once converted to HTML comments. Perhaps I should reach out to the developers of python-nmap to see if there's a way around this?

Thx,
TK

[buran]

Maybe you shouldn't supply -oG option in arguments, Nowhere in the examples and the docs they force -oG (i.e. Grepable output format).
Looking at the source code, they force -oX option.

[PythonNmap]

Hm. Tried to use -oX . python-nmap tried to wrap XML into XML.

Now I just tried using without -oG and without -oX. Now it works.

Cheers,
TK