MagicTree Documentation: Semantic Structure

Node Types | Table of Contents | Currently Supported XML Formats >

MagicTree root node is a branch node and is always called "magictree".

All data about the test targets gathered in the course of a project is kept under "testdata" node which is a child node of "magictree".

Nodes for hosts, fully qualified domain names (fqdn) and network blocks (netblock) are created directly under "testdata".

Network Blocks

Network blocks are represented by simple nodes

The network blocks have "netblock" as type and the IP address and netmask as value. The is represented using the dotted decimal notation followed by slash and the number of "1" bits in netmask, for example "192.168.1.0/24" (but not "192.168.1.0/255.255.255.0").

XML:

<netblock>10.1.1.128/25</netblock>

Fully Qualified Domain Names

Fully qualified domain names are represented by simple nodes

Fully qualified domain names have "fqdn" as type and the domain name, including the trailing dot as value, for example "www.gremwell.com.".

XML:

<fqdn>www.gremwell.com.</fqdn>

Hosts

Hosts are represented by simple nodes

Hosts have "host" as type and the IP address as value, for example "192.168.1.1".

XML:

<host>10.1.1.1</host>

In MagicTree hosts are identified by IP address, not by DNS name. All the data related to a host is stored under the "host" node.

We are aware that a DNS name may resolve to more than one IP address, a physical host may have multiple IP addresses, and one IP address may have multiple host names (virtual hosts). We believe that in a context of a network security assessment, an IP address is usually a more convinient way of identifying entities being tested. To store one or more DNS names that a given IP address may have, we use hostname objects.

Hostnames

DNS host names are represented by simple nodes and are located directly under a host node. One host can have multiple DNS names and therefore multiple hostname objects. DNS host names do not need a trailing dot.

XML:

<host>192.168.1.1
<hostname>shiny2.gremwell.com</hostname>
</host>

MAC addresses

MAC addresses are represented by simple nodes and are located directly under a host node.

XML:

<host>192.168.1.1
<mac>00:21:70:a0:47:17</mac>
</host>

Operating System

Operating systems are represented by simple nodes and are located directly under a host node.

XML:

<host>192.168.1.1
<os>Linux</os>
</host>

A host may hve more then one "os" nodes, because different tools may have different opinion on the operating syustem of the remote host. Even one tool may be in doubt about it and return several guesses.

NetBIOS names

NetBIOS names are represented by simple nodes and are located directly under a host node.

XML:

<host>192.168.1.1
<nbname>SHINY2</nbname>
</host>

TCP and UDP ports

TCP ports are located under <ipproto>tcp</ipproto> node under host node. UDP ports are located under <ipproto>udp</ipproto> node under host node. Both are represented by simple nodes.

XML:

<host>192.168.1.1
<ipproto>tcp
<port>80</port>
</ipproto>
<ipproto>udp
<port>161</port>
</ipproto>
</host>

Ports may have states, such as "open", "closed" or "filtered". They are represented by a "state" node under the port node:

<port>80
<state>open</state>
</port>

IP protocols

IP protocols are located directly under the "host" node and have "ipproto" type. TCP and UDP are represented as <ipproto>tcp</ipproto> and <ipproto>udp</ipproto> respectively. Other IP protocols may be represented using the protocol number instead of protocol name, for example <iproto>1</ipproto> if the name is not known.

XML:

<host>
<ipproto>tcp
<port>80</port>
</ipproto>
<iproto>icmp</ipproto>
<iproto>99</ipproto>
</host>

IP protocols, similar to ports, may have state,such as "open", "closed" or "filtered". They are represented by a "state" node under the protocol node:

<ipproto>99
<state>open</state>
</ipproto>

Application Protocols / Serices

Application level protocols (a.k.a. services) are located under the "port" node where the service was detected, for example

<iproto>tcp
<port>80
<service>http</service>
</port>
</ipproto>

An exception to this rule are services that work over SSL (for example, HTTPS, LDAPS, etc.). The nodes for those services are located under a <tunnel>ssl</tunnel> node, which is located directly under the port node:

<iproto>tcp
<port>443
<tunnel>ssl
<service>http</service>
</tunnel>
</port>
</ipproto>

Please note that the service above is listed as "http", not "https". This is a convention we follow for all SSL services. Instead of using a different service name for an SSL-based service, we use the same name as its non-SSL counterpart and wrap it in an <tunnel>ssl</tunnel>. This allows us to handle similar SSL and non-SSL services together (for example, run a CGI scan for both HTTP and HTTPS servers), and also allows us easily select all SSL-based services, for instance, to test for the presence of SSLv3/TLSv1 renegotiation vulnerbility.

In order to map the names of the SSL-based services returned by various tools to non-SSL names, we keep a list of mappings in the file called service-map.xml, stored in the XSLT directory. The file also serves to unify the service names used by different tools. For example, various tools may use "www" or "http" or "http-alt" for HTTP service. "www" and "http-alt" are mapped to "http" in service-map.xml

Software

If the software name and version is detected, it can be listed under the service:

<service>http <software>Apache <version>2.2.2</version> </software> </service>

Vulnerability Scanner Findings

Vulnerability scanner (Nessus, Nikto, Burp, Qualys etc.) findings are represented by Text objects and are located under the service node to which the finding apply or under the host node, if the finding applies to the host as a whole.

Qualys findings are an unfortunate exception, because Qualys XML format does not allow to determine easily which port and service the finding applies to, so Qualys findings generally end up under the host node.

If you are writing an XSLT transform for vulnerability scanner output, try whenever possible to place findings under the appropriate port and service node.

The "title" attribute of the Text object representing the finding contains a short title of the vulnerability, for example "SSL/TLS Renegotiation is Enabled". The text of the Text object contains the description of the finding.

The element name for findings is always "finding".

Findings should have a child node called "source" that indicates which vulnerability scanner has returned this finding.

Another child node called "id" represents the finding id returned by the scanner. For example, for Nessus, this will be Nessus plugin id.

Findings also have "source-severity" child node, that can have the values of "high", "medium", "low", "info" and "unknown". This node indicates the severity of the finding as reported by the source vulnerability scanner. Scanners may use other ways of grading the findings, so they have to be mapped to "high", "medium", "low" or "unknown" in XSLT.

The CVE IDs of the findings can be represented by one or more "cve" nodes located under the "finding" node. The BugTraq IDs can be represented by one or more "bid" nodes under "finding" node. OSVDB IDs can be represented by "osvdb" nodes.

If the vulnerability scanner outputs a short summary of vulnerability in addition to a more detailed description, it is represented by "synopsis" Text node. The title attribute of this node is set to "Synopsis".

If the vulnerability scanner outputs sulution or recommendations, , it is represented by "solution" Text node. The title attribute of this node is set to "Solution".

References to external resources, such as URLs of advisories, are represented by "reference" nodes.

Vulnerability scanners often output some information that is specific to a particular tested host/port. For example, Nessus provides "plugin output", and Burp provides "issue detail". This data is represented by "output" Text node with title attribute set to "Details".

MagicTree allows grouping similar findings in reports, so that information that is the same for multiple hosts and ports (description, solution, references, etc.) is listed only once, while host/port specific output is still clearly reported. The sample report template "summary-of-findings-with details" provides an example of such grouping.

If the vulnerability scanner outputs CVSS information, it can be represented by "cvss vector" and "cvss base score" nodes.

As menitioned above findings are represented by Text nodes.Text nodes are compared by name and title, the text content is not used for comparison. Sometimes a vulnerability scanner can output multiple findings with the same title for a given endpoint. For example, multiple XSS vulnerabilities may be found on a given HTTP port. When an XML file is loaded by MagicTree, the sibling nodes that are equal are always merged together. However, multiple XSS findings are separate entities and really should not be merged together. To solve this problem we have introduced a special attribute "mergeID". It's value can be an arbitrary string. When two nodes are compared for equality for the purpose of merging, the presense of mergeID is checked. If both have mergeID and the two mergeIDs are different, the two nodes are considered not equal and are not merged together. In all other cases (no mergeID is present, or only one of the two nodes have mergeID) the two nodes are compared using the usual rules. If you are wrting an XSLT and you have nodes that are equal acording to MagicTree node comparison rules, while in fact they represent separate real world entities, add a unique mergeID attribute to each one of them, so that they don't get merged together. We do this in Burp XSLT using Burp's finding serial number for mergeID value.

Example:

<finding title="SSL Certificate with Wrong Hostname" class="MtTextObject">
The commonName (CN) of the SSL certificate presented on this port is for a different machine.
<source class="MtSimpleObject">nessus</source>
<id class="MtSimpleObject">45411</id>
<source-severity class="MtSimpleObject">medium</source-severity>
<output title="Details"class="MtTextObject">
The following hostnames were checked :
www.example.com
</output>
<synopsis title="Synopsis"class="MtTextObject">
The SSL certificate for this service is for a different host.
</synopsis>
<solution title="Solution" class="MtTextObject">
Purchase or generate a proper certificate for this service.
</solution>
<cvss vector class="MtSimpleObject">
CVSS2#AV:N/AC:L/Au:N/C:N/I:P/A:N
</cvss vector>
<cvss base score class="MtSimpleObject">
5.0
</cvss base score>
</finding>

Node Types | Table of Contents | Currently Supported XML Formats >