MagicTree

MagicTree 1.3 - important bug fixes and support for IBM Rational AppScan

alla — Thu, 14 Mar 2013 10:20:45 +0000

MagicTree 1.3 - important bug fixes and support for IBM Rational AppScan

We have released MagicTree 1.3. It fixes several nasty bugs that may lead to data corruption. We recommend everybody who uses MagicTree to upgrade. New features include support for AppScan XML and better handling of Imperva Scuba XML

Here is the full change log:

Fix for #307 "Cannot create a working report template in LibreOffice 3.5.4.2".
Better parsing of Imperva Scuba XML
Fixed NullPointerException in FileFilter
Added debugging to idTracker and sanity checking to TreeController to catch the intergity bug
Fix for NullPointerException when handling MtSimpleObjects with no text
Fixes for data integrity bugs causing duplicated ids and broken xrefs
Added support for AppScan XML - contributed by VienHa Tran

Download MagicTree 1.3

alla Thu, 03/14/2013 - 11:20

MagicTree 1.2 Is Out

alla — Wed, 26 Sep 2012 15:15:05 +0000

MagicTree 1.2 Is Out

MagicTree 1.2 is available for download. New features in this release:

Metasploit XML import (issue #228)
Support "critical" severity from Nessus 5 (issue #254)
XSLT export. MagicTree data can now be exported as arbitrary XML. An XSLT for nmap-format export is provided. Use case: merge multiple nmap files, then export as one file to use in Nessus scan (issue #77)
Importing exploitability data from Nessus
Added "Save file as..." button to XML file view
Support for importing output of nmap scripts. Thanks to c4rt

Bug fixes and other changes:

Properly tracking when data needs saving
Properly add URLs to CVEs and BIDs in Nessus imports
Better handling of table selection during copy-paste in Table View - select first column if no columns are selected
Properly update tree view on setStatus
Line separator in copy-paste data from tables should be \n, not \r
Fix for #294 "Open and Merge Files dialogs should have *.mt and *.xml in filters"
Two-pass reading of MagicTree XML to better handle cross-references and detect integrity problems
XSLT bug fixes
Removed "tree promote" feature
Removed "Create sibling" menu
Some refactoring of the core, removing unused classes and methods

alla Wed, 09/26/2012 - 17:15

MagicTree 1.1 Released

alla — Tue, 14 Feb 2012 15:32:09 +0000

MagicTree 1.1 Released

MagicTree 1.1 is released and available for download. This release includes:

Rapid 7 NeXpose XML import (both simple XML and full XML formats are supported)
Arachni XML import (as of 0.4.0.2. Thanks to Herman Stevens of Astyran for contribution)
OWASP Zed Attack Proxy XML import (development snapshot as of 6-Feb-2012)
New matrix query interface
Bug fix (#224) Remove orphan projects does not work anymore
Bug fix (#226) NPE in dumpData()
Bug fix (#239) "Uncaught exception in Swing thread: null. null" when saving a custom query into the repo
Bug fix (#241) Corrupted reference links in report templates
Bug fix (#242) Updated report templates to honour "ignore" status

The new Matrix Query feature allows generating tables like the one shown below and exporting them to Excel or Calc just by copy/pasting.

alla Tue, 02/14/2012 - 16:32

NeXpose XML - A Rant

alla — Thu, 05 Jan 2012 21:34:04 +0000

NeXpose XML - A Rant

As promised here I am working on XSLT for Rapid7 NeXpose XML reports.

There is one great big problem though. "NeXpose Simple XML" format (which is the only XML format available, at least in community edition) contains almost no vulnerability information.

That is:

* It does not contain a human readable vulnerability name, only an id, like "FTP-GENERIC-0007"
* It does not contain a description of a vulnerability
* It does not contain severity or risk rating (high/medium/low or anything along those lines)
* It does not contain any information specific to the particular instance of vulnerability. By this I mean something similar to Nessus plugin output - data that shows some evidence of the vulnerability
* It does not contain impact, recommendations, or any human readable text whatsoever

In fact, with regards to vulnerabilities, it only contains an internal test identifier, like "FTP-GENERIC-0007", and references to CVE, BugTraq, OSVDB and so on.

This makes it pretty useless from report generation point of view. At most, the data from it can be used for port scan results.

I wonder what NeXpose though this XML will be used for? I (probably naively) assumed that XML data a tool generates is for interoperability with third-party tools. Like, you can take the data, feed it to another tool and do something useful with it. What kind of use NeXpose XML may be put to, I have no idea.

By the way, I also failed to find any description of the NeXpose XML format. Not that it is unusual :(

Update: I got XML samples for NeXpose full XML format (only available in commercial versions of NeXpose) and for Metasploit from Rapid7. We'll support all three (simple NeXpose XML, full NeXpose XML, Metasploit XML) in MagicTree 1.1, which is coming out real soon now.

alla Thu, 01/05/2012 - 22:34

MagicTree 1.0 Released

alla — Fri, 30 Sep 2011 09:37:23 +0000

MagicTree 1.0 Released

We are happy to announce that MagicTree version 1.0 is released and available for download.

We would like to thank everybody who submitted bug reports, feature requests or just wrote to tell us that they love MagicTree. You helped a lot!

Version 1.0 includes a lot of bug fixes and a number of new features, such as:

* Support for Acunetix data import
* Support for W3AF data import
* Support for OpenVAS 4 XML format
* Importing data from flat text files
* Simplified manual creation of ports
* Copy/paste and drag and drop support for tree nodes, table view data, queries and tasks
* mt:sort() custom XPath function for sorting data, such as findings, in TableView and reports
* More sophisticated auto-creation of tree nodes. We now support netblocks in various formats (192.168.1.1/24 , 192.168.1.0-192.168.1.255, 192.168.1.0/255.255.255.0), DNS names, IP addresses and URLs.
* Search in output files panel
* Creating cross-references by drag and drop
* Better support for KDE and XFCE desktop environments on Linux. View in Browser and opening reports now works on both.

The full Change Log for version 1.0 is available here.

alla Fri, 09/30/2011 - 11:37

MagicTree Build 1559

alla — Fri, 16 Sep 2011 17:50:17 +0000

MagicTree Build 1559

I've just uploaded MagicTree build 1559, which includes fixes for bugs we have found while working on the PenTest Magazine article.

We are working hard on the next release of MagicTree. We hope to have it out before the end of September.

alla Fri, 09/16/2011 - 19:50

Taming Vulnerability Data - Our article on MagicTree in PenTest Magazine

alla — Thu, 15 Sep 2011 11:07:41 +0000

Taming Vulnerability Data - Our article on MagicTree in PenTest Magazine

Update 2011/09/17: MagicTree build 1559 mentioned in the article is available for download.

PenTest Magazine has published our article Taming Vulnerability Data in its September extra issue along with a MagicTree review by Aby Rao.

In the article we explain how to use MagicTree to analyze Nessus vulnerability scan results and generate a custom report. Download the article here.

In his review Aby Rao points out MagicTree's "clear user interface design and intuitive data structuring", mentions "the simplicity in [...] generating results and then saving the report" and concludes that MagicTree data import capability "makes it a powerful interoperable tool". He also has several suggestions for features such as compliance report templates, PDF and HTML reporting and the ability to generate graphs in reports. All in all, very positive.

In the meantime we are working hard to push out the next release and hope to have it out before the end of September.

alla Thu, 09/15/2011 - 13:07

MagicTree FAQ and Build 1487

alla — Thu, 17 Mar 2011 09:32:36 +0000

MagicTree FAQ and Build 1487

We have started a FAQ page for MagicTree. If you have a questions that should be added, please comment.

We have also posted MagicTree build 1487 for download. It contains various bug fixes, in particular in XML parsing, and minor UI improvements.

alla Thu, 03/17/2011 - 10:32

MagicTree vs. Dradis

alla — Sat, 19 Feb 2011 16:21:24 +0000

MagicTree vs. Dradis

Note: this post is unfinished - two videos are missing
Correction: Dradis can do reports in Word format

Several people have noted that MagicTree is similar to Dradis. In this post I will try to make a point by point comparison, outlining out both similarities and differences. Obviously, I have a bias - being MagicTree developer, I know MagicTree a lot better than Dradis. Feel free to correct me or point out the features that I have missed.

Both MagicTree and Dradis are trying to solve the similar set of problems - managing penetration testing data and report generation. Both MagicTree and Dradis are allow importing the data produced by various penetration testing tools, allow the user add data manually and support report generation. Both MagicTree and Dradis store the data in a tree-like structure.

That being said, there are significant differences between the two tools.

Design, Architecture and Technology

Architecture

Dradis (version 2.6) is a web application. Older versions used to have a console client, but it has been discontinued. Dradis has a central server, where multiple clients connect. Thus it is possible for multiple testers to work on the same project instantly sharing the data they collect. If only one tester is working on a project using Dradis, he/she can run the Dradis server on his/her own computer.

MagicTree is a classic desktop application. There is no server, no database and no listening sockets. MagicTree does not support instant data sharing the way Dradis does, but follows a different approach. At any point in time during the test, one tester can take the MagicTree project file created by someone else and merge it with his/her own, getting all the data obtained by the other tester.

Data Structure

Though both MagicTree and Dradis store the data in a tree, there is a fundamental structural difference. Dradis puts the data in the tree according to its source. MagicTree structures the data according to the real-world object it describes. Let me explain that.

Say, for example, you have ran a nmap TCP port scan, and then thought that you need a UDP scan as well, and ran that too for the same targets. Importing these two files into Dradis will create two tree branches - one for each imported file:

When the same two files are imported in MagicTree it will merge together the results of two scans, so that both open TCP and UDP ports will appear under the hosts that they belong to:

The same goes for all data that gets merged in MagicTree, regardless of where it came from or what tool produced it. Any piece of data always appears under the object (host, port, service, etc.) it describes. This approach to storing data is fundamental to MagicTree and in particular allows querying of the data. I will describe the queries and other features build on top of that later in this article.

Extensibility by user

Dradis is an open-source application written in Ruby. A user can modify and extend it any way he likes. Doing so obviously requires some knowledge of Ruby and the ability (and desire ;) ) to read and understand the existing Ruby code. In particular, if you want to extend Dradis to support importing data from some tool, you'll need to write an upload plugin.

MagicTree is a closed-source application. However, it is possible for a user to extend it to be able to import data from tools it does not support out of the box. Two possibilities exist. If the tool you want to import data from produces XML output, you can write an XSLT transform and add it to MagicTree. The procedure for this is described here. You can use the XSLT files that come with MagicTree (in $HOME/.magictree/xslt directory) as an example. Alternatively, if the tool does not output XML or you don't feel like writing XSLTs, you can make a wrapper in any programming language you like that runs the tool, reads its output, parses it and outputs MagicTree XML that can be directly consumed by MagicTree. The MagicTree XML format structure and semantics are described here. Several sample scripts come with MagicTree and can be found in $HOME/.magictree/snippets directory.

Features

Dradis allows importing vulnerability descriptions from sources such as OSVDB and MediaWiki. This video demonstrates this feature. MagicTree currently does not do this, this feature is on our "to do" list.

Now let me demonstrate several MagicTree features that are absent in Dradis. The first of those is data querying. MagicTree query engine allows getting answers to questions such as "show me all http hosts and ports","are there any Apache servers running on Linux" and so on. Let's see how it's done:

The second feature unique to MagicTree that I want to show here is command execution. It is tightly linked with queries, allowing the user to extract the necessary data from the tree and feed it to command-line tools (show query, launching multiple commands, remote execution, data import).
[video comes here]

The last thing I would like to show is MagicTree's approach to report generation.
[video comes here]

Summary

The following table gives a side by side comparison of MagicTree and Dradis

	MagicTree	Dradis
General
Platform support	Multi-platform: Java	Multi-platform: Ruby
Architecture	Desktop application	Client-server. A fat client and a web interface are available
License	Proprietary. Distributed free of charge	Open source. GNU GPL.
Supported import formats
Nmap	Yes	Yes
Nikto	Yes	Yes
Nessus XML version 1	Yes	Yes
Nessus XML version 2	Yes	No
Burp	Yes	Yes
OpenVAS	Yes	No. User-contributed plug-in exists
Qualys	Yes	No
Imperva Scuba	Yes	No
Typhon	No	Yes
NeXpose	No	No. User-contributed plug-in exists
Netsparker	No	No. User-contributed plug-in exists
Supported report formats
Microsoft Word	Yes	Yes
OpenOffice	Yes	No
HTML	No	Yes
Other features
Adding file attachments to nodes	Yes	Yes
Searching data in the tree	Yes	Yes
Dradis-only features
Vulnerability data import: Dradis supports importing vulnerability information from OSVDB and MediaWiki
Online collaboration: Dradis supports multiple testers accessing the same project database on-line
MagicTree-only features
Task execution: MagicTree supports running shell commands straight from the GUI, capturing command output
Data analysis: MagicTree allows querying the collected data and feeding it to shell commands
Knowledge reuse: MagicTree allows saving queries and commands to be reused in this or future projects

alla Sat, 02/19/2011 - 17:21

Video: Using MagicTree for Analysing Data

alla — Mon, 14 Feb 2011 12:04:17 +0000

Video: Using MagicTree for Analysing Data

This video was going to be the first in a series of three. However I got stuck with the second one, so instead of waiting for the inspiration to hit me, I thought I'll publish this one anyway. Enjoy.

alla Mon, 02/14/2011 - 13:04