Samb Business Intelligence Blog

I had a chance this week between a zillion conference calls to put some time against Firefly. The objective for me this week was two fold: get the signature going and get the layouts under control.

What's a signature you ask? Well, I'm glad you asked. When we load the network, we perform a histogram on the entire network. This is shown in the Edge Influence Evaluation Tool midway down the left.

Here's a close up:

The signature shows the overall "connectedness" of the network. In this case, there are zero cases where nodes have zero edges and 4 cases where nodes share one edge, but the majority of the nodes have one edge and one (at the end) has 5 edges. Come to think of it - it's easier to show than to explain. I really like showing the histogram when allowing filters and I'll do the same for the metrics (coming soon - perhaps Monday). This provides a bit of analytics and I also have some thoughts about suggesting various layouts to the user based on the overall signature.

Speaking of Layouts, we added a bunch.

Here's the Fruchterman/Reingold Layout...

And the ever popular circular layout

A Grid Layout

A Sugiyama Layout (this reminds me of a suspension bridge with trusses)

A Sinusoidal Layout (There are Horizontal and Vertical variants of this)...

And a Spiral layout. Other layout methods can be added later.

There are a few things you notice about this work, one is that the comments get compressed and occlude one another. I plan on using the layout engine to attach the comments to the nodes and edges - since the layout engine is outside the data we can use it to layout things that are not exclusively nodes or edges - cool, huh?

The Layout engine is using NodeXL (codeplex) - this is available to all. It's exceptionally easy to use and very versatile. Here's a quick example of how to add nodes, edges and perform a layout (in C#) - notice that no visualization is occurring here - this is almost something that could be a service in the cloud.

using System;
using System.Drawing;
using Microsoft.NodeXL.Core;
using Microsoft.NodeXL.Layouts;

namespace PopulateAndLayOutGraph
{
class Program
{
    static void Main(string[] args)
    {
        // Create a graph.  The graph has no visual representation --
        // it is just a data structure.

        Graph oGraph = new Graph(GraphDirectedness.Directed);
        IVertexCollection oVertices = oGraph.Vertices;
        IEdgeCollection oEdges = oGraph.Edges;

        // Add three vertices.

        IVertex oVertexA = oVertices.Add();
        oVertexA.Name = "Vertex A";
        IVertex oVertexB = oVertices.Add();
        oVertexB.Name = "Vertex B";
        IVertex oVertexC = oVertices.Add();
        oVertexC.Name = "Vertex C";

        // Connect the vertices with directed edges.

        IEdge oEdge1 = oEdges.Add(oVertexA, oVertexB, true);
        IEdge oEdge2 = oEdges.Add(oVertexB, oVertexC, true);
        IEdge oEdge3 = oEdges.Add(oVertexC, oVertexA, true);

        // Lay out the graph within a 100x100 rectangle.  This sets
        // the IVertex.Location property of each vertex.

        ILayout oLayout = new FruchtermanReingoldLayout();

        LayoutContext oLayoutContext =
            new LayoutContext(new Rectangle(0, 0, 100, 100));

        oLayout.LayOutGraph(oGraph, oLayoutContext);

        // List the results.

        foreach (IVertex oVertex in oVertices)
        {
            Console.WriteLine("The location of {0} is {1}.",
                oVertex.Name,
                oVertex.Location);
        }
    }
}
}

Special Thanks to Tony Capone for his help on wiring this up.

Feel free to send your comments...the source code will be available soon.

I have a demo of Firefly today and decided that I would do a quick post to help myself "rehearse" a bit of the features we will talk through.

For the record, this is about half way through - which is code for "kind of messy". I prefer to code and get the hard stuff our off the way and wired up and then work through the polish and beautification stage. It drives some people batty...

First off let's take a look at the overall application...The red cross hair is used for selection purposes. Double-clicking on a node will center the cross hair on the node - we use animation to smoothly tween between positions and it comes across nice - almost like a video game. The view also shows commenting (with Text and Links - more on that in a minute) and Viewpoints - the large hollow white square on the left, bottom portion of the graph.

The Left control panel has been almost completely redone. Starting from the top left:

Size to Fit - reset the zoom to include the whole graph
Zoom Slider - that's kind of obvious
Pan & Selection modes - important when zoomed in and selecting and, well, er...panning.

Below that I have a button for persisting the zoomed and x,y positions of the current viewpoint. I'll be honest, I'm struggling a bit with the visual vocabulary and usability here. There's a part of me that says that the user should be able to draw the rectangle on the display and then name it - but dealing with different aspect ratios on X and Y for something like this will not be a WYSIWYG kind of thing.

Then we can customize the whole display. Show or Hide the nodes, the edges, the labels, the metrics (the blue ellipses) and the comments. You can also change the opacity and the color of all these things.

Here's an example of full opacity (solid) metrics, no nodes and faint edges and full labels. By altering these things you can get very different displays and focus on different elements of the graph in a better fashion.

There is now a search by name (just type the nodes name - or a part of it), or simply look it up and select it...

Here's Z selected - notice the red cross hairs...

OK, so comments are really important in this app. Comments can be added to a node, edge, or a region of the network.

Comments come in two flavors: Text and Links. Running your mouse over a comment expands it fully so you can read it. For a link, clicking on it will fire the browser and navigate you to that node - great for incorporating Reporting Services or any BI tool that accepts an ID for drilling down - this keeps the app light and flexible - almost like a visual wiki.

More coming, the layout and weighting aspects are next up.

Since InfoMesa is essentially a content visualization concept, the question of visualization comes up a lot. I tend to be a purist when it comes to the term visualization. In other words, to me, conditional formatting in Microsoft Excel is *not* visualization.

There's a concept of not recreating the wheel in InfoMesa, but really staking out new ground - ground that has been heretofore too expensive or too exclusive for many customers and making it pervasive and easily accessible and flexible across many different domains. InfoMesa certainly has its roots in Life Science, which I still consider the high water mark for complex knowledge work. The need to ingest voluminous datasets and find digital needles in digital haystacks and also "telling stories" with data sets is increasingly important in the sciences.

One of the technology sketches that I have been working on for the next phase of InfoMesa is to understand if we could insert a "generic" visualization mechanism inside of InfoMesa that would make InfoMesa relevant not just to Life Science but to any industry dealing with "gnarly" data all the time.

So here is an idea that I'll fly past you. What if...there was a visualizer tool that launches a composer inside of InfoMesa. This composer will allow a scientist to connect directly to diverse data sets, but also multiple data sets - in this way, this composer is actually creating a visual datawarehouse in a sense by meshing heterogeneous data types and sets. With data being pulled from both proprietary data sets on premises and also from cloud data sources this seems critical.

This diagram imports an image (in this case an image of the planet Mars at high resolution) and maps it onto a 3D sphere and then outputs to the display. the display in the final analysis would actually be inside of InfoMesa, but you get the general idea. Pressing Play will create the image below...it is a 3D navigation window - the model can be rotated, zoomed and panned and perhaps most importantly, multiple views can be stored and replayed.

This is a simple example, but let's get a bit more complicated...Here's a diagram that creates a sphere that maps the texture of the Earth to it and then ingests the recorded history of earthquakes from SQL Server (lat/long based) which also includes depth of the quake.

The diagram places the depth of the earthquakes as "spikes" that extend beyond the crust of the Earth to show where the quake occurred (by lat/long) and how deep the quake was. Here's the end visualization.

OK, so let's explore how flexible this method can be. Let's do something more analytic. Let's ingest data from three different datasets and map them together visually. This is a Cardiovascular study that shows patients by Gender (blue/pink) by condition of Cholesterol, Blood Pressure and Max Heart Rate.

Here's the visual. The Axes get "Auto-Generated" and the plots clearly show general behavior and outliers. So, the method seems to map quite broadly across the problem domain set.

Here's another diagram that creates a Naval Electronic Warfare example - the idea of telling a story with data...This diagram imports two pre-built 3D models (Akula and USS Dallas - hello, Hunt for Red October) and then ingests sonar mapping and textures the seabed floor. There is a lot going on here - tens of thousands of data points are in play.

Here's the resulting image from a "birdseye" view...

By rotating the viewer we can look from USS Dallas' perspective against the sea wall...

...here's a view from the stern of the Akula...the detail is striking...

OK, enough wackiness with submarines - back to reality - can we start to get close to what a molecular viewer does? Let's import an e.Coli protein with X,Y,Z of atom and atom type (Nitrogen, Carbon, Sulfur, etc.) and see what we can come up with.

Here's the result...not too bad - but still a long way from cartoon views, backbones, etc, but you can see how we would handle waters and possibly even iso-potentials.

I'll continue to work on this concept, but wanted to socialize it to get some basic feedback...

Introducing Firefly, a project I've been working on with LifeScience/Pharma companies for a while now. Like InfoMesa, the goal is to make this available broadly, so that the community can see Microsoft's strengths in visualization and sensemaking capabilities. Firefly is designed to ingest any network - any collection of nodes and their associated edges, perform readable layouts, provide navigation and implement social networking capabilities to bring insight into these typically complex and intimidating data structures.

Firefly is a Windows Presentation Foundation Application, just like InfoMesa. In fact, Firefly is likely to become part of InfoMesa as a network visualizer in the near future. This is the first post and I'm not quite ready to post the code for general availability, but that's the goal and I think we will be there soon. Firefly will allow someone to point the tool at any database (be it SQL Server, Access, Microsoft Azure Storage [SSDS], or Oracle) and define what the nodes are, what the edges are and Firefly does the rest. Firefly performs a layout of the data, persists it and allows navigation and annotation of the space.

Whenever visualization is employed there is a mantra or sorts that should be remembered: Overview-Detail-Filter-Drill for attributes. This mantra gives the basic needs of any visualization: provide a context for the whole data set, allow drill down into that universe, Allow filters to that data set and bring additional, linked values to the surface.

1. Overview - a zoomable, pannable look at the universe of nodes and edges in the entire graph
2. Alter Visualization - Emphasize/Under emphasize visual elements using color, opacity and visibility - i.e. make the edge lines yellow and faint.
3. Edge Analytics - Find most influential nodes - i.e. Show only the nodes that have 5 to 20 edges - the high influence nodes
4. Search for Nodes by Name - just type or select a node name and Firefly finds it for you - useful when the graph is large
5. Metrics weighed to each Node - allows visualization of addition Channels of Information as an overlay on the graph - radius and color can be mapped
6. Cross Hair for selecting nodes for detail (double-click will pin the cursor to a node)
7. Stored Views - Clicking create View in the top left corner will store the window constraints (x,y, wide, high, scale) to create quickly navigable neighborhoods of nodes and edges

Once you select a node, you can switch to the detail tab (located at the top of the screen) and view just that node and it's associated edges.

Clicking on any node moves that selection to the center and the graph "blooms" to show that nodes associations.

A word about layout and mapping. Since Networks and Graphs are without orientation (or coordinate systems) it is important to create a sense of constancy for the user - in other words, where stuff tends to be is always where it is. Geography is this way and so are star maps (yes, I know there are variations, but the orientation of constellations nearby one another makes it possible to find collections and constellations rather easily).

So, after Firefly creates a layout we freeze that layout and store it (or persist) for exploration - this is different than many other applications that simply reapply layouts - for small networks, persistence doesn't matter, but for large ones you quickly loose orientation.

I will post more information as we go and a new InfoMesa post will be up in a week or so.