Samb Business Intelligence Blog

I had a chance this week between a zillion conference calls to put some time against Firefly. The objective for me this week was two fold: get the signature going and get the layouts under control.

What's a signature you ask? Well, I'm glad you asked. When we load the network, we perform a histogram on the entire network. This is shown in the Edge Influence Evaluation Tool midway down the left.

Here's a close up:

The signature shows the overall "connectedness" of the network. In this case, there are zero cases where nodes have zero edges and 4 cases where nodes share one edge, but the majority of the nodes have one edge and one (at the end) has 5 edges. Come to think of it - it's easier to show than to explain. I really like showing the histogram when allowing filters and I'll do the same for the metrics (coming soon - perhaps Monday). This provides a bit of analytics and I also have some thoughts about suggesting various layouts to the user based on the overall signature.

Speaking of Layouts, we added a bunch.

Here's the Fruchterman/Reingold Layout...

And the ever popular circular layout

A Grid Layout

A Sugiyama Layout (this reminds me of a suspension bridge with trusses)

A Sinusoidal Layout (There are Horizontal and Vertical variants of this)...

And a Spiral layout. Other layout methods can be added later.

There are a few things you notice about this work, one is that the comments get compressed and occlude one another. I plan on using the layout engine to attach the comments to the nodes and edges - since the layout engine is outside the data we can use it to layout things that are not exclusively nodes or edges - cool, huh?

The Layout engine is using NodeXL (codeplex) - this is available to all. It's exceptionally easy to use and very versatile. Here's a quick example of how to add nodes, edges and perform a layout (in C#) - notice that no visualization is occurring here - this is almost something that could be a service in the cloud.

using System;
using System.Drawing;
using Microsoft.NodeXL.Core;
using Microsoft.NodeXL.Layouts;

namespace PopulateAndLayOutGraph
{
class Program
{
    static void Main(string[] args)
    {
        // Create a graph.  The graph has no visual representation --
        // it is just a data structure.

        Graph oGraph = new Graph(GraphDirectedness.Directed);
        IVertexCollection oVertices = oGraph.Vertices;
        IEdgeCollection oEdges = oGraph.Edges;

        // Add three vertices.

        IVertex oVertexA = oVertices.Add();
        oVertexA.Name = "Vertex A";
        IVertex oVertexB = oVertices.Add();
        oVertexB.Name = "Vertex B";
        IVertex oVertexC = oVertices.Add();
        oVertexC.Name = "Vertex C";

        // Connect the vertices with directed edges.

        IEdge oEdge1 = oEdges.Add(oVertexA, oVertexB, true);
        IEdge oEdge2 = oEdges.Add(oVertexB, oVertexC, true);
        IEdge oEdge3 = oEdges.Add(oVertexC, oVertexA, true);

        // Lay out the graph within a 100x100 rectangle.  This sets
        // the IVertex.Location property of each vertex.

        ILayout oLayout = new FruchtermanReingoldLayout();

        LayoutContext oLayoutContext =
            new LayoutContext(new Rectangle(0, 0, 100, 100));

        oLayout.LayOutGraph(oGraph, oLayoutContext);

        // List the results.

        foreach (IVertex oVertex in oVertices)
        {
            Console.WriteLine("The location of {0} is {1}.",
                oVertex.Name,
                oVertex.Location);
        }
    }
}
}

Special Thanks to Tony Capone for his help on wiring this up.

Feel free to send your comments...the source code will be available soon.

I have a demo of Firefly today and decided that I would do a quick post to help myself "rehearse" a bit of the features we will talk through.

For the record, this is about half way through - which is code for "kind of messy". I prefer to code and get the hard stuff our off the way and wired up and then work through the polish and beautification stage. It drives some people batty...

First off let's take a look at the overall application...The red cross hair is used for selection purposes. Double-clicking on a node will center the cross hair on the node - we use animation to smoothly tween between positions and it comes across nice - almost like a video game. The view also shows commenting (with Text and Links - more on that in a minute) and Viewpoints - the large hollow white square on the left, bottom portion of the graph.

The Left control panel has been almost completely redone. Starting from the top left:

Size to Fit - reset the zoom to include the whole graph
Zoom Slider - that's kind of obvious
Pan & Selection modes - important when zoomed in and selecting and, well, er...panning.

Below that I have a button for persisting the zoomed and x,y positions of the current viewpoint. I'll be honest, I'm struggling a bit with the visual vocabulary and usability here. There's a part of me that says that the user should be able to draw the rectangle on the display and then name it - but dealing with different aspect ratios on X and Y for something like this will not be a WYSIWYG kind of thing.

Then we can customize the whole display. Show or Hide the nodes, the edges, the labels, the metrics (the blue ellipses) and the comments. You can also change the opacity and the color of all these things.

Here's an example of full opacity (solid) metrics, no nodes and faint edges and full labels. By altering these things you can get very different displays and focus on different elements of the graph in a better fashion.

There is now a search by name (just type the nodes name - or a part of it), or simply look it up and select it...

Here's Z selected - notice the red cross hairs...

OK, so comments are really important in this app. Comments can be added to a node, edge, or a region of the network.

Comments come in two flavors: Text and Links. Running your mouse over a comment expands it fully so you can read it. For a link, clicking on it will fire the browser and navigate you to that node - great for incorporating Reporting Services or any BI tool that accepts an ID for drilling down - this keeps the app light and flexible - almost like a visual wiki.

More coming, the layout and weighting aspects are next up.

Since InfoMesa is essentially a content visualization concept, the question of visualization comes up a lot. I tend to be a purist when it comes to the term visualization. In other words, to me, conditional formatting in Microsoft Excel is *not* visualization.

There's a concept of not recreating the wheel in InfoMesa, but really staking out new ground - ground that has been heretofore too expensive or too exclusive for many customers and making it pervasive and easily accessible and flexible across many different domains. InfoMesa certainly has its roots in Life Science, which I still consider the high water mark for complex knowledge work. The need to ingest voluminous datasets and find digital needles in digital haystacks and also "telling stories" with data sets is increasingly important in the sciences.

One of the technology sketches that I have been working on for the next phase of InfoMesa is to understand if we could insert a "generic" visualization mechanism inside of InfoMesa that would make InfoMesa relevant not just to Life Science but to any industry dealing with "gnarly" data all the time.

So here is an idea that I'll fly past you. What if...there was a visualizer tool that launches a composer inside of InfoMesa. This composer will allow a scientist to connect directly to diverse data sets, but also multiple data sets - in this way, this composer is actually creating a visual datawarehouse in a sense by meshing heterogeneous data types and sets. With data being pulled from both proprietary data sets on premises and also from cloud data sources this seems critical.

This diagram imports an image (in this case an image of the planet Mars at high resolution) and maps it onto a 3D sphere and then outputs to the display. the display in the final analysis would actually be inside of InfoMesa, but you get the general idea. Pressing Play will create the image below...it is a 3D navigation window - the model can be rotated, zoomed and panned and perhaps most importantly, multiple views can be stored and replayed.

This is a simple example, but let's get a bit more complicated...Here's a diagram that creates a sphere that maps the texture of the Earth to it and then ingests the recorded history of earthquakes from SQL Server (lat/long based) which also includes depth of the quake.

The diagram places the depth of the earthquakes as "spikes" that extend beyond the crust of the Earth to show where the quake occurred (by lat/long) and how deep the quake was. Here's the end visualization.

OK, so let's explore how flexible this method can be. Let's do something more analytic. Let's ingest data from three different datasets and map them together visually. This is a Cardiovascular study that shows patients by Gender (blue/pink) by condition of Cholesterol, Blood Pressure and Max Heart Rate.

Here's the visual. The Axes get "Auto-Generated" and the plots clearly show general behavior and outliers. So, the method seems to map quite broadly across the problem domain set.

Here's another diagram that creates a Naval Electronic Warfare example - the idea of telling a story with data...This diagram imports two pre-built 3D models (Akula and USS Dallas - hello, Hunt for Red October) and then ingests sonar mapping and textures the seabed floor. There is a lot going on here - tens of thousands of data points are in play.

Here's the resulting image from a "birdseye" view...

By rotating the viewer we can look from USS Dallas' perspective against the sea wall...

...here's a view from the stern of the Akula...the detail is striking...

OK, enough wackiness with submarines - back to reality - can we start to get close to what a molecular viewer does? Let's import an e.Coli protein with X,Y,Z of atom and atom type (Nitrogen, Carbon, Sulfur, etc.) and see what we can come up with.

Here's the result...not too bad - but still a long way from cartoon views, backbones, etc, but you can see how we would handle waters and possibly even iso-potentials.

I'll continue to work on this concept, but wanted to socialize it to get some basic feedback...

Introducing Firefly, a project I've been working on with LifeScience/Pharma companies for a while now. Like InfoMesa, the goal is to make this available broadly, so that the community can see Microsoft's strengths in visualization and sensemaking capabilities. Firefly is designed to ingest any network - any collection of nodes and their associated edges, perform readable layouts, provide navigation and implement social networking capabilities to bring insight into these typically complex and intimidating data structures.

Firefly is a Windows Presentation Foundation Application, just like InfoMesa. In fact, Firefly is likely to become part of InfoMesa as a network visualizer in the near future. This is the first post and I'm not quite ready to post the code for general availability, but that's the goal and I think we will be there soon. Firefly will allow someone to point the tool at any database (be it SQL Server, Access, Microsoft Azure Storage [SSDS], or Oracle) and define what the nodes are, what the edges are and Firefly does the rest. Firefly performs a layout of the data, persists it and allows navigation and annotation of the space.

Whenever visualization is employed there is a mantra or sorts that should be remembered: Overview-Detail-Filter-Drill for attributes. This mantra gives the basic needs of any visualization: provide a context for the whole data set, allow drill down into that universe, Allow filters to that data set and bring additional, linked values to the surface.

1. Overview - a zoomable, pannable look at the universe of nodes and edges in the entire graph
2. Alter Visualization - Emphasize/Under emphasize visual elements using color, opacity and visibility - i.e. make the edge lines yellow and faint.
3. Edge Analytics - Find most influential nodes - i.e. Show only the nodes that have 5 to 20 edges - the high influence nodes
4. Search for Nodes by Name - just type or select a node name and Firefly finds it for you - useful when the graph is large
5. Metrics weighed to each Node - allows visualization of addition Channels of Information as an overlay on the graph - radius and color can be mapped
6. Cross Hair for selecting nodes for detail (double-click will pin the cursor to a node)
7. Stored Views - Clicking create View in the top left corner will store the window constraints (x,y, wide, high, scale) to create quickly navigable neighborhoods of nodes and edges

Once you select a node, you can switch to the detail tab (located at the top of the screen) and view just that node and it's associated edges.

Clicking on any node moves that selection to the center and the graph "blooms" to show that nodes associations.

A word about layout and mapping. Since Networks and Graphs are without orientation (or coordinate systems) it is important to create a sense of constancy for the user - in other words, where stuff tends to be is always where it is. Geography is this way and so are star maps (yes, I know there are variations, but the orientation of constellations nearby one another makes it possible to find collections and constellations rather easily).

So, after Firefly creates a layout we freeze that layout and store it (or persist) for exploration - this is different than many other applications that simply reapply layouts - for small networks, persistence doesn't matter, but for large ones you quickly loose orientation.

I will post more information as we go and a new InfoMesa post will be up in a week or so.

One of the coolest parts of my job is meeting with exceptionally smart people in the Life Science industry. Last week I had an opportunity to talk with James Averback of Life Science Integration Partners about Models and the integration of this important concept in the InfoMesa fabric. James is an expert in Regulated Documents, the acknowledge gems and output of the R&D process. As we talked and I demoed InfoMesa to him, we struck on the notion of models as a grouping mechanism.

There are a million definitions of a model, but in this case let's define it as how you do something, who does it, and what you need to accomplish the task. It's important to note that virtually all the things we have in InfoMesa and all the things we do during the work day can fit into a larger grouping of things.

Ideas, Hypothesis, Experiments, Protocols, Procedures, Stages, Departments - they could all be considered models that contain various elements - People, documents, paragraphs, definitions, molecules, atoms, viewpoints, etc.

We've heard a number of times that people enjoy the freeform nature of the InfoMesa Whiteboard, but the question of how to establish law & order in the wild west of free space is a concern.

We think that models are the ticket. Here's how they work...let's look at a freeform assembly of data first...

What are the tasks being done, who is doing the work, when does something happen before something else (precedence)?...models are the key...

These visuals are only to help communicate the basic idea (which we are still working through), the end result will likely be a bit difference.

In general Models are containers - miniature whiteboards (or perhaps better stated: branded whiteboards).

The characteristics of these containers are as follows:

• These containers have type (Idea, Hypothesis, Protocol, Department, etc.)
• These containers can contain any grouping of elements (molecule viewers, images, documents, sticky notes, video annotations, etc.)
• Containers can link to other containers (just as whiteboards link to other whiteboards)
• The linkage of containers can indicate concurrent process, serial process or sequence. Now we have a way to show how things need to progress - sketch out a flow and fill in the blanks.
• A container can have membership of People - like a department or a committee.
• A container can be tagged, just as Elements.
• Voting and Ranking of Type. A container can have attributes of the social environment.  How many people like this idea or hypothesis or how promising is a new idea?
• Models can have linkages to Human Amplification Methods - namely web services and workflows that can work on a researchers behalf-not AI, just processes.

We don't yet know everything about this new approach but here's a high-level graphic to show a basic idea...

And here's an idea of linkage and progression...

These Models need to easily created, assembled, grouped and maintained - I see them as higher level than a Tool, which is a custom application. Anyone should be able to wire together a new model. This has real estate implications and I think that the Services and Tools tabs will combine and a new tab, Models, will be placed on the Toolbar.

James will be providing guidance as we move through these waters and I welcome his input and yours. What are your thoughts about this direction?