Results for moritz

  • Multivariate Beer

    Posted to Projects  |  Tags: , ,

    Can you experience data? Sometimes visualization gets you part of the way there, putting data into context, serving as a trigger for your memory, and all that. But only so much can happen through the computer screen.

    I want to feel data the way I want to taste the food in pictures. It's one thing to see something good, and it's another to be at a restaurant to taste a dish direct from the source.
     Continue Reading 

  • A more visual world data portal

    Posted to Data Sources  |  Tags: , ,

    One of the most annoying parts of downloading data from large portals is that you never quite know what you're gonna get. It's a box of chocolates. It's government data sites. It's lists of datasets with vague or unhelpful titles with links to download. Of course, I'd rather have a hodgepodge than nothing at all, but as with most things, there's room for improvement.

    The OECD, which maintains and provides data on the country level, takes steps towards a more helpful portal that makes data grabs less of a headache. With the help of Raureif, 9elements, and Moritz Stefaner, the new portal is still in beta, but there's plenty to like.
     Continue Reading 

  • Data Cuisine uses food as the medium

    Posted to Data Art  |  Tags: ,

    Ditch the computer screen for your data. It's all about the food. Moritz Stefaner and prozessagenten, process by art and design ran a second round of the Data Cuisine workshop to explore how food can be used as a medium to communicate data. Naturally, you've got your basic visual cues, but when you introduce food, you open lots more possibilities.

    [W]e have all kinds of sculptural 3D possibilities. We can work with taste — from the basic tastes of sweet, sour, salty, bitter, umami to complex combinations or hotness. There is texture — immensely important in cooking! Then we have all the cultural connotations of ingredients and dishes (potatoes, caviar, …). We can work with cooking parameters (e.g. baking temperature or duration). Or the temperature of the dish itself, when served!

    The above shows piece of bread shows youth unemployment in Spain. See more data dishes here.

  • An exploration of selfies

    Posted to Data Art  |  Tags: , ,

    Selfiecity, from Lev Manovich, Moritz Stefaner, and a small group of analysts and researchers, is a detailed visual exploration of 3,200 selfies from five major cities around the world. The project is both a broad look at demographics and trends, as well as a chance to look closer at the individual observations.
     Continue Reading 

  • FIFA development work around the world

    Posted to Mapping  |  Tags: , ,

    Studio NAND and Moritz Stefaner, along with Jens Franke explore FIFA development programs around the world.

    The FIFA Development Globe visu­al­ises FIFA's world­wide involve­ment in supporting foot­ball through educa­tional and infra­struc­tural projects. Using a 3D globe in combin­a­tion with inter­con­nected inter­face and visu­al­iz­a­tion elements, the applic­a­tion provides multiple perspect­ives onto an enormous dataset of FIFA's activ­ities, grouped by tech­nical support, perform­ance activ­ities, and devel­op­ment projects.

    The globe itself is an icosahedron, or essentially a spherical shape made up of triangles. Triangles in each country represent programs and are colored by the three above categories, and you might recognize Moritz' elastic lists in the sidebar to filter through programs, by country, organization, and type. There's also a timeline view, which shows program development over the past five years.

    Give it a go here. I should warn you though that it runs in Flash (a client requirement), and it could run sluggish depending on your machine. Sometimes I was disorientated by the interaction and animation, especially when I clicked and nothing happened until a few seconds later.

  • Data sculpture shows emotional response to Olympics

    Posted to Data Art  |  Tags: , ,

    During the Olympics, Studio NAND, Moritz Stefaner, and Drew Hemment tracked Twitter sentiment with Emoto. This interactive installation and data sculpture is the last leg of the project.

    The emoto data sculp­ture repres­ents message volumes, aggreg­ated per hour and senti­ment level in hori­zontal bands which move up and down according to the current number of Tweets at each time. This resulted in simpli­fied 3-dimensional surfaces which allows visitors to identify patterns in message frequency distri­bu­tion more easily. And while not being specific­ally designed in this direc­tion, the surfaces also nicely support haptic exploration.

    The sculpture itself is black and unchanging, and it's used as a projection surface to display a heat map and overlay text. The projection is controlled by the user, which makes for an interesting blend of physical and digital.

  • How to Make an Interactive Network Visualization

    Networks! They are all around us. The universe is filled with systems and structures that can be organized as networks. Recently, we have seen them used to convict criminals, visualize friendships, and even to describe cereal ingredient combinations. We can understand their power to describe our complex world from Manuel Lima's wonderful talk on organized complexity. Now let's learn how to create our own.

    In this tutorial, we will focus on creating an interactive network visualization that will allow us to get details about the nodes in the network, rearrange the network into different layouts, and sort, filter, and search through our data.

    In this example, each node is a song. The nodes are sized based on popularity, and colored by artist. Links indicate two songs are similar to one another.

    Try out the visualization on different songs to see how the different layouts and filters look with the different graphs.

    Technology

    This visualization is a JavaScript based web application written using the powerful D3 visualization library. jQuery is also used for some DOM element manipulation. Both frameworks are included in the js/libs directory of the source code.

    If you hate CoffeeScript, you can always compile the code to JavaScript and start there.The code itself is actually written in CoffeeScript, a little language that is easy to learn, and compiles down to regular JavaScript. Why use CoffeeScript? I find that the reduced syntax makes it easier to read and understand what the code is doing. While it may seem a bit intimidating to learn a whole new 'language', there are just a few things you need to know about CoffeeScript to be a pro.

    Quick CoffeeScript Notes

    Functions

    First and foremost, This is what a function looks like:

    functionName = (input) ->
      results = input * 2
      results
    

    So the input parameters are inside the parentheses. The -> indicates the start of the implementation. If a function's implementation is super simple, this can all go on one line.

    cube = (x) -> x * x * x
    

    A function returns the last thing executed, so you typically don't need a return statement, but you can use one if you like.

    Indentation matters

    The other main syntactical surprise is that, similar to Python, indentation is significant and used to denote code hierarchy and scope. We can see an example of this in the function above: The implementation is indented.

    In practice, this isn't too big of an issue: just hit the Tab key instead of using curly braces – {, } – and you are set.

    Semicolons and Parentheses

    Taking a page from Ruby, semicolons are not needed and should be avoided in CoffeeScript.

    Also, parentheses are optional in many places. While this can get confusing, I typically use parentheses unless they are around a multi-line function that is an input argument into another function. If that doesn't make sense, don't worry – the code below should still be easy to follow.

    For other interesting details, spend a few minutes with the CoffeeScript documentation. You won't be disappointed.

    In-browser CoffeeScript

    Using CoffeeScript is made simpler by the fact that our CoffeeScript code can be compiled into JavaScript right in the browser. Our index.html includes the CoffeeScript compiler which in turn compiles to JavaScript any script listed as text/coffeescript :

    <script src="js/libs/coffee-script.js"></script>
    <script type="text/coffeescript" src="coffee/vis.coffee"></script>
    

    It's just that simple. When vis.coffee loads, it will be parsed and compiled into JavaScript before running. For production, we would want to do this compilation beforehand, but this let's us get started right away.

    Setting up the Network

    You don't have to organize your code like this, if it seems like too much work. But the encapsulation makes it easier to change your input data laterOk, let's get started with coding this visualization. We are going to be using a simplified version of Mike Bostock's (the creator of D3) reusable chart recommendations to package our implementation. What this means for us is that our main network code will be encapsulated in a function, with getters and setters to allow interaction with the code from outside. Here is the general framework:

    Network = () ->
      width = 960
      height = 800
      # ...
      network = (selection, data) ->
        # main implementation
    
      update = () ->
        # private function
    
      network.toggleLayout = (newLayout) ->
        # public function
    
      return network
    

    So the Network function defines a closure which scopes all the variables used in the visualization, like width and height. The network function is where the main body of the code goes, and is returned by Network at the end of the implementation.

    Functions defined on network, like network.toggleLayout() can be called externally while functions like update are 'private' functions and can only be called by other functions inside Network. You can think of it as the same abstraction that classes provide us in object-oriented programing. Here is how we create a new network :

    $ ->
      myNetwork = Network()
      # ...
    
      d3.json "data/songs.json", (json) ->
        myNetwork("#vis", json)
    

    So here, myNetwork is the value Network() returns – namely the function called network. Then we call this network function, passing in the id of the div where the visualization will live, and the data to visualize.

    Network Data Format

    From above, we see we are passing in the data from songs.json to be visualized. How is this data organized?

    Since a network defines connections between nodes as well as the data contained in the nodes themselves, it would be difficult to define as a simple 'spreadsheet' or table of values. Instead, we use JSON to capture this structure with as little overhead as possible.

    The input data for this visualization is expected to follow this basic structure:</p

    {
      "nodes": [
        {
          "name": "node 1",
          "artist": "artist name",
          "id": "unique_id_1",
          "playcount": 123
        },
        {
          "name": "node 2",
          # ...
        }
      ],
      "links": [
        {
          "source": "unique_id_1",
          "target": "unique_id_2"
        },
        {
          # ...
        }
      ]
    }
    

    This is a JSON object (just like a JavaScript object). If you haven't looked at JSON before, then let's take a look now! The format is pretty straight forward (as it's just JavaScript).

    This object requires two names, nodes and links. Both of these store arrays of other objects. nodes is an array of nodes. Each node object needs some fields used in the visualization as well as an id which uniquely identifies that node.

    The objects in the links array just need source and target. Both of these point to @id@'s of nodes.

    The traditional/default method in D3 of defining a link's source and target is to use their position in the nodes array as the value. Since we are going to be filtering and rearranging these nodes, I thought it would be best to use values independent of where the nodes are stored. We will see how to use these id's in a bit.

    Moved by the Force

    We will start with the default force-directed layout that is built into D3 . Force-based network layouts are essentially little physics simulations. Each node has a force associated with it (hence the name), which can repel (or attract) other nodes. Links between nodes act like springs to draw them back together. These pushing and pulling forces work on the network over a number of iterations, and eventually the system finds an equilibrium.

    Force-directed layouts usually result in pretty good looking network visualizations, which is why they are so popular. D3's implementation does a lot of work to make the physics simulation efficient, so it stays fast in the browser.

    Example of force-directed layout from our song network demo
    Force Directed Layout

    To start, as is typical with most D3 visualizations, we need to create a svg element in our page to render to. Lets look at the network() function which performs this action:

    First we declare a bunch of variables that will be available to us inside the Network closure. They are 'global' and available anywhere in Network. Note that our D3 force directed layout is one such global variable called force.

    Inside our network function, we start by tweaking the input data. Then we use D3 to append an svg element to the input selection element. linksG and nodesG are group elements that will contain the individual lines and circles used to create the links and nodes. Grouping related elements is a pretty common strategy when using D3. Here, we create the linksG before the nodesG because we want the nodes to sit on top of the links.

    The update function is where most of the action happens, so let's look at it now.

      # The update() function performs the bulk of the
      # work to setup our visualization based on the
      # current layout/sort/filter.
      #
      # update() is called everytime a parameter changes
      # and the network needs to be reset.
      update = () ->
        # filter data to show based on current filter settings.
        curNodesData = filterNodes(allData.nodes)
        curLinksData = filterLinks(allData.links, curNodesData)
    
        # sort nodes based on current sort and update centers for
        # radial layout
        if layout == "radial"
          artists = sortedArtists(curNodesData, curLinksData)
          updateCenters(artists)
    
        # reset nodes in force layout
        force.nodes(curNodesData)
    
        # enter / exit for nodes
        updateNodes()
    
        # always show links in force layout
        if layout == "force"
          force.links(curLinksData)
          updateLinks()
        else
          # reset links so they do not interfere with
          # other layouts. updateLinks() will be called when
          # force is done animating.
          force.links([])
          # if present, remove them from svg
          if link
            link.data([]).exit().remove()
            link = null
    
        # start me up!
        force.start()
    

    The final version of this visualization will have filtering and sorting capabilities, so update starts with filtering the nodes and links of the total dataset. Then sorts if necessary. We will come back and hit these functions later. For the basic force-directed layout without all these bells and whistles to come, all we really care about is:

    force.nodes(curNodesData)
    updateNodes()
    
    force.links(curLinksData)
    updateLinks()
    

    The force's nodes array is set to our currently displayed nodes, erasing any previous nodes in the simulation. Then we update the visual display of the nodes in the visualization. This same pattern is then followed for the links.

    Remember: the force layout doesn't add circles and lines for you. It just tells you where to put them.It is important to realize that in D3, the nodes and links in the force layout don't automatically get visualized in any way. This is to say that there is a separation between the force-directed physics simulation and any visual mapped to that simulation.

    To create this visual representation associated with this force-directed simulation, we will need to bind to the same data being used, which we do in updateNodes and updateLinks:

      # enter/exit display for nodes
      updateNodes = () ->
        node = nodesG.selectAll("circle.node")
          .data(curNodesData, (d) -> d.id)
    
        node.enter().append("circle")
          .attr("class", "node")
          .attr("cx", (d) -> d.x)
          .attr("cy", (d) -> d.y)
          .attr("r", (d) -> d.radius)
          .style("fill", (d) -> nodeColors(d.artist))
          .style("stroke", (d) -> strokeFor(d))
          .style("stroke-width", 1.0)
    
        node.on("mouseover", showDetails)
          .on("mouseout", hideDetails)
    
        node.exit().remove()
    
      # enter/exit display for links
      updateLinks = () ->
        link = linksG.selectAll("line.link")
          .data(curLinksData, (d) -> "#{d.source.id}_#{d.target.id}")
        link.enter().append("line")
          .attr("class", "link")
          .attr("stroke", "#ddd")
          .attr("stroke-opacity", 0.8)
          .attr("x1", (d) -> d.source.x)
          .attr("y1", (d) -> d.source.y)
          .attr("x2", (d) -> d.target.x)
          .attr("y2", (d) -> d.target.y)
    
        link.exit().remove()
    

    Finally, some D3 visualization code! Looking at updateNodes, we select all circle.node elements in our nodeG group (which at the very start of the execution of this code, will be empty). Then we bind our filtered node data to this selection, using the data function and indicating that data should be identified by its id value.

    The enter() function provides an access point to every element in our data array that does not have a circle associated with it. When append is called on this selection, it creates a new circle element for each of these representation-less data points. The attr and style functions set values for each one of these newly formed circles. When a function is used as the second parameter, like:

    .attr("r", (d) -> d.radius)
    

    The d is the data associated with the visual element, which is passed in automatically by D3. So with just a few lines of code we create and style all the circles we need.

    Because we will be filtering our data to add and remove nodes, there will be times where there is a circle element that exists on screen, but there is no data behind it. This is where the exit() function comes into play. exit() provides a selection of elements which are no longer associated with data. Here we simply remove them using the remove function.

    If the concepts of enter() and exit() are still not clicking, check out the Thinking With Joins and three little circles tutorials. These selections are a big part of D3, so it is worth having a feel for what they do.

    Configuring the Force

    There has to be a ton of Star Wars jokes I should be making... but I can't think of any.In order to get the force-directed graph working the way we want, we need to configure the force layout a bit more. This will occur in the setLayout function. For the force-directed layout, our force configuration is pretty simple:

    force.on("tick", forceTick)
    .charge(-200)
    .linkDistance(50)
    

    Here, charge is the repulsion value for nodes pushing away from one another and linkDistance is the maximum length of each link. These values allow the nodes to spread out a bit.

    The forceTick function will be called each iteration (aka 'tick') of the simulation. This is where we need to move our visual representations of the nodes and links of the network to where they are in the simulation after this tick. Here is forceTick:

      # tick function for force directed layout
      forceTick = (e) ->
        node
          .attr("cx", (d) -> d.x)
          .attr("cy", (d) -> d.y)
    
        link
          .attr("x1", (d) -> d.source.x)
          .attr("y1", (d) -> d.source.y)
          .attr("x2", (d) -> d.target.x)
          .attr("y2", (d) -> d.target.y)
    

    Pretty straightforward. The D3 simulation is modifying the x and y values of each node during the simulation. Thus, for each tick, we simply need to move the circles representing our nodes to where x and y are. The links can be moved based on where their source and target nodes are.

    Setting Up Data

    Speaking of source and target, we need to go back and see how to deal with our initial data where we were using the id of a node in place of the node's index in the nodes array. Here is setupData which is the very first thing executed in our network code:

      # called once to clean up raw data and switch links to
      # point to node instances
      # Returns modified data
      setupData = (data) ->
        # initialize circle radius scale
        countExtent = d3.extent(data.nodes, (d) -> d.playcount)
        circleRadius = d3.scale.sqrt().range([3, 12]).domain(countExtent)
    
        data.nodes.forEach (n) ->
          # set initial x/y to values within the width/height
          # of the visualization
          n.x = randomnumber=Math.floor(Math.random()*width)
          n.y = randomnumber=Math.floor(Math.random()*height)
          # add radius to the node so we can use it later
          n.radius = circleRadius(n.playcount)
    
        # id's -> node objects
        nodesMap  = mapNodes(data.nodes)
    
        # switch links to point to node objects instead of id's
        data.links.forEach (l) ->
          l.source = nodesMap.get(l.source)
          l.target = nodesMap.get(l.target)
    
          # linkedByIndex is used for link sorting
          linkedByIndex["#{l.source.id},#{l.target.id}"] = 1
    
        data
    

    setupData is doing a few things for us, so let's go through it all. First, we are using a d3.scale to specify the possible values that the circle radii can take, based on the extent of the playcount values. Then we iterate through all the nodes, setting their radius values, as well as setting their x and y values to be within the current visualization size. Importantly, nodes are not automatically sized by any particular data value they contain. We are just adding radius to our data so we can pull it out in updateNodes. The x and y initialization is just to reduce the time it takes for the force-directed layout to settle down.

    Finally, we map node id's to node objects and then replace the source and target in our links with the node objects themselves, instead of the id's that were in the raw data. This allows D3's force layout to work correctly, and makes it possible to add/remove nodes without worrying about getting our nodes array and links array out of order.

    Radializing the Force

    Now we have everything needed to display our network in a nice looking, fast, force-directed layout. It might have been a lot of explanation, but we did it in a pretty small amount of code.

    The force-directed layout is a great start, but also a bit limiting. Sometimes you want to see your network in a different layout – to find patterns or trends that aren't readily apparent in a force-directed one. In fact, it would be really cool if we could toggle between different layouts easily – and allow our users to see the data in a number of different formats. So, lets do that!

    Here's a basic idea: we are going to hijack D3's force-directed layout and tell it where we want the nodes to end up. This way, D3 will still take care of all the physics and animations behind the scenes to make the transitions between layouts look good without too much work. But we will get to influence where the nodes go, so their movements will no longer be purely based on the underlying simulation.

    Radial layout example, grouping song nodes by artistRadial Network Layout

    Always force-jack with extreme cautionTo aid in our force-jacking, I've created a separate entity to help position our nodes in a circular fashion called RadialPlacement. Its not really a full-on layout, but just tries to encapsulate the complexity of placing groups of nodes. Essentially, we will provide it with an array of keys. It will calculate radial locations for each of these keys. Then we can use these locations to position our nodes in a circular fashion (assuming we can match up our nodes with one of the input keys).

    RadialPlacement is a little clumsy looking, but gets the job done. The bulk of the work occurs in setKeys and radialLocation :

    # Help with the placement of nodes
    RadialPlacement = () ->
      # stores the key -> location values
      values = d3.map()
      # how much to separate each location by
      increment = 20
      # how large to make the layout
      radius = 200
      # where the center of the layout should be
      center = {"x":0, "y":0}
      # what angle to start at
      start = -120
      current = start
    
      # Given a set of keys, perform some
      # magic to create a two ringed radial layout.
      # Expects radius, increment, and center to be set.
      # If there are a small number of keys, just make
      # one circle.
      setKeys = (keys) ->
        # start with an empty values
        values = d3.map()
    
        # number of keys to go in first circle
        firstCircleCount = 360 / increment
    
        # if we don't have enough keys, modify increment
        # so that they all fit in one circle
        if keys.length < firstCircleCount       increment = 360 / keys.length     # set locations for inner circle     firstCircleKeys = keys.slice(0,firstCircleCount)     firstCircleKeys.forEach (k) -> place(k)
    
        # set locations for outer circle
        secondCircleKeys = keys.slice(firstCircleCount)
    
        # setup outer circle
        radius = radius + radius / 1.8
        increment = 360 / secondCircleKeys.length
    
        secondCircleKeys.forEach (k) -> place(k)
    
      # Gets a new location for input key
      place = (key) ->
        value = radialLocation(center, current, radius)
        values.set(key,value)
        current += increment
        value
    
      # Given an center point, angle, and radius length,
      # return a radial position for that angle
      radialLocation = (center, angle, radius) ->
        x = (center.x + radius * Math.cos(angle * Math.PI / 180))
        y = (center.y + radius * Math.sin(angle * Math.PI / 180))
        {"x":x,"y":y}
    

    Hopefully the comments help walk you through the code. In setKeys our goal is to break up the total set of keys into an inner circle and an outer circle. We use slice to pull apart the array, after we figure out how many locations can fit in the inner circle.

    radialLocation does the actual polar coordinate conversion to get a radial location. It is called from place, which is in turn called from setKeys.

    Toggling Between Layouts

    Lets the user explore different layouts interactivelyToggle Layout

    With RadialPlacement in tow, we can now create a toggle between our force-directed layout and a new radial layout. The radial layout will use the song's artist field as keys so the nodes will be grouped by artist.

    In the update function described above, we saw a mention of the radial layout:

     if layout == "radial"
          artists = sortedArtists(curNodesData, curLinksData)
          updateCenters(artists)
    

    Here, sortedArtists provides an array of artist values sorted by either the number of songs each artist has, or the number of links. Let's focus on updateCenters, which deals with our radial layout:

      updateCenters = (artists) ->
        if layout == "radial"
          groupCenters = RadialPlacement().center({"x":width/2, "y":height / 2 - 100})
            .radius(300).increment(18).keys(artists)
    

    We can see that we just pass our artists array to the RadialPlacement function. It calculates locations for all keys and stores them until we want to position our nodes.

    Now we just need to work on this node positioning and move them towards their artist's location. To do this, we change the tick function for the D3 force instance to use radialTick when our radial layout is selected:

      # tick function for radial layout
      radialTick = (e) ->
        node.each(moveToRadialLayout(e.alpha))
    
        node
          .attr("cx", (d) -> d.x)
          .attr("cy", (d) -> d.y)
    
        if e.alpha < 0.03       force.stop()       updateLinks()   # Adjusts x/y for each node to   # push them towards appropriate location.   # Uses alpha to dampen effect over time.   moveToRadialLayout = (alpha) ->
        k = alpha * 0.1
        (d) ->
          centerNode = groupCenters(d.artist)
          d.x += (centerNode.x - d.x) * k
          d.y += (centerNode.y - d.y) * k
    

    We can see that radialTick calls moveToRadialLayout which simply looks up the location for the node's artist location from the previously computed groupCenters. It then moves the node towards this center.

    This movement is dampened by the alpha parameter of the force layout. alpha represents the cooling of the physics simulation as it reaches equilibrium. So it gets smaller as the animation continues. This dampening allows the nodes repel forces to impact the position of the nodes as it nears stopping – which means the nodes will be allowed to push away from each other and cause a nice looking clustering effect without node overlap.

    We also use the alpha value inside radialTick to stop the simulation after it has cooled enough and to have an easy opportunity to redisplay the links.

    Because the nodes are different sizes, we want them to have different levels of repelling force to push on each other with. Luckily, the force's charge function can itself take a function which will get the current node's data to calculate the charge. This means we can base the charge off of the node's radius, as we've stored it in the data:

    charge = (node) -> -Math.pow(node.radius, 2.0) / 2
    

    The specific ratio is just based on experimentation and tweaking. You are welcome to play around with what other effects you can come up with for charge.

    Filter and Sort

    Force-directed and Radial layouts after filtering for obscure songsFiltering Example

    The filter and sort functionality works how you would expect: we check the networks current setting and perform operations on the nodes and links based on these settings. Let's look at the filter functionality that deals with popular and obscure songs, as it uses a bit of D3 array functionality:

      # Removes nodes from input array
      # based on current filter setting.
      # Returns array of nodes
      filterNodes = (allNodes) ->
        filteredNodes = allNodes
        if filter == "popular" or filter == "obscure"
          playcounts = allNodes.map((d) -> d.playcount).sort(d3.ascending)
          # get median value
          cutoff = d3.quantile(playcounts, 0.5)
    
          filteredNodes = allNodes.filter (n) ->
            if filter == "popular"
              n.playcount > cutoff
            else if filter == "obscure"
              n.playcount
        filteredNodes
    

    filterNodes defaults to returning the entire node array. If popular or obscure is selected, it uses D3's quantile function to get the median value. Then it filters the node array based on this cutoff. I'm not sure if the median value of playcounts is a good indicator of the difference between 'popular' and 'obscure', but it gives us an excuse to use some of the nice data wrangling built into D3.

    Bonus: Search

    Search bar - Simple searching made easySearch

    Search is a feature that is often needed in networks and other visualizations, but often lacking. Given a search term, one way to make a basic search that highlights the matched nodes would be:

      # Public function to update highlighted nodes
      # from search
      network.updateSearch = (searchTerm) ->
        searchRegEx = new RegExp(searchTerm.toLowerCase())
        node.each (d) ->
          element = d3.select(this)
          match = d.name.toLowerCase().search(searchRegEx)
          if searchTerm.length > 0 and match >= 0
            element.style("fill", "#F38630")
              .style("stroke-width", 2.0)
              .style("stroke", "#555")
            d.searched = true
          else
            d.searched = false
            element.style("fill", (d) -> nodeColors(d.artist))
              .style("stroke-width", 1.0)
    

    We just create a regular expression out of the search, then compare it to the value in the nodes that we want to search on. If there is a match, we highlight the node. Nothing spectacular, but its a start to a must-have feature in network visualizations.

    We can see that updateSearch is a public function, so how do we connect it to the UI on our network visualization code?

    Wiring it Up

    The other button groups use very similar code.There are a lot of ways we could connect our buttons and other UI to the network functionality. I've tried to keep things simple here and just have a separate section for each button group. Here is the layout toggling code:

      d3.selectAll("#layouts a").on "click", (d) ->
        newLayout = d3.select(this).attr("id")
        activate("layouts", newLayout)
        myNetwork.toggleLayout(newLayout)
    

    So we simply active the clicked button and then call into the network closure to switch layouts. The activate function just adds the active class to the right button.

    Our search is pretty similar:

      $("#search").keyup () ->
        searchTerm = $(this).val()
        myNetwork.updateSearch(searchTerm)
    

    It just uses jQuery to watch for a key-up event, then re-runs the updateSearch function.

    Thanks and Goodnight

    Hopefully we have hit all the highlights of this visualization. Interactive networks are a powerful visual, and this code should serve as a jumping off point for your own amazing network visualizations.

    Other placement functions could be easily developed for more interesting layouts, like spirals, sunflowers, or grids. Filtering and sorting could be extended in any number of ways to get more insight into your particular dataset. Finally, labelling could be added to each node to see what is present without mousing over.

    I hope you enjoyed this walk-through, and I can't wait to see your own networks!

  • Worldwide mood around London 2012

    Posted to Data Art  |  Tags: , ,

    No doubt there is going to be a lot of tweeting about the Olympics during the next couple of weeks, but sometimes it's hard to get a sense of what people are talking about because of the high volume. Emoto, a team effort by Drew Hemment, Moritz Stefaner, and Studio NAND, is a Twitter tracker that aggregates sentiment around topics.
     Continue Reading 

  • River flow simulation

    Posted to Data Art  |  Tags: ,

    In Newcastle, there's a floating tide mill building on the River Tyne. The mill turns to generate power for the building, and in that flow of water are four sensors for oxygen, acidity, nitrates and salinity. Values for these metrics, along with wheel speed, are captured about every thirty minutes. Stephan Thiel of Studio NAND, in collaboration with Moritz Stefaner, visualized this data in an abstracted simulation of the flow through the tidemill.

    Particles are continuously moving from right to left, being attracted or repelled by four circular zones representing the sensor values. The overall behavior of the particles is influenced by the turning speed of the waterwheel. If the value of one sensor is above its mean value, particles are repelled. If the value is below the mean, particles are attracted towards the center of the zone.

    For example, if all four values are greater than the mean, you end up with four circular swells around these zones. In the above, oxygen is below the mean, so the simulated flows head towards the center of the oxygen zone instead of move around it like with the three zones before. So you end up with a sort of fingerprint for each window of data capture.

    The data itself is probably of little interest to anyone who doesn't work at the mill, but the aesthetics of the piece is calming and certainly evokes the context of what the data represents.

    The wind map by Wattenberg and Viegas and Drawing Water by Wicks come to mind. Oh, and also perpetual ocean.

  • Updated OECD Better Life index

    Posted to Visualization  |  Tags: , ,  |  Kim Rees

    The OECD's Better Life Index which debuted last year to much fanfare has been updated with some great new features by Moritz Stefaner.

    The concept and beauty of the original piece remain intact. However, the experience is made better by the ability to compare to different demographics. For instance, after I adjust my Better Life settings, I can see how my settings compare to other women my age in the US, or to French men. It's fun to compare to different people around the world and watch the flowers readjust themselves to the various comparisons. It invokes a sense of global community and humanity.
     Continue Reading 

  • Data and visualization blogs worth following

    Posted to Visualization  |  Tags: ,

    About three years ago, I shared 37 data-ish blogs you should know about, but a lot has changed since then. Some blogs are no longer in commission, and lots of new blogs have sprung up (and died).

    Today, I went through my feed reader again, and here's what came up. Coincidentally, 37 blogs came up again. (Update: added two I forgot, so 39 now.) I'm subscribed to a lot more than this since I don't unsubscribe to dried up feeds. But this list is restricted to blogs that have updated in the past two months and are at least four months old.
     Continue Reading 

  • Learning data visualization

    I listen to a lot of podcasts. They make my workouts much more enjoyable. For the most part though, I only listen to ones about sports and more general podcasts about design, technology, and working from home. However, a couple of months ago, Enrico Bertini and Moritz Stefaner started Data Stories, a podcast on visualization. Enrico is a researcher in the area and Moritz is more of a practitioner, so it's a good contrast between the two.

    Neither had experience producing podcasts before this, so it was rough around the edges at first. But each episode has been getting better. I highly recommend it.

    In the most recent episode, with Andy Kirk, they discuss the most common question from people new to the field: how to get started. Go ahead and listen. It's a good one if you're itching to get your feet wet.

    One thing I'd add (that maybe I missed as cars drove past me) is that it's important to establish what you want to learn visualization for. The purpose will change what methods to use and what software to learn. Monitoring server load for a web service is going to be different than say, designing an atlas.

  • Watching ‘wtf Wikipedia’ as SOPA/PIPA blackout begins

    Posted to Network Visualization  |  Tags: , , ,

    While SOPA and PIPA are no laughing matter (join the strike), the reaction from those on Twitter who don't know what's going on is great entertainment. Do a search on 'wtf wikipedia' for tweets from confused individuals who are trying to find information on stuff. I'm just going to leave Twitter trackers Revisit and Spot, by Moritz Stefaner and Jeff Clark, respectively, open all day. "OMG I'm doing homework and Wikipedia is blacked out wtf !!!!!!!!!!!!!!!!!!!!!"

  • The Best Data Visualization Projects of 2011

    Posted to Visualization  |  Tags: , ,

    I almost didn't make a best-of list this year, but as I clicked through the year's post, it was hard not to. If last year (and maybe the year before) was the year of the gigantic graphic, this was the year of big data. Or maybe we've gotten better at filtering to the good stuff. (Fancy that.) In any case, data graphics continue to thrive and designers are putting more thought into what the data are about, and that's a very good thing.

    So here are my favorites from 2011, ordered by preference. The order could easily scramble depending when you ask me.
     Continue Reading 

  • Data visualization freelancing

    In an interview with Enrico Bertini, Moritz Stefaner, whose work you'll probably recognize, talks about his experiences as a freelancer and how he got started. Some of the highlights include how to get your name out there, important skills, and the demand for people who know data, visualization, and aesthetics. At nearly an hour long, there's a lot of good information in there.

    The main takeaway: get started now, play with data, hone your skills, and the work will come.

    [Fell in Love with Data]

  • The Vizosphere

    There are lots of people on Twitter who talk visualization. Moritz Stefaner had some fun with Gephi for a view of a whole lot of those people. He calls it the Vizosphere.
     Continue Reading 

  • Challenge: Visualize the impact of Wikipedia

    Posted to Contests  |  Tags: ,

    If you're like me, you've probably used Wikipedia at least once in the past week (or day... or hour). It's had a huge impact on how we find information and keep history up-to-date. The online encyclopedia turned 10 this year, and to celebrate, WikiSym and the Wikimedia foundation recently launched a challenge: WikiViz 2011.

    WikiViz 2011 is about visualizing the impact of Wikipedia. We want to see the most effective, compelling and creative data-driven visualizations of how Wikipedia impacted the world with its content, culture and open collaboration model. Potential topics include: the imprint of Wikipedia on knowledge sharing and access to information; its impact on literacy and education, journalism and research; on the functioning of scientific and cultural organizations and businesses, as well as the daily life of individuals around the world.

    There are lots of small datasets within Wikipedia articles, but Wikipedia itself is also one giant (open) dataset. For example, we've seen the history of the world according to tagged events as well as back and forth discussions for deletion.

    Can you find something good? Judged by Moritz Stefaner, Andrew Vande Moere, and Kim Rees, among others, winners get to attend WikiSym on the house and of course get a mention or two.

  • Custom maps in Processing

    Posted to Quicklinks  |  Tags: , ,

    Till Nagel teaches you how to design custom maps in Processing with TileMill. Could come in handy one day. Saving for later. [via]

  • Better Life Index measures well-being across countries

    Posted to Visualization  |  Tags: , , , ,

    OECD, with the help of Moritz Stefaner and Raureif, promote a well-being index beyond GDP in the Better Life Initiative:

    There is more to life than the cold numbers of GDP and economic statistics — This index allows you to compare well-being across countries, based on 11 tpoics the OECD has identified as essential, in the areas of material living conditions and quality of life.

    Based on topics such as health, housing, and education, each country is represented with a flower, and each petal on a flower represents a metric. The higher the index, the higher the flower appears on the vertical axis, and if the flower metaphor is too abstract for you, roll over each flower to see the breakdown by bar graph.
     Continue Reading 

  • Ben Fry on visualization future and data literacy

    Posted to News  |  Tags: , , ,

    Ben Fry, co-creator of Processing and head of Fathom Design, talks data visualization with O'Reilly Radar editor Mac Slocum. When asked about the concern over visualization and analysis getting into amateur hands:

    I think it’s kind of funny… The same argument has been made with any technological leap since the beginning of time. Books printed in mass had a similar reaction. The internet came along and everybody could post things on the internet and wouldn’t that be the end of the world… The important thing is to focus on the literacy aspect of it. The more that people are doing the work — it all kind of goes to improve the conversation of what’s good, bad useful and what’s not.

    When asked how he sees visualization developing over the next couple of years:

    I think the real thing that's going to change is that we're going to start understanding that visualization isn't this sort of monolithic thing... I like to look at it a lot like writing. You have novels and poetry and haikus. You know there's lots of different types of writing and styles of writing — and I think the same thing happens in visualization... some things are tools for analysis and some things are purely for entertainment, and there's not so much a spectrum that there is different ways of addressing it.

    Watch the short eight-minute interview below. There are some other interesting soundbites in there. I especially like the tidbit at the end about snippy discussions within the visualization sphere. Similar sentiments in a recent Q&A with Moritz Stefaner.
     Continue Reading