tutorial 5.2:

Mapping and animation

In this tutorial we will look at a few techniques for mapping datapoints in relation to time. Processing is primarily intended as a platform for producing interactive, animated data visualizations, and you're encouraged to explore the built-in capabilities of Processing as well as the functionality added by its many contributed libraries. Unlike the previous examples, we will not apply any transforms in this example; instead we're plotting the datapoints sequentially based on their associated timestamps.


1) Time-based animation


Here's the Processing code for this example:

locationDataMappingSequentialCSV

Many of the datasets you've been using in class are sequential - they associate a timestamp with each datapoint - and it can sometimes be interesting to visualize the data as a sequence in time. To do this, you first need to get the timestamp into a numeric format that can be used in simple arithmetic operations. Because date is usually provided as a text string with variable formatting, you will need to parse this text string in order to produce an integer-based representation of time such as Unix time. Java provides tools for parsing strings into date objects; this can also be done in various other scripting languages such as PHP or Perl. I've chosen to do this in PHP because it provides a particularly flexible function for making a string to date conversion: here's the code I used to do this:

<?php

$dataFileName = 'tracksData.csv';
$outDataFileName = 'output.csv';

$tableINPUT = fopen($dataFileName, "r") or die("can't open file");
$tableOUTPUT = fopen($outDataFileName, "w");
$lineCount = 0;

print("working ... <br>");
while (!feof($tableINPUT) ) {

    $line_of_text = fgetcsv($tableINPUT, 1024);
    $lonTemp = $line_of_text[2];
    $latTemp = $line_of_text[3];
    $altTemp = $line_of_text[4];
    $accuracyTemp = $line_of_text[6];
    $dateTime = $line_of_text[8];

    $timeStamp = strtotime($dateTime);

    $stringData = $lonTemp . "," . $latTemp . "," . $altTemp . "," . $accuracyTemp . "," . $timeStamp . "\n";
    fwrite($tableOUTPUT, $stringData);
    print ($timeStamp . "<br>");
    $lineCount ++;
    //print ($line_of_text[0] . $line_of_text[1]. $line_of_text[2] . "<br>");
}
fclose($tableOUTPUT);
print(" lines printed");

?>

This script in PHP reads in 'tracksData.csv' and outputs a new CSV called 'output.csv' that reformats the timestamp as a Unix time value. It can often save a lot of time to use scripts to format your data before bringing it in to Processing. It's very useful to know a language like Perl, Python or PHP as a tool for quickly formatting data files. Some formatting can also be accomplished quickly in Excel, including date formatting, and this is what I recommend trying first. Unfortunately Excel's ability to recognize and interpret date strings is quite limited compared with Perl or PHP, so if you get stuck with Excel you may need to contact me about translating your timestamps into a usable format. If Excel recognizes your date strings as dates you can use the following function in Excel to convert the date to a Unix timestamp:

=(B2-DATE(1970,1,1))*86400

... where B2 is the cell storing your date.

Once your data has the timestamp in a numeric format you can use this information to sequentially display the data points in your Processing sketch. This sequnetial drawing of the datapoints is done through a small change in the functions that draw the tweets and track-points from Phil Langley's 'locationDataMappingCSV' example that we looked at last week. This is the important change:

if (myTime<=tempTimeStamp) { //DRAW EACH POINT IN THE WALK AND A LINE BETWEEN THEM
  if (tempPosX!=0) {
    stroke(0, 255, 0, 50);
    strokeWeight(4);
    line(tempPosX, tempPosY, posX, posY);
    noStroke();
    fill(255, 0, 0, 50);
    ellipse(posX, posY, dia, dia);
  }
}

Note that the datapoints are only being drawn if the associated timestamp is <= a running time-counter that starts with the first Unix time value in the data.