tutorial 4.1:

Simple projection: XML

In this tutorial we will look at a simple method for representing geolocated data in your Processing sketches. In this example we will read a data stream in XML format (Extensible markup language). Many live data streams are available only in XML format, so it's useful to become familiar with the tools in Processing for reading XML data. The XML data can be read into an object known as the XML object: this is similar to the table object in that it stores the XML data and provides a range of methods for accessing and working with your data.


1) Data sources


Many social media services with location tracking, such as Foursquare and Twitter, offer a way to download data recording your own history of activity, usually in XML format. For example, Foursquare provides an RSS feed (this is a specific type of XML) of user data that can be accessed at this url: https://foursquare.com/feeds/. To access live twitter data you will need to sign up as a developer; there's also a useful list of API's here for accessing twitter data.

There are also numerous open data search tools such as Google Fusion Charts, ManyEyes, etc that can be used to search for geolocated datasets online.

Many sources of open data government data provide geolocated data, for example the London Datastore which offers live geolocated data streams.

You can also use mobile apps to generate your own geolocated data. The 'My Tracks' Android app allows you to record a path as a series of geolocated points and download the data as a CSV, which can then be loaded into Processing. The Google Maps mobile app also allows you to track your location and download the data as a KML file (XML format that can be converted to CSV in Google Earth).

The map data used in the examples below can be found on OpenStreetMap, an open source mapping project that allows you to download map data in XML format. The Unfolding library for Processing allows you to easily work with OpenStreetMaps data, as well as geographic data from several other providers such as Microsoft and Google; unfortunately this is available for Processing 1.5.1.

2) Code Examples


The following example provides basic functions for plotting geographic coordinates as dots in a Processing sketch. In this example, there is a direct translation of latitude and longitude coordinates to y and x coordinates, a very simple form of map projection but one that is accurate enough for mapping over small distances, where the curvature of the earth is not a major distorting factor for distance, area and angular measurements.

Here is the code:

/*

 A sketch written by Phil Langley

 Data Used :
 openStreetMap data downloaded from http://www.openstreetmap.org/

 This function maps latitude and longitude coordinates to y and x coordinates

*/


int W=600;
int H=600;
int BOARDER = 200;

float posX;
float posY;
float dia;
float col;

Table dataTable;
Table dataTableTweets;
XML xmlData;
XML[] childrenNode;

float lonMin = 100000000;
float lonMax = -100000000;
float latMin = 100000000;
float latMax = -100000000;

int SCALEFACTOR = 15000;

PFont Calibri;
PImage twtImg;

void setup(){
  smooth();
  readData();
  sortXML();
  size(W,H);
  Calibri = loadFont("Calibri-12.vlw");
}

void draw(){
  noLoop();
  background(0);
  drawXML();
}

There are three custom functions in this code which I'll describe briefly here:

void readData(){
   xmlData = loadXML("dataMap.xml");
   childrenNode = xmlData.getChildren("node");
}

The readData() function reads an XML data file into an XML object in Processing. childrenNode is an array containing all the values in the XML data source corresponding to the 'node' tag.

void sortXML(){ 
  float lonTemp;
  float latTemp;
  for (int i = 0; i < childrenNode.length; i++) {
    lonTemp = childrenNode[i].getFloat("lon");
    if (lonTemp > lonMax){
      lonMax = lonTemp;  
    }
    if (lonTemp < lonMin){
      lonMin = lonTemp;
    }
    latTemp = childrenNode[i].getFloat("lat");
    if (latTemp > latMax){
      latMax = latTemp;
    }
    if (latTemp < latMin){
      latMin = latTemp;
    }
  }

  W = int((lonMax-lonMin)*SCALEFACTOR);
  H = int((latMax-latMin)*SCALEFACTOR);

} 

The sortXML() function reads data from an XML datafile containing openStreetMaps data and determines the width and height of the Processing window based on the minimum and maximum values of latitude and longitude.

void drawXML(){
  float lon;
  float lat;
  for (int i = 0; i < childrenNode.length; i++) {
    lon = ((childrenNode[i].getFloat("lon"))-lonMin)*SCALEFACTOR;
    lat = H-((childrenNode[i].getFloat("lat"))-latMin)*SCALEFACTOR;

    fill(255,255,255,10);
    noStroke();
    ellipse(lon, lat, 6,6); 
    fill(255,255,255,20);
    noStroke();
    ellipse(lon, lat, 3,3); 
    fill(255,255,255);
    noStroke();
    ellipse(lon, lat, 1,1);   
  } 
}

The drawXML() function reads data from an XML datafile containing openStreetMaps data and draws the individual datapoints to a map as x,y coordinates.



The SCALEFACTOR variable has not been rigorously determined and is just a way to convert latitude and longitude values into Processing coordinates in a way that results in a legible map.

You can download the code and data from this example here.