September 7, 2013 • #

Inspired by a couple of others, I released a micro project of mine called Terra, to provide a fast way to run several geospatial tools on your computer.


Because I work with a variety of GIS datasets, I end up writing lots of scripts and small automation utilities to manipulate, convert, and merge data, in tons of different formats. Working with geo data at scale like this challenges the non-software developer to get comfortable with basic scripting and programming. I’ve learned a ton in the last couple years about Unix environments, and the community of open source geo tools for working with data in ways that can be pipelined or automated. Fundamental knowledge about bash, Python, or Ruby quickly becomes critical to saving yourself countless hours of repetitive, slow data processing.

GDALThe renaissance tool of choice for all sorts of data munging is GDAL, and the resident command line suites of GDAL and OGR. The GDAL and OGR programs (for raster and vector data, respectively) are super powerful out of the box, once you understand the somewhat obtuse and involved syntax for sending data between datasources, and the myriad translation parameters. But these get extra powerful as multitools for all types of data is when you can read from, and sometimes write to, proprietary data formats like Esri geodatabases, ECW files, MrSID raster images, GeoPDFs, SpatiaLite, and others. Many of these formats, though, require you to build the toolchain from source on your own, including the associated client libraries, and this process can be a giant pain, particularly for anyone who doesn’t want to learn the nuances of make and binary building1. The primary driver for building Terra was to have a simple, clean, consistent environment with a working base set of geotools. It gives you a prebuilt configuration that you can have up and running in minutes.

Terra uses Vagrant, for provisioning virtual machines, and Chef, an automation tool for batching up the setup, and maintaining its configuration. Vagrant is really a wrapper around VirtualBox VMs, and uses base Linux images to give you a clean starting point for each deployment. It’s amazing for running dev environments. It’s supports both Chef and Puppet, two CM tools for automating installation of software. I used Chef since I like writing Ruby, and created recipes to bootstrap the installs.

This all started because I got sick of setting up custom GDAL builds on desktop systems. Next on the list for this mini project is to provision installs of some other open geo apps, like TileMill and CartoDB, to run locally. Try it out on your computer, all you need is VirtualBox and Vagrant installed, and install is a few simple commands. Check it out on GitHub, follow the README to get yourself set up, and post an issue if you’d like to see other functionality included.

  1. Don’t mean to bag on Makefiles, they’re fantastic

Topics:   projects   code   open source