The purpose of map generalisation is to represent spatial data in a way that makes it possible to effectively view the data at scales smaller than that for which it was originally intended. In the case of the Ordnance Survey’s MasterMap product you have data at an incredibly fine level of spatial resolution, which is ideally viewed at a scale of approximately 1:1000 give or take 500. When you are reliant on MasterMap, but need to create map of a wider area you are faced with the challenge of generalising data so that it can be ably understood, this means reducing the complexity of components of the spatial data, for instance smoothing wiggly lines, or transforming complicated polygons into simpler ones, as well as aggregating an abundance of small features into larger ones. Such interventions are necessary because it is increasingly difficult to resolve fine detail as map scale decreases, leading to complex shapes appearing messy and disordered when visualised at smaller scales that that which they were intended for. The actual scale of a map gives us an insight into the types of objects that it is possible to resolve at different scales; at a scale of 1:1000 a physical distance of 100cm represents 1km, at 1:10000 and 1:100000 the distance of 1km is covered by 10cm and 1cm respectively. If we conservatively suggest that we can resolve features that are 5mm across, then at 1:1000; 1:10000; and 1:100000, the smallest real world objects that can be represented are 5m,  50m, and 500m respectively. These distances equate to real world objects such as large cars and trucks (c. 5m in length), Olympic-sized swimming pools and office buildings (50m), whilst a distance of 500m is twice the span of Tower Bridge. Evidently, there are significant difference in what constitutes appropriate detail at each of these scales.

I’ve been dealing with one such problem recently, involving the representation of MasterMap building outlines at a scale of 1:10000, somewhat smaller that the 1:1000ish that it was intended for. In order to create an effective map I did needed to generalise the building outlines, however, unfortunately I don’t have access to ESRI’s ArcGIS “simplify building” tool due to licensing restrictions, so I had to come up with another solution. Initially I attempted the classic line generalisation procedure – the Douglas-Peucker algorithm, which simplifies by reducing the number of points in a curve subject to some pre-specified threshold value. However, buildings are strong geometric shapes, often rectangular and orthogonal, so an algorithm such as the Douglas-Peucker can have the effect of disrupting the geometric regularity of building outlines, removing corners etc. What is required is a polygon simplification algorithm that preserves orthogonality, however I couldn’t find anything that did this whilst being accessible, instead I had to come up with a procedure to approximate a generalisation of the building polygons by another methods. The image below reveals the result, which I think is successful enough to use.

In the image above, A is the raw data, and B is the generalised data. I experimented with a few approaches, but the one I assessed as being the best was to position an enclosing rectangle around each building polygon, so that the area of the enclosing rectangle was minimised, and subsequently buffer the result to close any small gaps, choosing to dissolve as well in order to further reduce the complexity. Subsequently I removed the particularly small buildings. The generalisation is more in evidence in the image below, in which A and B are the same as before.

I am reasonably pleased with the result, which was achieved after a little trial and error. Whilst technical approaches to orthogonal simplification exist I can’t imagine them being much more effective at this scale, although perhaps at smaller scales they would be more appropriate as they can create meaningful aggregations of building based upon characteristics such as nearest-neighbour distance.

I work a lot with flows and spatial interactions, one thing that I’ve wanted to do for a while is compute a mean flow direction surface. Unfortunately, arithmetic means don’t work for angular data, this is because it cannot account for the circular nature of the distribution of angular measurements. For instance the angles 5 degrees and 355 degrees are seperated only by 10 degrees, but their arithmetic mean is 180 degrees -w ay off, it should be 0 degrees!

Luckily, Brunsdon and Charlton have published on this very subject, so I took it upon myself to implement a weighted circular mean function in Python. The key obstacle was learning about complex numbers, about which, up until this point, I had no idea about at all!

The first thing to do is calculate the angle between a set of candidate points (such as people) and a set of services (such as Medical Centres). This is simple enough to do using, and would look something like:

import math

math.atan2((y2-y1),(x2-x1))

In which the pair (x1,y1) is the location of the candidate point, and (x2,y2) the location of the allocated service for that candidate point. The line linking these two points defines a flow from a candidate point, to a servcie and vice versa.

Having calculated all of the angles, I used ArcGIS to create an output grid, at the extent of the candidate points, using the “fishnet” function which creates a vector grid of prespecified dimensions.

The beauty of Brunsdon and Charlton’s method is that it uses a local method of approximation, this means that for each cell in the output grid, a mean direction can be calculated based upon the values of nearby points, applying a weighting allows for more distance points to have less of an effect on the mean direction.

Firstly, I read all the candidate points into a KDTree structure, this allows me to search for local points, at the same time I also create an array of the angles for those candidate points.

from scipy.spatial import cKDTree
import numpy as np

tree = cKDTree(treepoints)
res, idx = tree.query(testpoint,300000,0,2,100)
res = res[0][np.where(res[0] < np.Inf)[0]]
idx = idx[0][:len(res)]

The tree takes a numpy array of coordinate pairs, and the query method returns an array of distances to points (res) and their index value in the original array of coordinates (idx). The testpoint is a cell in the vector grid; 300000 is the k-number of nearest neighbours to find, here I have simply set it arbitrarily high in the context of my dataset; 0 is for approximate nearest neighbours, here I’ve specified exact; 2 indicates the use of euclidian distance; and 100 is the threshold, neighbours won’t be returned if they are further than 100 metres away. The penultimate line simply returns an array that is shortened to just those values which are less than 100m away (i.e. less than infinity) – points over 100m away are returned as value Inf.

The next step is to actually compute the mean direction, this requires a special approach using complex numbers however. Brunsdon and Charlton show that a direction can be stated as a complex number z in which z = exp(iθ) this is effectively: z = cos(θ) + i sin(θ)  in which i is an imaginary number. We can restate our directions in Python using:

import cmath

thetas = angles[idx]
cThetas = []
for i in xrange(0,len(thetas)):
    cThetas.append(complex(np.cos(thetas[i]),np.sin(thetas[i])))
cThetas = np.array(cThetas)

Here, the complex function allows the complex number representing an angle to be stored in a list, which I convert (lazily) to a numpy array. The first term, thetas, is using the idx array from the cKDTree to cleverly index the relevant angle records from the angles array which stores all the angle values in the order of entries for the cKDTree.

Next a temporary variable is created which calculates the mean direction:

temp = np.sum(cThetas)/np.absolute(np.sum(cThetas))
MeanDir = np.angle(temp, deg = True)

The mean direction is given by the argument (Arg) of the resultant complex number, Python implements this with the np.angle function, where deg = True returns the angle in degrees, and False in radians.

So far this is the unweighted mean, aggregating directional observations within a 100m disk (see also: uniform disk smoothing). To introduce weighting we must first define a weighting scheme, I’ve used the one suggested by Brunsdon and Charlton, which is Gaussian, and might look at bit like this:

def gaussW(dists,band):
    out = np.zeros(dists.shape)
    for i in xrange(0,len(out)):
        temp = np.power(dists[i],2)/(2.0*np.power(float(band),2))
        out[i] = np.exp(-1.0 * temp)
    return out

weight = gaussW(res,100)

Quite simply, I pass the distance array res to the gaussW function and it gives me back an array of weights for that ordering of distances. Using this I can redo the mean direction thus:

temp = np.sum(weight*cThetas)/np.absolute(np.sum(weight*cThetas))
MeanDir = np.angle(temp, deg = True)

There you have it! Attached is the script I used. Obviously, Brunsdon and Charlton implement a variance and a couple of visualisation devices, but these should be simple enough to implement now!

I created an output for flows of patients to GPs in Southwark, visualised using one of ESRI’s circular/direction colour ramps from colour ramp pack 2. Not sure how best to visualise the legend at this point though. NB. 90 is north, -90 is South, 0/-0 is East and 180/-180 is West. The map is visualised to show the 4 cardinal directions, but the output is in fact continuous.

My example script is here. Note that I am using dbfpy to read and write to shapefile DBF tables directly.

The intent of this post is not simply to uncover where the highest density of people belonging to a particular ethnic group are, but rather to use the ‘location quotient’ (LQ) technique to compare the ethnic density in any one area to the overall ethnic density in Southwark, thus providing a relative insight into where the density of particular groups is more, less or as dense as expected.

Location Quotients tend to work with areal units, characterising different areas subject to a larger region and providing a basic insight into where functions are clustered. Because the Southwark patient register data is address geocoded, we would be losing some spatial information if we choose to aggregate the data, not to mention the question of which areal aggregation is best. More info on how to create location quotients here.

A Location Quotient has 3 possible interpretations; if it is around 1 then the ethnic population in that area is at the level we would expect given what we observe nationally. If the LQ is less than 1 then that area has a lesser population of a particular ethnic group that what we would expect based upon national figures. Finally, in the LQ value is over 1 this suggests a concentration of the ethnic group in the area which is greater than we would expect given nationally observed levels. A LQ is quite simply a rate-ratio.

Instead of the standrad areal approach, the maps here use a density estimation approach in which disaggregate point data is transformed into a representation of the continuous density function of the point distribution. The LQ can then be computed for each cell based on the density of that cell with respect to the total density of the surface. This creates a smoothed LQ surface which is readily interpretable in the same manner as above. The Kernel Density Estimation used to create the ethnic and total population density surfaces should be parameterised in the same way; these examples use a 250m bandwidth and a 25m cells size, which is largely empirically redundant, based on the input dataset’s spatial resolution, but creates a more aesthetically appealing mapped representation. Naturally, the procedure works well for clustered data, in Southwarks case for the African and Muslim groups.

I saw a thought provoking presentation recently, given by Wenwen Li of the University of California Santa Barbara, the talk was a wide ranging insight into Cyber Infrastructure, its uses for geospatial information, and some of the computational techniques that underpinned the project. One element of the project involved zone design for the greater Los Angeles region, and involved the implementation of an algorithm that was intended to aggregate small areal units into larger zones whilst meeting a number of conditions, principle among these conditions was ‘compactness’. The output looked very much like a single hierarchy of Christaller hexagons, and this got me thinking about the nature of space and compactness.

Christaller’s hexagons are the defining illustration of something called ‘central place theory’, a geographical abstraction that idealises settlement pattern based upon an underlying space which is assumed to be isotropic. The assumption of spatial isotropy is the big leap in this model, it assumes that the ‘friction of distance’ from any given point increases at an equal rate whichever way you go from that point. Clearly such a suggestion is not applicable to Los Angeles, where huge freeways and interchanges can make adjacent parcels of land remote neighbours, and increase the connection between advantageously placed non-adjacent sites? Surely a city in which sprawl and ribbon development, as well as segregated communities should be modeled differently? Why then do many of our zoning algorithms favour compact ‘circular’ shapes, very much in the christaller mould, and why do we reject uncompact areal features as ugly slivers? In short, how did the circle come to be the ideal shape of zone in regional studies? Certainly, it is easier, both implementationally and conceptually, to model circles than to consider optimising a zone system over an n zone by n zone similarity matrix pertaining to variables which may be important to aggregating any set of areal units. However, as we explore more and more the complex systems defined by cities and regions, surely there is a need to start integrating a more realistic anisotropic view of space, one in which the friction of distance from any given point in any given direction is defined by the underlying demography, built environment and/or infrastructure.

One such attempt at this, AMOEBA (A Multidirectional Optimum Ecotope-Based Algorithm), developed by Aldstadt and Getis, is worth noting. In this algorithm, zones are defined via the Getis-Ord Gi* statistic, which is a local statistic for identifying clustering, thus zones are defined by local conditions, which are free to vary anistropically across space, rather than by a predefined preference for circles. Spectacularly this algorithm is implemented in the superb clusterpy python module for spatially constrained clustering.

I’ve been doing a bit of work recently on segregation with Pablo Mateos, and having gone through the motions with aspatial indices of segregation (the classics): dissimilarity, exposure and so on, I decided to investigate the more explicitly spatial ones. Taking a lead from Reardon and O’Sullivan’s (2004) paper “Measures of Spatial Segregation” in sociological methodology, I got in touch with David O’Sullivan and he, and his student Seong-Yun Hong, helped me with the implementation of some spatial measures of segregation. This post specifically concerns spatially weighted entropy – a measure of population diversity. Reardon and O’Sullivan define spatially weighted entropy as:

This equation describes the ‘entropy’, derived from Shannon’s information theory, for each grid cell in the image (below) in which each cell value results from the entropy computed for a 1km ‘neighbourhood’ p around each cell (essentially a circular buffer). The ethnic group in question is given by ‘m’ (with the pi representing the proportion of a given group in a given neighbourhood) and relates to ethnic groups defined from the Southwark patient register using Onomap, the groups defined are: African, East Asian and Pacific, European, Muslim, South Asian, British, Eastern European, Hispanic, and Unclassified or Other. The Onomap software is able to apply this classification by looking at the forename and surname combination of patients registered to use Southwark GPs, or patients living in Southwark but using GPs outside of Southwark. The cells in the image relate directly to the residential locations of patients, who were geocoded to their household using the Ordnance Survey’s Address Layer 2 product, therefore empty cells are areas within which no recorded patients were found, such as parks, and transport infrastructure. As the data underlying this is from patient registrations with GPs, we have to accept that the data is likely to be partial, with potentially systematic biases in those people who have registered – young men and people from countries where GPs as a method of primary care do not exist- may have been omitted.

In the image, higher values of entropy indicate greater diversity of population by ethnic group, the resultant images is unsurprising in terms of Southwark, with the Dulwich Village area showing as the least diverse place, home as it is to more affluent, generally ‘British’ groups. Likewise historical factors regarding access to housing have shaped the lower entropy scores in the middle of the borough – home to African populations and the North East, home to the British working classes who were rehoused from the now more African areas in the middle of the borough. Finally, the greater Waterloo- Elephant and Castle region in the north-west shows up as the ethnic melting pot in the borough.

In the image above, the 1km neighbourhood defined in the spatially weighted entropy score has a smoothing effect, I experimented with smaller values for the neighburhood size, and found that the resultant output did not change dramatically from that obtained above. At the end of the day, the selection of neighbourhood size is largely arbitrary and will depend on sociocultural factors of the area and it’s people. Similarly, as there is no data for the regions outside of Southwark we are more uncertain of the values at the edges than in the middle of the borough as we are only sampling from within Southwark itself. Nonetheless, this representation of Southwark goes somewhat beyond what is possible using the commonly used output zones defined by the census.

The government is set to release a bill detailing how it is they expect the proposed GP Consortia to work. GP Consortia, groups of GPs working together, are set to replace the current structure of Primary Care Trusts (PCTs) and Strategic Health Authorities (SHAs) as the mechanism through which primary healthcare is provided to the public, and services are commissioned. Recently, the planned wholesale changes to the NHS have come under a sustained attack from the media, professional bodies and MPs, meanwhile the plans for GP consortia have moved into a trial phase in which different setups are being tested for their effectiveness. The trial consortia demonstrate the extent to which the plans represent a completely new venture, with a broad spectrum of possibilities being tested in terms of consortia templates, from a ‘consortia’ of a mere 3 GP practices, to a vast group of 83 GP practices. There seems little reasoning behind how Consortia are allowed to form at the moment, thus I saw an interesting opportunity to consider the ‘GP Consortia Problem’ as a geographic question. This is most evident in the fact that the NHS is mandated to provide an equitable and universal service, and an unmetered potential for GPs to ‘consort’ may well lead to increasing inequities in healthcare provision.

I see the ‘GP Consortia Problem’ as solvable through a zone-design approach. To do this, I identify contiguity between all English GPs and employ spatially constrained clustering. The following assumptions are made:

• Distance is important, GP consortia should be space covering without holes or islands, therefore a ‘neighbour’ approach to contiguity is advocated using graphs.
• As a preliminary test, GPs are considered to be equal, although there is scope in the future to develop measures of dissimilarity and homegeneity which will provide better, or more appropriate solutions to the GP Consortia problem.
• Based on the trials, I assume that Consortia must consist of at least 35 GPs, the average number of GPs per consortia in the trial phase.

I have used two approaches to creating contiguity amongst the English GP practices, both of them graph theoretical concepts based upon geometric analyses: the delaunay triangulation, and the gabriel graph. I believe that the gabriel graph is a sub graph of the delaunay triangulation, as such it is sparser than the delaunay graph. The two graphs are defined as:

• Delaunay Triangulation – for a set of nodes (GP practices) the delaunay triangulation is the set of triangles created by drawing a circle with 3 nodes (which define the triangles edges) on the circle’s perimeter, in which the circle does not contain any other points- iterated for all sets of 3-points.
• Gabriel Graph- 2 nodes are connected if they form the start and end-point of the diameter of a circle, and the circle does not contain any other points – iterated for all pairs of points.

In this sense, both the Delaunay triangulation and the Gabriel graph are nearest proximity measures. Having obtained the graph, the differences can be seen below. Note both graphs have been constrained for the English boundary.

Having created the ‘contiguity’ graphs, I wrote a short python script to extract the realtionships between GPs and write the output as a ‘.gal’ file for use with pySAL. I utilised the pySAL regionalisation module to compute the consortia solutions, I have used this previously in my blog, so I won’t go into detail on it. I paramterised the solution using the contiguity matrices created, assuming equality amongst GP practices, and looking for groups of at least 35 GPs. The regionalisations were then joined to a special areal geography I created for visualisation, this is simply the Voronoi diagram of the English GPs clipped to the English boundary. The results are below:

In these results it is notable that the Gabriel graph gives a cleaner result, the density of the delaunay-based contiguity matrix means that the result is subject to some sliver-like polygons in the regionalisation, and ‘spikier’ regions in general.

Of course, this is just a test, but it does point at the potential to create a rationalised system fo GP Consortia. Naturally, the biggest issue with these maps is that they only establish an areal depiction of consortia, one that is largely irrelevant. This is because the actual service areas of GPs tend to overlap and extend beyond any given GP’s voronoi defined footprint. Therefore the geography of patients requires a subsequent treatment once a geography of COnsortia has been established, and only in the interaction of the two can issues pertaining to equity be understood.

In Finland, municipalities are incredibly powerful; like local authorities in the UK, municipalities are responsible for local administration, but they also levy an income tax and are responsible for providing most public services. Municipalities were founded on the assumption of equality, which forms the basis for the reform considerations currently ongoing in the Finnish government. The fact is, municipalities simply aren’t equal, they vary wildly in population size and area; numbering 342 they are of rough numerical equivalence to UK local authorities, despite Finland itself having around 1/11th of the UK’s population. The population skew is greatly emphasised by the presence of cities such as Helsinki which cross municipal boundaries, and more remote municipalities whose geographical extent was set out by horse-and-cart distance. It is therefore understandable that the Finnish government would be interested in the possibility of mergers to reduce the number of municipalities, and create a system in which muncipalities serve a similar number, or at least a threshold, population. It is believed that this would make administration more efficient, as services can be centralised to a greater degree, and small municipalities which already share services can formalise this.

Some mergers have already taken place, and there are governmental incentives for merging. However, with the specified model that municalities should be reformed such that they have a base population of 20-30,000 people, from the Association of Finnish Local and Regional Authorities (Kuntaliitto), we can apply automated zone design scenarios to test the ‘what if’ aspect of creating a new Finnish municipal system based on preserving different characteristics.

The zone design tool that I use to test a few basic scenarios is the regionalization library in pySAL, a spatial analysis module for Python. This implements the max-P algorithm for spatially contrained clustering subject to a similarity matrix, and a threshold value. I specified the research design so that I was testing for the optimal new aggregation of the pre-existing muncipalities and tested 3 different scenarios: 1) No similarity measure (all municipalities assumed equal, but for population) 2) Similarity based on municipal tax regime, and 3) Similarity based upon municipal tax regime and % non-finnish speakers, which accounts culturally for the Sami people of Lapland and ethnically Swedish Finns.

The regionalisation requires that you create a contiguity matrix for the zones, I arbitrarily chose the queen case, and added bespoke contiguity for Finnish islands based upon proximity, this is easy to do in GeoDa and it outputs a .gal file which you can read into python. Then all you really need to do is the following:

import pysal
import numpy as np #required as pysal uses numpy arrays

#Read in your population and similarity data in some way,
#I tend to create a python list from a csv.

#convert the population and similarity data into
#numpy arrays, from lists called pop and sim
pop = np.asarray(pop)
sim = np.asarray(sim)

# Read in your precomputed Weights matrix
w = pysal.open("...\\QueenWeights.gal").read()

# Create an (optional) array of 1s to represent equality
# (replace sim in Maxp function call)
nosim = np.ones((342,1))

# Run solutions for maxp algorithms with specific parameters
r= pysal.Maxp(w,sim,floor = 20000,floor_variable = pop,initial=100)

# Write r.regions to outfile to get regional assignments,
#this can be joined to shp in ArcGIS.

The maxP algorithm works by first randomly creating a set of possible zoning configurations, then it chooses the current optimum and seeks to refine it using a computationally expensive zone-swapping method. Optimality in this case is defined by minimising dissimilarity whilst obeying the threshold population constraint. The API reference for regionalisation in pysal is very good.

Here are some of the results I produced for this basic approach:

Zone Design - Pop > 20,000 with assumption of Municipal Equality

Zone Design - Pop > 20,000 with similarity of tax regime and language

Unlike some of the more advanced zone-design algorithms, pySAL doesn’t yet provide a way of preserving or optimising area shape characteristics, so you can get sliver-like polygons forming. Nonetheless it presents an interesting insight and a set of functions from which a more advanced/bespoke algorithm could be built.As it turns out the islands that I arbitrarily allocated to the contiguity matrix are actually quite a contentious topic and given their strategic significance to Sweden are in fact neutral territories which would be untouched by any redesign of municipal structure.

As ever, local knowledge is important, and for the economists at VATT this is a real task to undertake. They won’t be doing anything quite so crude, they do however have a curious spatial problem to deal with.

Python has been more tightly integrated in the new release of ArcGIS 10, allowing scripting to occur directly through a Python process without even opening up ArcMap. Admittedly this was available before, but now everything is more tightly coupled and a lot cleaner in it’s implementation. However, what has really interested, and indeed confused me of late is how to use Python in the ‘field calculator’.

Field Calculator is a really useful tool, when you are looking at an attribute table for a shapefile in ArcGIS and you want to derive a value for each object in the file based on a function you can input the function into the field calculator and it will work it out for you row by row. Sometimes the value you want to derive is a bit more complicated than simple arithmetic and you need to write a script. Previously you could do this in VBA, but I always found it limited and confusing, now however you can do it in Python – much simpler!

There are a few pitfalls to using Python in ArcGIS field calculator, and so I’m going to specify how to write simple field calculator python scripts in ArcGIS from my early experience.

Firstly, for Python in field calculator the way to do it seems to be in write a Python function, and then call it for each row. In addition to this, because you are writing a function you have to give it the relevant parameters (i.e fields) with which to do the computation. Finally, and annoyingly you have to write your function in a little box, and use a consistent indentation standard (1 space works best for reasons of space) as Python requires.

Here is a basic recipe for achieving field calculations in ArcGIS using Python. Obvious this is overly simplistic as you do not need a script to do this calculation, but it serves as an introductory example.

1) Name a function and parameterise it with the fields to base the calculation on. Do this in the lower box.

In the image you can see I’ve input: density( !sum_pop!, !Area!) This means send the values in the fields called sum_pop and Area to the function called density.

2) Define the function you are calling in the larger upper box.

You define a function in python using the “def” command. In the image i have defined the “density” function by writing the line: def density( pop,area):

This function definition means: define a function called density which takes the parameters pop and area. The parameters could be called anything, but it is useful to call them something that makes sense for use in the function. These parameters are variable names that the function uses to identify the fields you have passed the function when you called it, as in 1).

Normally you’d do some sort of calculation within the function, however this example is so simple that all we need to do is “return” a value to the function call. This function is the density defined as population over area: pop/area.

Looking at the field calculator I have found that you are limited to the basic, math and datetime modules in python, without the ability to import other modules. You can however define several functions and call them from within your main function.

For details on the basic syntax of using python, this site is particularly good: http://www.tutorialspoint.com/python/index.htm

Here is a quick confirmation that you can use Python to draw very detailed maps; using the previously specified method I was unable to get python to draw all UK OAs due to their great number (c.220,000) and high complexity (c.50,000,000) vertices. Additionally I was unable to use the generalised OA boundaries for the UK from UKBorders as they contain topological errors that the shapefile reader cannot deal with. ArcGIS is obviously a bit clever in how it handles bad topologies. So I extracted all the vertices and fed them into shapely polygons, and visualised them in the same way, but without reading shapefiles directly into python and was able to output this:

This method has had an impact on the speed of computation as it can take roughly 25 minutes to output this map. The map looks pretty good, aside from a slightly odd polygon in the Bristol channel. Nevertheless, coupled with the operations that shapely, and other geo-libraries, can do this si increasing indication of the maturity of GIS in a variety of platforms. Oh, and it’s all free!

The British National Archives has released a large number of files relating to UFO sightings between 1994 and 2000. These previous classified documents detail (often amusingly or excruciatingly) the reports made by members of the public to the MoD regarding the sighting of Unidentified Flying Objects (UFOs). As a geographer and a user of GIS, one of the overriding beliefs is that there exists an abundance of location information stored in documents waiting to be captured and analysed spatially. The records concerning UFO sightings are one such example, hundreds of reports of sightings that all give a location in addition to a lot of other information including numerous drawings of the ‘craft’ that were seen. Because I was interested in testing out the new “Unlock” geocoding service available though Edina Digimap for academic subscribers I decided to extract some of the place information from the first UFO file and use unlock to geocode the sightings.

For those who need clarification,geocoding refers to the process whereby a textual reference to a location, such as a place name, an address, a postcode etc. is given a spatial reference, i.e. a pair of coordinates that can be represented on a map. What “Unlock” does is take a given placename, look for it in a gazetteer – a dictionary of all known places- and when it finds it, returns the coordinate inforamtion associated with that placename in the gazeteer. The Unlock service can be found at: http://digimap.edina.ac.uk/unlock/

Using this service allowed me to create the following map:

I decided to classify the sightings fairly crudely by season as the impression I got from the data was that there were very few sighting in the summer, and more in the autumn and winter. These sightings are mostly from 1994, part of the earliest tranche of sightings released and are to be found in the first file released by the National Archive. As a result this only accounts for about a quarter of all sightings in this year. The MoD also like to make maps, however their techniques aren’t quite so well defined as we can now achieve with GIS and geocoding technologies:

Nevertheless the map is quite illustrative of the major patterns in UFO reporting (as this may be different from raw sightings).

The only downside to this whole process is that “Unlock” the site I used to geocode some of the data is actually quite poor. The site looks nice and the data is undoubtedly very strong, however it is a terrible user experience, there didn’t seem to be any accessible intructions on how to use the site and when I copied from the examples given to try and geocode my 50 places as a batch the output seemed to randomly drop some places in favour of several options for others, in spite of this individual searches showed that the dropped places did exist. An assessment of the sites usability is required – I can’t imagine it being used currently by anyone beyond real specialists, certainly no one will be unlocking much data unless they have a good familiarity with a number of web protocols and data structures. Hopefully in time though it will become more friendly.