I found a nice little time saving device when testing different numbers of nearest neighbour weights in the excellent PySAL library in python, so I thought I’d share it.

Basically I wanted to test a number of different values of k when choosing a weighting scheme for spatial smoothing using nearest neighbours, but I didn’t want to have to continually recalculate the weight’s matrix for different values of k. Here’s what I did:

1. Calculate a k nearest neighbour vector for a high value of k, this should be the same size, or larger than what you anticipate as being your maximum value of k.
2. Store this table in a database, the database is useful as for large values of k for a large set of data you create k x n rows. Databases are optimised to store and query this amount of data in a way that text files aren’t!
3. Create the weights matrix in PySAL from first principles: grab all the data from the database and order it by the ‘from’ neighbour id, and then the weights of the ‘to’ neighbours, Create the weights matrix as specified in the PySAL docs on weighting, but only use as many observations as you want to test by slicing the input matrixes.

Here is my code showing how this works:

import _mysql    #Library that connects to Mysql
from pysal import W    #Weights part of pysal

# Important - Database connection!
db = _mysql.connect(host="localhost",user="root",passwd="",db="data")

# Now Create a Spatial Weights Object

# These first 4 lines pull in the weights data from my MySQL database
# and store it as a list of tuples called 'resultWeight'
getWeights = "select * from `spatialweight` order by `polyID`,`weight`"
db.query(getWeights)
r = db.store_result()
resultWeight = r.fetch_row(maxrows=0)

nList = []    # Empty list for neighbours
wList = []    # Empty list for weights
neighbours = {}    # Empty dictionary to hold neighbours
weights = {}    # Empty dictionary to hold weights

# Now iterate through the results to form dictionaries with lists of
# neighbours and weights for each relevant point in the dataset

recNum = 1
while recNum < (len(resultWeight)+1):
    # append data from resultWeights until the limit
    # for k is reached for each point
    nList.append(int(resultWeight[recNum-1][1]))
    wList.append(float(resultWeight[recNum-1][2]))

    if recNum % 50 == 0:    # 50 as I used a maximum value of k = 50
        # Slice nList and wList depending on the value of k to test
        neighbours[int(resultWeight[recNum-1][0])] = nList[0:20]
        weights[int(resultWeight[recNum-1][0])] = wList[0:20]
        # Reset nList and wList for the next point in the weights data.
        nList = []
        wList = []
    recNum += 1

# Now simply create the weights matrix for the value of k specified.
w = W(neighbours,weights)

#the order the matrix for use later.
if not w.id_order_set:
    w.id_order = range(1,n)
# Where n = number of observations in the dataset (assuming
# point IDs are sequential integers starting at 1.)

I’ve spent a chunk of time recently address geocoding the Southwark PCT patient register to Ordnance Survey Address Layer 2 data. What this means is that I can start identifying and (later) classifying households, this will allow me to ask questions about how different households approach healthcare. More broadly it allows me an insight into the demographic character of Southwark.

The data actually extends past the Southwark boundary as people in Lambeth, Lewisham, Bromley and Croyden do also to some extent use Southwark primary healthcare services (GPs) this means that although Southwark’s population is only c.300,000 the datset I’m using is for just over 340,000 people. There is some uncertainty in the data naturally, this results from the two datasets used; on the one hand addresses recorded in the Southwark patient register are not all necessarily complete, for example there is sometimes a failure to record which particular subdivision of a house someone lives in, or which flat in a larger block of social housing. On the other hand the AddressLayer2 data, although very rich, is not necessarily complete, this could be due to the prescence of unacknowledged subdivisions in residential housing, and although most social housing estates seem well documented, some commercial developments are not necessarily registered beyond the building level. Similarly, there are a number of instances of social institutions, such as the Salvation Army and St. Mungos, or marinas and dormitories having a single registered address for a high number of residents. This may have the effect of skewing the data slightly. With this in mind I created the following graph from the dataset of Number of households against number of inhabitants per household:

This shows that there is still a major trend for single-person households, but equally that around a quarter of all households are co-habited. The long tail in the graph (which i have truncated here) is caused by a few special cases, some examples of which are acknowledged in the previous paragraph. The average household size of 3.10 is itself higher than the UK average household sizes reported after the 2001 census which was 2.36; at the time the borough of Newham in East London had the highest household occupancy rate at 2.64. Of course there are any number of reasons why these data are not comparable, to start with the census took place 8 years before the Southwark dataset was created, similarly the uncertainty in the Southwark dataset is higher as it was not created with the primary purpose that it be able to successfully locate all patients as more often than not patients go to the Doctor and not vice-versa, whereas the census is distributed at a household level to each individual. The Southwark dataset does also include particularly tranisient communities which are missed by the census, such as the homeless who don’t have a fixed address (and hence may be using shelter or hostel addresses) but still require medical treatment at times.

Nevertheless, an interesting first look. The next steps will involve evaluating and validating the dataset to the best of my ability and then moving on to look at ways of examining and classifying household structure.

I’ve spent some time recently working towards a situation in which the whole dataset for patients registered to General Practices in the London Borough of Southwark is coded to address level. Previously I had been working with the data at postcode level and I wanted to start investigating the effects of households on uptake of service, and well as profiling patients at a finer granularity and integrating geographically more sensitive analyses. The geocoding project obeyed the general rules set out for this kind of work; it was reasonably easy, in the end, to address match 92% of the data by scripting, somewhat frustrating to push that total up to 99% (through semi-automated methods of address matching) and all but impossible to match the last 0.5% of patients.

This last group, roughly equivilant to 1500 people who have given addresses which i cannot, even manually, match. This tends to be because, perhaps unwittingly, the postcode doesn’t exist, there is too much uncertainty meaning it could relate to several possible places or the house or the road simply does not exist. In some cases it was easy to clean up the data, for instance it became clear that in a number of cases the addresses actually related to boats moored in South Dock Marina, London’s largest marina. Obviously people that live on boats still need health care, but do not have an address as such, in this case I registered boats to the Dock Office. Similar issues occured with students registered as living in one of Southwark’s numerous student residences, the student’s transient nature meant that there were numerous different ways of recording their residences. In a similar vein it was interesting to deal with the fairly substantial group of people who were either registered as NFA (no fixed abode) and to the GP surgery’s postcode, or to one of several shelters or missions such as the Salvation Army or St. Mungos. This aspect of the data gives an insight that is otherwise quite hard to get at, naturally homeless people require health care from time to time, and it order to receive it they need to go into the system in some way, the fixed address structure of registration means these people occur as somewhat anomolous results within the database. This has the potential to give an insight into the homeless situation in Southwark. Finally there seemed to be some trouble matching patietns that were registered as living in care homes, again these were easy to address match, it was simply that the address information itself had been misreported, or simply read the name of the particular care home in question.

Having gone through the unmatched patients and weeded out cases such as those above that were valid patients, but who didn’t neatly fit into a database with an address-based structure I was left with what appeared to be whole sets of estates that were completely unmatched. I ran a series of wildcard searches on the AddressLayer2 database I have set up in order to try and find these estates, but kept returning empty sets of results. One of the estates that I couldn’t match was the “Sumner Estate”, this rang a bell as I used to live in Peckham and cycled through this estate everyday on the way to LSE, I vaguely remembered reading about its scheduled demolition in The Economist in about 2006-2007. I did a quick google search and found that it was in fact part of the Aylesbury Regeneration scheme, a £2.5bn regeneration by Southwark Council that aimed to clear and rebuild some of Southwark’s worst and most notorious social housing estates. This estate was bad from the beginning and in fact lasted fewer than 50 years, with the most recent 20 being acknowledged as in a state of critical decay.

Aylesbury Estate. Source: http://www.flickr.com/photos/se9

I conducted a number of further searches on google of the following places: Wood Dene; Alison House; Marchant House; Yeoman House; Saul House; Sharpness House; Rainswick Court; Lambourne House; Silwood Estate; Kingshill; Dobson House; Dufrey House; Ayton House; Habington House; Hordle Promenade South and North; North Peckham Estate. I found that all of these houses or estates had been demolished at some point in the mid 2000s. This accounted for around 600-700 patients in my dataset, the larger issue here is data uncertainty: if there exists people in the dataset that don’t actually exist in reality then we have an issue. Having said that, the 600 people that I uncovered as having a defunct registered address only accounts for 0.17% of the dataset, so maybe it’s not too bad. What I actually wanted to focus on here is the hidden nature of these regenerated places.

In conducting internet-based searches for information on the various housing estates listed above I found a very dark picture. To start with inforamtion is very scarce, there is little record on Southwark Council website express regarding regeneration and which blocks were torn down. Some information came from copies of local papers and bulletins. Sadly a great deal was also associated with news media that was reporting the regeneration of an estate as an aside to far graver news, most notably the murder of Damilola Taylor on the North Peckham Estate. Indeed several estates were conspicious in their absence from any online resource or comment other than court documents acknowledging that a defendant heralded from such an estate. In redeveloping large estates, whole roads were removed, the aforementioned Hordle Promenade North and South, as well as Clanfield Way or Walkford Way. The legacy these roads leave however is quite interesting, Hordle Promenade North is a Google maps POI despite no longer existing. Similarly the postcode for Clanfield Way – SE15 6EW remains a poi, allocated to a different stretch of road now, as well as SE16 6EY former postcode for the now demolished Walkford Way. Occasionally planning documents deal with house and estate clearing in a very matter of fact way. There is almost not voice online for any of the inhabitants of these places.

It is easy to view the city as a static entity, it changes so slowly compared to the pace of life, and yet when changes do occur they are easily assimilated into our internal map, as if a change never occured. However, these estates still linger, as hidden reminders of the palimpsestic nature of the city – slums torn down and regenerated, deprivation papered over, tragic events of the past lapsing into memory, slowly forgotten as the city turns over and adjusts its morphology.

A little while ago I created some basic measures of spatial equity for my main study site in Southwark, London. Spatial equity in this case relates to a measure of the ‘fairness’ of spatial distribution of services. The NHS as a public institution has a requirement in its universal terms of service to provide a fair service to all.

The following maps aim to show how different areas in Southwark, in this case output areas (OAs), have different characterisitics in terms of: the level of primary care provision available, and the distance to centres of primary healthcare. Following Truelove (1993), Talen and Anselin (1998) and Ricketts et al (1994) the first 3 maps use a buffer-approach to spatial equity, whilst the final shows a gravity model approach.

Figure 1: Spatial equity measured with a 500m buffer around GPs

This first map (figure 1) demonstrates that large parts of Southwark do not have access to healthcare services within 500 metres (euclidian distance), whereas the best served areas have access to more than one GP surgery and as many as 24 individual doctors.

Figure 2: Spatial equity measured with a 750m buffer around GPs

Figure two demonstrates that with a 750m buffer most areas are served, although there are still unserved areas, particularly in the south of the borough. The most well-served areas not have access to as many as 48 doctors.

Figure 3: Spatial equity measured with a 1000m buffer around GPs

A 1km buffer still shows areas of Southwark which are unserved, particularly in the south. My recent working paper features a map which confirms that residents of these areas are less likely to use Southwark services than those in the more core areas in the centre of the borough.

Figure 4: Spatial Equity measured by Log of the Gravity Potential

This final map uses a distance decay function rather than a buffer to represent spatial equity and is specified thusly (Talen and Anselin, 1998 p.600):

where Sj is the size of a facility (measured by number of doctors, operating capacity etc.) at location j and d is a distance decay factor between area i and facility j with a friction parameter alpha, here set to 2.

The result is not hugely different to the buffered approaches, giving a similar account of affairs. It is notable that in all cases the spatial equity correlates with provision of social housing. In the UK context Southwark is a special case, being amongst the most deprived Local authorities by IMD07 rank, in which fair access to services is skewed towards the needs of the more deprived, whether or not uptake, or ability to uptake actually reflects this is another question.

References

]]> http://danieljlewis.org/2009/12/08/basic-equity-maps-for-southwark/feed/ 0 http://danieljlewis.org/2009/12/04/casa-working-paper-150-now-available/ http://danieljlewis.org/2009/12/04/casa-working-paper-150-now-available/#comments Fri, 04 Dec 2009 12:58:27 +0000 Daniel Lewis http://danieljlewis.org/?p=81

My first CASA working paper is now available. It is the result of a large amount of work I did for my upgrade from MPhil to PhD study at UCL. The topic is “Choice and the Composition of General Practice Patient Registers“. The abstract is as follows:

Choice of general practice (GP) in the National Health Service (NHS), the UKs universal healthcare service, is a core element in the current trajectory of NHS policy. This paper uses an accessibilitybased approach to investigate the pattern of patient choice that exists for GPs in the London Borough of Southwark. Using a spatial model of GP accessibility it is shown that particular population groups make non-accessibility based decisions when choosing a GP. These patterns are assessed by considering differences in the composition of GP patient registers between the current patient register, and a modelled patient register configured for optimal access to GPs. The patient population is classified in two ways for the purpose of this analysis: by geodemographic group, and by ethnicity. The paper considers choice in healthcare for intra-urban areas, focusing on the role of accessibility and equity.

The paper is accessible here.

Curiously, about a week after I posted on the limitations of the GP choice element of NHS Choices, the services has been upgraded. Today users were met with the following box:

This is clearly a step forward in that it allows a basic, albeit subjective, assessment of quality for patients. It remains to be seen whether the uptake of ratings will be as successful as they have been for hospitals. Bear in mind that there are roughly 10158 GP surgeries in the UK, clearly a large number of reviews are needed for it to be a viable resource.

Coupled with this, the presentation of search results for GPs has taken a step forward. As the next image demonstrates:

The new layout of search results for GPs brings NHS choices much more in line with its Hospital search. It is notable that NHS choices is now pushing the choices that people can make regarding the type of GP they want to use. The tick boxes at the top allows for a number of options geared towards choice of GP including available clinics and surgery accessibility requirements, as well as opening times. Further the initial dialogue in each returned search result offers more, by way of number and sex of doctors, languages spoken, and the potential for patient reviews. The other import from the hospitals search functionality is the ability to shortlist and compare services.

What hasn’t changed however are the methods of ordering GPs, this is still fundamentally a distance based list, although it is now more modifiable. The only other option is to list GPs alphabetically, because you might want to use a GP whose name is higher in the alphabet?! Coupled with this, there are still overtones of locality and community in the way services are distributed, as well as references to defacto catchment areas. Getting more information is also tricky, I decided to learn more about the “number of GPs” from the link on the right hand side and was confronted with the following information:

This does not actually tell me anything that I didn’t already know from the presence of the fact itself. In fact it seems markedly rushed and incomplete. Unfortunately, it would seem that beyond this new shiny facade, much of the information hasn’t changed. I still can’t find out anything new when I drill down, it is simply that the frontend has been improved. The sections that hint at GP quality are still absent or not very useful, and there are links to the NHS Information Centre for Quality and Outcomes Framework (QoF) data which is at best confusing and intimidating.

Nevertheless, I may revisit the patient reviews as a data source in the future and investigate how good a source of data they may be. They may add value to a mash-up of primary care services, or contribute to a discussion of neo-geography.

I am growing very interested in how the NHS actually facilitate choice within the primary care system. This post investigates the apparent asymmetry between choosing a hospital and choosing a GP using NHS Choices, the NHS’s online health portal. Figure 1 is a screen capture of the results list presented when searching for a GP, figure 2 is the equivalent results for a hospital search.

The GP search creates a basic proximity ranking of the potential GPs available from an individual’s postcode, as per the example of a GP search shown in Figure 1. Search results cannot be manipulated in any other ways, meaning that non-accessibility based criteria which prospective patients might wish to fulfil cannot be met. Options related to choice are not-present at the start (these could include patient list size, number of doctors, services offered etc), although some of this information is available when you ‘drill-down’ by selecting a specific GP.

When searching for hospitals it is possible to search not only by postcode but also by specialty. When search results are presented they are given with an indicator of “Quality of Service” from the NHS Annual Health Check, which measures Hospital performance. Further, the results also include the standardised mortality ratio, and some wiki-style reviews of the hospital. All this information exists for Hospitals without drilling down, and when you do there are further measures to help patients choose a suitable hospital.

GPs, unlike hospitals, have very limited quality criteria confined to opening and closing times. The NHS seems to limit the possibility of defining a GP as good or bad through use of waiting times, understandable prevalence data and such. This attitude in particular seems contra to the choice agenda being pushed. However GPs (surgeries) are a partnership between the public NHS trust and the private General Practioner(s) (GPs – doctors) that run them so such a black and white classification may be difficult for the NHS to achieve politically, despite the likely patient benefit.

Figure 1: NHS Choices, GP search results for a postcode in North London (http://www.nhs.uk)

Figure 2: NHS Choices, Hospital search results for a postcode in North London (http://www.nhs.uk)

Recently I’ve been doing some work involving the transportation problem. The Transportation Problem is an allocation optimisation problem that requires the optimal assignment of demand, in my case patients by Output Area, to known, fixed, supply points, in my case doctors surgeries (General Practices). Rather than use a euclidian or manhattan metric to model distance from the demand site to the supply site I have used public transport travel times from TfL.

Initially this seemed a difficult task, and early attempts only provided partial, or non-optimal, solutions. However, once I had found the linear programming functionality in R through the package LPSolve it became very easy to create a model with the constraints I wanted and get a solution very quickly. Key to the success of the R package was the ability to set the constraints I needed, crucially integer constraints so that people were not subdivided, and constraints on the number of patients doctors could take.

Mapping the outcomes in ArcGIS was straightforward due to R’s built in csv-export funtionality.

Here is an example of the output.

The legend denotes the 44 physical practices in the London borough of Southwark, some doctors exist on the same site and so these practices were agglomerated. The grey areas represent unallocated demand caused by capping the size of the General Practices. In another definition of the model I ran I set the GPs up as uncapacitated so that all the demand would be satisfied. This model uses data from 2006, I have data from 2009 for which I will also run the model.

Some earlier work I did on this is available in the Proceedings of GISRUK ’09.