Postcodes

Fascinating debate, if you’re into the power of information agenda, over at Simon’s site on the use of post codes and the availability of the register. I’m told that the licensing of the national postcode register is an ‘old chestnut’. I’m going to spend some time trying to understand just why it can’t be available for free or at marginal cost. Feel free to air your views in the comment section.

29 comments ↓

#1 Greg on 06.29.08 at 7:50 pm

Thanks Tom. I’ve worked with postcodes for years, and it’s a real shame that the UK has one of the best systems for postal zoning, but the worst possible licensing regime.
The Post Office absolutely requires postal codes in order to make its sorting and delivery efficient. What’s more, as it is the originator, noone else can create them. ie. it’s a true natural monopoly with zero marginal cost.
The only reason we have to pay for them is that it suits the Post Office budget to exploit this monopoly – it would cost them nothing to simply upload the postcode file they use for their own purposes, and let businesses, charities and individuals use them to deliver innovative products and services, and better databases (from which, btw, the Post Office benefits).
In terms of your desires – may I plea for free. Free is magnitudes better than low cost for two reasons:
– it allows experimentation (I have an idea, but I need the data to test it. The cost of the data prevents me trying, so the idea never gets tested).
– it avoids transaction costs, legal costs, etc. The Post Office spends a fortune on sales people, administrators and lawyers to protect and exploit its data; and on the flip side potential users spend a fortune working out what they can and can’t do within license. Even low cost products incur these fixed costs; plus the infrastructure for users to monitor their customers’ usage and charge for it. These costs are totally eliminated when data is free.
The USPS allows you to download their ZIP code data in a whole bunch of formats – free – as well as providing it on CD Rom and via APIs. We should be embarrassed when we see how we hamper ourselves.

#2 Tim Bailey on 06.29.08 at 10:57 pm

I would like to add my voice to the ‘block to innovation’ argument about the present licensing costs of the post code database – it is about £5,000 p.a. per server – and we are a Charity… We have paid in the past, but can not justify the cost any longer. Which is a pity, as the post code is is ubiquitous and well understood by users when they are looking to find advice services though one of your databases.

#3 Mitch on 06.30.08 at 10:27 am

Greg,
Not sure what your involvement with postocdes has been. I worked for many years on the Royal Mail Postocde team and would make the following points:

1. The postcode system is designed solely for the collection, sorting and delivery of mail. Postcodes are also used for other purposes e.g. by insurance companies to categorise risk, but they are not designed to be so used and may not be suitable.

2. The maintenance of postocdes is not a small task and is definitely not done at marginal cost. By definition, address information changes regularly and each change has a significant impact to Royal Mail systems and processes. There is also a significant impact on local authorities and local businesses and residents. This an ongoing and expensive process.

3. The postal address file (PAF) is a significant asset to Royal Mail and it’s reasonable for them to exploit it (this government has encouraged Royal Mail to be more independent and commercial). There is a web site which allows anyone to interrogate for free the address file – http://postcode.royalmail.com/portal/rm/postcodefinder?catId=400145. Royal Mail feels this is more than enough for any domestic user. Business customers have to pay for greater access. So what?

4. Currently Royal Mail is ‘owned’ by the government; it’s still a nationalised industry. In that case, I’m not sure why Tom needs to investigate anything. If the government wants the postcode file to be available free or at marginal cost they only have to say the word. They won’t, of course, and Tom knows this.

There are some common misconceptions about postocdes, some of which I’ve tried to address here. The use (and mis-use) of this information is often blamed on Royal Mail but in my opinion they don’t explain themselves enough.

#4 Brian Hughes on 06.30.08 at 10:40 am

Why should it be free? I guess the PO employ loads of people to keep the system going and so it’s not unreasonable for it to charge especially commercial companies such as insurance and marketing ones who gain great value from the things. The argument that the PO has to produce them anyway and thus they should be free is dangerous. You could equally argue that they have to empty my local post box and will almost certainly be delivering other post to the people I send letters to and therefore I shouldn’t have to pay to send my post. Or that the train I take to London would have to run anyway and therefore should be free for me to use…

#5 Robert Mugabe on 06.30.08 at 4:08 pm

You don’t mean ‘marginal cost’ Tom, you utter moron – how did you get elected?

#6 sam on 07.01.08 at 1:23 am

for the purpose of “find my nearest…” databases, it is possible to use free API based geocoders provided by the likes of Google/Yahoo (i.e. free = 50000 lookups a day), it will usually sucessfully resolve a postcode to 1000meters of the street.

This level of accuracy is suitable for this type of application. But overall, its a shame to have to use American companies to access UK data for free.

Maybe the postoffice is clocking on that with all this web 2.0 generation systems coming out nowadays, the concept of post won’t be around forever.

#7 Mr Jabberwock on 07.02.08 at 10:42 am

Bob

Tom my be an utter… as you say and I certainly wonder how he got elected, but he did, I think. mean “marginal cost”.

Marginal cost is the additional cost incurred in the activity, in this case the additional cost of making it available, being the cost over and above the cost they have as a fixed cost of maintaining it for their own purposes.

Right or wrong that is I think what Tom said and meant.

Mr J

#8 paul canning on 07.02.08 at 9:16 pm

Of course I agree ;]

I think the key element in Simon’s argument is ‘usable’.

postcodes are truly ‘customer focussed’. I don’t know if there’s proof but it makes complete b****y sense to me that absolutely every citizen knows theirs.

Therefore a postcode based GIS is *by definition* ‘customer centric’.

Give Simon a medal I say!

#9 Ian Tempest on 07.03.08 at 4:47 pm

In response to Mitch’s defence of the Royal Mail’s charging for postcode data.

My own involvement with postcodes relates to various mashup websites, plotting local information onto Google Maps (or Yahoo Maps, or Microsoft Live Maps). There are several methods for plotting addresses onto these online maps, but they are either inaccurate or long-winded without the postcode coordinates.

1. Mitch states that postcodes may not be suitable for non-mail purposes. I would suggest that, by providing a geographic location accurate to within 20 addresses, postcodes are eminently suitable for all manner of location-specific services.

2. “By definition, address information changes regularly”. Surely by definition, the vast majority of address information does *not* change regularly. I would happily use postcode data that was one or two years old, if only it was free.

3. “There is a web site which allows anyone to interrogate for free the address file… Business customers have to pay for greater access. So what?” So there are users – such as the charity mentioned by Tim above, other not-for-profits, and innumerable web applications – for whom interrogating the website is inadequate or inappropriate, but who cannot afford the very large costs that the Royal Mail attaches to the postcode file. (Some websites use advertising to help cover the costs of their hosting and bandwidth – perhaps this qualifies them as businesses, but the revenues are minimal, and would not cover the cost of the postcode data except in the case of the very largest, most popular websites).

4. I find it encouraging that a Cabinet Office Minister has mentioned this debate at all, if only on a blog. The ‘Show Us a Better Way’ competition announced this week asked “What would you create with public information?”, and “Ever been frustrated that you can’t find out something that ought to be easy to find?” Opening up the postcodes would allow these government datasets to be presented in a clear and obvious manner – NHS Choices ‘Find services’, ONS neighbourhood statistics, London Gazette notices, and others.

There may be misconceptions about postcodes, but what is certain is that each postcode has a corresponding coordinate (latitude and longitude), and that it would be both useful and proper if this data was in the public domain.

#10 Greg on 07.05.08 at 9:20 pm

In response to Mitch.

My involvement with postcode data has been four-fold:
– I’ve run a direct mail data business, providing companies with data gathering, addressing and mailing services
– I’ve assisted charities in campaigning and targeting precious funds at recruiting potential donors through the post
– My parents run a micro-business which dispatches orders by post
– Both through work and independently, I’ve prepared various mashups

To your points:
1. Postcodes are designed solely for mail-related purposes:
– a: If I wish to mail anything, I should use a postcode. Note that if I’m a business or charity, I’ll incur costs to get hold of the postcodes. Conversely, doing so actually saves the Royal Mail money (yes Mitch, I’m aware of Mailsort, etc for bulk mailers). ie. the Royal Mail charges me to use its monopoly data in order to save itself money in sorting and delivery. That does seem a little perverse.
– b: The data has some extremely useful purposes beyond that; for example, a local authority may use postcodes to group addresses for rubbish collection. If someone decides the data is fit for a purpose; great.

2. The maintenance may not be a small task – but its something the Royal Mail has to do anyway. ie. I’m not asking the Royal Mail to spend an extra penny to meet wider needs. I’m merely asking that once they’ve built the data to meet their own needs, they release it – complete with any disclaimers they feel necessary – so that others can use it. The mere act of not making it free means that the Royal Mail spends a lot of money on licensing, protection, enforcement, ‘value adding’ (in quotes because the Royal Mail is rarely the best organisation to add value to the data), selling, etc.

3. The issue with the Royal Mail treating PAF as an asset is that it’s necessarily a monopoly production; if other people could produce competing PAFs then perhaps charging for it would be acceptable; but of course, postcodes only work because there’s one definitive set. The payment for greater access by business is not acceptable for three reasons:
a. It favours large companies enormously – the fixed costs of setting up PAF access are prohibitive to small companies and start ups. You’ll find many large businesses have websites which autocomplete your address from its postcode. Most smaller companies’ websites don’t. This is why. It’s an instance of the ways in which the UK’s bureaucracy makes it harder for small businesses to start and grow than is the case in the US./
b. The incurred costs can be very high. We had an instance of a client using postcode lookups, paying on a ‘per lookup’ basis. A small bug in the code meant that each lookup was being done multiple times; the mere act of looking up postcodes cost c. £1 per lookup, and the total cost was many thousands of pounds before it was fixed. Of course, bugfree code would have avoided that, but bugfree code isn’t cheap either. Some sort of incidental costs always occur when data is not free – and the government’s job should be to help reduce the cost of doing business, not increase it. After all, we’re all customers or employees of businesses.
c. It’s not just businesses – charities, local authorities and government departments spend a fortune on this stuff. Of course, it’s just taxpayers money going round in circles – but the admin is not cheap.

4. Of course, you’re right. The government could choose to free the data – but it’s a useful stealth tax, which generates visible income rather than the much less visible but far greater benefits that freeing the data would provide. Tom’s great for sticking his neck out and looking into this.

The points you make, Mitch, are unfortunately so reminisecent of the innovation-stifling opinions of inward-facing bureaucrats which have been such a major contributor to Britain’s loss of economic advantage over the years. Examples which are now so clear include the fact that we invented public-key encryption long before the US, but kept it a government secret rather than using it to gain an edge in commerce; or that Frank Whittle invented the jet engine only to find that closed-minded bureaucrats couldn’t see it working. Bureaucrats are rarely the best people to judge whether something has a place in propelling innovation and competitiveness. The fact that there’s so much energy on my side of the postcodes debate says it all.

Mitch; you should be proud that you worked on a world-leading data source. It’s just such a shame its wings are crippled by its owners.

#11 Free Our Data: the blog » Blog Archive » The postcode debate, summed up beautifully on Tom Watson’s blog on 07.07.08 at 10:40 pm

[...] He mused, “I’m going to spend some time trying to understand just why [PAF] can’t be available for free or at marginal cost. Feel free to air your views in the comment section.” [...]

#12 Mo on 07.07.08 at 11:49 pm

It’s been a fair while since I’ve commented here (hi Tom!), but I should note that I’m a web developer by trade, and I work primarily on e-commerce sites. Thus, I have a vested interest in this.

The PAF contains a lot of data… a lot of that data is useless to many people, but one particular aspect of it is very very useful to anybody who collects British addresses online: a complete address including a postcode. With that, forms can be made massively easier and less error-prone, which reduces the number of rejected transactions, which helps your business run smoothly. Bottom line: people are rubbish at entering their own addresses into sites.

So, I thought, there’s been success in collaborative projects in the past; why don’t I just ask people to anonymously submit their complete postal addresses to me? I don’t care about who they are (unless they’re a business, perhaps), but being able to have a database of all of the addresses with the postcode W12 7RJ which can be queried by anybody is immensely powerful and useful.

Except I can’t do that, because even though I’d be collecting people’s “own” addresses from them, I’d be breaching the PAF copyright.

(Of course, it would also take quite a long time before it was useful). Personally, and from speaking to others in the same industry as me, we could largely care less about the additional data in the PAF, because it’s entirely specific to Royal Mail—the Post Code, on the other hand, is ubiquitous… and that, to me, tells me that it cannot be commercially-held in a modern society which wants to support business growth and development.

#13 Rob Dunfey on 07.08.08 at 10:56 am

Far more useful would be access the the National Land and Property Gazetteer (NLPG) currently under the ownership of the Improvement and Development Agency (idea.gov.uk) and is compiled from contributions by local councils. This provides definitive addresses for the whole of the UK, even addressing addresses that don’t have a postal address.

One benefit to this would be a massive saving of money for the Ordnance Survey who attempt to maintain a similar product.

Rob

#14 Richard Fairhurst on 07.08.08 at 1:04 pm

http://www.npemap.org.uk is your friend for an imperfect, but very, very usable free postcode database.

#15 Gerv on 07.08.08 at 6:36 pm

Tom: you ask “why it can’t be available for free or at marginal cost”. (The disadvantages of “marginal cost” have been well-described above by Greg, so for simplicity I’ll just use “free”.)

The only argument for not making it available for free is that it would reduce the income of the Royal Mail. This is possibly true, although it’s hard to quantify what effect freeing the data would have on use of the postal service. It certainly wouldn’t be negative.

However, we shouldn’t look at income, but profit. What greater percentage of sent items would have accurate postcodes on them (which reduces RM’s costs) if the data were freely available? Again, hard to quantify, but a point worth considering.

But even above that, particularly as RM is owned by the government, we should not be looking at just the RM’s profits, but everyone else’s, and the general social good. Freeing the data reduces the costs of every business which uses it, and increases the capabilities of every business which currently isn’t using it because it’s too expensive but could if it were free. And that’s before you add in the community groups, charities, and so on.

The UK is, unlike many countries, blessed with a comprehensive country-wide, well-used system for locating places down to a couple of hundred yards. It should be as widely used as possible.

#16 Denis Payne on 07.09.08 at 7:53 am

NLPG should be included in this debate – Councils maintain it, and since it drives electoral registration (from later this year) and Council Tax collection, they’re well motivated to get it complete. Its therefore likely to have more addresses than PAF.

However, it includes Postcodes, and RM therefore want their fees from anyone who wants to use it, and Ordnance Survey, who are producing a competing product “Address Layer 2″ are also muddying the waters.

So, we have at least 3 publicly funded bodies (RM, OS, Local Government) producing address databases, exchanging money between themselves (as well as paying for sales, marketing, legal etc etc), and UK plc not being able to take full advantage of what’s there.

Just how did this all happen? Can someone please call STOP to this madness?

#17 More on Latent Communities for Government Websites : Tree of Knowledge on 07.09.08 at 8:19 am

[...] Tom Watson MP asked on his blog: “I’m told that the licensing of the national postcode register is an ‘old chestnut’. I’m going to spend some time trying to understand just why [PAF](TW – essentially post code data) can’t be available for free or at marginal cost. Feel free to air your views in the comment section.” [...]

#18 steve on 07.09.08 at 4:52 pm

Denis is correct. The selling of address information between organisations within this country is madness!

Local Authorities are charged for the use of the postcode in the NLPG – They are contractually obliged to purchase Ordnance Survey’s Addresspoint (PAF geocoded). The vast majority do not want this information, not do they use it. They obtain postcodes directly from Royal Mail when they inform them of the addresses for new properties.

PAF is maintained largely (although indirectly) by local authorities through the submission of street naming and numbering information. This is at zero cost to Royal Mail.

Local Authorities (through the IDe&A) have been trying to provide a national electronic feed of this information to Royal Mail (and others), which could be exploited to remove a large amount of the cost in maintaining PAF. This idea is being strongly rejected by Royal Mail… why? Because it will mean that Royal Mail will no longer be able to claim the IPR in addresses.

Ordnance Survey argues that as a trading fund they are required to profit from selling data (including Addresspoint and Address Layer 2). The problem is that around 60% of the revenue generated by Ordnance Survey comes from public funded organisations. The other 40% is probably equal to the costs involved in selling the data.

It is similar with postcode data. Royal Mail spend huge amounts of money maintaining PAF, but much of this work is duplicated within Local Authorities, and much more is due to the inefficient way that Royal Mail’s address development centre works.

Why should one public funded organisation, have to buy data from another public funded organisation? Especially as LA’s provides data essential in the allocation of postcodes to Royal Mail free of charge! The only group that should be able to claim ownership of postcodes or addresses is the public, who to a large extent, have paid for the information to be collected in the first place!

The solution is so simple – Address information (including postcodes) is made free to all.

#19 The Royal Mail PAF, that old Chestnut… on 07.10.08 at 12:03 pm

[...] Tom Watson…MP for West Bromwich East, the UK’s first Parliamentary blogger, and critically, the cabinet office minister responsible for delivering on the Power Of Information report, recently noted interest in the issues surrounding access to the Royal Mail Postcode Address File (PAF); Tom was promptly informed it was an ‘old chestnut’, what on Earth could that mean? [...]

#20 Rob Dunfey on 07.10.08 at 12:13 pm

Tom, Dennis, Steve,

I’ve elaborated on the NLPG and the ‘Postcode Chestnut’ with a blog post; I’d be really keen to hear any comment, especially the outcome of your research into the issue Tom.

All the best,

Rob

#21 Greg on 07.12.08 at 2:35 pm

This debate really is getting some traction – the blog posts at Free Our Data and gisconsultancy.com are taking it further; and Tim Watts’ take at TreeOfKnowledge moves it into the general area of allowing Ministers to get insights from interested or knowledgeable parties for free.
Reading through here and the external posts, I’ve learnt more about the way public authorities are tied in knots with each other, passing around data, money and ridculously complex licensing arrangements.
I wonder, Tom, if the last time the Treasury looked at this they looked at the total system cost of PAF licensing within the public sector alone? That is to say, my view is that the cost to the economy of PAF licensing far exceeds the Royal Mail’s revenue. However, the Treasury will typically look at the narrow balance sheet for the Royal Mail (ie. if the RM receives £500m a year for licensing PAF, they want to keep that public sector income, however much more money could be made for UK PLC if it were opened up). Perhaps, though, the picture might be more conducive if the Treasury looked at the following calculation:
Revenue from PAF licensing
minus Amount paid by public sector organisations for access to PAF
minus Costs incurred by public sector organisations in legal and transactional costs
minus Opportunity costs incurred by public sector in not using PAF data as widely as they would if it were free
minus Amount spent by public sector on other sources of postcode data (QAS, Experian, etc)
minus Costs to RM in preparing, maintaining, marketing and negotiating licenses
May be time for a FOI request.

#22 ARDN on 07.14.08 at 4:11 pm

The PAF is PSI and covered by SI 1515 of 5 July 2005 which transposed the EU Directive on Re-use of Public Sector Information, 2003.

The legislation allows charging for making PSI available for re-use provided that the price is the same for all re-users and is no greater than marginal cost. The cost of PAF for RM’s public task – roughly equivalent to its statutory duty to deliver a univesal postal service – must be incurred and covers the maintenance of the file.

The marginal cost of re-use of the PAF, given that it has to be complete, accurate and up to date to support the public task, is almost zero. If anybody in the public or private sector gets it for free, then all re-users are entitled to get it free. If it is already on line, the marginal cost of searching and retrieving all, part or individual items increases with the server power needed to cope with the demand. If it is distributed on CD, there are costs for the media, postage and handling: say a few pounds.

The counter-argument to this analysis goes as follows:

#23 ARDN on 07.14.08 at 4:36 pm

The counter-argument to this analysis goes as follows:

RM is not a public body within the meaning of the Directive.

Even if it is, its public task includes selling the PAF to maximise its revenues or profits.

It is not obliged to increase its costs to lower the marginal costs for re-users.

The re-use regulations allow it to withdraw PAF as a service to outside organistions completely, leaving customers to find postcodes some other way as they do for e-mail and fax numbers, for example. (There are independent businesses which publish directories.)

ARDN signing off.

#24 Grant on 07.21.08 at 12:37 pm

Given that Local Authorities are statutorily the creators of a house name/number and street name and that Royal Mail (RM) simply add the postcode to speed up their sorting, a valid address actually consists of the name/number, street and town. Note that this does not include the post-town, which often does not match the geographic town and can be very confusing.

RM have done a good job of selling the postcode as a requirement for your “official” address, assisted by companies which refuse anything without a postcode as they use PAF to determine address validity. However RM will deliver mail without a postcode (it just takes longer and presumably costs them more).

Imagine then if enough people refused to use postcodes on mail and let RM pick up the cost of the extra sorting. I think this would quickly eat into the revenues from the PAF file and make the free release of the data more attractive.

#25 Free Our Data: the blog » Blog Archive » Want the Postcode Address File for free? Just ask on 07.21.08 at 11:04 pm

[...] Obviously, we would not condone using it in ways that breach the Royal Mail licence. We’re aiming to do this legally. But it’s definitely another success for the Power of Information taskforce and Tom Watson in the Cabinet Office. He said he’d have a go on June 29th; now he’s achieved it. Three weeks? For government and licensing regimes, that’s fast. The following posts may be related…(the database guesses): In today’s Guardian: who will address the postcode mess? (13 July 2006; score: 67.83%)Will the Post Office and Ordnance Survey ever agree about house names? (10 July 2006; score: 54.06%)Postcomm makes recommendations on future of Postcode Address File: it should make a profit (19 April 2007; score: 53.3%)Revealed: how profitable Royal Mail’s Postcode Address File (PAF) really is (26 April 2007; score: 50.18%)Paying twice for data? Through your council, you might be paying EIGHT times (21 March 2006; score: 33.52%) [...]

#26 Dave on 07.25.08 at 12:42 pm

Emergency Services also use the PAF data and currently in Wales PAF overwrite supplied primary Welsh addresses with English which causes no end of problems. Free the data please TOM

#27 Fran Knight on 10.16.08 at 2:25 pm

Hi When postcodes were first brought out I and others were led to believe you would then only need a name a house number and the postcode to send mail, it has never been like that. Why not?
Fran

#28 Going Hyper-Local - Location Based Internet | WOWNDADI on 11.26.08 at 5:34 pm

[...] space is not without its issues (see this post about post code data), with access to data and privacy being major converns. However, the next few [...]

#29 Michael J Brook on 10.26.09 at 10:36 am

Our local area post code HD2 is according to the insurance companies and reflected in our insurance premiums, – in the top 20 list of worst post codes for crime in the UK – and I would not argue this. Yet on its perimeter HD2 also contains the Birkby-Edgerton Conservation Area with some of the best surviving examples of Victorian/Edwardian/Art deco and Art Nouveau properties in the UK. As this also adjoins HD1 and HD3 postal areas, it should not be difficult for the Conservation area to be ‘exchanged’ into another adjoining area, particularly as post codes are not used to assist local delivery of mail. So who do I contact to solicit some negotiation? Gordon Brown/ Sir Humphrey Appleby ? A chap called Crozier who with help from the unions is Beeching-like intent on closing RM down. Actually it has been effectively ‘down’ for many years in certain parts of London.

Leave a Comment

Protected with IP Blacklist CloudIP Blacklist Cloud