Experimenting on yourself

August 29, 2014

A recent post for the What Works Centre that I thought would be good here too.


At the What Works Centre we’re keen on experiments. As we explain here, when it comes to impact evaluation, experimental and ‘quasi-experimental’ techniques generally stand the best chance of identifying the causal effect of a policy.

Researchers are also keen to experiment on themselves (or their colleagues). Here’s a great example from the Journal of Economic Perspectives, where the editors have conducted a randomised control trial on the academics who peer-review journal submissions.

Journal editors rely on these anonymous referees, who give their time for free, knowing that others will do the same when they submit their own papers. (For younger academics, being chosen to review papers for a top journal also looks good on your CV.)

Of course, this social contract sometimes breaks down. Reviewers are often late or drop out late in the process, but anonymity means that such bad behaviour rarely leaks out. To deal with this, some journals have started paying reviewers. But is that the most effective solution? To find out, Raj Chetty and colleagues conducted a field experiment on 1,500 reviewers at the Journal of Public Economics (where Chetty is an editor). Here’s the abstract:

We evaluate policies to increase prosocial behavior using a field experiment with 1,500 referees at the Journal of Public Economics. We randomly assign referees to four groups: a control group with a six-week deadline to submit a referee report; a group with a four-week deadline; a cash incentive group rewarded with $100 for meeting the four-week deadline; and a social incentive group in which referees were told that their turnaround times would be publicly posted. We obtain four sets of results.

First, shorter deadlines reduce the time referees take to submit reports substantially. Second, cash incentives significantly improve speed, especially in the week before the deadline. Cash payments do not crowd out intrinsic motivation: after the cash treatment ends, referees who received cash incentives are no slower than those in the four-week deadline group. Third, social incentives have smaller but significant effects on review times and are especially effective among tenured professors, who are less sensitive to deadlines and cash incentives. Fourth, all the treatments have little or no effect on rates of agreement to review, quality of reports, or review times at other journals. We conclude that small changes in journals’ policies could substantially expedite peer review at little cost. More generally, price incentives, nudges, and social pressure are effective and complementary methods of increasing pro-social behavior.


What can we take from this?

First, academics respond well to cash incentives. No surprise there, especially as these referees are all economists.

Second, academics respond well to tight deadlines – this may surprise you. One explanation is that many academics overload themselves and find it hard to prioritise. For such an overworked individual, tightening the deadline may do the prioritisation for them.

Third, the threat of public shame also works – especially for better-paid, more senior people with a reputation to protect (and less need to impress journal editors).

Fourth, this experiment highlights some bigger issues in evaluation generally. One is that understanding the logic chain behind your results is just as important as getting the result in the first place. Rather than resorting to conjecture, it’s important to design your experiment so you can work out what is driving the result. In many cases, researchers can use mixed methods – interviews or participant observation – to help do this. Another is that context matters. I suspect that some of these results are driven by the power of the journal in question: for economists the JPubE is a top international journal, and many researchers would jump at the chance to help out the editor. A less prestigious publication might have more trouble getting these tools to work. It’s also possible that academics in other fields would respond differently to these treatments. In the jargon, we need to think carefully about the ‘external validity’ of this trial. In this case, further experiments – on sociologists or biochemists, say – would build our understanding of what’s most effective where.


A version of this post originally appeared on the What Works Centre for Local Economic Growth blog.

Spaces of Evidence seminar, 26 September

June 27, 2014

(c) richard serra / max nathan

I’m speaking at Goldsmiths in September, at one of the ESRC Spaces of Evidence seminars which will look at different types of economic evidence, their characteristics and limitations, and their uses in policy-making.

Will Davies, the organiser, has put together a nice lineup including Angus Deaton (Princeton), Suzy Moat (Warwick), Martin Giraudeau (LSE), Tiago Mata (UCL), Zsuzsanna Vargha (Leicester) and Vera Ehrenstein (Goldsmiths).

Here’s the blurb:

Economics and economists have a long history of providing a scientific basis or justification for public policy decisions. Concepts derived from welfare economics, such as ‘market failure’, have provided a language through which politicians and government officials can understand where and why the state might (and might not) intervene in market processes. The efficiency of potential regulation can be tested through the use of models, based on neo-classical assumptions.

However, events such as the financial crisis have thrown a renewed scepticism upon the capacity of orthodox economic theories to adequately model situations. At the same time, a new empiricism has emerged, which makes a bold appeal to data and field trials, which are purportedly less cluttered by normative assumptions about causality and probability. ‘Big Data’ and randomised controlled trials are at the forefront of new efforts to probe economic activity, in search of policies which ‘work’. The distinction between ‘model’ and ‘reality’ is abandoned, and the economy becomes treated as a zone of experimentation and data-mining, such that behavioural patterns can be discerned.

The seminar will explore the implications of these new directions in economic evidence, and ask what they mean for the authority of public policy, how they reconfigure expertise, and what types of epistemological and political assumptions they conceal.

It’s open to all, but you’ll need to register. Full details are here.

Same Difference?

June 25, 2014


I have a new article out in the Journal of Economic Geography. Originally one of my PHD papers, it looks at the demography of innovation, particularly the roles of what I call ‘minority ethnic inventors’.

Here’s the abstract:

Minority ethnic inventors play important roles in US innovation, especially in high-tech regions such as Silicon Valley. Do ‘ethnicity–innovation’ channels exist elsewhere? Ethnicity could influence innovation via production complementarities from diverse inventor communities, co-ethnic network externalities or individual ‘stars’. I explore these issues using new UK patents microdata and a novel name-classification system. UK minority ethnic inventors are spatially concentrated, as in the USA, but have different characteristics reflecting UK-specific geography and history. I find that the diversity of inventor communities helps raise individual patenting, with suggestive influence of East Asian-origin stars. Majority inventors may benefit from multiplier effects.

The full paper is here. You can also read the working paper version, though bear in mind there are some differences to the final edit. I’ll upate this post at some point with a proper pre-print.


My new book

May 23, 2014

I have a book out: Urban Economics and Urban Policy, written with Henry Overman and Paul Cheshire, and published by Edward Elgar.

In a nutshell, it’s ‘economic urbanism’. We bring together last two decades of work by economists and economic geographers on urban issues, and distill some high-level lessons for policymakers. We look at trends in city growth and change, spatial disparities and urban housing/labour markets, as well as evaluating a range of urban policies.

The focus is on the UK, and especially work done at LSE’s Spatial Economics Research Centre since 2008. You can read the first chapter here.


The book began as a kind of greatest hits compilation for SERC, but has morphed into a broader attempt to show what economists (and economic geographers like me) can bring to cities and urban analysis.

Economics’ influence on urban policy has historically been very limited: urban thinking has been dominated by architects, planners and governance types.

In part, this is because economists haven’t been very interested in space until recently. As Paul pointed out at the book’s launch, economics 101 classes mention the three factors of production – capital, labour and land – after which land is rarely (if ever) discussed again. That has only really begun to change in the last decade or so, with the very obvious death of ‘death of distance’ arguments, and people like Paul Krugman and Ed Glaeser making their influence felt in the profession. (Ed kindly wrote the foreword for our book.)

It’s also because spatial economic concepts and techniques are fiddly and difficult to explain. Dealing with spatial autocorrelation is rarely as glamorous or compelling as iconic buildings or big political personalities. Evan Davies did economic geographers everywhere a great service with the Mind the Gap series, which did a bravura job of distilling agglomeration, knowledge spillovers and path-dependence into everyday language.

And of course lessons from spatial economics aren’t always ones policymakers want to hear. Urban systems tend to build in spatial differences, and these inequalities are self-reinforcing and hard for policy to reverse. Many urban policies are effective, but many popular ones – such as Enterprise Zones or cluster programmes – often don’t have much impact.


In turn, that highlights both the advantages and limitations in the economic urbanist’s approach. City leaders should take economic ideas and analysis seriously, especially when making decisions about housing, planning or development. The book is an attempt to put economic thinking back in the room. But we can’t reduce cities to purely economic processes: as objects or systems, they are too complex and chaotic for that. And as Max Weber says:

… The explanation of everything by economic causes alone is never exhaustive in any sense whatsoever, not in even in the … economic sphere itself. In principle, a banking history of a nation which adduces only economic motives for explanatory purposes is naturally just as unacceptable as an explanation of the Sistine Madonna as a consequence of the social-economic basis of the culture of the epoch in which it was created. 

That logic also applies to policy choices. In practice we often have to trade off economic, social and environmental goals – when planning new roads or houses, for instance. Citizens’ welfare is rather wider than economic welfare, and we should avoid collapsing the first into the second.

Given those complexities, economists need to be mindful of real-world priorities and politics when giving policy advice. (As do others – Richard Rogers’ reductionist readings of Jane Jacobs have not been very helpful in the UK, for example.) The What Works Centre for Local Economic Growth, which I’m helping to run, is one attempt to translate quantitative academic analysis from a range of fields into feasible, pragmatic policy ideas.

As an economic geographer, co-authoring a book with two economists proper is a rewarding experience – and a challenging one. The three of us didn’t agree on everything: as you can imagine, my views on regeneration, brownfield development and place-based policies are more optimistic than some of my co-authors. In the book we carefully flag who led on each chapter, and which work is genuinely joint.


I hope all that’s encouraged you to take a further look. The hardback is painfully expensive, as academic books always are. The ebook edition is quite a lot cheaper. Either way, order it from the EE website and use the code CHES35 to get yourselves a 35% discount. Happy reading!

New articles published in Economic Geography, European Urban and Regional Studies

April 29, 2014

(c) Max Nathan 2011

Since last summer I’ve been pretty focused on helping get the What Works Centre off the ground, so I’m posting these two articles rather late in the day.

The first is online at European Urban and Regional Studies. It’s my over-ambitious attempt to build a framework for thinking about the economic impacts of diversity in cities, drawing on my own work as well as the growing international literature. Specifically, I critique and try to move beyond Richard Florida’s thinking on these issues.

Here’s the abstract:

In recent years, most European countries have experienced substantial demographic changes and rising cultural diversity. Understanding the social and economic impacts of these shifts is a major challenge for policymakers. Richard Florida’s ideas have provided a popular – and pervasive – framework for doing so. This paper assess Florida’s legacy and sets out a ‘post-Florida’ framework for ‘technology, talent and tolerance’ research. The paper first traces the development of Florida’s ideas. ‘Florida 1.0’, encapsulated by the Three Ts framework, has performed badly in practice. There are problems in bringing causality to the fundamental relationships, and in consistently replicating the results in other countries. ‘Florida 2.0’, though suggests that Creative Class metrics have value as alternative measures of human capital. This create space for a post-Florida agenda based on economic micro-foundations. I argue that the growing body of ‘economics of diversity’ research meets these conditions, and review theory and empirics. Urban ‘diversity shocks’ shift the size and composition of populations and workforces, with impacts operating via labour markets, and through wider production and consumption networks. While short-term labour market effects are small, over time low-value industrial sectors may become migrant-dependent. Diversity may help raise productivity and wages through innovation, entrepreneurship, market access and trade channels. Bigger, more diverse cities help generate hybridised goods and services, but may also raise local costs through crowding. All of this presents new challenges for policymakers, who need to manage diversity’s net effects, and address both economic costs and benefits.

The full piece is here. There’s no pre-print available, but you can access this paper in the IZA Journal of Migration which covers some of the same ground.


The second piece is out in Economic Geography, and is co-authored with Neil Lee. This is an empirical study testing links between firm-level demographics, innovation and entrepreneurship. We use a recent sample of London businesses, and uncover some small but robust diversity effects.

Here’s the abstract:

A growing body of research is making links between diversity and the economic performance of cities and regions. Most of the underlying mechanisms take place within firms, but only a handful of organization-level studies have been conducted. We contribute to this underexplored literature by using a unique sample of 7,600 firms to investigate links among cultural diversity, innovation, entrepreneurship, and sales strategies in London businesses between 2005 and 2007. London is one of the world’s major cities, with a rich cultural diversity that is widely seen as a social and economic asset. Our data allowed us to distinguish owner/partner and wider workforce characteristics, identify migrant/minority-headed firms, and differentiate firms along multiple dimensions. The results, which are robust to most challenges, suggest a small but significant “diversity bonus” for all types of London firms. First, companies with diverse management are more likely to introduce new product innovations than are those with homogeneous “top teams.” Second, diversity is particularly important for reaching international markets and serving London’s cosmopolitan population. Third, migrant status has positive links to entrepreneurship. Overall, the results provide some support for claims that diversity is an economic asset, as well as a social benefit.

The full piece is here, and a pre-print version is here.

Migrant entrepreneurs: fuzzy numbers and real impacts

March 24, 2014

The Centre for Entrepreneurs think tank recently made waves with this report on migrant entrepreneurship. The headlines are striking: 450,000 migrants set up 1 in 7 UK companies, at almost twice the rate of the UK population (17% vs 10%).

Here are some reactions. Overall, this is a welcome piece of work. However, I have some reservations about the numbers. And the report – understandably – doesn’t address some of the big issues where we still need answers.


First, praise where it’s due. This is completely new analysis which reflects serious effort. The actual analysis was done by Duedil, who crunched around 3m raw observations to get these numbers. (At NIESR we’ve been working with similar data: it’s a lot of work.).

The analysis has also shone some much-needed light on the role of migrants in entrepreneurship. As CFE point out, policymakers are only just starting to take this stuff seriously. Canada and Chile already have proper start-up visas. The US seems stuck in endless discussions. The UK is still getting there. And the underlying evidence base is under-developed.


Second, the data. It’s clear there’s a story here, and this is why the CFE work is welcome. But push a bit and the numbers are less convincing. It’s hard to tell whether the true numbers are higher or lower: the results are fuzzy.

One big issue is that there’s no correction for corporate structure – the raw data is legal entities, not businesses, and the true number of firms could be a lot smaller than the count of corporations. The relative contribution of non-UK entrepreneurs could then be higher or lower than before.

Another is that companies with migrant and UK-born founders are counted as migrant-founded. That makes sense. But without knowing what share of ‘migrant-founded’ companies are co-founded it’s hard to be clear on the true migrant contribution.

A third issue is that the data provides the nationality of company directors, not birth country. As CFE point out, this likely underestimates the count of migrant entrepreneurs (since many won’t take UK citizenship). But since there’s also many more migrants than non-UK nationals in the wider workforce (see here), this likely reduces the share  of migrant entrepreneurship. The report shows that 17% of non-UK nationals set up companies in the UK. Based on LFS figures, for that figure to hold for migrants the data would have to uncover a further 290,000-odd migrant entrepreneurs on top of the 450,000-odd non-UK nationals already identified. I’m not sure if that’s really plausible.

Full disclosure: We’ve asked DueDil for the raw data so we can see how they did the analysis. We’ve also put our questions to them: they haven’t responded yet, but I’ll update this post when they do. 


More broadly, this is descriptive analysis, which understandably doesn’t try to look at impacts (exploring these is a project in itself). CFE are upfront about this, although they suggest those impacts are likely to be large and positive.

I think there are (at least) four big questions for further research.

1/ What is the relative economic impact of migrant-founded companies versus co-founded and native-founded? Do they tend to do better or worse in terms of sales, productivity, attracting finance or coming up with new ideas?

2/ What explains this? Is it a ‘migrant x-factor’, or something about the company, or industry-level effect, location (say in a city)? Or some combination of these?

3/ What is the additional impact of migrant top team members? There’s no counterfactual here, but one possible workaround is to look at companies where a migrant director joined, and compare the change in their performance against similar companies where this didn’t happen. The data in the CFE report would provide a great basis to do this.

4/ What is the distributional impact of migrant (co)-founded companies? Do new migrant firms tend to complement or displace UK-founded companies?  The evidence suggests that more competition in an industry means some firms innovate out of trouble, while others exit.

Working out who are the winners and losers is politically crucial. As consumers, we may just want the best and cheapest goods and not care who makes them. But government needs to decide if they are interested in competitive markets per se, or the competitive position of UK businesses. Industrial policies often get into trouble for this reason. But the same issues likely apply to immigration policy too.

What have high-skilled migrants ever done for us?

March 2, 2014

(c) 2014 uncovention

I’ve got a new paper out in the IZA Journal of Migration.

It’s about high skilled migrants – by which I mean people with a degree, with specialist skills (e.g. scientists) or with rich experience in (say) entrepreneurship. It’s based on some recent work for the UK Government’s Migration Advisory Committee.

In the UK, immigration is once again a top 3 issue for public opinion, it’s pretty important to gather what evidence we have.

Here’s the abstract:

In recent years, the economics of migration literature has shown a substantial growth in papers exploring host country impacts beyond the labour market. Specifically, researchers have begun to shift their attention from labour market and fiscal changes, towards exploring what we might call ‘the wider effects of migration’ on the production and consumption sides of the economy – and the role of high-skilled migrants in these processes. This paper surveys the emerging ‘wider impacts’ literature, including studies from the US, European and other countries. It sets out some simple, non-technical frameworks, discusses the empirical findings and identifies avenues for future research.

Thanks to the joys of Open Access, you can read the whole thing here. In case you don’t have time, here are some of the main points:

1/ Skilled migrants now comprise around 30% of OECD migrants – that’s 25% more than in 2000/1. And the future trend is upward.

2/ We know quite a lot about immigration’s impact on wages and jobs (positive on the average, but more serious for some low skilled workers). However, we know almost nothing about the wider impacts of immigration – on productivity and its drivers, and on housing or public services – where high-skilled migrants are going to be central.

3/ To get a sense of these impacts, we need to think about immigration in terms of economic growth – that is, as a factor in production and something that shifts the size and composition of local populations.

4/ In theory, these wider impacts are ambiguous. For example, more diverse workforces could generate more new ideas; but diverse teams could have communication problems or lower trust.

5/ In practice, the international evidence suggests that aggregate wider impacts are positive (gains outweigh losses). In places like Silicon Valley, high skill migrants are a driving force in the local economy.

6/ That’s encouraging for pro-migration voices in the UK and elsewhere, but we still need much more evidence. In particular, we need to know more about the winners and losers in these aggregate outcomes.

7/ Ideally, the UK and other countries would develop experimental policies for high-skilled migrants, enableing us to cleanly identify these effects. Turning the Tier 1 entrepreneur route into a proper Startup Visa is one obvious option. Creating a market for wealthy investors via visa auctions is another. Both could allow for evaluation through randomised trials or quasi-experimental approaches. But the current politics of immigration are making this kind of thing increasingly difficult to do.

Do inventors talk to strangers?

January 15, 2014

Not such a productive meeting, (c) wikimedia

I’ve a new working paper out, written with my LSE colleagues Riccardo Crescenzi and Andrés Rodríguez-Pose. It’s available in three flavours, CEPR [£], IZA and SERC.

Here’s the abstract:

This paper investigates how physical, organisational, institutional, cognitive, social, and ethnic proximities between inventors shape their collaboration decisions. Using a new panel of UK inventors and a novel identification strategy, this paper systematically explores the net effects of all these ‘proximities’ on co-patenting. The regression analysis allows us to identify the full effects of each proximity, both on choice of collaborator and on the underlying decision to collaborate. The results show that physical proximity is an important influence on collaboration, but is mediated by organisational and ethnic factors. Over time, physical proximity increases in salience. For multiple inventors, geographic proximity is, however, much less important than organisational, social, and ethnic links. For inventors as a whole, proximities are fundamentally complementary, while for multiple inventors they are substitutes.

In other words, we find that physical proximity is critical to break the ice in a research collaboration; once the relationship has been established, however, other forms of proximity become more important. Crucially, for multiple inventors we find that co-location basically disappears as a driver of collaboration.

Obvious, you might think. But I’d argue that the multiple inventor group finding is pretty counter-intuitive. Our results also imply that the gains from incubators and research labs are strong for young researchers, but may fall off quite quickly. And our numbers chime with the ‘nursery cities’ hypothesis – basically, big cities are better for small young firms than older, larger ones.

The paper’s now with a journal, so fingers crossed. I’ll keep you posted when there’s news.

Agglomeration, clusters and industrial policy

November 25, 2013

Sou Fujimoto, Serpentine Pavilion. (c) Max Nathan 2013








I have a new article out in the Oxford Review of Economic Policy, joint with Henry Overman. It’s part of a special issue on ‘Government and Business’, with other contributions by Jonathan Haskel, Stian Westlake, Dieter Helm, Francesca Froy and Phil McCann.

You can see the whole lot here, and (for the moment) PDFs are free.

My piece with Henry is a constructive-critical take on clusters and the urban level of innovation policy. Here’s the abstract:

This paper considers the appropriate spatial scale for industrial policy. Should policy focus on particular places, targeting clusters of firms that are spatially concentrated? Or should it, instead, be ‘space neutral’, refusing to discriminate between different areas unless absolutely necessary? We provide an overview of the literature and identify two waves of literature that argue strongly in favour of a cluster approach. We argue that this approach rests on shaky theoretical and empirical foundations. In contrast, we suggest that more attention should be paid to the appropriate spatial scale for horizontal interventions. What can policy do to make cities work better, in ways that help firms to grow? That is, what is the appropriate role for ‘agglomeration’ rather than ‘cluster’ policy? Finally, we consider the possibility that some horizontal industrial policy objectives may be better served by specifically targeting particular places or from decentralized design or delivery.

Read the whole thing here.

… and we’re live

October 25, 2013

our London launch

We had the London launch of the What Works Centre yesterday. It went very well – full room, sharp discussion, plus strong contributions from LSE’s Director Craig Calhoun, from BIS and DCLG Ministers Michael Fallon and Kris Hopkins and from Joanna Killian from Essex.

We’re off to Manchester in a couple of weeks for a second launch session. Details here.

Now the hard work begins

In the meantime you can catch up on what we’re up to here and here.


Get every new post delivered to your Inbox.

Join 58 other followers

%d bloggers like this: