As an example, consider work the connection between diversity and public goods provision. In the 1990s and 2000s, at least a dozen papers in top econ or polisci journals have tested theoretical mechanisms suggesting that diversity would reduce the amount of public goods provided by the state. The papers generated a number of different ways to measure social and economic diversity (like ELF, gini-based measures, and measures of inequality between groups; one paper even looked at age diversity at the American county level) and a number of ways to measure service provision (GDP growth, surveys that measure service completion, satellite pictures of lights at night), and employed a diverse variety of controls.
Basically every cross-national paper found the same thing: diversity and service provision were negatively correlated. But some people got suspicious. All the regressions were cross national, and the tests were unable to falsify a number more specific theories. Gibson and Hoffman decided to test the theory using local data from Zambia, and found a positive correlation between diversity and services, even controlling for some obvious confounds like population density. A bunch of new papers established that autocratic countries were responsible for most of the negative correlation. Regression of democracies produce positive correlations between diversity and services; regression of autocracies produce negative correlations. The evidence holds in studies within a number countries. Results from Bolivia, Botswana, Vietnam, and a number of other countries show that there is a diversity dividend in democracies and a diversity deficit in autocracies.
A new generation of studies is taking advantage of variation in the rules of municipal governments within democracies like Bolivia and South Africa. If different rules of vote aggregation correlate to different regression coefficients between services and diversity, it will yield new insights on potential institutional fixes to mitigate the damage caused by diversity elsewhere.
This literature is obviously not perfect and no one really argues that it has nailed causation in any precise way. But a number of theories about diversity and public goods have been conclusively falsified in the minds of everyone that contributes to the literature, and the new questions seem to be progressively narrowing the mechanisms that seem plausible. The robustness of the findings and the narrowing of the plausible questions strike me as obvious progress.
Some cross-national papers
Ajilore, O. & Smith, J. (2011). Ethnic Fragmentation and Police Spending. Applied Economics Letters, 18(4): 329–32.
Alesina, A., Baqir, R., & Easterly, W. (1997). Public Goods and Ethnic Divisions. NBER Working Paper #6009.
Alesina, A., Michalopoulos, S., & Papaioannou, E. (2015). Ethnic inequality. Journal of Political Economy, 123(3), 547-724.
Baldwin, K., & Huber, J. (2010). Economic Versus Cultural Differences: Forms of Ethnic Diversity and Public Goods Provision. American Political Science Review, 104(04): 644–62.
Easterly, W., & Levine, R. (1997). Africa's Growth Tragedy: Policies and Ethnic Divisions. The Quarterly Journal of Economics, 112(4): 1203–50.
Poterba, J. (1997). Demographic Structure and the Political Economy of Public Education. Journal of Policy Analysis and Management, 16(1): 48–66.
Some within-country papers
Gibson, C., & Hoffman, B. 2013. Coalitions Not Conflicts: Ethnicity, Political Institutions, and Expenditure in Africa. Comparative Politics, 45(3): 273–90.
Gibson, C., Ferree, K., & Hoffman, B. (2016). Social Diversity Meets Electoral Institutions in Africa: Testing the Interactive Hypothesis at the Subnational Level in South Africa and Ghana. Working Paper.
Gisselquist, R., Leiderer, S., & Niño-Zarazúa, R. (2016). “Ethnic Heterogeneity and Public Goods Provision in Zambia: Evidence of a Subnational ‘Diversity Dividend’.” World Development, 78: 308–23.
Gisselquist, R. (2013). Ethnic Divisions and Public Goods Provision, Revisited.” Ethnic and Racial Studies, 37(9): 1605–27.
Habyarimana, J., Humphreys, M., Posner, D., & Weinstein, J. (2007). Why Does Ethnic Diversity Undermine Public Goods Provision? American Political Science Review, 101(4): 709–25.
Huber, J. (2012). Measuring Ethnic Voting: Do Proportional Electoral Laws Politicize Ethnicity? American Journal of Political Science, 56(4): 986–1001.
Khwaja, A. I. (2009). Can Good Projects Succeed in Bad Communities? Journal of Public Economics, 93(7-8): 899– 916.
Miguel, E., & Gugerty, M. (2005). Ethnic Diversity, Social Sanctions, and Public Goods in Kenya. Journal of Public Economics, 89(11-12): 2325–68.