Evaluation Economics

Critical review of an article Replication:

Choose a published article (e.g. from the database JSTOR) which estimates the "causal" effect of a variable on an outcome, e.g. evaluating a policy

Make sure the authors of the article have made their data available - most of the best economic journals have a section of their website dedicated to storing data of published articles. For example, check the American Economic Association (AER, AEJ) or the Quarterly Journal of Economics.

Read the article carefully

Replicate what you believe are the main results of the paper. Especially any graph/regression that supports the validity of the identification assumption. If by any chance no test of the identification assumptions is provided, try to devise one.

Document your replication in a readable do-file and create formatted tables of your output which you can use in word or latex (whichever text processing format you use).

If your replication leads to different estimates/standard errors, try to find out why.

Critical review of the article:
Explain clearly and briefly (no more than a third of your review should be dedicated to this))
what is the research question,
what is the identification strategy in this paper
Which identifying assumptions are made?
Which econometric method is used to estimate the treatment effects in this paper?
Summarise the findings
Point out possible shortcomings or extensions.
Discuss how credible the identification strategy is.
Could you think of alternative tests?
Is the model specification adequate? Could you think of an alternative one? - justify why and estimate it.
Are the authors providing robustness checks, are they useful? Could you think of any other robustness check?
(The extension part) Present at least one set of estimates that is not reported in the paper. By using any of your ideas from above:
add or eliminate some of the variables
split the analysis into different groups (gender, ethnicity,...),
split the analysis by time period,...
do a novel robustness check
bring in a new dataset to answer the same question

bring in an improved method, or ideally:
bring as much novelty as possible (new data, new methods)

When submitting, you must provide:
your written work (work exceeding 3000 words will not be accepted, shorter works are desirable)

• an electronic copy of the article that you are replicating
• a copy of the data that you have used to replicate
• a copy of your replication R-file (or do-file) - please provide annotation (i.e. comments) so that it can be understood what you are doing.
• r and rData (or .do and .dta) files for your extensions.