top of page
Search

Addressing sample overlap using Generalized Weights should now be straightforward

Heiko Rachinger and I recently got our paper on addressing sample overlap in economics meta-analyses accepted for publication in the Journal of Economic Surveys (see here). The paper demonstrates how to apply our generalized-weights (GW) method (developed in a paper published in Research Synthesis Methods in 2020, see here) in cases of data aggregation, different estimation methods, or conversion to PCCs. By tackling these challenges, we hope to make the application of GW relatively straightforward.





For those unfamiliar with our GW method, it incorporates the covariance between sample-overlapping estimates in their weights in the meta-analysis. An estimate is given less weight not only when its variance is large, as in conventional inverse-variance weights, but also when its covariance with another estimate is large.


Thus far, meta-analysts have addressed sample overlap mostly by using clustered standard errors or multilevel models. These methods are satisfactory in same cases—e.g., when sample confined at the study level—but cannot completely address sample overlap in other cases. GW can tackle any case of sample overlap, within or between studies, no matter how complex it may be. Moreover, GW can easily be combined with those methods, if necessary to tackle estimate dependence caused by other factors (e.g., study-specific effects).


To make the implementation of GW as easy as possible, we have prepared Stata and R codes, which are publicly available on OSF (see here). They include companion documents with instructions for implementation. It's especially important to note how the input file must be structured for the codes to run smoothly.


Our method is very general and can tackle virtually any type of sample overlap. In cases where sample overlap takes different forms than those addressed in the paper, it may be necessary to tweak the input file slightly. With some ingenuity, we believe that virtually any sample overlap case can be addressed.


We invite you all to give it a try! If you encounter problems, please contact us at pedro.bom@deusto.es or heiko.rachinger@uib.es. Thanks!

21 views0 comments

Recent Posts

See All

Univariate PET-PEESE: Why Not Multivariate PET-PEESE?

[NOTE: The R programs that produced the results in this blog are publicly available here: https://osf.io/w2vxg/. “Program1” produced the results for single simulated datasets. “Program2” produced the

Methods Guidelines for Meta-Analysis

Zuzana Irsova (Charles University & Anglo-American University in Prague) As discussed at the Palma colloquium, together with Tom, Chris, and Tomas we prepared a brief practical guide on how to do mode

Comentarios


bottom of page