元分析的最重要特點是其可重複性和透明度，在實際研究實踐中其具體情況是什麼樣的呢？今天推薦的這篇文章就是為了回答這個問題。這篇文章2020年發表在期刊Perspectives on Psychological Science。感興趣的可以讀一下這篇文章。請關注微信公眾號「元分析」獲取更多精彩內容。
Transparency and Reproducibility of Meta-Analyses in Psychology: A Meta-Review
Systematic review and meta-analysis are possible as viable research techniques only through transparent reporting of primary research; thus, one might expect meta-analysts to demonstrate best practice in their reporting of results and have a high degree of transparency leading to reproducibility of their work. This assumption has yet to be fully tested in the psychological sciences. We therefore aimed to assess the transparency and reproducibility of psychological meta-analyses. We conducted a meta-review by sampling 150 studies from Psychological Bulletinto extract information about each review’s transparent and reproducible reporting practices. The results revealed that authors reported on average 55% of criteria and that transparent reporting practices increased over the three decades studied (b = 1.09, SE = 0.24, t = 4.519, p < .001). Review authors consistently reported eligibility criteria, effect-size information, and synthesis techniques. Review authors, however, on average, did not report specific search results, screening and extraction procedures, and most importantly, effect-size and moderator information from each individual study. Far fewer studies provided statistical code required for complete analytical replication. We argue that the field of psychology and research synthesis in general should require review authors to report these elements in a transparent and reproducible manner.
中文摘要
只有通過透明的原始研究報告，系統綜述和元分析才可能成為可行的研究技術。因此，人們可能會期望元分析人員以最佳實踐報告結果，並具有高度的透明度，從而可以確保工作的可重複性。這個假設尚未在心理學中得到充分檢驗。因此，我們旨在評估心理學元分析的透明度和可重複性。通過對《心理公報》中的150項研究進行取樣，提取有關每篇綜述的透明度和可重複性的報告實踐的信息，進行了一次元綜述。結果表明，作者報告標準的平均值為55％，並且透明度的報告實踐在三十年中有所增加（b = 1.09，SE = 0.24，t = 4.519，p <.001）。綜述作者一致報告了納入標準，效果量信息和綜合技術。但是，平均來說，綜述作者沒有報告具體的搜索結果、篩選和提取程序，最重要的是，沒有報告每個研究的效應量和調節變量信息。很少有研究提供可重複的完整分析所需的統計代碼。我們認為，心理學和研究綜合領域通常應要求綜述作者以透明和可重複性的方式報告這些要素。