Definition of Meta
analysis

Uses of meta analysis

Methods of meta analysis

Limitations of meta analysis

**Key words and key terms**

Review article

Summary quantitative statistic

Over conclusion

**AGENDA**

A. Definition And Historical Background

B. Advantages Of Meta Analysis

C. Steps In Meta Analysis

D. Difficulties Of Meta Analysis

E. Reading Results Of Meta Analysis

**PRE-TEST**

**1. The following statements are true about meta-analysis**

Meta analysis combines data from more than one study to produce a quantitative summary statistic

Meta analysis picks up missed statistical significance because of a larger effective sample
size

Meta-analysis enables study of variations of effect measures across different population
groups

The odds ratio or the regression coefficient are the usual summary measures

The summary effect measure is a weighted summary of the effect measures of individual studies

DEFINITION

Combining data from more than one study

Producing a quantitative summary statistic.

HISTORICAL BACKGROUND

Review articles

Written by discipline leaders

Subjective

Summarization of conclusions without looking at original data

Meta analysis

First developed for randomized clinical trials

Use in observational studies is more problematic

Difficulty of knowing and adjusting for confounders.

** **

ADVANTAGES OF META ANALYSIS

Computation of an effect estimate for a larger number of study subjects

Picking up statistical significance that would be missed by small individual studies

Study of variation across several population subgroups

Transparent reviews based on quantitative assessments.

STEPS IN META ANALYSIS

Identification of the variables of interest

Outcome

Exposure

Confounder

Intermediate

Modifying

Identification of relevant articles from data bases such as medline

Use of variables and key words

Inclusion/exclusion criteria

Determination of effect measures and population to be covered

Data abstraction on a standard form (raw data is preferred if available)

Display of effect measures (with 95% CI) for each study

Computation of the summary effect measure, OR or b

Each individual study is treated as a stratum

Tests of homogeneity

Compute by weighted logistic regression or MH weighted average

Weight of each measure the inverse of its precision ie 1/(se)^{2}

Adjustment of the summary statistic for bias

Confounding bias

Selection bias

Misclassification bias

Sensitivity analysis to test the robustness of the combined effect measure.

DIFFICULTIES OF META ANALYSIS

Differences in study designs, study analysis, and data quality

Over conclusion

Artifactual conclusion

Conclusion not supported by aggregated data

Bias

Publication bias

Positive results favored

Studies in academic or government institutions favored

Studies by pharmaceutical firms suspected

Selection bias

Inadequate search for reports

Multiple publications of the same study

Bias (conscious or unconscious) in study selection

Use of wrong methods

Conclusion based on the proportion of positive studies

Scatter plot of test statistics to show a general trend or correlation

Using SD as a standardized measure of the deviation from the center

Quality scoring of the various studies.

READING RESULTS OF META ANALYSIS

Methods clearly stated?

Search for articles comprehensive enough?

Criteria for selecting articles for review stated?

Criteria objective?

Criteria adhered to?

Possibility of bias in the selection of articles?

Methodologic quality of each article assessed?

Differences between studies explained?

Combination of results from the primary studies appropriate?

Conclusions of the reviewer supported by data?

SYNOPSIS

Meta analysis refers to methods used to combine data from more than one study to produce a quantitative
summary statistic. Meta analysis enables computation of an effect estimate for
a larger number of study subjects thus enabling picking up statistical significance that would be missed if analysis was based
on small individual studies. Meta analysis also enables study of variation across several
population subgroups since it involves several individual studies carried out in various countries and populations. Criteria
must be set for what articles to include or exclude. Information is abstracted from the articles on a standardized data abstract
form with standard outcome, exposure, confounder, or effect modifying variables. The first step is to display the effect measures
with their 95% confidence limits to get a general idea of their distribution before proceeding to compute summary measures.
The summary effect measure, OR or b, is computed from the effect measures of individual studies using weighted logistic regression or computing a
MH weighted average in which the weight of each measure is the inverse of its precision ie 1/(se)^{2}. In both the
logistic or MH procedures, each study is treated as a stratum. The combined effect measure is then statistically adjusted
for confounding, selection, and misclassification biases. Tests of homogeneity can be carried out before computing the summary
effect measure. Sensitivity analysis is undertaken to test the robustness of the combined effect measure.

** **

**A. DEFINITION and HISTORICAL BACKGROUND**

DEFINITION

Meta analysis refers to methods used to combine data from more than one study to produce a
quantitative summary statistic.

HISTORICAL BACKGROUND

Review articles written by discipline leaders were the most popular method of combining
findings from various studies. They however had two serious draw-backs. They were subjective and therefore prone to error
and bias. The reviewer was free to make decisions on what data and conclusions to emphasize. Secondly they dealt with summarization
of conclusions without looking at the data on which the conclusions were based. Meta analyis was first developed for randomized clinical trials. Its use in observational studies
is more problematic mainly because of the difficulty of knowing and adjusting for confounders.

**B. ADVANTAGES OF ****META**** ANALYSIS**

**Meta-analysis** has become popular with the proliferation of epidemiological studies on particular subjects. Writers of review articles
and practising epidemiologists would like to have some form of consensus or summary of the findings of various studies. Meta analysis enables computation
of an effect estimate for a larger number of study subjects thus enabling picking up statistical significance that would be
missed if analysis was based on small individual studies. Many clinical trials especially with invasive intervention can not
recruit enough patients in one center to reach statistical significance. Meta analysis also enables study of variation across several population subgroups since it
involves several individual studies carried out in various countries and populations. Meta analysis
makes the process of reviewing several studies with view to reaching a general
conclusion very transparent because it is based on quantitative assessments.

**C. STEPS IN META ANALYSIS**

The first step is to identify the variables of interest such as outcome, exposure, confounder,
intermediate, and effect modifying variables. The variables are used together with other relevant key words to identify relevant
articles from the data bases of such as medline or from other unpublished sources. Criteria must be set for what articles
to include or exclude. These include definition of what effect measures are used and the populations covered. It is best if
there is as much homogeneity in the population as is practical. Data collection is carried out by abstracting information
from the articles on a standardized data abstract form with standard outcome, exposure, confounder, or effect modifying variables.
The ideal is to abstract raw dta and reanalyze it in a standard way to enable combination of several studies to get one summary
effect measure. In practice the raw data is not available and only effect measures are available. The first step is to display
the effect measures with their 95% confidence limits to get a general idea of their distribution before proceeding to compute
summary measures. The summary effect measure, OR or b, is computed from the effect measures of individual studies using weighted logistic regression or computing a
MH weighted average in which the weight of each measure is the inverse of its precision ie 1/(se)^{2}. In both the
logistic or MH procedures, each study is treated as a stratum. The combined effect measure is then statistically adjusted
for confounding, selection, and misclassification biases. Tests of homogeneity can be carried out before computing the summary
effect measure. Sensitivity analysis is undertaken to test the robustness of the combined effect measure.

**D. DIFFICULTIES OF META
ANALYSIS **

Meta analysis is methodologically complex because of different study designs, study analysis,
and even different data quality. The major problems are: overconclusion and bias (publication bias, selection bias), and use
of wrong methods. Overconclousion arises when a conclusion is artifactual and is not supported by the aggregated data.The
results of metaanalysis based on published sources may not reflect the true situation because of existence of publication
bias. Positive findings are more likely to be published than negative ones. Studies carried out in academic or government
institutions are thought to be more credible and are therefore more likely to be published whereas studies by pharmaceutical
firms have a lower publication rates. Inadequate search for reports may lead to bias just as multiple publications of the same study data leads to bias. Bias, conscious or unconscious, may occur in the
selection of studies for analysis. In some cases the methods used for metaanalysis are wrong or inappropriate such as reaching
a conclusion based on the proportion of positive studies, a scatterplot of test statistics to show a general trend or correlation,
use of the standard deviation as a standardized measure of the deviation of the effect measures from the center, or quality
scoring of the various studies.

**E. READING RESULTS OF ****META**** ANALYSIS**

The following questions should be considered when reading review or meta-analysis articles:
are the methods clearly stated (b) was the search for articles comprehensive enough?,
were the criteria for selecting articles for review stated, were the criteria objective and were they adhered to?,
was there a possibility of bias in the selection of those articles?, was the
methodologic quality of each article assessed?, were differences between studies explained or were they just glossed over?, was the combination of results from the primary studies appropriate?, and were the
conclusions of the reviewer supported by data?