Overwhelming heterogeneity in systematic reviews of observational anti-epileptic studies

Purpose: Observational studies may provide important information on the long-term effects of treatments for epilepsy, but systematic reviews of observational studies may be more prone to heterogeneity and biases. These issues were investigated in a systematic review of non-randomised add-on anti-epileptic drug studies. Methods: Searches of MEDLINE (1966-2006), EMBASE (1974-2006), CINAHL (1982-2006), the Cochrane database of systematic reviews, the Cochrane Controlled Trials register, the DARE database and hand-searching congress proceedings were conducted. Randomised controlled trials, follow-on randomised controlled trials and prospective and retrospective cohort studies of gabapentin, topiramate, or levetiracetam as add-on therapy in adults (>12 years aid) were identified. Outcomes were 50% responders and proportion seizure free. Results: Thirty-eight non-randomised gabapentin studies, 82 topiramate and 84 levetiracetam studies were identified. There was marked heterogeneity of effect estimates from observational studies which prohibited the pooling of estimates in random effects models. Median effect estimates were larger and more varied for observational studies than randomised placebo-controlled trials (RCTs). For example, the median value (10(th) and 90(th) percentile) for 50% responders for gabapentin was 36% (15 and 71%) compared to 23% (19 and 38%) for gabapentin RCTs. Patient and study covariates in meta-regression models could not explain the vast heterogeneity. Publication bias was evident and a sensitivity analysis, allowing for the effects of publication bias, showed that effect estimates could increase by up to 6% for seizure freedom rates. Discussion: Reports of observational anti-epileptic studies give limited information on patient selection and characteristics. Systematic reviews of observational studies are prone to significant heterogeneity and bias which cannot adequately be explained by reported study characteristics. Reporting standards for observational studies of anti-epileptic drugs could be improved by following guidelines for reporting non-randomised studies of interventions. (C) 2008 Elsevier B.V. All rights reserved.