Advertisement

Application of bivariate meta-analytic approach for pooling effect measures of correlated multiple outcomes in medical research

Open AccessPublished:March 24, 2022DOI:https://doi.org/10.1016/j.cegh.2022.101029

      Abstract

      Background

      Multivariate meta-analysis is used when multiple correlated outcomes are reported in a systematic review. This study explored the application of multivariate meta-analysis in such a context. The objectives of the present study were to compare the summary findings and decisions between univariate and bivariate meta-analyses, as well as to assess how much sensitive the results are towards the strength of the correlation between the outcome variables.

      Methods

      A systematic review that reported two correlated outcomes, Intact parathyroid hormone levels and serum phosphate was chosen for demonstrating the applications of bivariate meta-analysis. Both univariate and bivariate meta-analyses with fixed effect and random effect models were carried out and the results were compared. A sensitivity analysis was performed for a wide spectrum of correlations from −1 to +1 to assess the impact of correlation on pooled effect estimates and its precision.

      Results

      Pooled effect estimates generated through bivariate meta-analysis were found to be varying when compared to those obtained through univariate meta-analysis. The confidence interval of the pooled effect estimates obtained through bivariate meta-analysis was wider than in univariate meta-analysis. Further, the value of the pooled effect estimates along with its confidence intervals also differed for varied levels of correlations.

      Conclusions

      This study observed that when we have multiple correlated outcome variables to answer a single question bivariate meta-analysis could be a better approach. The magnitude of the correlation between the outcome variables also plays a vital role in meta-analysis.

      Keywords

      1. Introduction

      Systematic reviews and meta-analyses have been considered as the crest of the evidence pyramid. Systematic review collates the evidence from several studies addressing the same research question of interest.
      If sufficient data is acquired from these studies, a meta-analysis may be applied to generate pooled effect estimates and associated confidence intervals (C.I.).
      • Glass G.V.
      Primary, secondary, and meta-analysis of research.
      The methodology of systematic reviews and meta-analysis has been well developed and applied in evidence-based medicine since the past two decades.
      In medical research, we often measure more than one primary outcome variable to answer a single research question. Most often such variables are statistically correlated like systolic blood pressure and diastolic blood pressure (correlation coefficient, ‘r’ ranged from 0.68 to 0.83), triglyceride and high-density lipoprotein (HDL) (r = −0.46) or low-density lipoprotein (LDL) and Non HDL (r = 0.961).
      • Kannel B.
      • Gordon T.
      • Schwartz J.
      Systolic versus diastolic blood pressure and risk of coronary heart disease. :12.Kannel WB, Gordon T, Schwartz MJ. Systolic versus diastolic blood pressure and risk of coronary heart disease: the Framingham study.
      • Hatahet W.
      • Khosla P.
      • Fungwe T.V.
      Prevalence of risk factors to coronary heart disease in an Arab-American population in Southeast Michigan.
      • Al-Waili K.
      • Al-Zakwani I.
      • Al-Dughaishi T.
      • et al.
      Comparison of therapeutic lipid target achievements among high-risk patients in Oman.
      Dependency might arise between these variables either because the multiple outcomes are measured from the same set of individuals or the same outcome measured at multiple time points. For example, heart rate, VAS score and pulmonary function (FEV1) measured at different time points were found to be highly correlated.
      • Albanese M.
      • Neofytou M.
      • Ouarrak T.
      • Schneider S.
      • Schöls W.
      Evaluation of heart rate measurements in clinical studies: a prospective cohort study in patients with heart disease.
      • Karabis A.
      • Nikolakopoulos S.
      • Pandhi S.
      • et al.
      High correlation of VAS pain scores after 2 and 6 weeks of treatment with VAS pain scores at 12 weeks in randomised controlled trials in rheumatoid arthritis and osteoarthritis: meta-analysis and implications.
      • Juwara L.
      • Boateng J.
      Assessing the effects of exposure to sulfuric acid aerosol on respiratory function in adults.
      However, while analysing such data, the researcher analyses each outcome variable separately (we call it as univariate analysis), compute the effect measure along with C.I. and inference regarding that particular variable will be drawn. One of the hidden assumptions for such analysis is that these variables are independent. However, in reality, many times these outcome variables are correlated and hence, the primary assumption gets violated.
      • Bland J.M.
      • Altman D.G.
      Statistics Notes: correlation, regression, and repeated data.
      Multivariate analysis which accounts for these correlations in making inferences is the correct method for analysing such data and various authors have attempted this in medical data analysis.
      • Saccenti E.
      • Hoefsloot H.C.J.
      • Smilde A.K.
      • Westerhuis J.A.
      • Hendriks M.M.W.B.
      Reflections on univariate and multivariate analysis of metabolomics data.
      ,
      • Mishra A.
      • Harichandrakumar K.T.
      • Vs B.
      • Satheesh S.
      • Nair N.S.
      Multivariate approach in analyzing medical data with correlated multiple outcomes: an exploration using ACCORD trial data.
      Further, the above mentioned correlation of the primary outcome variables is likely to exist while pooling such effect measures in meta-analysis. Hence, not adjusting for this correlation in the meta-analysis might produce effect measures with compromised precision. However, in the traditional systematic reviews, even if correlated multiple effect measures are present, usually a univariate meta-analysis is performed separately for each outcome.
      • Mavridis D.
      • Salanti G.
      A practical introduction to multivariate meta-analysis.
      Even though the lipid parameters were correlated as mentioned above, meta-analyses were done and forest plots were constructed for each outcome separately.
      • Wild R.A.
      • Rizzo M.
      • Clifton S.
      • Carmina E.
      Lipid levels in polycystic ovary syndrome: systematic review and meta-analysis.
      ,
      • Adams S.P.
      • Tiellet N.
      • Alaeiilkhchi N.
      • Wright J.M.
      Cerivastatin for lowering lipids. Cochrane hypertension group.
      Such a meta-analytic approach that ignores the correlation structure existing between these endpoints within studies may compromise the validity.
      • Riley R.D.
      Multivariate meta-analysis: the effect of ignoring within-study correlation.
      Moreover, this may also end up in selective outcome reporting bias. The appropriate approach would be to apply the multivariate meta-analysis method which incorporates the correlation between the variables and get simultaneous estimates for correlated variables. Such estimates will have better statistical properties and be estimated using a single modeling framework.
      • Jackson D.
      • Riley R.
      • White I.R.
      Multivariate meta‐analysis: potential and promise.
      Also, when missing outcomes exist, it helps in reducing the effect of outcome reporting bias.
      • Frosi G.
      • Riley R.D.
      • Williamson P.R.
      • Kirkham J.J.
      Multivariate meta-analysis helps examine the impact of outcome reporting bias in Cochrane rheumatoid arthritis reviews.
      However, multivariate meta-analysis methods are often less used in systematic reviews probably due to their complexity and difficulty in computation, understanding, and interpretation compared to the traditional univariate approach.
      • Riley R.D.
      Multivariate meta-analysis: the effect of ignoring within-study correlation.
      The objective of the current study is to demonstrate the utility of multivariate meta-analysis and compare the pooled effect measures obtained by multivariate meta-analysis of correlated multiple outcomes with that of univariate meta-analysis. The study also demonstrates how the multivariate approach results vary for a spectrum of correlations.

      2. Materials and methods

      2.1 Data source

      Data for the present study was taken from a systematic review that studied the effects and safety of Calcimimetics in end Stage Renal Disease Patients with Secondary Hyperparathyroidism.
      • Zhang Q.
      • Li M.
      • You L.
      • et al.
      Effects and safety of calcimimetics in end stage renal disease patients with secondary hyperparathyroidism: a meta-analysis. Burdmann EA, editor.
      In patients with chronic kidney disease, secondary hyperparathyroidism is one of the most common mineral metabolism disorders. The effectiveness of the calcimimetics therapy in end stage renal disease patients with secondary hyperparathyroidism was assessed through intact parathyroid hormone (iPTH) and serum phosphate (Pho) levels. For demonstration, the data on calcimimetics (Table 1.), which reported both the outcomes were taken from a systematic review.
      • Zhang Q.
      • Li M.
      • You L.
      • et al.
      Effects and safety of calcimimetics in end stage renal disease patients with secondary hyperparathyroidism: a meta-analysis. Burdmann EA, editor.
      Even though the original systematic review comprised nine studies, only seven studies that reported both the outcomes were considered for the analysis. These studies were conducted during the period from 2003 to 2008, the sample size ranged from 13 to 370 in the control group and 19 to 371 in the treatment group. The effect size (difference in means) and standard errors obtained from the studies are given in Table 1.
      Table 1Summary of Calcimimetics data set of Parathyroid hormone (pg/ml) and Serum phosphate (mg/dl).
      Study idParathyroid hormone (pg/ml)Serum Phosphate (mg/dl)
      Mean ± SDMean Difference (SE)Mean ± SDMean Difference (SE)
      TreatmentControlTreatmentControl
      Quarles et al., 2003451 ± 431.49552 ± 484.40−101.00(113.60)5.8 ± 1.175.7 ± 1.100.10 (0.28)
      Block et al., 2004374 ± 365.97693 ± 442.41−319.00(29.83)5.6 ± 1.936.0 ± 1.92−0.40(0.14)
      Lindberg et al., 2005525.5 ± 510.81852 ± 551.00−326.50(60.52)5.5 ± 1.705.8 ± 1.00−0.30(0.18)
      Martin et al., 2005385 ± 357.95698 ± 472.49−313.00(41.40)5.7 ± 1.436.0 ± 1.43−0.30(0.14)
      Sterrett et al., 2007294 ± 258.70683 ± 380.35−389.00(45.44)5.8 ± 1.995.9 ± 1.05−0.10(0.22)
      Malluche et al., 2008307 ± 218.38829 ± 543.00−522.00(137.79)5.9 ± 1.576.1 ± 1.19−0.20(0.51)
      Messa et al., 2008264 ± 168.00519 ± 281.00−255.00(19.18)5.1 ± 1.605.4 ± 1.50−0.30(0.14)
      SD-Standard deviation, SE- Standard error of difference in means.

      2.2 Meta-analysis framework

      The original review conducted a univariate meta-analysis for each variable separately and conclusions were drawn. However, a study conducted by Arora K. et al. reported a significant positive correlation between these two outcomes (r = 0.38;p = 0.003).
      • Arora K.
      • Goyal G.
      • Soin D.
      • Kumar S.
      • Arora H.
      • Garg C.
      Correlation of parathyroid hormone levels with mineral status in end-stage renal disease patients.
      This suggests the requirement for incorporation of correlation in the derivation of summary effect estimates. Both univariate fixed effect (UFMA) and random effect meta-analysis (URMA) were performed for both outcomes. Bivariate fixed effect meta-analysis (BFMA) was carried out using generalized least squares method, assuming that only within study variance covariance structure exists and bivariate random effect meta-analysis (BRMA) was carried out using both Maximum likelihood (ML) and, Restricted maximum likelihood (REML) estimation procedures.
      • Schwarzer G.
      • Carpenter J.R.
      • Rücker G.
      Meta-Analysis with R.
      The between-study correlation which depicts how the true underlying endpoint summary values are related across studies was also estimated in BRMA.
      • Riley R.D.
      • Abrams K.R.
      • Sutton A.J.
      • Lambert P.C.
      • Thompson J.R.
      Bivariate random-effects meta-analysis and the estimation of between-study correlation.
      Further, both BFMA and BRMA were carried out assuming different within-study correlations for all the possible range of correlations, ranging from −0.90, +0.90. The exact positive and negative correlations considered in the study were 0.25, 0.50, 0.75, 0.90 and −0.25, −0.50, −0.75, −0.90 respectively. The impact was assessed in terms of the change in the treatment effect estimates and its precision. When the correlation between outcomes is zero, the multivariate meta-analysis model shrinks to a univariate model. The dependency between outcomes was modeled through the variance-covariance structures as follows:

      2.2.1 The bivariate fixed effect meta-analysis model
      • Schwarzer G.
      • Carpenter J.R.
      • Rücker G.
      Meta-Analysis with R.
      ,
      • Berkey C.S.
      • Hoaglin D.C.
      • Antczak‐Bouckoms A.
      • Mosteller F.
      • Colditz G.A.
      Meta‐analysis of multiple outcomes by regression with random effects.
      • Raudenbush S.W.
      • Becker B.J.
      • Kalaian H.
      Modeling multivariate effect sizes.
      • van Houwelingen H.C.
      • Arends L.R.
      • Stijnen T.
      Advanced methods in meta-analysis: multivariate approach and meta-regression.

      The bivariate fixed effect meta-analysis model is given by,
      [θˆm1θˆm2]=[1001][θF1θF2]+[εm1εm2]


      where,
      [θˆm1θˆm2] denotes the bivariate outcome obtained from mth study (m = 1,2, …n),
      [θF1θF2]is the fixed effect parameter vector for the two outcomes and
      [εm1εm2]N{0,Ωm=[σm12σm12σm21σm22]}


      Ωm is the within study variance covariance matrix for the mth study.
      The vector of pooled fixed effect estimate is given by,
      θˆF=(m=1nwm)1(m=1nwmθˆm)


      where, wm=(Ωˆm)1is the weight given for mth study and
      the variance of the pooled effect estimate is given by,
      Var(θˆF)=(m=1nwm)1


      2.2.2 The bivariate random effect meta-analysis model

      The bivariate random effect meta-analysis model is given by,
      [θˆm1θˆm2]=[1001][θR1θR2]+[Um1Um2]+[εm1εm2]


      where,
      [θR1θR2] is the random effect parameter vector for the two outcomes
      [Um1Um2]N{0,T=[τ12τ12τ21τ22]}


      [εm1εm2]N{0,Ωm=[σ12σ12σ21σ22]}


      T is the between study variance covariance matrix.
      The vector of random effect pooled estimate is given by,
      θˆR=(m=1nwm)1(m=1nwmθˆm)


      where, wm=(Tˆ+Ωˆm)1is the weight assigned to the mth study and
      variance of the pooled effect estimate is given by,
      Var(θˆR)(m=1nwm)1


      In univariate analysis, 95% C.I and in bivariate analysis, simultaneous C.I (S.C.I) were computed. S.C.I. is a way for estimating individual C.I.s so that the joint confidence level for the family parameters remains (1-alpha).
      • Johnson R.A.
      • Wichern D.W.
      Applied Multivariate Statistical Analysis.
      For individual study estimates 100 (1-α) S.C.I. for the difference of mean is given by,
      a(X1X2)±χp2(α)a(1n1+1n2)Spooleda


      where,
      Spooled=(n11)s1,a2+(n21)s2,a2(n1+n22)


      p = number of variables.
      For pooled effect estimates the S.C.I. is given by,
      a(X1X2)Pooled±χp2(α)SE(X1X2)Pooled


      Univariate and bivariate meta-analyses were carried out using meta & mvmeta packages and forest plots were plotted using forest plot and ggplot2 packages in R software version 4.0.5.
      • Gasparrini A.
      Mvmeta: Multivariate and Univariate Meta-Analysis and Meta-Regression.
      ,
      R Core Team
      R: A Language and Environment for Statistical Computing.

      2.2.3 Ethics statement

      The Institute Ethics Committee (IEC) of Jawaharlal Institute of Postgraduate Medical Education and Research (JIPMER), Puducherry reviewed and exempted this study (IEC protocol no: JIP/IEC/2017/0360 dated December 30, 2018)

      3. Results

      Table 1 summarise, Calcimimetics data set of Parathyroid hormone (pg/ml) and Serum phosphate (mg/dl). The 95% C.I. and 95% S.C.I. were estimated using the effect estimates and standard error obtained from the individual studies.
      Results of both univariate and bivariate meta-analyses are given in Table 2. The BFMA and BRMA were carried out for a wide range of correlations varying from −0.90 to +0.90. UFMA and URMA results in comparison with BFMA and BRMA with assumed varying correlations are also depicted in Table 2.
      Table 2Univariate fixed and random effect meta-analysis results in comparison with Bivariate Fixed and Random effect meta-analysis with assumed varying correlations.
      MethodAssumed correlationiPTH (MD)SE95% CI1 or SCIPho (MD)SE95% CI1 or SCI
      UFMA−291.1913.72(-318.08, −264.31)−0.280.07(-0.41, −0.15)
      BFMA−0.90−279.0111.28(-306.61, −251.41)−0.360.06(-0.50, −0.22)
      −0.75−286.9712.82(-318.35, −255.59)−0.320.06(-0.47, −0.16)
      −0.50−290.4313.47(-323.40, −257.47)−0.300.07(-0.46, −0.14)
      −0.25−291.3113.67(-324.76, −257.86)−0.290.07(-0.45, −0.12)
      0.25−290.2013.67(-323.65, −256.75)−0.270.07(-0.43, −0.10)
      0.50−287.7313.47(-320.70, −254.77)−0.250.07(-0.42, −0.09)
      0.75−280.6812.82(-312.06, −249.30)−0.220.06(-0.37, −0.06)
      0.90−265.5811.28(-293.18, −237.98)−0.140.06(-0.28, 0.00)
      URMA−310.5027.35(-364.10, −256.90)−0.280.07(-0.41, −0.15)
      BRMA (ML)−0.90−300.6037.49(-392.37, −208.82)−0.270.09(-0.49, −0.06)
      −0.75−300.8733.11(-381.92, −219.82)−0.270.08(-0.47, −0.07)
      −0.50−305.1824.70(-365.64, −244.71)−0.280.07(-0.45,-0.11)
      −0.25−306.5722.16(-360.80, −252.34)−0.280.07(-0.45, −0.11)
      0.25−309.2722.65(-364.71, −253.84)−0.280.07(-0.45, −0.11)
      0.50−312.9423.83(-371.27, −254.61)−0.270.07(-0.44, −0.10)
      0.75−320.6427.11(-387.00, −254.27)−0.270.07(-0.44, −0.10)
      0.90−329.6131.64(-407.06, −252.17)−0.270.07(-0.44, −0.10)
      BRMA (REML)−0.90−300.3441.93(-402.97, −197.70)−0.270.09(-0.49, −0.06)
      −0.75−300.6637.80(-393.19, −208.13)−0.270.08(-0.47, −0.07)
      −0.50−303.9829.59(-376.41, −231.56)−0.280.07(-0.45, −0.11)
      −0.25−307.8525.03(-369.12, −246.57)−0.280.07(-0.45, −0.11)
      0.25−311.6125.27(-373.46, −249.76)−0.280.07(-0.45, −0.11)
      0.50−315.4226.72(-380.83, −250.02)−0.270.07(-0.44, −0.10)
      0.75−322.3929.60(-394.84, −249.95)−0.270.07(-0.44, −0.11)
      0.90−330.3733.73(-412.93, −247.80)−0.260.07(-0.43, −0.09)
      Note: iPTH, Parathyroid hormone (pg/ml); Pho, Serum Phosphate (mg/dl); 1In univariate it is 95% confidence interval (95% CI) and bivariate simultaneous confidence interval (95% SCI); MD, difference in means; SE, standard error of difference in means; UFMA, univariate fixed-effect meta-analysis; BFMA, bivariate fixed-effect meta-analysis; URMA, univariate random-effects meta-analysis; BRMA, bivariate random-effects meta-analysis; ML, Maximum likelihood estimator; REML, Restricted maximum likelihood estimator.

      3.1 Comparison of Univariate & Bivariate random effect meta-analysis for r = 0.50 along with the individual study results

      Forest plots of the results obtained from individual studies and those from the URMA and BRMA (REML) for a correlation of 0.50 are given in Fig. 1A and Fig. 1B, for outcomes iPTH and Serum phosphate respectively. The blue squares with the whiskers indicate the effect estimate obtained from individual studies and its 95% S.C.I. The whiskers attached to the red circle represent the 95% C.I. corresponding to univariate. Among the seven studies, Malluche et al. and Quarles et al. had less precise estimates when compared to the rest of the studies. Also, for the studies conducted by Martin et al. and Messa et al., the results were found to be significant in the univariate analysis whereas turned out to be non-significant in the bivariate for the Serum phosphate outcome. Summary effect estimates obtained from the BRMA and URMA are represented by the blue and red diamonds respectively. The width of the red and blue diamonds represents the 95% C.I. and 95% S.C.I. In both URMA and BRMA, calcimimetic agents effectively improved iPTH levels (MD, −310.50 pg/mL, 95% CI;-364.10, −256.90 and −315.42 pg/mL, 95% CI; −380.83, −250.02). BRMA estimate for iPTH indicates a larger difference between treatment groups when compared to the URMA estimate. The univariate and bivariate random pooled effect estimates were −0.27 mg/dl (95% CI; −0.44, −0.10) and −0.28 mg/dl (−0.41, −0.15) for serum phosphate. Forest plot for comparison of univariate and bivariate pooled effect estimate (r = 0.50) for (A) Intact parathyroid hormone (pg/ml) and (B) Serum phosphate (mg/dl) is depicted by Fig. 1.
      Fig. 1
      Fig. 1Forest plot for comparison of univariate and bivariate pooled effect estimate (r=0.50) for (A) Intact parathyroid hormone (pg/ml) and (B) Serum phosphate (mg/dl).

      3.2 Comparison of univariate with bivariate meta-analysis for varying correlations

      3.2.1 Comparison of univariate and bivariate fixed effect results

      Pooled effect sizes along with their 95% CI or SCI for univariate and bivariate meta-analysis for varying correlations are given in Fig. 2 and Fig. 3 respectively for the iPTH and Serum phosphate outcomes. As the correlation proceeded from 0.25 to 0.9 a decreasing trend was observed in the values of pooled effect estimates (from −290.20 pg/ml to −265.58 pg/ml) for iPTH and (from −0.27 to −0.14 mg/dl) for serum Phosphate in the BFMA. The corresponding standard errors also decreased for both outcomes. When negative correlation moved from −0.25 to - 0.90, the pooled effect estimate was found to decrease for iPTH (−291.31 to −279.01 pg/ml) whereas increased for serum phosphate (−0.29 to −0.36 mg/dl) in BFMA. However, the standard error decreased for an increase in negative correlation in both outcomes as in positive correlation. Thus, the value of the pooled estimate for iPTH outcome shifted upward (moved towards null value) for those obtained using BFMA when compared to those derived from UFMA for an increase in the magnitude of the correlation (Fig. 2). Also, as the correlation progressed from −0.90 to +0.90, values of the pooled effect estimates attained a parabolic pattern for iPTH outcome in BFMA as given. The pooled effect estimate of the serum phosphate formed a nonlinear pattern when the correlation moved from −0.90 to +0.90 (Fig. 3).
      Fig. 2
      Fig. 2Pooled effect measures and 95%CI or SCI for Intact parathyroid hormone (pg/ml) using different models and at different levels of assumed correlations.
      Fig. 3
      Fig. 3Pooled effect measures and 95%CI or SCI for Serum phosphate (mg/dl) using different models and at different levels of assumed correlations.

      3.2.2 Comparison of univariate and bivariate random effect results

      The value of the pooled effect estimate for iPTH outcome increased from −309.27 to −329.61 pg/ml and from −311.61 to −330.37 pg/ml respectively under ML and REML methods as correlation increased from 0.25 to 0.90. The corresponding standard errors also increased. Whereas the negative correlation varied from −0.25 to −0.90, the value of the pooled effect estimate decreased from −306.57 to −300.60 pg/ml and from −307.85 to −300.34 pg/ml respectively under ML and REML methods. The standard errors in both cases increased drastically. The BRMA estimates shifted downward compared to URMA estimate for an increase in positive correlation whereas shifted upward for an increase in negative correlation for the iPTH outcome (Fig. 2). The shift clearly shows the impact of the correlation on the estimates. For an increase in the magnitude of the correlation, the value of the pooled effect estimate of serum phosphate outcome decreased (−0.28 to −0.26 mg/dl) under both methods of estimation. Even though a hike happened in the standard error for an increase in negative correlation, it did not differ much for an increase in the positive correlation (Fig. 3). The between-study correlation was estimated to be −1 or +1 for different positive and negative within-study correlations respectively.

      4. Discussion

      The objective of the present study was to demonstrate the utility of multivariate meta-analysis in the context of correlated multiple outcomes. The data from an existing systematic review originally been analysed using univariate meta-analysis have been reanalyzed using multivariate meta-analysis. The S.C.Is were wider when compared to the univariate C.I.s in individual studies. Also, in general, the C.I. of the pooled effect estimate obtained through the bivariate meta-analytic approach was found to be wider when compared to the univariate approach. In the univariate meta-analysis, the weights given to individual studies were based on the variance alone, whereas in bivariate both variance and covariance were taken into account. So in BRMA, the weights assigned to individual studies were the inverse of the variance-covariance matrix, resulting in wider estimates. This phenomenon was similar to the inflated standard errors obtained in cluster randomised trials after incorporating the design effect.
      In the current study, both the pooled effect estimate and standard error decreased with an increase in positive correlation for BFMA. The value of the pooled effect size was found to increase for serum phosphate, but appears to decrease for iPTH in the case of an increase in negative correlation. The standard error for both outcomes also decreased for an increase in negative correlation. Thus, for an increase in correlation, regardless of its sign, the standard error was found to reduce in BFMA for both the outcomes. In other words, an increase in both negative and positive correlation in BFMA resulted in a decrease in the magnitude of the width of the C.I. for both variables. Also, the lower limit of the C.I. moved towards the null value. This evidence was further supported by a study done by Frozi et al. where, a decrease in standard errors was reported in the multivariate fixed effect meta-analysis model (MFMA) for all outcomes when compared to the UFMA.
      • Frosi G.
      • Riley R.D.
      • Williamson P.R.
      • Kirkham J.J.
      Multivariate meta-analysis helps examine the impact of outcome reporting bias in Cochrane rheumatoid arthritis reviews.
      Also, a change in the pooled effect estimates was reported, i.e., the value of the pooled effect increased for a few outcomes whereas decreased for a few outcomes in MFMA compared to UFMA. The present study also demonstrated a shift in the pooled effect estimates under BFMA for different correlations when compared to UFMA.
      In the same study of Frozi et al. as mentioned above, the standard errors were reported to be higher in multivariate random effect meta-analysis (MRMA) for six out of seven outcomes when compared to URMA estimates. Jackson D. and Riley R. D also demonstrated the applications of MRMA using three different case studies.
      • Jackson D.
      • White I.R.
      • Riley R.D.
      Quantifying the impact of between‐study heterogeneity in multivariate meta‐analyses.
      In the first case study, the pooled effect estimates obtained for both outcomes were lesser, whereas standard error was either higher or remained the same in MRMA compared to URMA. In the present study, we also got higher standard errors for BRMA when compared to URMA. The BRMA pooled estimates and the corresponding standard error were found to be higher for an increase in positive correlation under both methods of estimation for iPTH outcome. Thus, in BRMA, for an increase in the magnitude of the correlation, the width of the C.I. of iPTH outcome also increased under both methods of estimation. The pooled effect estimates tend to decrease while the related standard errors increased for an increase in negative correlation. Under both the restricted maximum likelihood method and maximum likelihood method of estimation, the standard error increased with an increase in both negative and positive correlation in BRMA for iPTH outcomes. In contrast to iPTH, the pooled effect estimates of serum phosphate outcome appear to decrease for an increase in correlation irrespective of its sign. The corresponding standard error had an increasing trend for an increase in negative correlation whereas not much change was visible in the case of serum phosphate. Since there was not much heterogeneity across studies in Serum phosphate outcome, no remarkable differences were observed between URMA and BRMA estimates. The values of the pooled effect estimates and its standard errors of serum phosphate were almost the same and unlikely to change for both methods of estimation. Also, in such instances, minimal difference was visible in the values of the pooled effect estimates for different correlations. The pooled effect estimates and standard errors obtained for the restricted maximum likelihood method were found to be bigger when compared to that of the maximum likelihood method. A similar pattern of change was observed in the value of the estimates and its precision in the case of maximum likelihood and restricted maximum likelihood methods for both the outcomes. In the second case study, the value of the pooled effect size and the standard error was found to be higher for one outcome in MRMA when compared to the URMA whereas a reduction was observed in both the estimates for the second outcome. Under the third case study, standard error decreased for both outcomes in the bivariate when compared to the univariate. But the value of the pooled effect estimate increased for one outcome whereas decreased for the other outcome in the MRMA. The third case study produced similar results to those of a study conducted by Wei Y. and Higgin J. P, whereas an increase in the precision of estimated treatment effects was observed in the bivariate when compared to the univariate method.
      • Wei Y.
      • Higgins J.P.
      Estimating within‐study covariances in multivariate meta‐analysis with multiple outcomes.
      The present study reveals that the result of bivariate meta-analysis was found to be sensitive to the varying values of correlation coefficients. Hence, multivariate meta-analysis can be considered as the preferred solution in contrast to multiple univariate analyses.
      • Moran J.L.
      Multivariate meta-analysis of critical care meta-analyses: a meta-epidemiological study.

      5. Conclusion

      This study demonstrates the importance of using multivariate meta-analysis and its application in health science evidence synthesis in the presence of multiple correlated outcomes. The current study suggests that the correlation between the outcome variables has an impact on the summary estimates and hence, the final decision. We conclude that if the outcomes are correlated, bivariate meta-analysis appears to be a better approach to get more reliable results.

      Declaration of conflicting interest

      The authors have no conflicts of interest to declare for this study.

      Funding

      This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

      Acknowledgments

      The authors would like to thank Dr. Vishnu Vardhan (Doctoral committee member) and Dr. Amala R for their valuable suggestions and help rendered during the project.

      References

      1. Higgins J.P.T. Thomas J. Chandler J. Cochrane Handbook for Systematic Reviews of Interventions. The Cochrane Collaboration, 2021 (Version 6.2 (updated February 2021))
        • Glass G.V.
        Primary, secondary, and meta-analysis of research.
        Educ Res. 1976; 5: 3-8https://doi.org/10.3102/0013189X005010003
        • Kannel B.
        • Gordon T.
        • Schwartz J.
        Systolic versus diastolic blood pressure and risk of coronary heart disease. :12.Kannel WB, Gordon T, Schwartz MJ. Systolic versus diastolic blood pressure and risk of coronary heart disease: the Framingham study.
        Am J Cardiol. 1971; 27: 335-346
        • Hatahet W.
        • Khosla P.
        • Fungwe T.V.
        Prevalence of risk factors to coronary heart disease in an Arab-American population in Southeast Michigan.
        Int J Food Sci Nutr. 2002; 53: 325-335
        • Al-Waili K.
        • Al-Zakwani I.
        • Al-Dughaishi T.
        • et al.
        Comparison of therapeutic lipid target achievements among high-risk patients in Oman.
        Angiology. 2014; 65: 430-435
        • Albanese M.
        • Neofytou M.
        • Ouarrak T.
        • Schneider S.
        • Schöls W.
        Evaluation of heart rate measurements in clinical studies: a prospective cohort study in patients with heart disease.
        Eur J Clin Pharmacol. 2016; 72: 789-795
        • Karabis A.
        • Nikolakopoulos S.
        • Pandhi S.
        • et al.
        High correlation of VAS pain scores after 2 and 6 weeks of treatment with VAS pain scores at 12 weeks in randomised controlled trials in rheumatoid arthritis and osteoarthritis: meta-analysis and implications.
        Arthritis Res Ther. 2016; 18: 73
        • Juwara L.
        • Boateng J.
        Assessing the effects of exposure to sulfuric acid aerosol on respiratory function in adults.
        (ArXiv190604296 Stat.)
        • Bland J.M.
        • Altman D.G.
        Statistics Notes: correlation, regression, and repeated data.
        BMJ. 1994; 308 (896): 896
        • Saccenti E.
        • Hoefsloot H.C.J.
        • Smilde A.K.
        • Westerhuis J.A.
        • Hendriks M.M.W.B.
        Reflections on univariate and multivariate analysis of metabolomics data.
        Metabolomics. 2014; 10: 361-374
        • Mishra A.
        • Harichandrakumar K.T.
        • Vs B.
        • Satheesh S.
        • Nair N.S.
        Multivariate approach in analyzing medical data with correlated multiple outcomes: an exploration using ACCORD trial data.
        Clin Epidemiol Glob Health. 2021; 11: 100785
        • Mavridis D.
        • Salanti G.
        A practical introduction to multivariate meta-analysis.
        Stat Methods Med Res. 2013; 22: 133-158
        • Wild R.A.
        • Rizzo M.
        • Clifton S.
        • Carmina E.
        Lipid levels in polycystic ovary syndrome: systematic review and meta-analysis.
        Fertil Steril. 2011; 95 (e11): 1073-1079
        • Adams S.P.
        • Tiellet N.
        • Alaeiilkhchi N.
        • Wright J.M.
        Cerivastatin for lowering lipids. Cochrane hypertension group.
        in: Cochrane Database Syst Rev.2020https://doi.org/10.1002/14651858.CD012501.pub2
        • Riley R.D.
        Multivariate meta-analysis: the effect of ignoring within-study correlation.
        J R Stat Soc Ser A Stat Soc. 2009; 172: 789-811https://doi.org/10.1111/j.1467-985X.2008.00593.x
        • Jackson D.
        • Riley R.
        • White I.R.
        Multivariate meta‐analysis: potential and promise.
        Stat Med. 2011; 30: 2481-2498
        • Frosi G.
        • Riley R.D.
        • Williamson P.R.
        • Kirkham J.J.
        Multivariate meta-analysis helps examine the impact of outcome reporting bias in Cochrane rheumatoid arthritis reviews.
        J Clin Epidemiol. 2015; 68: 542-550
        • Zhang Q.
        • Li M.
        • You L.
        • et al.
        Effects and safety of calcimimetics in end stage renal disease patients with secondary hyperparathyroidism: a meta-analysis. Burdmann EA, editor.
        PLoS One. 2012; 7e48070
        • Arora K.
        • Goyal G.
        • Soin D.
        • Kumar S.
        • Arora H.
        • Garg C.
        Correlation of parathyroid hormone levels with mineral status in end-stage renal disease patients.
        Indian J Endocrinol Metab. 2018; 22: 735
        • Schwarzer G.
        • Carpenter J.R.
        • Rücker G.
        Meta-Analysis with R.
        Cham: Springer International Publishing, 2015 (Use R!)
        • Riley R.D.
        • Abrams K.R.
        • Sutton A.J.
        • Lambert P.C.
        • Thompson J.R.
        Bivariate random-effects meta-analysis and the estimation of between-study correlation.
        BMC Med Res Methodol. 2007; 7: 3
        • Berkey C.S.
        • Hoaglin D.C.
        • Antczak‐Bouckoms A.
        • Mosteller F.
        • Colditz G.A.
        Meta‐analysis of multiple outcomes by regression with random effects.
        Stat Med. 1998; 17: 2537-2550
        • Raudenbush S.W.
        • Becker B.J.
        • Kalaian H.
        Modeling multivariate effect sizes.
        Psychol Bull. 1988; 103: 111-120
        • van Houwelingen H.C.
        • Arends L.R.
        • Stijnen T.
        Advanced methods in meta-analysis: multivariate approach and meta-regression.
        Stat Med. 2002; 21: 589-624https://doi.org/10.1002/sim.1040
        • Johnson R.A.
        • Wichern D.W.
        Applied Multivariate Statistical Analysis.
        sixth ed. Pearson Prentice Hall, Upper Saddle River, N.J2007: 773
        • Gasparrini A.
        Mvmeta: Multivariate and Univariate Meta-Analysis and Meta-Regression.
        2019
        • R Core Team
        R: A Language and Environment for Statistical Computing.
        R Foundation for Statistical Computing, Vienna, Austria2019
        • Jackson D.
        • White I.R.
        • Riley R.D.
        Quantifying the impact of between‐study heterogeneity in multivariate meta‐analyses.
        Stat Med. 2012; 31: 3805-3820
        • Wei Y.
        • Higgins J.P.
        Estimating within‐study covariances in multivariate meta‐analysis with multiple outcomes.
        Stat Med. 2013; 32: 1191-1205
        • Moran J.L.
        Multivariate meta-analysis of critical care meta-analyses: a meta-epidemiological study.
        BMC Med Res Methodol. 2021; 21: 148https://doi.org/10.1186/s12874-021-01336-4