Update to the association between Oral Hormone Pregnancy Tests, including Primodos, and congenital anomalies

On 18 March 2019, C Heneghan and JK Aronson discussed the findings of the Heneghan et al systematic review “Oral hormone pregnancy tests and the risks of congenital malformations: a systematic review and meta-analysis” [1] at a meeting of an ad hoc expert group convened by the Medicines and Healthcare products Regulatory Agency (MHRA) in London.

The meeting sought to assess the suitability and robustness of the methods used, including the selection and application of the quality scores and any clinical implications.

Because we had inadequate time to respond to questions raised for the first time during the meeting, we have set out our detailed response to criticisms in this report. We also present a pooled analysis of data that were included in the report of the UK’s Commission on Human Medicines independent Expert Working Group (EWG) [2], based on data obtained through an FOI request.

A. Critical appraisal of the Heneghan et al. Systematic review

On 5 March, we were sent the MHRA’s review of the Heneghan et al systematic review. [1]  The questions raised about the review were:

1. The selection of controls.
2. The selection of confounding variables across studies.
3. The analysis from studies that took account of a previous history of congenital malformations.

1. Selection of controls

Of the 16 case-control studies, Heneghan et al did not include data for 40 participants from two studies in which there was the potential to select an alternative group for comparison. Therefore, 40 of 17,194 available items of patient data (0.23%) were not included in the analysis.

These 40 items came from two of the 26 studies:

• in Ferencz 1980, 20 disease controls were not included in the analysis because none had used hormone pregnancy tests;
• in Greenberg 1977, 20 subjects were reported as having been exposed to hormones in both the case and control groups, and we considered it likely that these were exposed twins or family members.

Of the 10 cohort studies, we did not include data from 3132 subjects from 4 studies in which  an alternative group could have been selected for comparison. Therefore 3132 of the 55,974 items of patient data (5.61%) were not included in the analysis.

These 3132 items came from four of the 26 studies:

• in Fleming 1978, we excluded 140 doubtful malformations, which were mostly rhesus incompatibility (n = 37) and stillbirths (n = 100);
• in Michaelis 1983, we excluded 108 patients who had been exposed not only to Duogynon but also to other hormones;
• in Rumeau-Rouquette 1978, we excluded 1224 patients in whom other estrogen-progestogen derivatives were used that were not hormone pregnancy tests;
• in Torfs 1981, we excluded patients in whom serum tests (n = 689) or urine tests (n = 332) had been used; we included 17,057 non-affected controls.

Thus, Heneghan et al used 95.7% (69,996/73,168) of the available control data. The main reasons for omitting the rest were non-use of hormones or other tests or, as set out in our protocol (computer dated 23 October 2018), we extracted data for the controls that were most closely matched to the cases. The exclusion of 4.33% of the control data had minimal impact on the effect estimate and does not remove the statistical significance.

Protocol: Hormone pregnancy test use in pregnancy and risk of abnormalities in the offspring: a systematic review protocol. Carl Heneghan, Elizabeth Spencer, Bennett Holman, Igho Onakpoya. 25 March 2019. CEBM 

2. Selection of confounding variables across studies

Confounding variables for matching were reported in 19 of the 26 studies (see Table 1. ). As we described in our paper, we consider that of the 16 case-control studies, 12 controlled for the most important factor (item 5a in the Newcastle-Ottawa Scale, NOS for non-randomized studies) and nine controlled for important additional factors (item 5b).  Of the ten cohort studies, six controlled for the most important factor (item 5a) and four controlled for important additional factors (item 5b). Table 1 sets out the confounding variables collected and notes on matching/adjustments made in each individual study.

“A further assessment of bias in studies of harms: a case study of Primodos and congenital malformations” is set out in  BMJ EBM Spotlight * (published 15 March). This post discusses in detail the assessment of quality in assessing associations of harms and the use of the NOS.

* Heneghan C, Assessing bias in studies of harms: a case study of Primodos and congenital malformations. 15 March 2019. https://blogs.bmj.com/bmjebmspotlight/2019/03/15/assessing-bias-in-studies-of-harms-a-case-study-of-primodos-and-congenital-malformations/

3. Analysis of the data from studies that took account of a previous history of congenital malformations

Two studies took account of a previous history of congenital malformations in their analysis:

Gal et al. 1972: Excluding cases of previous malformed babies and those with a history of infertility did not affect the statistical significance: cases 15/85 vs control 4/97 (P = 0.01 to 0.001). Greenberg et al. 1977: After exclusion of all case-control pairs with a family history of congenital malformations in either or both families, use of HPTs by case mothers remained statistically significant: cases 64/743 vs control 35/781 (χ2 = 9.42; P < 0.01). Cases and controls were matched for all factors, except a history of previous offspring with abnormalities in the study families.

B. Meta-analysis of results presented in the EWG report

1. Obtaining the raw data extracted by the EWG

After discussions of the APPG on 21 January 2019, Marie Lyon sent an FOI request to the MHRA on 30 January 2019, asking them to release the raw data from the EWG report (see exhibit 1); she wrote again on 4 February 2019.

“The APPG supporting the Association have put in an FOI request for the raw data used in the Forest plot conclusions.  I was not aware this information was excluded in the EWG document. Would you please ask the MHRA/CHM why the raw data was not included and would you also please ask them to expedite the request for this information. The EWG pledged to allow ‘full public scrutiny and to publish all evidence which had been gathered, together with the assessments of the data.’  Exclusion of the raw data does not fulfil this pledge.” 

She received a response on 6 February: “the request will be forwarded to the appropriate department”. A copy of the FOI request was sent to IMMDS on 20 February. The issue of the availability of the raw data was raised by Lord Alton in the House of Lords on 28 February. On 5 March, the IMMDS emailed Marie Lyon to ask if the FOI request had been actioned. She confirmed that it had not.

Marie Lyon sent a further request on 6 March: “Would you please let me know if the raw data I requested has been actioned yet.” She received a response from the MHRA on 8 March, but this did not include the attachment containing the raw data. On 8 March, she asked for the attachment and finally received the raw data on 11 March.

We were interested in analysing these data, because we had noted a footnote in Figure 2 of the original EWG report, a forest plot of data on heart defects, which stated that “weights are from random effects analysis”, although neither weights nor pooled analyses were presented in the final report.

Having received the raw data that had been extracted by the EWG, we now present the results of a random-effects meta-analysis.

2. Results of the meta-analysis of the data extracted by the EWG

(a) Congenital heart defects

Analysis of the data presented in the EWG report shows a significant association of oral HPTs with a risk of congenital heart defects: OR = 1.92 (95% CI = 1.36 to 2.68; I2 = 22%; P = 0.0002; data from 9 case-control studies and 6 cohort studies).

(b) Any congenital malformation

Analysis of the data presented in the EWG report shows a significant association of oral HPTs with a risk of any congenital malformation: OR = 1.34 (95% CI = 1.13 to 1.60; I2 = 0%; P = 0.0008; data from 4 case-control studies and 8 cohort studies).

(c) Urogenital defects

Analysis of the data presented in the EWG report shows a non-significant association of oral HPTs with a risk of urogenital defects: OR = 2.22  (95% CI = 0.82 to 6.02; I2 = 0%; P = 0.12; data from 2 cohort studies).

(d) “Other” defects

Analysis of the data presented in the EWG report shows a significant association for congenital “other defects”; OR = 3.62 (95% CI = 1.11 to 11.82; data from 5 case-control studies). However, significant heterogeneity (I2 = 76%) across these 5 studies suggests that these results should not be combined; the term “other” probably reflects very different outcomes.

3. Conclusions

The results of the meta-analysis of the data presented in the EWG review* [2] are similar to those found in the Heneghan et al systematic review [1]. Both reviews show significant associations of HPTs with all congenital malformations and congenital heart defects, and a non-significant association with urogenital defects.

The criteria for including studies differed between the two meta-analyses, as Heneghan et al focused the question solely on exposure to HPTs and excluded exposure to other hormones.

However, both systematic reviews show that the use of oral HPTs in pregnancy is associated with increased risks of congenital malformations (Table 2).

Table 2. A comparison of analyses of the data presented in the EWG report and those presented by Heneghan et al.

* Although the EWG data reported the ORs with 95% CI estimates for outcomes involving three studies (Lammer 1986, Sainz 1987, and Tummler 2014), the raw data for the events rates in these studies were not reported in the paper. These studies  were, therefore not included in the meta-analysis

• Lammer 1986: nervous system; orofacial clefts; digestive and abdominal wall; limb defects;
• Sainz 1987: nervous system;
• Tümmler 2014: nervous system; urinary system; limb defects.

Authors

Carl Heneghan
Professor of Evidence-Based Medicine
University of Oxford

Jeffrey K Aronson
Consultant Physician and Clinical Pharmacologist
University of Oxford

Further information

Conflicts of interest

CH is an advisor to the All-Party Parliamentary Group on Hormone Pregnancy Tests,and he has presented the results of the systematic review at the UK Houses of Parliament and The Independent Medicines and Medical Devices Safety Review. CH has received expenses and fees for his media work, including BBC Inside Health. He holds grant funding from the NIHR, the NIHR School of Primary Care Research Evidence Synthesis Working Group (project 390), The NIHR Oxford BRC, and the WHO. The CEBM jointly runs the EvidenceLive Conference with the BMJ and the Overdiagnosis Conference with some international partners; these are based on a non-profit model. CH is Editor in Chief of BMJ Evidence-Based Medicine and an NIHR Senior Investigator

JKA has published many review articles and original papers in peer-reviewed journals on adverse drug reactions; he has edited textbooks, for which he has received royalties; he has been remunerated for acting as an expert witness in cases involving suspected adverse drug reactions; he has presented the results of the systematic review discussed above to members of the Bundestag; he is a member of the Centre for Evidence-Based Medicine (see above).

The views expressed are those of the authors and not necessarily those of the NIHR or the Department of Health and Social Care.

Acknowledgements:
Igho Onakpoya provided input to this letter and analysed the data using RevMan.

References:

[1] Heneghan C, Aronson JK, Spencer E, et al. Oral hormone pregnancy tests and the risks of congenital malformations: a systematic review and meta-analysis [version 2; peer review: 3 approved]. F1000Research 2019, 7:1725 (https://doi.org/10.12688/f1000research.16758.2)

[2] Report of the Commission on Human Medicines’ Expert Working Group on Hormone Pregnancy Tests. GOV.UK. https://www.gov.uk/government/publications/report-of-the-commission-on-human-medicines-expert-working-group-on-hormone-pregnancy-tests (accessed March 2019).