Experience in machine learning application to predict pregnancy loss after assisted reproductive technologies

According to the latest data, about 15–20% of all pregnancies end in miscarriages, and 80% of them happens before the 12th week of pregnancy [1]. Pregnancy loss may happen in healthy married couple. However, in cases of two or more spontaneous miscarriages before 22 weeks, additional examination is required, as well as special preparation for the next pregnancy and treatment [2]. It should be noted that the chance of subsequent miscarriage increases significantly, if there are two or more terminations of pregnancy in medical history. The risk of repeated miscarriage after two miscarriages is 24%, after three – 30%, after four – 40% [3, 4]. The main factors leading to miscarriage include embryonic/fetal   chromosomal aneuploidies, anatomical, infectious, endocrine, immunological and other factors. In addition, men’s reproductive health and ejaculate quality make a significant contribution to pregnancy outcome. The risk of having a miscarriage is in pregnancy resulting from assisted reproductive technologies (ART) [5]. It is noteworthy that tendency towards increasing age of parents, who seek infertility treatment using in vitro fertilization (IVF), as well as chromosomal abnormalities in embryos, are among the main factors that raise the risk of miscarriage after ART [6].

Analysis of correctable risk factors for miscarriage helps to significantly optimize preparation of the married couple for ART program, reduce financial costs of IVF in the context of reducing the number of ART attempts to increase the number of live births, as well as improve the effectiveness of infertility treatment. At present, the systems based on neural network modeling are of considerable interest for practical healthcare in the field of ART, and make it possible to perform integral analysis of correlation relationships and identify hidden patterns in a large data array [7]. Prediction of miscarriage after ART program using machine learning (ML) tools, divides the analyzed data, depending on their characteristics, into subsets based on characteristic values. Despite the fact that the decision tree can find non-linear dependencies and can be easily interpreted by the operator, the major limitations of this method include potential instability caused by data change, as well as the risk of overfitting [8]. To identify the most reliable trend in prediction of treatment outcome, it is necessary to compare and combine ML algorithms and classical mathematical models, such as linear regression.

Thus, the objective of the study was prediction and detection of risk factors of miscarriage in patients undergoing ART program based on clinical, anamnestic and embryological parameters, using the decision tree algorithm combined with linear regression.

Materials and methods

The retrospective study included 1021 married couples aged 21–44 years, who underwent infertility treatment using ART. Written informed consent to participate in the study and process their personal data was obtained from each couple. Inclusion criteria in the study were the following: normal karyotype of the spouses, ovarian stimulation using gonadotropin-releasing hormone (GnRH) antagonist protocol, selective embryo transfer. Exclusion criteria were abnormally shaped uterus, as well as contraindications to undergo ART program. The patients enrolled in the study underwent preliminary medical examination in accordance with the Order No 803n of the Ministry of Health of Russia. The team of mathematicians was involved in development of ML algorithm and modeling.  

The patients included in the study underwent standard ovarian stimulation using GnRH antagonist protocol on day 2 or 3 of the menstrual cycle. Transvaginal puncture of the ovaries was performed 36 hours after the trigger shot with subsequent in vitro fertilization of the obtained oocytes (5.6% of patients), intracytoplasmic sperm injection (ICSI) (81.9%), and physiological ICSI (PICSI) (12.5%). Selective embryo transfer was done using the standard technique on day 5 after fertilization. Three weeks after embryo transfer, when there was positive test result for the β-subunit of human chorionic gonadotropin, pelvic ultrasound was performed to diagnose clinical pregnancy. The following criteria were analyzed as embryological parameters: sperm concentration, the percentage of progressive motility, non-progressive motility, immotility, morphologically normal sperm on the day of puncture, the number of cumulus–oocyte complexes (COCs), mature MII oocytes, zygotes, embryo quality, the number of excellent-grade, good-grade and average-grade blastocysts, as well as the number of embryos that stopped developing.

Statistical data processing and the algorithm used for constructing the decision tree

Microsoft Excel 15.0 was used to perform statistical analysis and regression analysis of the obtained data. The results of statistical analysis were interpreted at p-value equal to 0.05 as the cut-off for significance. Decision tree was used as ML method with Interactive Python Notebook (ipynb). The data were analyzed using Gini impurity. Scikit-learn, Numpy, Pandas, Shap, Matplotlib libraries were used for the Python programming language.

The methodology of assessing model quality in the study included the following steps.

1. Splitting dataset into training and test subsets:

*The train_test_split method is used in the data science library scikit-learn to split the data into training and test sets. In each step of the cycle, the data is split once again. Training data is 80% of the dataset, and test data is 20%. The random_state parameter equal to 42 ensures reproducibility of the random splitting results in each step.

2. Building and training ML model:

* DecisionTreeClassifier model is used for classification. The maximum depth of the tree (max_depth) equals to 55. This model is trained on the training data set (X_train and y_train).

3. Prediction:

*After the model is trained using training dataset, it is used to predict the labels (y_pred) of the test set (X_test).

4. Model quality assessment:

*The quality of the model is assessed using two metrics: f1_score – the harmonic mean of precision and recall, that makes this metric appropriate for imbalanced classes; accuracy_score is a percentage of correct predictions among all predictions.

5. Cross-validation:

*Model training and evaluation of model performance is repeated 5 times according to the number of cross-validation splits (n_cv_splits) to be performed.  After the end of the cycle, mean and standard deviation for each of the metrics (f1_score and accuracy_score) is calculated and displayed for all 5 iterations, that allows to evaluate the stability and overall performance of the model.

Results

In the first step of the study clinical and laboratory parameters of patients enrolled in the study were analyzed. The data are represented in Tables 1 and 2. Clinical and laboratory characteristics of patients included woman’s age, man’s age, Anti-Mullerian Hormone (AMH) level, body mass index (BMI),  bad habits, age at menarche, duration of menstrual bleeding, menstrual cycle length, sexual debut, number of pregnancies, abortions, miscarriages, ectopic pregnancies in history, presence of sexually transmitted infections (STIs), adnexitis, endometriosis, uterine fibroids, polyps, chronic endometritis, polycystic ovary syndrome (PCOS), ovarian cysts, ovarian resection, tubectomy in history, infertility duration, IVF attempts, factor of infertility.

Pregnancy and miscarriage rates in patients enrolled in the study are shown in Table 3.

In the next step of the study, together with the mathematicians and bioinformatics specialists a linear regression model was constructed, and p-value was assessed. The clinical and laboratory values that are statistically significant for pregnancy, are represented in Table 4. The regression coefficient reflects positive and negative impact parameters on the frequency of miscarriage. According to the linear regression model, the frequency of miscarriage is influenced by woman's age, the number of pregnancies in history, the number of preterm births, inability to get pregnant from the particular partner, stimulation duration, embryo quality, dual trigger for final oocyte maturation, as well as fertilization of the obtained oocytes through ICSI.

In the next step, machine learning model using the decision tree, that reflects the significance of the analyzed parameters and their importance in determining pregnancy rate, was built to identify hidden patterns. The following parameters were of maximum importance: woman’s age, man’s age, stimulation duration, total gonadotropin dose, spermogram parameters (the percentage of progressive motility, morphologically normal sperm, sperm concentration, immotility), infertility duration, sexual debut, the parameters of stimulated cycle (the number of 2PN, MII, COCs), BMI, embryo quality according to morphological evaluation of embryos based on the Gardner score, the number of high quality embryos, fertilization through ICSI, AMH level, uterine fibroids, endometritis, STIs in history, the number of abortions, as well as inability to get pregnant from the particular partner in history (Table 5).

Thus, according to the two constructed models, the indicators of maximum significance in determining the frequency of miscarriage are the age, pregnancies from the particular partner in history, stimulation duration, embryo quality, and the method of fertilization.

Model accuracy:

• mean: 0.84
• standard deviation: 0.06

f1 score:

• mean: 0.775
• standard deviation: 0.09

Discussion

The models predicting the outcome of infertility treatment are widely used in medicine. Logistic and linear regression, as well as ML are among the main algorithms used to solve this objective. It should be noted that traditional prediction models have limited accuracy. In 2019, the results of the study showed that the area under the curve in the model for prediction of embryo implantation rate in patients undergoing ART program, that was constructed using logistic regression was 0.66, and 0.74 using ML. [9].  In 2020, Barnett-Itzhaki Z. et al.  also assessed the accuracy of the model to predict the effectiveness of ART, that was built using ML (0.69–0.90) and traditional statistical methods (0.34–0.74), respectively [10]. It should be noted that currently data storage technologies are being improved, so the introduction of ML allows more efficient and rapid big data processing without involving a large number of medical resources, selection of necessary information fragments from a large data array, creation of structured databases and, based on the previous experience, creation of more reliable prediction models. In contrast to ML, linear regression has more limitations for analysis of a large number of multi-class parameters and variables. In addition, classical models are based on deterministic equations and have no ability for autonomous and independent learning. For this reason, these models cannot be used to create decision support systems [11].

The first step of our study was linear regression model building. It showed that the frequency of miscarriage after ART is influenced by the age, the number of pregnancies, the number of preterm births in history, stimulation duration, inability to get pregnant from the particular partner, embryo quality, the use of dual trigger for final oocyte maturation, as well as fertilization of the obtained oocytes through ICSI.

According to the created machine learning model,  the indicators of maximum significance in determining miscarriage rate are the age, stimulation duration, total gonadotropin dose, spermogram parameters (the percentage of progressive motility, morphologically normal sperm, sperm concentration, immotility),  infertility duration, sexual debut, the parameters of stimulated cycle (the number of 2PN, MII, COCs), BMI, embryo quality, the number of high quality embryos, fertilization through ICSI, AMH level, uterine fibroids, endometritis, CTIs in history, the number of abortions, as well as inability to get pregnant from the particular partner in history.

It is noteworthy that stimulation duration, drug dose, the number of oocytes and zygotes influence miscarriage. The results of previous studies have shown that these parameters have a positive impact on embryo implantation rate [12]. Nevertheless, the influence of the stimulated cycle parameters on the frequency of miscarriage can be associated with the fact that increasing the length of stimulation duration causes increases in drug dose and the number of oocytes obtained during transvaginal ovarian puncture, leading to the risk of ovarian hyperstimulation syndrome (OHSS). The linear regression has shown that the use of a double trigger reduces miscarriage rate most likely as a result of optimal prevention of the risk of OHSS. The changes associated with OHSS can lead to miscarriage due to the development of procoagulant state and formation of microthrombi in the placental vessels, disruption of blood supply in the placental vessels, and can cause local infarction, ischemia and hypoxia [13]. Moderate physical activity, as well as low molecular weight heparins can be recommended for patients at risk for OHSS not only to prevent progression of OHSS during pregnancy, but also miscarriage.

The results of building the two models showed that fertilization of obtained oocytes through ICSI reduces the risk of miscarriage. According to the Order No.803n of the Ministry of Health of Russia “On the procedure for using assisted reproductive technologies, contraindications and limitations to their use”, fertilization  using the ICSI technique should be performed according to the following indications: male  factor infertility, that  manifests itself in significantly reduced ejaculate volume; the use of surgically obtained sperm; the use of oocytes after cryopreservation; preimplantation genetic testing using polymerase chain reaction, low fertilization rate in the previous IVF program. There is a large number of studies showing that with mild male factor infertility, the effectiveness of ART programs does not change, when oocytes are fertilized using IVF or ICSI, despite the fact that initially the percentage of fertilized oocytes through ICSI is higher [14].  Given the fact that ML model showed that male factor infertility influences miscarriage rate, it is most likely, that using ICSI for sperm selection also allows to optimize the best sperm selection. The obtained results require further research for objective interpretation of the spermogram parameters on the day of transvaginal puncture to identify the groups of patients who can undergo IVF without reducing live birth rate.

In previous steps of our study, the constructed machine learning model showed that sperm concentration in 1 ml is of maximum significance among the main spermogram parameters, that influence clinical pregnancy rate, as all as embryonic outcomes. At the same time, the percentage of morphologically normal sperm, as well as the percentage of progressively motile sperm has insignificant influence on these data. Our study showed that miscarriage rate largely depends on the percentage of progressively motile and morphologically normal sperm, and sperm concentration is of less significance. In addition, linear regression showed no statistically significant contribution of any spermogram parameter to miscarriage rate. The obtained results are consistent with the published data, which reflect the important contribution of the percentage of morphologically normal sperm to live birth rate, but not to pregnancy rate [15]. The described patterns allow to take a different look at preparation of men for ART programs, since a personalized algorithm of preparation and differentiated interpretation of treatment results based on spermogram data facilitates a targeted and comprehensive approach to infertility treatment associated with male factor depending on medical history of the married couple.

It is traditionally believed that ovarian reserve parameters directly influence the effectiveness of ART due to selection of the best quality embryo in patients with a high ovarian reserve [16]. ML model showed that AMH and FSH levels, infertility duration, the number of good/best quality embryos appropriate for vitrification also have a large impact on miscarriage rate. According to the systematic review in 2020, ovarian reserve parameters, especially AMH level, can be a predictor of early reproductive losses [17].

According to previous studies, this parameter autonomously influences the effectiveness of ART and live birth rate even with age adjustment of a married couple [18]. It is most likely associated with the fact that prolonged infertility duration in a married couple may be due to the complex of factors, including ejaculate quality and endometrial receptivity.

Machine learning model demonstrated that infertility duration, factor of infertility, as well as inability to get pregnant from the particular partner in history have influence on miscarriage. In the previous steps of the study it was found that reduced effectiveness of ART in patients with inability to become pregnant from the particular partner in history can be due to complex molecular abnormalities in endometrial receptivity in this married couple, fertilizing ability of sperm, changes in oocyte quality, and abnormal embryo cleavage. The results of our study also showed that primary infertility is associated with the risk of miscarriage, that is consistent with the results of previous studies [19].

In the context of miscarriage, it should be noted that patients with primary infertility, in addition to interaction between impaired gamete production and embryo implantation, could be unthoroughly examined, and the causes of miscarriage can be determined only after pregnancy loss.  One of the largest studies that used ML to predict miscarriage rate found that age, AMH level, BMI, infertility duration, pregnancies and births in history and factor of infertility were most significant among 108 variables to build a predictive model [20]. It is noteworthy that this study did not find any influence of ejaculate quality on miscarriage rate.  It can be due to the fact that general definition without a detailed explanation (teratozoospermia, normozoospermia, asthenozoospermia, etc.) was used as a criterion for ejaculate quality.

 In conducting similar studies, it is worth to pay attention to the practical significance of the obtained results. In clinical practice, when consulting a married couple, the doctor most often predicts live birth based only on age-related data. Undoubtedly, woman's age has a great influence on live birth rate. At the same time, according to the results of a number of similar studies, man's age has no influence on miscarriage. The results obtained in our study are consistent with the results of the study showing that man’s age also makes a significant contribution to determination of final ART success and live birth rates [21]. Research in this area, especially using ML tools for data processing enables to make personalized and integrated prognosis for each married couple, since the frequency of miscarriage may vary significantly among different patients of similar age.  The obtained results can optimize the state’s financial and economic expenditures to conduct ART cycles at the expense of Compulsory Health Insurance for different groups of patients. In addition, a clear and unified algorithm can facilitate the targeted impact on the most probable cause of miscarriage, taking into account optimization of preparation time and achievement of maximum effect to reduce miscarriage rate in the subsequent programs. Convenient and simple algorithm with nderstandable interface, developed together with mathematicians, allows less experienced specialists to use this tool as an auxiliary program for objective preparation and treatment of a married couples facing infertility.

Conclusion

Despite significant advances in creating the systems based on traditional regression analysis, their accuracy does not exceed 65% on average. The use of ML has limitations and disadvantages, but integration of such systems and their application in daily medical practice can significantly improve not only the quality of predicting the effectiveness of ART, live births, but also help more effectively work with accumulated data and constantly updated and added information without involvement of a large number of resources. Indeed, currently the use of ML techniques is impossible without involvement of applied mathematicians and specialists in the field of programming and bioinformatics. However, software product development and user interface understandable to the doctor can become a convenient auxiliary tool allowing more objective, integral and comprehensive approach to infertility treatment using ART.