Introduction

Cancer recurrence is considered to be an important cancer outcome metric to measure the burden of the disease and success of (neo)adjuvant therapies. Despite this, high-quality breast cancer recurrence rates currently remain unknown in most countries, including Belgium. To date, cancer recurrence is not systematically registered in most population-based cancer registries, due to the difficulty and labor-intensity of registering follow-up for recurrences.

Recurrence definitions used for registration purposes differ among countries, due to the lack of consensus regarding a standardized clinical definition. Defining recurrence clinically is a challenge, since various methods exist to detect recurrences after (neo)adjuvant treatments of a patient such as physical examination, pathological examination, imaging, or tumor markers. Unlike the guidelines and definitions that currently exist in the clinical trial setting,1,2 no guidelines are set to correctly and consistently register a recurrence in a patient with stage IIII breast cancer at diagnosis.

Real-world recurrence data could give an estimation of cancer burden and efficacy of cancer treatment modalities outside a conventional clinical trial setting, which could eventually lead to improvements in quality of care.3,4 Administrative data from health insurance companies on medical treatments and procedures, also known as bill claims, and hospital discharge data could represent an alternative source for the assessment of disease evolution after breast cancer treatment.

Recently, machine learning algorithms based on classification and regression trees (CART) have been developed to detect cancer recurrence at the population level using claims data.5 However, only in a limited number of countries, research teams were able to successfully construct algorithms to detect breast cancer recurrences, and only for a small number of centers (USA,6,7 Canada,8,9 Denmark10,11 and Sweden)12 Our aim was to develop, test and validate an algorithm using administrative data features allowing the estimation of breast cancer recurrence rates for all Belgian patients with breast cancer.

To construct and validate an algorithm to detect distant recurrences, female patients with breast cancer diagnosed between January 1, 2009 and December 31, 2014 were included from nine different centers located in all three Belgian regions. We did not include patients with stage IV breast cancer at diagnosis, patients with a history of cancer (any second primary cancer, multiple tumors, and contralateral tumors), or patients who could not be coupled to administrative data sources. All breast cancers, regardless of molecular subtype, were included. Among the nine centers were centers from the Flemish region (University Hospitals Leuven, General Hospital Groeninge, Jessa Hospital, Imelda Hospital, and AZ Delta), Brussels-Capital region (Cliniques universitaires Saint-Luc and Institut Jules Bordet) and Walloon region (CHR Mons-Hainaut and CHU UCL Namur). For all nine centers, 300 patients were included per center, by randomly selecting from the study population 50 patients per incidence year. The study population of six centers was divided by randomization (6040% split-sample validation) into a training set to develop the algorithm, and an independent test set to perform an internal validation.13 The algorithm was additionally validated with an external validation set of the three remaining centers, to check reproducibility of the algorithm in a dataset with patients from other centers.

For the selection of the nine centers, we aimed for a reasonable variety of center characteristics based on teaching vs non-teaching hospital, the spread across the three regions in Belgium, and center size.

For each patient in the study population, recurrence status (yes, no, unknown) and recurrence date (day, month, year) were extracted and collected from electronic medical files and reviewed by trained data managers from each of the nine hospitals. Recurrence was defined as the occurrence of a distant recurrence or metastasis between 120 days after the primary diagnosis and within 10 years of follow-up after diagnosis or end of study (December 31, 2018). Data managers were instructed to consider death due to breast cancer in our definition of a recurrence. Loco-regional recurrence, was not considered as an outcome in our study. Both patients with a progression (without a disease-free interval) and patients with a recurrence (with a disease-free interval) were considered as outcome in our definition of recurrence. Patients with an unknown recurrence status, due to the lack of follow-up for example, were excluded from the analysis. Patients with a recurrence within 120 days were considered de novo stage IV and therefore excluded because interference of first-line treatment complicates recurrence detection. Starting from diagnosis to detect recurrent disease might cause more false positive recurrence cases due to the treatment of the initial breast cancer overlapping with the immediate first-line treatment due to metastatic disease. Recurrence diagnosis date was the time-point (described in day, month, and year), confirmed by pathological examination, imaging (CT, PET-CT, bone scintigraphy or MRI scan), or defined by physicians in the multidisciplinary team meeting (MDT).

In the course of an extensive data linking process with pseudonymization of the patient data, the recurrence data from the hospitals (i.e., gold standard) were linked to several population-based data sources. These included cancer registration data from the Belgian Cancer Registry (BCR), and administrative data sources, including claims or reimbursement data (InterMutualistic Agency, IMA),14 hospital discharge data (Technische Cel, TCT),15 information on vital status (Crossroads Bank for Social Security, CBSS)16 and cause of death (Agentschap Zorg en Gezondheid, Observatoire de la Sant et du Social de Bruxelles-Capitale, and Agence pour une Vie de Qualit AVIQ).17 Information on data sources and data used is presented in Appendix 1.

To build a robust algorithm to detect distant recurrences, pre-processing and extraction of features were performed. Expert-driven features to potentially detect recurrences in administrative data were created based on recommendations from breast oncologists (P.N. and H.W.). First, a comprehensive list of reimbursement codes for diagnostic and therapeutic procedures and medications was selected, and code groups were created based on their relevance for the diagnosis and/or treatment of distant metastasis in breast cancer patients (See Appendix 2).

Potential features were further refined based on the exploration of data from patients with a recurrence, including time-frames starting from time points after diagnosis (0 days, 90 days, 160 days, 270 days, and 365 days after diagnosis). We assessed different time-frames to obtain the most accurate feature to detect recurrences, and because starting from the date of diagnosis might result in noise from the treatment of the initial breast cancer. We additionally created features based on count of codes, by assessing the maximum number of codes per year or per pre-defined time-frame (starting from 0, 90, 160, 270, and 365 days after diagnosis) (Table 1). The best performing time-frame was selected for each feature by maximizing the Youdens J index:18

Table 1 List of Potential Markers for Recurrence (Available Within Administrative Data) Based on Recommendations from Breast Oncologists

After a feature list was obtained (as described in previous section), this list was narrowed down based on the ensemble method of bootstrapping.19 In total 1000 bootstrap samples were used to generate 1000 classification and regression trees (CART) using the same training set, and to select best-performing features based on the frequency of the features.19,20

Cost-complexity pruning was applied for each bootstrap sample, to obtain the best performing model and avoid over-fitting of the model to the dataset.20 CART inherently uses entropy for the selection of nodes or features. The higher the entropy, the more informative and useful the feature is.20 A 10-fold cross-validation was also performed to ensure robustness of the model in different training sets. Collinearity of the selected features was accounted for by the one standard error (1-SE) rule, to eliminate redundant features. The 1-SE rule selects the least complex tree that is within 1 standard error from the best performing tree.21

Based on the selected features from the bootstrapping, a principal CART model was built to classify patients as having a recurrence or not by using the complete training set.

Sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and classification accuracy was calculated for evaluating and comparing the performance of the principal CART model. All models were created and trained in SAS 9.4 (SAS Institute, Cary, NC, USA) within the SAS Enterprise Guide software (version 7.15 of the SAS System for Windows).

Data for a total of 2507 patients could be retrieved from nine Belgian centers and were included in the final dataset to train, test and externally validate the algorithm (Figure 1 and Table 2). The mean follow-up period was 7.4 years. For the split sample validation, the patients from six centers were split into the training set (N = 975 of which 78 distant recurrences, 8.0%) and internal validation set (N = 713 of which 56 distant recurrences, 7.9%). The external validation set consisted of three independent centers with 819 patients, of which 82 had distant recurrences (10.0%). The training, internal validation, and external validation sets did not have differences in distribution of baseline tumor and patient characteristics (Table 2).

Table 2 Baseline Patient and Tumor Characteristics

Figure 1 Patient inclusion flow diagram.

Based on bootstrap aggregation, 1000 CART models were built using the following features: (1) Presence of a follow-up MDT meeting, starting from 270 days after diagnosis (feature present in 975 out of 1000 CART models), (2) Maximum number of CT codes present (with a moving average over time) of 5 or more times a year (851 CART models), and (3) Death due to breast cancer (412 CART models) (see Supplementary Figure 1). Afterwards, the final CART model was constructed with these three features and calculated by using all data of the training set (Figure 2).

Figure 2 Final CART model to detect recurrences based on the three selected features after bootstrapping. Nodes represent selected features by the algorithm to classify patients.

Abbreviations: MDT, multidisciplinary team meeting; CT, computed tomography scan.

The sensitivity of the principal CART model to detect recurrences for the training set was 79.5% (95% confidence interval [CI] 68.887.8%), specificity was 98.2% (95% CI 97.199.0%), with an overall accuracy of 96.7% (95% CI 95.497.7%) (Table 3), and an AUC (area under the curve) of 94.2%. After 10-fold cross-validation within the training set, we found a sensitivity of 71.8% (95% CI 66.486.7%), specificity of 98.2% (95% CI 96.398.5%) and overall accuracy of 96.1% (95% CI 94.797.2%). The internal validation (i.e. based on test set) resulted in a sensitivity of 83.9% (95% CI 71.792.4%), a specificity of 96.7% (95% CI 95.098.9%), and accuracy of 95.7% (95% CI 93.997.0%). After external validation was performed on three additional centers, the sensitivity was 84.1% (95% CI 74.491.3%), with a specificity of 98.2% (95% CI 97.099.1%) and accuracy of 96.8% (95% CI 95.497.9%).

Table 3 Performance of Training Set, Cross Validation, Internal Validation Set and External Validation Set

In this study, we were able to successfully develop a machine learning algorithm to detect distant recurrence in patients with breast cancer, achieving accuracy of 96.8% after external validation in multiple centers across Belgium. The final list of detected parameters were presence of a follow-up MDT meeting, CT scan (max 5 times a year), and death due to breast cancer. Recurrence data are lacking in many population-based cancer registries due to the cost and labor-intensity of registration.3 True incidence of cancer recurrence should be known across age groups and regions in Belgium, to measure burden of illness and eventually improve quality of care. Current recurrence numbers are often extrapolated from clinical trials, which typically exclude older and frail patients. Older patients are more susceptible to receive under-treatment and to recurrences22,23 and recurrence numbers could therefore be underestimated.

The administrative data sources used in our algorithm virtually cover all residents of Belgium,14 which was useful to achieve population-based recurrence data. We were also able to accomplish a multi-centric study by developing the training model and performing an external validation based on data of multiple centers. Likewise, it is highly important to have a relatively large population and reliable gold standard to develop and train a machine learning model in these studies, to avoid prolonging and complicating the feature selection process due to conflicting recurrence and treatment data occurrence.

The definition of a distant recurrence in medical files was the occurrence of a distant recurrence or metastases after a period of 120 days. This time-frame until detection of recurrence varied among previous studies.2427 Most common exclusions were done either from 120 days (Chubak et al 2012) or 180 days after diagnosis (Amar et al 2020). Disease progression can be difficult to measure accurately and can be overestimated because of timing of therapeutic procedures that might be delayed. The limitation of our study was that we could not make a distinction between disease progression and disease recurrence. Defining medical recurrence in the clinic is a challenge, which makes it more difficult to define recurrence with a proxy based on administrative data.28 Therefore, setting a clear definition of window of treatment and the time-frame for detection of recurrence is considered important for future studies.

We chose to restrict our definition to distant recurrences to achieve a straightforward feature selection. We included death due to breast cancer as an outcome in our definition of recurrences. Cause-specific death and accurate source of cause of death is of utmost importance when studying recurrences, since recurrence and death are closely related to each other.29

The machine learning algorithm used in this study was a decision tree, i.e. the Classification And Regression Tree (CART) with the ensemble method. Ensemble learning combines multiple decision trees sequentially (boosting) or in parallel (bootstrap aggregation). The key advantages of using bootstrap aggregation are: better predictive accuracy, less variance, and less bias than a single decision tree. Similarly, latest studies more often make use of ensemble methods.7,9,12

Within the recurrence detection features that were selected from the bootstrapping method for the cohort of six different Belgian centers, no treatment features were selected, which could indicate that there are more inter-center similarities for diagnostic regimens and more differences in terms of treatment regimens. During pre-processing of the features, we did additional checks of features to improve accuracy of the model. For instance, we generated a treatment feature that only included metastases-specific chemotherapy agent codes. However, this feature was not included in the final model. Next, we tried out a model without diagnostic features, but this did not improve accuracy. Previous studies mostly make use of metastatic diagnosis codes (secondary malignant neoplasm or SMN code from ICD-9 or ICD-10) in their algorithm, which would be useful if highly reliable. We also checked subgroups by testing out different models for patients younger or older than 70 years, and different incidence years. We applied the algorithm on subgroups based on age or incidence years, to check if the algorithm accuracy performed better in specific subgroups. As expected, we found higher performance in younger patients (Supplementary Table 1).

Our algorithm performance was comparable to previous studies using decision trees.9,12,24,3032 We found greater accuracy compared with the pooled accuracy of previous algorithms.5

Although algorithms with highest overall accuracy are often sought-after in earlier studies, some studies also provide multiple algorithms to choose from based on their preference, e.g. high-sensitivity or high-specificity algorithms.6,10,24,26,30 Finally, we also investigated the false negative cases from University Hospitals in Leuven to explain why these cases were misclassified. We found that in most false negative cases, the patients were missed due to the lack of attestation of the claims or management of the patients procedures. These cases were most likely patients for which there was a decision to withhold treatment because of comorbid disease, older age, the prognosis of the recurrence, or patients treatments were reimbursed by the sponsor of a clinical trial.

Previously, algorithms based on administrative claims data to detect breast cancer recurrences at the population level have been established.5,710,12 For example research groups from the USA, Canada, and Sweden have built algorithms to detect recurrences in a delimited region within a population. Recent results from these groups have proven that machine learning algorithms based on administrative data can be used to detect recurrences, in the absence of systematic registration. These studies, however, only encompassed a few centers and were thus not validated in a larger cohort of a population. Moreover, most of these algorithms included complete metastasis-specific International Classification of Diseases (ICD)-codes to detect recurrences. Since metastasis-specific codes are not complete in our database, we were not able to use this code in our algorithm. Particularly, the Danish registry has actively collected recurrence information in the Danish Breast Cancer Group (DBCG) clinical database, which they were able to use to construct and validate population-based recurrence-algorithms to complete their recurrence database.10,11 Additionally, they were able to look into long-term recurrences beyond 10 years after incidence date.4,33

The objective of this study was to develop an algorithm that could be used on a nation-wide level to estimate population-wide distant recurrences. Compared with other studies, we used a large sample size and reported both internal and external validation, which was hardly reported in earlier studies.5 Another strength of our study was that unlike many other studies from the USA using Medicare claims,3438 we were able to include all eligible patients with a breast cancer diagnosis, and not just patients older than 65 years.

Although we used different diagnosis and treatment code sources, it should be noted that treatment regimens often change over time and adaptation of the features should be performed for later use. Adapting the algorithm based on changes in diagnosis or treatment regimens might be necessary to obtain accurate recurrence rates of more incidence years in the future. Ideally, we would also prefer to have long-term follow-up and claims data for patients to detect long-term recurrences. However, due to regulations and the large bulk of data that is generated, a longer follow-up of the codes was not possible within the current study. Longer follow-up of recurrences and administrative data would likely improve the accuracy and lead to a more robust algorithm.

In conclusion, our machine learning algorithm to detect metastatic breast cancer recurrences performed with high accuracy after external validation. Claims data are available for medical procedures and medications, hospital discharge data, vital status and cause of death data on the whole population level, which allows the development of models for Belgium. This substantiates the feasibility to develop and validate recurrence algorithms at the population level and might encourage other population-based registries to develop recurrence models or actively register recurrences in the future as these become progressively important. These rates are valuable to gain more insights about recurrences outside the clinical trial setting and might unveil the importance of active registration of recurrences.

AUC, Area under the curve; ATC, Anatomical Therapeutic Chemical classification; AVIQ, Agence pour une Vie de Qualit; BCR, Belgian Cancer Registry; CA15-3, Cancer antigen 15-3; CART, Classification and regression tree; CBSS, Crossroads Bank for Social Security; CT, Computed tomography; FN, False negatives; FP, False positives; ICD, International Classification of Diseases and Related Health Problems; IMA, InterMutualistic Agency; MDT, Multidisciplinary team meeting; MRI, Magnetic Resonance Imaging; MZG, Minimale Ziekenhuis Gegevens; NPV, Negative predictive value; PPV, Positive predictive value; PET-CT, Positron emission tomography computed tomography; SE, Standard error; SMN, Secondary malignant neoplasm; TN, True negatives; TP, True positives.

The data that support the findings of this study are available upon reasonable request. The data can be given within the secured environment of the Belgian Cancer Registry, according to its regulations, and only upon approval by the Information Security Committee.

This retrospective chart review study involving human participants was in accordance with the ethical standards of the institutional and national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. This study was approved by the Ethics Committee of University Hospitals Leuven (S60928). Informed consent for use of data of all participants was obtained.

All authors made a significant contribution to the work reported, whether that is in the conception, study design, execution, acquisition of data, analysis and interpretation, or in all these areas; took part in drafting, revising or critically reviewing the article; gave final approval of the version to be published; have agreed on the journal to which the article has been submitted; and agree to be accountable for all aspects of the work.

This work was supported by VZW THINK-PINK (Belgium).

The authors report no conflicts of interest in this work.

1. Gourgou-Bourgade S, Cameron D, Poortmans P, et al. Guidelines for time-to-event end point definitions in breast cancer trials: results of the DATECAN initiative (Definition for the Assessment of Time-to-event Endpoints in CANcer trials). Ann Oncol. 2015;26(5):873879. doi:10.1093/annonc/mdv106

2. Eisenhauer EA, Therasse P, Bogaerts J, et al. New response evaluation criteria in solid tumours: revised RECIST guideline (version 1.1). Eur J Cancer. 2009;45(2):228247. doi:10.1016/j.ejca.2008.10.026

3. Warren JL, Yabroff KR. Challenges and opportunities in measuring cancer recurrence in the United States. J Natl Cancer Inst. 2015;107:djv134djv134. doi:10.1093/jnci/djv134

4. Negoita S, Ramirez-Pena E. Prevention of late recurrence: an increasingly important target for breast cancer research and control. J Natl Cancer Inst. 2021. doi:10.1093/JNCI/DJAB203

5. Izci H, Tambuyzer T, Tuand K, et al. A systematic review of estimating breast cancer recurrence at the population level with administrative data. J Natl Cancer Inst. 2020;112:979988. doi:10.1093/jnci/djaa050

6. Ritzwoller DP, Hassett MJ, Uno H, et al. Development, validation, and dissemination of a breast cancer recurrence detection and timing informatics algorithm. J Natl Cancer Inst. 2018;110:273281. doi:10.1093/jnci/djx200

7. Amar T, Beatty JD, Fedorenko C, et al. Incorporating breast cancer recurrence events into population-based cancer registries using medical claims: cohort study. JMIR Cancer. 2020;6(2):110.

8. Cairncrossh ZF, Nelson G, Shack L, Metcalfe A. Validation in Alberta of an administrative data algorithm to identify cancer recurrence. Curr Oncol. 2020;27(3):e343e346. doi:10.3747/co.27.5861

9. Lambert P, Pitz M, Singh H, Decker K. Evaluation of algorithms using administrative health and structured electronic medical record data to determine breast and colorectal cancer recurrence in a Canadian province: using algorithms to determine breast and colorectal cancer recurrence. BMC Cancer. 2021;21(1):110. doi:10.1186/s12885-021-08526-9

10. Pedersen RN, ztrk B, Mellemkjr L, et al. Validation of an algorithm to ascertain late breast cancer recurrence using Danish medical registries. Clin Epidemiol. 2020;12:10831093. doi:10.2147/CLEP.S269962

11. Rasmussen LA, Jensen H, Virgilsen LF, et al. A validated algorithm for register-based identification of patients with recurrence of breast cancer-Based on Danish Breast Cancer Group (DBCG) data. CANCER Epidemiol. 2019;59:129134. doi:10.1016/j.canep.2019.01.016

12. Valachis A, Carlqvist P, Szilcz M, et al. Use of classifiers to optimise the identification and characterisation of metastatic breast cancer in a nationwide administrative registry. Acta Oncol. 2021;60(12):16041610. doi:10.1080/0284186X.2021.1979645

13. Steyerberg EW, Vergouwe Y. Towards better clinical prediction models: seven steps for development and an ABCD for validation. Eur Heart J. 2014;35:19251931. doi:10.1093/eurheartj/ehu207

14. Het Intermutualistisch Agentschap [The Intermutualistic Agency] (IMA) LAgence InterMutualiste (AIM). https://ima-aim.be/.

15. Technische Cel voor het beheer van de MZG-MFG data [Technical cel for management of MZG-MFG data]- La Cellule Technique pour la gestion des donnes RHM-RFM. https://tct.fgov.be/.

16. CBSS - Crossroads Bank for Social Security. Available from: https://www.ksz-bcss.fgov.be/nl/documents-list. Accessed April 28, 2023.

17. Agence pour une Vie de Qualit [Walloon Agency for quality of life] (AViQ). https://www.aviq.be/.

18. Smits N. A note on Youdens J and its cost ratio. BMC Med Res Methodol. 2010;10(1):14. doi:10.1186/1471-2288-10-89

19. Sutton CD. Classification and regression trees, bagging, and boosting. Handb Stat. 2005;24:303329.

20. Breiman L, Friedman JH, Olshen RA, Stone CJ. Classification and regression trees. Classif Regres Trees. 1984;20:1358.

21. Chen Y, Yang Y. The one standard error rule for model selection: does it work? Stats. 2021;4(4):868892. doi:10.3390/stats4040051

22. Enger SM, Soe ST, Buist DSM, et al. Breast cancer treatment of older women in integrated health care settings. J Clin Oncol. 2006;24(27):43774383. doi:10.1200/JCO.2006.06.3065

23. Han Y, Sui Z, Jia Y, et al. Metastasis patterns and prognosis in breast cancer patients aged 80 years: a SEER database analysis. J Cancer. 2021;12(21):6445. doi:10.7150/jca.63813

24. Xu Y, Kong S, Cheung WY, et al. Development and validation of case-finding algorithms for recurrence of breast cancer using routinely collected administrative data. BMC Cancer. 2019;19(1):110. doi:10.1186/s12885-019-5432-8

25. Chubak J, Onega T, Zhu W, et al. An electronic health record-based algorithm to ascertain the date of second breast cancer events. Med Care. 2017;55:e81e87. doi:10.1097/MLR.0000000000000352

26. Kroenke CH, Chubak J, Johnson L, et al. Enhancing breast cancer recurrence algorithms through selective use of medical record data. J Natl Cancer Inst. 2016;108. doi:10.1093/jnci/djv336

27. Cronin-Fenton D, Kjrsgaard A, Nrgaard M, et al. Breast cancer recurrence, bone metastases, and visceral metastases in women with stage II and III breast cancer in Denmark. Breast Cancer Res Treat. 2018;167(2):517528. doi:10.1007/s10549-017-4510-3

28. In H, Bilimoria KY, Stewart AK, et al. Cancer recurrence: an important but missing variable in national cancer registries. Ann Surg Oncol. 2014;21(5):15201529. doi:10.1245/s10434-014-3516-x

29. Nout RA, Fiets WE, Struikmans H, et al. The in- or exclusion of non-breast cancer related death and contralateral breast cancer significantly affects estimated outcome probability in early breast cancer. Breast Cancer Res Treat. 2008;109(3):567572. doi:10.1007/s10549-007-9681-x

30. Chubak J, Yu O, Pocobelli G, et al. Administrative data algorithms to identify second breast cancer events following early-stage invasive breast cancer. J Natl Cancer Inst. 2012;104(12):931940. doi:10.1093/jnci/djs233

31. Nordstrom B, Whyte J, Stolar M, Catherine Mercaldi JK, Kallich JD. Identification of metastatic cancer in claims data. Pharmacoepidemiology. 2012;21(2):2128. doi:10.1002/pds.3247

32. Nordstrom BL, Simeone JC, Malley KG, et al. Validation of claims algorithms for progression to metastatic cancer in patients with breast, non-small cell lung, and colorectal cancer. Pharmacoepidemiol Drug Saf. 2015;24(1, SI):511.

33. Pedersen RN, Ozt Rk Esen B, Mellemkjaer L, et al. The incidence of breast cancer recurrence 1032 years after primary diagnosis. J Natl Cancer Inst. 2021. doi:10.1093/JNCI/DJAB202

34. Lamont EB, Ii JEH, Weeks JC, et al. Measuring disease-free survival and cancer relapse using medicare claims from CALGB breast cancer trial participants (Companion to 9344). J Natl Cancer Inst. 2006;98(18):13351338. doi:10.1093/jnci/djj363

35. Chawla N, Yabroff KR, Mariotto A, et al. Limited validity of diagnosis codes in Medicare claims for identifying cancer metastases and inferring stage. Ann Epidemiol. 2014;24(9):666672.e2. doi:10.1016/j.annepidem.2014.06.099

36. Hassett MJ, Ritzwoller DP, Taback N, et al. Validating billing/encounter codes as indicators of lung, colorectal, breast, and prostate cancer recurrence using 2 large contemporary cohorts. Med Care. 2014;52(10):e65e73. doi:10.1097/MLR.0b013e318277eb6f

37. Sathiakumar N, Delzell E, Yun H, et al. Accuracy of medicare claim-based algorithm to detect breast, prostate, or lung cancer bone metastases. Med Care. 2017;55:e144e149. doi:10.1097/MLR.0000000000000539

38. McClish D, Penberthy L, Pugh A. Using Medicare claims to identify second primary cancers and recurrences in order to supplement a cancer registry. J Clin Epidemiol. 2003;56(8):760767. doi:10.1016/S0895-4356(03)00091-X

View original post here:
Machine Learning to Estimate Breast Cancer Recurrence | CLEP - Dove Medical Press

Manually evaluating transaction monitoring models is slow and error-prone, with mistakes resulting in potentially large fines. To avoid this, banks are increasingly turning to automated machine learning.

Regulators increasingly expect banks and financial institutions to be able to demonstrate the effectiveness of their transaction monitoring systems.

As part of this process, banks need to evaluate the models they use and verify (and document) that theyre up to the task. Institutions that fail to maintain a sufficiently effective anti-money laundering program arefrequently hit with huge fines, including several that have totaled over USD1 billion.

Lisa Monaco, the deputy attorney general at the US Department of Justice (DoJ) while announcing arecent fine for Danske Bank, said to expect companies to invest in robust compliance programs. Failure to do so may well be a one-way ticket to a multi-billion-dollar guilty plea.

Such threats are putting added pressure on smaller banks and FIs. While the larger institutions often will struggle less because of their army of data scientists, model validation and evaluation can be a burden for players with more limited resources.

What is a model?

In the US, banks commonly monitor transactions using a rule-based system of parameters and thresholds. Common rules detect the value of transactions over a period of time or an increase in the volume or value of transactions. If sufficient conditions are met, an alert is triggered.

Even in their simplest incarnation, regulators consider such systems to be models. According to supervisory guidanceOCC 2011-12, a model is defined as any quantitative approach that processes inputs and produces reports. In practice, a typical rule-based transaction monitoring system involves multiple layers of rules.

Regardless of complexity, banks must manage model risks appropriately. There are three main types of model risk that banks need to consider:

These are easy questions to ask, but answering them can be extremely challenging. The OCC supervisory guidance stipulates that banks should manage model risks just like any other type of risk, which includes critical analysis by objective, informed parties who can identify model limitations and assumptions and produce appropriate change.

This guidance states that banks should ensure their models are performing as expected, in line with their design objectives and business uses. It defines the key elements of an effective validation framework as:

Regulatory compliance

Regulators have continued to raise the bar as the US seeks to restrict access to sanctioned countries and individuals, as well as cracking down on financial crime in general.

Since 2018, the New York State Department of Financial Services has required boards or senior officers to submitan annual compliance finding that certifies the effectiveness of an institutions transaction monitoring and sanctions filtering programs.

Taking this a step further, the DoJ announced in 2022 that it was considering a requirement for chief executives and chief compliance officers to certify the design and implementation of their compliance program. With continued geopolitical tensions as the war in Ukraine drags on, the potential cost of a compliance failure is only going to increase.

The regulation of models comes under these broad requirements for effective risk controls. While the approach that banks take to evaluate models will vary on a case-by-case basis, the general principles apply equally.

Similarly, the frequency of model evaluation should be determined using a risk-based approach, typically prompted by any significant changes to the institutions risk profile, such as a merger or acquisition, or expansion into new products, services, customer types or geographic areas. However, regulators increasingly expect models to be evaluated as often as every 12-18 months.

Model evaluation challenges

Rule-based models are being asked to do much more as the nature and volume of financial transactions has evolved. As new threats have emerged, models have become more and more complex (though not more effective). Unfortunately, many are not up to the task.

In many cases, the model has become a confusing black box that few people within the institution understand. Over the years, changes to data feeds, scenario logic, system functions, and staffing can mean that documentation explaining how the model works is incomplete or inaccurate. All of this can make evaluation very difficult for smaller banks. A first-time assessment will almost certainly be time-consuming and costly, and possibly flawed.

However, the challenges are not going away. Changes in consumer behavior, which accelerated during the pandemic, are here to stay. Banks and FIs have digitized their operations, vastly increasing their range of online services and payment methods. Consumers are also showing greater willingness to switch to challenger banks with digital-first business models.

These changes have created more vulnerabilities. Competitive pressures are putting compliance budgets under pressure, while the expansion of online services has created more opportunities for AML failures. To keep up, FIs need to respond quickly and flexibly to new threats.

Better model evaluation with Automated Machine Learning

This process of model evaluation can be optimized using automated machine learning (AutoML). This allows models to be evaluated continuously (or on short cycles) with a standardized process, which leads to higher quality evaluations. By contrast, the manual approach is slow and very error prone.

AutoML models take huge sets of data, learn from the behaviors encoded in that data and reveal patterns that indicate evidence of money laundering. The rapidly changing landscape of AML regulations, in combination with the growing number of transactions and customers, leaves almost no room for a traditional manual project-by-project approach. That is why the industry is increasingly looking at a more disruptive approach:models that are trained with customers' good behavior. The results of this non-traditional method in combination with AutoML let banks adaptto the new reality and stay ahead of almost any new criminal pattern.

Continued here:
Securing weak spots in AML: Optimizing Model Evaluation with ... - Finextra

Presentation at The Eleventh International Conference on Learning Representations (ICLR) debuts new findings for end-to-end neural network Trojan removal techniques

LOS ALAMITOS, Calif., May 5, 2023 /PRNewswire/ -- Today, at the virtual Backdoor Attacks and Defenses in Machine Learning (BANDS) workshop during The Eleventh International Conference on Learning Representations (ICLR), participants in the IEEE Trojan Removal Competition presented their findings and success rates at effectively and efficiently mitigating the effects of neural trojans while maintaining high performance. Evaluated on clean accuracy, poisoned accuracy, and attack success rate, the competition's winning team from the Harbin Institute of Technology in Shenzhen, with set HZZQ Defense, formulated a highly effective solution, resulting in a 98.14% poisoned accuracy rate and only a 0.12% attack success rate. This group will be awarded the first-place prize of $5,000 USD.

"The IEEE Trojan Removal Competition is a fundamental solution to improve the trustworthy implementation of neural networks from implanted backdoors," said Prof. Meikang Qiu, chair of IEEE Smart Computing Special Technical Committee (SCSTC) and full professor of Beacom College of Computer and Cyber Science at Dakota State University, Madison, S.D., U.S.A. He also was named the distinguished contributor of IEEE Computer Society in 2021. "This competition's emphasis on Trojan Removal is vital because it encourages research and development efforts toward enhancing an underexplored but paramount issue."

In 2022, IEEE CS established its Emerging Technology Fund, and for the first time, awarded $25,000 USD to IEEE SCSTC for the "Annual Competition on Emerging Issues of Data Security and Privacy (EDISP)," which yielded the IEEE Trojan Removal Competition (TRC '22). The proposal offered a novel take on a cyber topic, because unlike most existing competitions that only focus on backdoor model detection, this competition encouraged participants to explore solutions that can enhance the security of neural networks. By developing general, effective, and efficient white box trojan removal techniques, participants have contributed to building trust in deep learning and artificial intelligence, especially for pre-trained models in the wild, which is crucial to protecting artificial intelligence from potential attacks.

With 1,706 valid submissions from 44 teams worldwide, six groups successfully developed techniques that achieved better results than the state-of-the-art baseline metrics published in top machine-learning venues. The benchmarks summarizing the models and attacks used during the competition are being released to enable additional research and evaluation.

"We're hoping that this benchmark provides diverse and easy access to model settings for people coming up with new AI security techniques," shared Yi Zeng, the competition chairof the IEEE TRC'22, research assistant atBradley Department of Electrical and Computer Engineering, Virginia Tech, Blacksburg, Va., U.S.A. "This competition has yielded new data sets consisting of trained poisoned pre-trained models that are of different architectures and trained on diverse kinds of data distributionswith really high attack success rates, and now developers can explore new defense methods and get rid of remaining vulnerabilities."

During the competition, collective participant results yielded two key findings:

These findings point to the fact that for the time being, a generalized approach to mitigating attacks on neural networks is not advisable. Zeng emphasized the urgent need for a comprehensive AI security solution: "As we continue to witness the widespread impact of pre-trained foundation models on our daily lives, ensuring the security of these systems becomes increasingly critical. We hope that the insights gleaned from this competition, coupled with the release of the benchmark, will galvanize the community to develop more robust and adaptable security measures for AI systems."

"As the world becomes more dependent on AI and machine learning, it is important to deal with the security and privacy issues that these technologies bring up," said Qiu. "The IEEE TRC '22 competition for EDISP has made a big difference in this area. I'd like to offer a special thanks to my colleagues on the steering committeeProfessors Ruoxi Jia from Virginia Tech, Neil Gong from Duke, Tianwei Zhang from Nanyang Technological University, Shu-Tao Xia from Tsinghua University, and Bo Li from University of Illinois Urbana-Champaignfor their help and support."

Ideas and insights coming out of the event, along with the public benchmark data, will help make the future of machine learning and artificial intelligence safer and more dependable. The team plans to run the competition for a second year, and those findings will further strengthen the security parameters of neural networks.

"This is precisely the kind of work we want the Emerging Technology Fund to fuel," said Nita Patel, 2023 IEEE Computer Society President. "It goes a long way toward bolstering iterative developments that will strengthen the security of machine learning and AI platforms as the technologies advance."

For more information about the Emerging Technology Grants Program overall, visit https://www.computer.org/communities/emerging-technology-fund.

About IEEE Trojan Removal CompetitionThe IEEE TRC'22 aims to encourage the development of innovative end-to-end neural network backdoor removal techniques to counter backdoor attacks. For more information, visit https://www.trojan-removal.com/.

About IEEE Computer SocietyThe IEEE Computer Society is the world's home for computer science, engineering, and technology. A global leader in providing access to computer science research, analysis, and information, the IEEE Computer Society offers a comprehensive array of unmatched products, services, and opportunities for individuals at all stages of their professional careers. Known as the premier organization that empowers the people who drive technology, the IEEE Computer Society offers international conferences, peer-reviewed publications, a unique digital library, and training programs. Visit computer.org for more information.

SOURCE IEEE Computer Society

Read the original post:
IEEE Computer Society Emerging Technology Fund Recipient ... - Benzinga

Services like Google Translate can help millions of people communicate in over 100 languages. Users can type or speak words to be translated, or even translate text in photos and videos using augmented reality.

Now, computer science professor Andrea Salgian and Ben Guerrieri 26 are working to add one more language to the list: American Sign Language.

Using computer vision and machine learning, the researchers are setting out to create a program to serve as a Google Translate tool for ASL speakers to sign to the camera and receive a direct translation.

Right now, were looking at recognizing letters and words that have static gestures, Salgian said, referring to letters in the ASL alphabet with no hand movement. The program will act more like a dictionary at first. The pair will then develop the automated translation, she explained.

Salgians research utilizes a free machine-learning framework called Mediapipe, which is developed by Google and uses a camera to detect joint locations in real time. The program tracks the users movements, provides the coordinates of every single joint in the hand, and uses the coordinates to extract gestures that are matched to ASL signs.

Computer science major Ben Guerrieri 26 discovered Salgians project shortly after arriving at TCNJ and is now working alongside her in this AI research.

Its such a hands-on thing for me to do, he said of his contribution to the project, which consists of researching and developing the translator algorithms. We get to incrementally develop algorithms that have super fascinating real-time results.

This project is part of Salgians on-going interest and research into visual gesture recognition that also includes applications to musical conducting and exercising.

ASL is a fascinating application, especially looking at the accessibility aspect of it, Salgian said. To make communication possible for those who dont speak ASL but would love to understand would mean so much, Salgian said.

Kaitlyn Bonomo 23

See the original post here:
Computer science research team explores how machine learning ... - The College of New Jersey News

Its happening already. Following Dropboxs move to lay off 500 employees as it shifts its focus to AI, IBM now plans to replace 7,800 jobs with AI technology and pause hiring for roles that could be automated. Company CEO Arvind Krishna stated that most back-office positions, such as HR and accounting, will be replaced.

Layoffs due to AI were inevitable, but amid lingering job losses, new jobs are also being created. A report by the World Economic Forum states that demand for AI and machine learning specialists will grow at the fastest rate in the next five years. The organisation has also listed a number of AI jobs that are expected to see massive growth in the coming years. Lets take a look at them.

AI and machine learning specialists: These are professionals who design, develop, and implement AI and ML systems and applications. They use various tools and techniques to analyse data, build models, and optimise algorithms. The demand for AI and machine learning specialists will grow at the fastest rate in the next five years, the WEF report says.

Big data specialists: They specialise in managing, analysing and interpreting large and complex data sets. They use cutting-edge technologies to organise, store, and retrieve vast amounts of information, turning it into valuable insights that can drive business decisions. They work with a variety of industries such as healthcare, finance, and technology, to help them understand and leverage the power of data.

Data engineers: They are responsible for the design, construction and maintenance of the data infrastructure that supports an organisations data management and analytics needs. They develop and manage data pipelines, work with large datasets, and ensure that data is available and accessible to those who need it. They also work with other data professionals to design and implement data architectures that meet the needs of the organisation.

Data analysts and scientists: These are experts who collect, process, and interpret large and complex datasets to generate insights and solutions for various problems and domains. They use statistical methods, programming languages, and visualisation tools to manipulate and communicate data. Data analysts and scientists are expected to see a 32% growth in demand by 2023.

Apart from the aforementioned jobs listed by the World Economic Forum, heres a list of other jobs AI is expected to create in the near future.

AI trainers: They are responsible for teaching machines to learn from data effectively. They also help to ensure that the AI models accurately interpret the data, providing businesses with valuable insights that can drive informed decisions.

AI ethicists: They use their expertise to ensure that AI systems are developed and deployed responsibly. They also identify potential ethical concerns related to privacy, fairness, and transparency, and work to address them through policy and guidelines.

AI user experience designers: They create interfaces and experiences that are intuitive and user-friendly for AI-driven products and services. They also work to ensure that users can easily interact with AI systems, making their experiences more enjoyable and productive.

AI security analysts: They focus on ensuring the safety and integrity of AI-driven solutions. They also identify potential threats, vulnerabilities, and attacks that could compromise AI systems and develop strategies to mitigate them.

Robotics engineers: They design, build, and program autonomous machines that can perform a wide range of tasks, from assembly line work to surgical procedures. By incorporating AI capabilities such as computer vision and natural language processing, they create intelligent machines that can work alongside humans in new and exciting ways.

Of course, these are just a few examples of the new jobs that AI is expected to create. As AI continues to evolve and become more integrated into various industries, its likely that even more new job opportunities will emerge.

IE Online Media Services Pvt Ltd

First published on: 03-05-2023 at 19:39 IST

Continue reading here:
From machine learning to robotics: WEF report predicts the most lucrative AI jobs - The Indian Express

GuiltyTargets-COVID-19 web tool

We start by providing a high level overview about the capabilities of the GuiltyTargets-COVID-19 web tool. The web application initially allows the user to browse through a ranked list of potential targets generated using six bulk RNA-Seq and three single cell RNA-Seq datasets applied to a lung specific proteinprotein interaction (PPI) network reconstruction. Our website is also equipped with several filtering options to allow the user to quickly obtain the most relevant results. The candidate targets were ranked using a machine learning algorithm, GuiltyTargets19, which aims to quantify the degree of similarity of a candidate target to other known (candidate) drug targets. Further details about GuiltyTargets are outlined in the Methods section of this paper.

The user can retrieve a consensus ranking of any combination of datasets desired (Fig. 1). For each protein listed, its level of differential gene expression (upregulated, downregulated, no differential expressed) is displayed using a color coding system in addition to its association with COVID-19 as described in the literature. This latter feature is accomplished using an automated web search of scientific articles from PubMed that mention the protein in combination with COVID-19.

Though we provide nine different RNA-Seq datasets to explore, our tool also allows one to upload their own gene expression data. Uploaded data is sent through the GuiltyTargets algorithm and, after a short period of time, a ranking of candidate proteins is made available to the user to download and explore.

To further elucidate their linkage to known disease mechanisms, GuiltyTargets-COVID-19 enables one to explore the neighborhood of any given candidate target within the lung tissue specific PPI network reconstruction (Fig. 2). The network is labeled with information about known disease associations in humans in addition to virus-host interactions.

Importantly, in order to present the user with a list of possible drug candidates for a given protein, we parsed the ChEMBL database to generate a mapping of known ligands for each of the prioritized proteins and included this information in our web application. Direct links to the ligands description pages were added to GuiltyTargets-COVID-19 so that researchers can quickly explore the each compounds profile.

To point out potential target related safety issues, GuiltyTargets-COVID-19 includes a list of adverse effects for each target-linked compound, all of which were derived from the NSIDES database20. By making this information readily available, the user can quickly decide which compounds for a given target are most viable.

Altogether, GuiltyTargets-COVID-19 implements a comprehensive workflow involving computational target prioritization supplemented with annotations from several key databases.

Screenshot of the GuiltyTargets-COVID-19 web application available at https://guiltytargets-covid.eu/.

In the following sections, we demonstrate the utility of GuiltyTargets-COVID-19 based on the analysis of 6 bulk RNA-Seq and 3 single cell RNA-Seq datasets. A detailed overview of the data and workflow can be found in the Differential gene expression section of the Methods. In brief, GuiltyTargets-COVID-19 maps differentially expressed genes in each of these datasets to a lung tissue specific, genome-wide PPI network, which was constructed using data from BioGRID21, IntAct22 and STRING23 (see PPI Network Construction in Methods). Users can choose a combination of these datasets and the tool will present a ranking of each protein for each selected dataset based on its similarity to known drug targets. Additionally, a consensus ranking is also calculated if multiple datasets were selected.

For our analysis, we initially performed a ranking for each individual dataset. This ranking was performed using the GuiltyTargets positive-unlabeled machine learning algorithm19, which combines a PPI network, a differential gene expression (DGE) dataset, and a list of included nodes that are labeled as putative targets. Based on these results, GuiltyTargets then quantifies the probability that a candidate protein could be labeled as a target as well. In order to create a usable model, GuiltyTargets-COVID-19 was trained using a set of 218 proteins targeted by small compounds extracted from ChEMBL. This set of proteins was previously found to be involved in cellular response mechanisms specific to COVID-19 that have been shown to be transcriptionally dysregulated in several bulk RNA-Seq datasets15. The set of 218 proteins may thus be regarded as an extendable set of candidate targets. We chose this approach as there are currently very few approved drugs for COVID-19 (7 as of December 2022 in the European Union), hence making a machine learning model based ranking with respect to only known targets of approved drugs rather questionable.

In order to maximize transparency, GuiltyTargets-COVID-19 also reports the ranking performance of the GuiltyTargets machine learning algorithm that is calculated using the cross-validated area under receiver operator characteristic curve (AUC). As show in Fig. 6, the cross-validated AUCs found for each of the nine datasets used in this work were found to be between 85% and 90%, which align with the results reported in19. Additional details regarding the algorithms performance can be found in the Methods Section.

First degree neighbors of the (a) AKT3 and (b) PIK3CA proteins. Nodes are colored according to their associations: light orange means no virus or human association was found, dark orange indicates only human association, purple signifies viral association, and and dark blue nodes are proteins with associations to both viral mechanisms and human processes. The neighboring proteins and their associations for AKT3 and PIK3CA are outlined in Supplementary Data S1 and S2, respectively.

For our use case, we focused on proteins with a predicted target likelihood higher than 85% in each of the nine datasets. This resulted in 5167 candidate targets for each of the bulk RNA-Seq datasets and 4565 candidate targets for each of the scRNA-Seq datasets. By enabling the filter option novel in our web tool, we can select for those prioritized targets that are not among the original set of 218 proteins labeled as known targets and used for training the model.

Among these prioritized targets, there was a considerable difference between the analyzed bulk RNA-Seq data, with only a single protein target appearing among the top candidates for all 6 datasets: AKT3 (Fig. 3). AKT3 is of great interest in COVID-19 research as the PI3K/AKT signaling pathway plays a central role in cell survival. Moreover, researchers have observed an association between this pathway and coagulopathies in SARS-CoV-2 infected patients24. It has been suggested that the PI3K/AKT signaling pathway can be over-activated in COVID-19 patients either by direct or indirect mechanisms, thus suggesting this pathway may serve as a potential therapeutic target25.

To better understand the relationship of AKT3 with known COVID-19 disease mechanisms, the user can also download a CSV file comprised of the direct (first-degree) neighbors of AKT3 in the lung tissue specific PPI network used for our analysis. Each first-degree neighbor is additionally annotated to indicate whether the corresponding protein is associated with either the disease or with the virus itself. Figure 2a provides a visualization of the AKT3 neighbor network generated using Cytoscape 3.9.126.

Interestingly, a larger number of shared prioritized protein targets can be found among the scRNA-Seq data. Based on the 17 cell types identified in the three datasets, four common target candidates were identified: AKT2, AKT3, MAPK11, and MLKL. The presence of AKT3, as well as its isoform AKT2, in our list of prioritized targets supports the predicted association of the PI3K/AKT signaling pathway with COVID-19 as observed in our analysis of the bulk RNA datasets. Interestingly, our analysis of the single-cell datasets revealed two additional proteins of interest, MAPK11 and MLKL. MAPK11 is targeted by the compound losmapimod, which was tested against COVID-19 in a (terminated) phase III clinical trial (NCT04511819). The trial ended in August 2021 due to the rapidly evolving environment for the treatment of Covid-19 and ongoing challenges to identify and enroll qualified patients to participate (https://clinicaltrials.gov/ct2/show/NCT04511819). MLKL is a pseudokinase that plays a key role in TNF-induced necroptosis, a programmed cell death process. Recent evidence suggests that it can become dysregulated by the inflammatory response due to SARS-CoV-2 infection27. According to the DGldb database28 (which is cross-referenced by GuiltyTargets-COVID-19), the protein is also druggable and thus may serve as a therapeutic target.

Overall, these results demonstrate that GuiltyTargets-COVID-19 has the capability of identifying candidate targets with a clear disease association as well as assessing their potential druggability.

Venn diagram of the number of prioritized targets from the bulk RNA-Seq datasets.

After analyzing the top ranked protein targets shared by each group of RNA-Seq data, we next sought to characterize those candidates found in unique cell types (Table 1). Interestingly, we found that PIK3CA was only ranked among the top therapeutic candidates in goblet cells. Goblet cells are modified epithelial cells that secrete mucus on the surface of mucous membranes of organs, particularly those of the lower digestive tract and airways. Dactolisib is a compound targeting PIK3CA that has been tested in a phase II clinical trial for its ability to reduce COVID-19 disease severity (NCT04409327). The trial was terminated due to an insufficient accrual rate (https://clinicaltrials.gov/ct2/show/NCT04409327). Figure 2b depicts the PIK3CA protein and its first-degree neighbors as defined by the PPI network used in the GuiltyTargets-COVID-19 algorithm.

Another interesting drug we identified during our analysis is the compound varespladib, a compound that is currently being tested in a phase II clinical trial (NCT04969991) and which targets PLA2G2A, a potential protein target that primarily affects NKT cells (Table 1). To better support the user in finding more information about the disease context of such candidate targets, GuiltyTargets-COVID-19 also includes links to PubMed articles in which the protein and its roles in COVID-19 are discussed. Identification of relevant articles is discussed in the the Methods section.

Altogether, these results demonstrate that the tool presented here can be used for cell type specific target prioritization as well as aiding in characterizing the proteins in the context of COVID-19.

GuiltyTargets-COVID-19 also includes a feature for identifying small compound ligands from the ChEMBL database with reported activity (pChEMBL > 5) against candidate targets. In our use case, we were able to identify 186 ligands for AKT3, the top prioritized target across bulk RNA-Seq datasets. Furthermore, 126 ligands were mapped to the four candidate targets that were found among all single cell RNA-Seq datasets. A complete report of the number of ligands mapped to protein targets unique for a given cell type can be found in Table 2. We observed a high imbalance of mapped ligands for different cell types with secretory cells being targeted by the vast majority of compounds.

In total, these results demonstrate the ability of GuiltyTargets-COVID-19 to efficiently identify active ligands against candidate targets, thus supporting researchers in rapidly identifying potential new drugs for therapeutic intervention or repurposing.

An important factor that must be taken into consideration with new target candidates are the adverse events which are associated with the drugs targeting these proteins. To better assess the suggested therapeutics, we mapped significant adverse effects from the NSIDES database (http://tatonettilab.org/offsides) to the extracted ChEMBL compounds. Hence, each protein can be visualized in tandem with the ligands that target it, as well as any side effects found to be associated with the linked compounds. To showcase this feature, Fig. 4 depicts the AKT3 protein as well as its associated ligands and their side effects as shown in the GuiltyTargets-COVID-19 web application.

Screenshot of part of the adverse effect network for the AKT3 protein.

Read the rest here:
A machine learning method for the identification and ... - Nature.com

Credit providers have grappled with fraudsters since long before mobile banking. In a modern landscape, financial services businesses dedicate ample resources to thwart fraud attempts.

As fraudulent actors get smarter, machine learning can help companies stay one step ahead. But first, organizations need access to those tools.

Capital One is democratizing access to ML tools, encouraging workers to contribute to a common shared ecosystem to provide practitioners with easy access to ML and spur innovation. In the process, Capital One found opportunities for cross-unit collaboration and improved how the company detects fraud.

"The future is here," said Zach Hanif, vice president and head of enterprise machine learning models and platforms at Capital One. "But, historically, it hasn't always been distributed evenly."

ML tools keep humans focused on the tasks that require their attention, prioritizing resources through technology. Artificial intelligence capabilities are finding a role in financial services in particular.

Four in five companies in the sector have up to five AI use cases at work in their organization, according to an NVIDIA reportpublished in February. Nearly one-quarter are using AI to help detect fraud.

Hanif's team worked alongside the card fraud division to build homegrown and open-source ML algorithms and technologies. With ML tools, the company can quickly determine whether a transaction is benign or if it needs further investigation because of potential fraud.

"We were able to get these teams on the same stack and focused on collaboration, which made sure that we were able to bring down some silos," Hanif said. "We were able to prioritize the development of reusable components so when one team would build a component of their pipeline, other teams were able to immediately begin leveraging it and save themselves the time of that initial development."

Machine learning gives the company a way to quickly determine whether something needs to be investigated, Hanifsaid.

Picking a technology and spreading it throughout the organization isn't a turnkey task.

There are several barriers to easing access to ML throughout any organization, according to Arun Chandrasekaran, distinguished VP analyst at Gartner.

The top barriers are security and privacy concerns and the black-box nature of AI systems, as well as the absence of internal AI know-how, AI governance tools and self-service AI and data platforms, Chandrasekaran told CIO Dive in an email.

Despite the advancement of AI tools in the enterprise, activities associated with data and analytics including preparation, transformation, pattern identification, model development and sharing insights with others are still done manually at many organizations.

"Demands for more data-driven and analytics-enabled decision making, and the friction and technical hurdles of this workflow, limit widespread user adoption and achieving better business outcomes," Chandrasekaran said.

But changing how companies operate is a human problem as much as it is a technical one. Cultural factors can determine whether or not a company succeeds at democratizing the use of a technology tool such as ML.

"To be able to drive change across a large organization, you're trying to make a cultural alteration," Hanif said.

Leaders need to encourage employees to imagine what they can do with specific tools, he said. With that mindset, fear of change falls away and employees begin to think about how a new technology can be contextualized within the existing problem space.

"Standardizing a platform allows everyone to have a common operating environment and runbook," Hanif said. "That way, they can start and engage in that process in a standard, well-understood way. That makes so many different things inside of the organization go smoother, go faster, and reduce the overall risk."

View post:
How Capital One is democratizing machine learning to curb fraud - Banking Dive

May 3, 2023 by Diana Ambolis

123

Non-fungible tokens (NFTs) have exploded in popularity over the past year, with many investors seeking to capitalize on this emerging market. However, with NFT values often fluctuating rapidly, it can be difficult for investors to know when to buy or sell. Machine learning offers a potential solution to this problem, providing investors with insights and

Non-fungible tokens (NFTs) have exploded in popularity over the past year, with many investors seeking to capitalize on this emerging market. However, with NFT values often fluctuating rapidly, it can be difficult for investors to know when to buy or sell. Machine learning offers a potential solution to this problem, providing investors with insights and predictive models that can help inform investment decisions and maximize returns.

Machine learning algorithms can be trained to analyze a range of data points and variables that are relevant to NFT value. This could include factors such as the artists reputation, the rarity of the NFT, the size of the NFT market, and even social media sentiment around a particular NFT. By analyzing this data, machine learning algorithms can identify patterns and correlations that can be used to predict the future value of a given NFT.

Determining the true value of an NFT can be challenging, with many factors to consider, including the artists reputation, the rarity of the NFT, and social media sentiment around a particular NFT. Machine learning offers a potential solution to this problem, providing investors with insights and predictive models that can help determine the value of NFTs. In this article, well explore the top 10 benefits of using machine learning to determine NFT value.

Machine learning offers a range of benefits for investors seeking to determine NFT value. By providing accurate predictions, improving efficiency, and reducing bias, machine learning can help investors make more informed decisions about NFT investments. As the NFT market continues to evolve, it is likely that machine learning will become an increasingly important tool for investors seeking to capitalize on this emerging market.

Also, read The Top 5 Best NFT Products So Far: A Closer Look

One of the key benefits of using machine learning for NFT investment is that it can help investors make more informed decisions about which NFTs to buy or sell. By providing insights and predictions about future value, machine learning algorithms can help investors identify undervalued NFTs that have strong potential for growth, as well as overvalued NFTs that may be at risk of declining in value.

Another benefit of using machine learning for NFT investment is that it can help investors manage risk. By providing predictive models and insights, machine learning algorithms can help investors understand the potential risks and rewards associated with a given NFT investment, allowing them to make more informed decisions about how to allocate their resources.

There are also potential drawbacks to using machine learning for NFT investment. For example, the accuracy of predictive models can be influenced by a range of factors, including the quality and quantity of data used to train the algorithm. In addition, the NFT market is still relatively new and untested, making it difficult to predict how the market will behave over time.

Despite these potential drawbacks, many investors are turning to machine learning as a way to inform their NFT investment decisions. As the NFT market continues to grow and evolve, machine learning is likely to become an increasingly important tool for investors seeking to capitalize on this emerging market.

Machine learning has the potential to revolutionize the world of NFT investment, providing investors with new insights and predictive models that can inform investment decisions and maximize returns. By analyzing a range of data points and variables, machine learning algorithms can identify patterns and correlations that can be used to predict NFT value and manage risk. While there are potential drawbacks to using machine learning in this context, the benefits are significant, and it is likely that this technology will become an increasingly important tool for investors seeking to capitalize on the emerging NFT market.

Visit link:
Machine Learning And NFT Investment: Predicting NFT Value And ... - Blockchain Magazine

In episode 105 of the AI/Hyperautomation Minute, Toni Witt provides clarity behind generative AI, its underlying technology the GPT (generative pre-trained transformer) machine learning model and how its evolving.

This episode is sponsored by Acceleration Economys Generative AI Digital Summit, taking place on May 25. Registration for the event, which features practitioner and platform insights on how solutions such as ChatGPT will impact the future of work, customer experience, data strategy, cybersecurity, and more, is free. To reserve your spot, sign up today.

00:26 While there are many conversations about generative AI, those outside of the tech field may still have a misunderstanding of the underlying technology and how its evolving.

01:03 Toni clarifies that ChatGPT is an web-based tool that gives access to GPT-3, which is the underlying machine learning model. GPT-3 is a word predictor. Its a form of deep learning with capabilities that are essentially a subset of what machine learning and AI can do.

01:37 Machine learning started with prediction and classification. Most AI applications that give returns to companies are these classification or predictor models, Toni explains. The Netflix recommender algorithm is an example of this, as it uses data from previous movies and shows that youve liked in the past to recommend what to watch next.

02:12 GPT-3 is a transformer model. Theres a pretty big debate going on whether these transformer models are going to be the ones that reach what you might call AGI, or artificial general intelligence, that basically matches the intelligence level of a human, Toni says.

02:57 Sam Altman, CEO of OpenAI, pointed out a trend that there will be base-level models. The GPT series is already an indication that models will help train other models. Think of it like a tech stack, says Toni.

Looking for real-world insights into artificial intelligence and hyperautomation? Subscribe to the AI and Hyperautomation channel:

See the rest here:
How the GPT Machine Learning Model Advances Generative AI - Acceleration Economy

We want to have the results within a week, so that we can really accelerate decision-making for climate scientists, said Yu, who is an assistant professor in the Department of Computer Science and Engineering at the Jacobs School of Engineering and the Halcolu Data Science Institute.

Ambitious? Yes. But thats where artificial intelligence comes in. Thanks to a $3.6 million grant awarded in 2021 by the Department of Energy, Yu and two UC San Diego colleagues, Yian Ma and Lawrence Saul, have teamed up with researchers at Columbia University and UC Irvine to develop new machine learning methods that can speed up these climate models, better predict the future, and improve our understanding of climate extremes.

This work comes at a crucial time, as it becomes increasingly important that we develop an accurate understanding of how climate change is impacting our Earth, our communities and our daily livesand how to use that newfound knowledge to inform climate action. To date, the team has published more than 20 papers in both machine learning and climate science-related journals as they continue to push the boundaries of science and engineering on this highly consequential front.

To increase the accuracy of predictionsand quantify their inherent uncertaintythe team is working on customizing algorithms to embed physical laws and first principles into deep learning models, a form of machine learning that essentially imitates the function of the human brain. Its no small task, but its given them the opportunity to collaborate closely with climate scientists who are putting these machine learning methods into practical algorithms in climate modeling.

Because of this grant, we have established new connections and new collaborations to expand the impact of AI methods to climate science, said Yu. We started working on algorithms and models with the application of climate in mind, and now we can really work closely with climate scientists to validate our models.

More here:
The AI Revolution is Upon UsAnd UC San Diego Researchers Are ... - University of California San Diego

10 best ways to earn money through machine learning in 2023 are enlisted in this article

10 best ways to earn money through machine learning in 2023 take advantage of the early lifespan and its adoption may then leverage this into other apps.

Land Gigs with FlexJobs: FlexJobs is one of the top freelance websites for finding high-quality employment from actual businesses. Whether you are a machine learning novice or a specialist, you may begin communicating with clients to monetize your skills by working on freelancing projects.

Become a Freelancer or List your Company to Hire a Team on Toptal: Toptal is similar to FlexJobs in that it is reserved for top freelancers and top firms wanting to recruit freelance machine learning programmers. This is evident in the hourly pricing given on the site as well as the caliber of the programmers.

Develop a Simple AI App: Creating an app is another excellent approach to generating money using machine learning. You may design a subscription app in which users can pay to access certain premium features. Subscription applications are expected to earn at least 50% more money than other apps with various sorts of in-app sales.

Become an ML Educational Content Creator: You can make money with machine learning online right now if you start teaching people about machine learning and its benefits. To publish and sell your course, use online platforms that provide teaching platforms, such as Udemy and Coursera.

Create and Publish an Online ML Book: You may create a book to provide extraordinary insights on the power of 3D printing, robots, AI, synthetic biology, networks, and sensors. Online book publication is now feasible because of systems such as Kindle Direct Publication, which provides a free publishing service.

Sell Artificial Intelligence Devices: Another profitable enterprise to consider is selling GPS gadgets to automobile owners. GPS navigation services can aid with traffic forecasting. As a result, it can assist car users in saving money if they choose a different route to work. Based on everyday experiences, you may estimate the places likely to be congested with access to the current traffic condition.

Generate Vast Artificial Intelligence Data for Cash: Because machine learning can aid in the generation of massive amounts of data, you can benefit from providing AI solutions to various businesses. AI systems function similarly to humans and have a wide range of auditory and visual experiences. An AI system may learn new things and be motivated by dynamic data and movies.

Create a Product or a Service: AI chatbots are goldmines and a great method to generate money with machine learning. Creating chatbot frameworks for mobile phones in the back endand machine learning engines in the front end is an excellent way to make money quickly. Making services like sentiment analysis or Google Vision where the firm or user may pay after making numerous queries per month is another excellent approach to gaining money using ML.

Participate in ML Challenges: You may earn money using machine learning by participating in and winning ML contests, in addition to teaching it. If you are a guru or have amassed a wealth of knowledge on this subject, you may compete against other real-world machine-learning specialists in tournaments.

Create and License a Machine Learning Tech: If you can develop an AI technology and license it, you can generate money by selling your rights to someone else. As the licensor, you must sign a contract allowing another party, the licensee, to use, re-use, alter, or re-sell it for cash, compensation, or consideration.

Excerpt from:
10 Best Ways to Earn Money Through Machine Learning in 2023 - Analytics Insight

Endler, J. A. Signals, signal conditions, and the direction of evolution. Am. Nat. 139, S125S153 (1992).

Article Google Scholar

Romer, H. Ecological constraints for the evolution of hearing and sound communication in insects. Evol. Biol. Hear. 7993 (1992).

Slabbekoorn, H. & Smith, T. B. Bird song, ecology and speciation. Philos. Trans. R. Soc. B 357, 493503 (2002).

Article Google Scholar

Mendelson, T. C. & Shaw, K. L. Sexual behaviour: Rapid speciation in an arthropod. Nature 433, 375376. https://doi.org/10.1038/433375a (2005).

Article ADS CAS PubMed Google Scholar

Bradbury, J. W. & Vehrencamp, S. L. Principles of Animal Communication (Sinauer Associates, 2011).

Google Scholar

Mason, N. A. et al. Song evolution, speciation, and vocal learning in passerine birds. Evolution 71, 786796 (2016).

Article Google Scholar

Chen, Z. & Wiens, J. J. The origins of acoustic communication in vertebrates. Nat. Commun. 11, 369. https://doi.org/10.1038/s41467-020-14356-3 (2020).

Article ADS CAS PubMed PubMed Central Google Scholar

Beecher, M. D. & Brenowitz, E. A. Functional aspects of song learning in songbirds. Trends Ecol. Evol. 20, 143149 (2005).

Article PubMed Google Scholar

Catchpole, C. K. & Slater, P. J. B. Bird Song: Biological Themes and Variations 2nd edn. (Cambridge University Press, 2008).

Book Google Scholar

Seddon, N., Merrill, R. M. & Tobias, J. A. Sexually selected traits predict patterns of species richness in a diverse clade of suboscine birds. Am. Nat. 171, 620631. https://doi.org/10.1086/587071 (2008).

Article PubMed Google Scholar

Riebel, K. Song and female mate choices in zebra finchesA review. Adv. Stud. Behav. 40, 197238 (2009).

Article Google Scholar

Ackermann, H., Hage, S. R. & Ziegler, W. Brain mechanisms of acoustic communication in humans and nonhuman primates: An evolutionary perspective. Behav. Brain Sci. 37, 529546. https://doi.org/10.1017/S0140525X13003099 (2014).

Article PubMed Google Scholar

Woolley, S. M. N. & Moore, J. M. Coevolution in communication senders and receivers: Vocal behavior and auditory processing in multiple songbird species. Ann. N. Y. Acad. Sci. 1225, 155165 (2011).

Article ADS PubMed PubMed Central Google Scholar

Wilkins, M. R., Seddon, N. & Safran, R. J. Evolutionary divergence in acoustic signals: Causes and consequences. Trends Ecol. Evol. 28, 156166. https://doi.org/10.1016/j.tree.2012.10.002 (2013).

Article PubMed Google Scholar

Podos, J. & Patek, S. N. Acoustic signal evolution: biomechanics, size, and performance. Anim. Signal. Funct. Integr. Approach 175203 (2015).

Demery, A.-J.C., Burns, K. J. & Mason, N. A. Bill size, bill shape, and body size constrain bird song evolution on a macroevolutionary scale. Ornithology 138, 111 (2021).

Article Google Scholar

Simmons, A. M. Acoustic Communication (eds Simmons, A. M. et al.) 114 (Springer New York, 2003).

Searcy, W. A. & Nowicki, S. The Evolution of Animal Communication: Reliability and Deception in Signaling Systems (Princeton University Press, 2005).

Google Scholar

Dooling, R. J., Brown, S. D., Klump, G. M. & Okanoya, K. Auditory perception of conspecific and heterospecific vocalizations in birds: Evidence for special processes. J. Comp. Psychol. 106, 2028 (1992).

Article CAS PubMed Google Scholar

Hauber, M. E., Campbell, D. L. & Woolley, S. M. N. The functional role and female perception of male song in Zebra Finches. Emu-Austral. Ornithol. 110, 209218 (2010).

Article Google Scholar

Lynch, A. Ecology and Evolution of Acoustic Communication in Birds (eds Kroodsma, D. E. & Miller, E. H.) 181197 (Comstock Publishing Associates, 1996).

Catchpole, C. K. Acoustic Communication in Birds Vol. 1 (eds Kroodsma, D. E. & Miller, E. H.) 297319 (Academic Press, 1982).

Wiley, R. H. & Richards, D. G. Physical constraints on acoustic communication in the atmosphere: Implications for the evolution of animal vocalizations. Behav. Ecol. Sociobiol. 3, 6994 (1978).

Article Google Scholar

Pckert, M., Martens, J., Kosuch, J., Nazarenko, A. A. & Veith, M. Phylogenetic signal in the song of crests and kinglets (Aves: Regulus). Evolution 57, 616629 (2003).

PubMed Google Scholar

Arato, J. & Fitch, W. T. Phylogenetic signal in the vocalizations of vocal learning and vocal non-learning birds. Philos. Trans. R. Soc. B 376, 20200241 (2021).

Article CAS Google Scholar

Podos, J. Correlated evolution of morphology and vocal signal structure in Darwins finches. Nature 409, 185188. https://doi.org/10.1038/35051570 (2001).

Article ADS CAS PubMed Google Scholar

Grant, B. R. & Grant, P. R. Cultural inheritance of song and its role in the evolution of Darwins finches. Evolution 50, 24712487 (1996).

Article PubMed Google Scholar

Payne, R. B. Handbook of the Birds of the World vol. 15 (eds del Hoyo, A. E. J. & Christie, D. A.) 234377 (Lynx Edicions, 2010).

Payne, R. B., Payne, L. L., Woods, J. L. & Sorenson, M. D. Imprinting and the origin of parasitehost species associations in brood-parasitic indigobirds, Vidua chalybeata. Anim. Behav. 59, 6981 (2000).

Article CAS PubMed Google Scholar

Zann, R. A. Variation in the songs of three species of estrildine grassfinches. Emu-Austral. Ornithol. 76, 97108 (1976).

Article Google Scholar

Moore, J. M. & Woolley, S. M. N. Emergent tuning for learned vocalizations in auditory cortex. Nat. Neurosci. 22, 14691476 (2019).

Article CAS PubMed PubMed Central Google Scholar

Zann, R. A. The Zebra Finch: A Synthesis of Field and Laboratory Studies (Oxford University Press, 1996).

Google Scholar

Payne, R. B. & Payne, L. L. Song mimicry and species associations of west African indigobirds Vidua with Quail-finch Ortygospiza atricollis, Goldbreast Amandava subflava and Brown Twinspot Clytospiza monteiri. Ibis 136, 291304 (1994).

Article Google Scholar

Hosino, T. & Okanoya, K. Lesion of a higher-order song nucleus disruptsphrase level complexity in Bengalese finches. NeuroReport 11, 20912095 (2000).

Article CAS PubMed Google Scholar

Woolley, S. M. & Rubel, E. W. Bengalese finches Lonchura striata domestica depend upon auditory feedback for the maintenance of adult song. J. Neurosci. 17, 63806390. https://doi.org/10.1523/JNEUROSCI.17-16-06380.1997 (1997).

Article CAS PubMed PubMed Central Google Scholar

Honda, E. & Okanoya, K. Acoustical and syntactical comparisons between songs of the white-backed munia (Lonchura striata) and its domesticated strain, the Bengalese finch (Lonchura striata var. domestica). Zool. Sci. 16, 319326 (1999).

Article Google Scholar

Glaze, C. M. & Troyer, T. W. Temporal structure in zebra finch song: implications for motor coding. J. Neurosci. 26, 9911005 (2006).

Article CAS PubMed PubMed Central Google Scholar

Troyer, T. W., Brainard, M. S. & Bouchard, K. E. Timing during transitions in Bengalese finch song: Implications for motor sequencing. J. Neurophysiol. 118, 15561566 (2017).

Article PubMed PubMed Central Google Scholar

Ondracek, J. M., and Hahnloser, R. H. R. Insights from Comparative Hearing Research (eds Kppl, C. et al.) 347388 (Springer, 2013).

Lachlan, R. F., Van Heijningen, C. A., Ter Haar, S. M. & Ten Cate, C. Zebra finch song phonology and syntactical structure across populations and continentsA computational comparison. Front. Psychol. 7, 980 (2016).

Article PubMed PubMed Central Google Scholar

Tchernichovski, O., Nottebohm, F., Ho, C. E., Bijan, P. & Mitra, P. P. A procedure for an automated measurement of song similarity. Anim. Behav. 59, 11671176 (2000).

Article CAS PubMed Google Scholar

Wohlgemuth, M. J., Sober, S. J. & Brainard, M. S. Linked control of syllable sequence and phonology in birdsong. J. Neurosci. 30, 1293612949 (2010).

Article CAS PubMed PubMed Central Google Scholar

Chen, Y. & Sakata, J. T. Norepinephrine in the avian auditory cortex enhances developmental song learning. J. Neurophysiol. 125, 23972407. https://doi.org/10.1152/jn.00612.2020 (2021).

Article CAS PubMed Google Scholar

Rodriguez-Saltos, C. A. et al. Song preferences predict the quality of vocal learning in zebra finches. Sci. Rep. 13, 605. https://doi.org/10.1038/s41598-023-27708-y (2023).

Article ADS CAS PubMed PubMed Central Google Scholar

Sober, S. J., Wohlgemuth, M. J. & Brainard, M. S. Central contributions to acoustic variation in birdsong. J. Neurosci. 28, 1037010379 (2008).

Article CAS PubMed PubMed Central Google Scholar

James, L. S. & Sakata, J. T. Learning biases underlie universals in avian vocal sequencing. Curr. Biol. 27, 3676-3682.e3674. https://doi.org/10.1016/j.cub.2017.10.019 (2017).

Article CAS PubMed Google Scholar

MacLean, E. L. et al. How does cognition evolve? Phylogenetic comparative psychology. Anim. Cogn. 15, 223238 (2012).

Article PubMed Google Scholar

Olsson, U. & Alstrm, P. A comprehensive phylogeny and taxonomic evaluation of the waxbills (Aves: Estrildidae). Mol. Phylogenet. Evol. 146, 106757 (2020).

Article PubMed Google Scholar

Sorenson, M. D. & Payne, R. B. A single ancient origin of brood parasitism in African finches: Implications for host-parasite coevolution. Evolution 55, 25502567 (2001).

CAS PubMed Google Scholar

Sorenson, M. D., Balakrishnan, C. N. & Payne, R. B. Clade-limited colonization in brood parasitic finches (Vidua spp.). Syst. Biol. 53, 104153 (2004).

Article Google Scholar

Arnaiz-Villena, A. et al. Estrildinae finches (Aves, Passeriformes) from Africa, South Asia and Australia: A molecular phylogeographic study. Open Ornithol. J. 2, 2936 (2009).

Article CAS Google Scholar

Kagawa, H., Suzuki, K., Takahasi, M. & Okanoya, K. Domestication changes innate constraints for birdsong learning. Behav. Proc. 106, 9197 (2014).

Article Google Scholar

Podos, J. A performance constraint on the evolution of trilled vocalizations in a songbird family (Passeriformes: Emberizidae). Evolution 51, 537551 (1997).

Article PubMed Google Scholar

ten-Cate, C. Natures Music: The Science of Birdsong (eds Marler, P. R. & Slabbekoorn, H.) 300321 (Elsevier Academic Press, 2004).

Read the original here:
Machine learning and statistical classification of birdsong link vocal ... - Nature.com