Deep learning for prediction of mechanism in acute ischemic stroke using brain diffusion magnetic resonance image

Baik-Kyun Kim; Seung Park; Moon-Ku Han; Jeong-Ho Hong; Dae-In Lee; Kyu Sun Yum

doi:10.18700/jnc.230039

Journal of Neurocritical Care > Volume 16(2); 2023 > Article

Kim, Park, Han, Hong, Lee, and Yum: Deep learning for prediction of mechanism in acute ischemic stroke using brain diffusion magnetic resonance image

Original Article

J Neurocrit Care 2023; 16(2): 85-93.

Published online: December 26, 2023

DOI: https://doi.org/10.18700/jnc.230039

Deep learning for prediction of mechanism in acute ischemic stroke using brain diffusion magnetic resonance image

Baik-Kyun Kim, MD^1,^2,³

, Seung Park, PhD⁴

, Moon-Ku Han, MD, PhD¹

, Jeong-Ho Hong, MD, PhD⁵

, Dae-In Lee, MD⁶

, Kyu Sun Yum, MD⁷

¹Department of Neurology, Seoul National University Bundang Hospital, Seongnam, Korea

²Department of Public Health, Seoul National University Bundang Hospital, Seongnam, Korea

³Department of Neurology, Gyeonggi Provincial Medical Center, Icheon Hospital, Icheon, Korea

⁴Department of Biomedical Engineering, Chungbuk National University Hospital, Cheongju, Korea

⁵Department of Neurology, Keimyung University Dongsan Medical Center, Daegu, Korea

⁶Division of Cardiology, Department of Internal Medicine, Korea University Guro Hospital, Seoul, Korea

⁷Department of Neurology, Chungbuk National University Hospital, Chungbuk National University College of Medicine, Cheongju, Korea

Corresponding author: Kyu Sun Yum, MD Department of Neurology, Chungbuk National University Hospital, Chungbuk National University College of Medicine, 776 1sunhwan-ro, Seowon-gu, Cheongju 28644, Korea Tel: +82-43-269-6339, Fax: +82-43-269-6196, Email: alcest@gmail.com

Received November 13, 2023 Revised December 5, 2023 Accepted December 6, 2023

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Background

Acute ischemic stroke is a disease with multiple etiologies. Therefore, identifying the mechanism of acute ischemic stroke is fundamental to its treatment and secondary prevention. The Trial of Org 10172 in Acute Stroke Treatment classification is currently the most widely used system, but it often has a limitations of classifying unknown causes and inadequate inter-rater reliability. Therefore, we attempted to develop a three-dimensional (3D)-convolutional neural network (CNN)-based algorithm for stroke lesion segmentation and subtype classification using only the diffusion and apparent diffusion coefficient information of patients with acute ischemic stroke.

Methods

This study included 2,251 patients with acute ischemic stroke who visited our hospital between February 2013 and July 2019.

Results

The segmentation model for lesion segmentation in the training set achieved a Dice score of 0.843±0.009. The subtype classification model achieved an average accuracy of 81.9%, with accuracies of 81.6% for large artery atherosclerosis, 86.8% for cardioembolism, 72.9% for small vessel occlusion, and 86.3% for control.

Conclusion

We developed a model to predict the mechanism of cerebral infarction using diffusion magnetic resonance imaging, which has great potential for identifying diffusion lesion segmentation and stroke subtype classification. As deep learning systems are gradually developing, they are becoming useful in clinical practice and applications.

Key Words: Deep learning; Ischemic stroke; Etiology; Diffusion magnetic resonance imaging

INTRODUCTION

Acute ischemic stroke has various causes based on the causative mechanism, consisting of large artery atherosclerosis (LAA), cardioembolism (CE), small vessel disease, stroke of other determined etiologies, or stroke of undetermined etiology. Classification of acute ischemic stroke based on the cause is important for treatment and secondary prevention. The most widely used classification system is the Trial of Org 10172 in Acute Stroke Treatment (TOAST) classification [1,2]. However, this classification method shows moderate inter-rater reliability in classifying acute ischemic stroke and has a limitation of frequently classifying strokes as having an undetermined etiology [3]. To overcome this limitation, efforts are underway to develop a computerized algorithm for acute stroke diagnosis; however, these have not shown sufficient results [4,5].

Various diagnostic methods such as brain imaging and heart tests are required to determine the causative mechanisms of acute ischemic stroke. Early diagnosis of the stroke subtype using this classification system can positively affect treatment, prognosis, and secondary prevention [1]. Diffusion-weighted magnetic resonance imaging (MRI) is widely used to diagnose acute stroke. It has superior performance in detecting hyperacute lesions and very small ischemic lesions and in distinguishing chronic and acute lesions [6] compared to brain computed tomography (CT) and conventional MRI. Furthermore, simultaneous use of the apparent diffusion coefficient (ADC) map and diffusion-weighted imaging (DWI) allow for more accurate distinction of the lesion of acute ischemic stroke, providing important information about the time window of the lesion [7]. The diffusion imaging lesion pattern, which provides useful information for the early diagnosis of acute ischemic stroke, has been reported to be closely related to the stroke subtype [8,9].

Various deep learning algorithms based on convolutional neural networks (CNNs) have been proposed for diagnosing acute ischemic stroke in brain MRI images [10-21]. These studies have shown that deep learning can detect stroke lesions more accurately than traditional machine-learning techniques and can extract meaningful features for severity evaluation or prognosis prediction. Researchers have proposed lesion segmentation techniques for patients with acute ischemic stroke based on the U-Net architecture [16,17]. To efficiently exploit the contextual information of volumetric MRI data, Zhang et al. [18] proposed a stroke lesion segmentation technique using a three-dimensional (3D) fully connected-DenseNet. Although the aforementioned studies demonstrated that deep learning can classify patients with acute ischemic stroke via lesion segmentation, a classification technique for predicting the treatment mechanism of acute ischemic stroke has yet to be reported. In this study, we presented a 3D CNN-based model for stroke lesion segmentation and subtype classification using only DWI and ADC images from patients with acute ischemic stroke.

METHODS

Study population

The participants were 2,251 patients with acute ischemic stroke who visited our hospital between February 2013 and July 2019. Information on acute ischemic stroke was compiled from a registry. All patients with acute ischemic stroke were reviewed by at least three stroke specialists and classified according to the TOAST classification. There were 1,789 patients with LAA, CE, and small vessel occlusion (SVO), excluding stroke of other determined etiologies or stroke of undetermined etiology. Among them, 1,396 patients underwent DWI and ADC (Fig. 1). There were 608 patients with LAA, 441 with CE, and 359 with SVO. Among the healthy patients who visited the hospital during the same period, 400 who showed normal MRI findings at our clinic were included as controls. The patients’ sex, age, National Institutes of Health Stroke Scale score, and medical history, including stroke, hypertension, diabetes mellitus, and atrial fibrillation, are listed in Table 1. In the control group, brain images without clinical information were used. Baseline characteristics were presented as frequencies (percentages). Continuous variables with normal distributions are presented as means±standard deviation, whereas variables with non-normal distributions are presented as medians (interquartile ranges).

Imaging acquisition

MRI was performed using various machines including 1.5 T (Achieva, Philips Healthcare) and 3.0 T (Ingenia CX, Philips Healthcare; Achieva, Philips Healthcare) scanners. The parameters of the DWI sequence were as follows: repetition time, 2,500–3,000 ms; echo time, 80 ms; slice thickness, 3–5 mm; intersection gap, 1 mm; field of view, 220×220 mm; matrix size, 256×256 (approximately 2×2 mm in-plane resolution); and b values, 0 and 1,000 sec/mm². Each apparent ADC map was generated automatically using the manufacturer’s software.

Data preparation

To produce a “ground truth” reference standard for training and evaluating the subtype classification model, each patient was classified into four classes (LAA, SVO, CE, and Control) according to the TOAST classification system. Lesion areas in each DWI slide were manually annotated by two experienced neurologists using in-house annotation software. Finally, each lesion was cross-validated and labelled, with a final decision agreed upon by both raters. To address the data distribution and validation methodology in our study, we adopted a 5-fold Stratified Cross-Validation approach. This ensures the proportionality of the class distribution across each fold, which is crucial for maintaining the integrity of the validation process given the imbalanced nature of our dataset. The preprocessing pipeline was meticulously designed to normalize and standardize the MRI images obtained from various vendors with different acquisition parameters. Each patient's MRI data comprising varying numbers of slices were resampled to a uniform 3D voxel size of 256 (H) × 256 (W) × 128 (D). This resampling is pivotal for aligning the spatial dimensions across all the datasets. To address the intensity variations due to different magnetic resonance (MR) parameters and scanner calibrations, we performed intensity normalization using the window center and width values provided in the digital Imaging and communication in medicine (DICOM) file metadata. This step adjusts the pixel intensity values to a standard scale, thereby enhancing the image comparability. Additionally, DWI, which inherently have varying numbers of slices owing to different scanning protocols, were standardized by selecting a fixed number of slices that best represented the essential features required for accurate segmentation. This uniform pre-processing approach ensures that subsequent segmentation algorithms operate on data that reflect consistent anatomical structures and tissue characteristics, thereby enabling a more reliable and valid comparative analysis across all images. Practically, for each of the five folds, we allocated 60% of the data for training purposes, 20% for validation, and 20% for testing. This division was performed independently within each fold to confirm the generalizability and reliability of the model's performance.

Lesion segmentation mode

Our segmentation model, based on a 3D CNN called V-Net [22] is illustrated in Fig. 2. The model consists of an encoder that extracts feature maps from local 3D volumes and a decoder that predicts stroke lesions using feature maps. Because our model has a very deep architecture, we employed a residual block to alleviate the gradient vanishing and exploding problems. The residual block contains (1) a 3D convolutional layer with kernel size 3×3×3 and (2) a residual skip connection and two 3D convolutional layers with kernel size 3×3×3, each following batch normalization and a rectified linear unit, respectively. In the network encoder, residual blocks were utilized for feature extraction and max-pooling layers, with a stride of two to reduce spectral dimensionality. In contrast, the decoder consists of up-convolutional layers with strides of two, followed by residual blocks after feature-map concatenation. Skip connections from the layers of equal resolution in the encoder provide high-resolution features to the decoder. A sigmoid activation layer was connected to the last layer of the decoder to calculate a probability map of the stroke lesions.

The model was trained over 200 epochs with an Adam optimizer, an initial learning rate of 1e-5, and a batch size of 8. The model was trained from scratch without using pretrained weights. We tested various loss functions such as weighted cross-entropy loss, L1 loss, and Dice loss; Dice loss achieved the best performance. Furthermore, to address the data scarcity problem, data augmentation techniques such as rigid transformation, horizontal/vertical flip, Gaussian noise, and gamma correction were randomly triggered in each training session.

Subtype classification model

As illustrated in Fig. 3, our classification model predicted the probabilities of the four classes: LAA, SVO, CE, and Control. For feature extraction, we adopted a residual block in the lesion-segmentation model. In addition, to guide the network to focus on the lesions, feature maps were enhanced using the lesion prediction results provided by the segmentation model. Specifically, the enhanced feature map Fenh is obtained by

Fenh = F × (1 + h(A)), (1)

where F and A are the feature maps extracted by each residual block and lesion segmentation result, respectively. h(·) is a bilinear interpolation to match the spatial resolutions between F and A. This attention mechanism significantly improves the classification performance of the model by guiding the network to focus on lesion areas to predict stroke subtypes. The model was trained over 400 epochs with an Adam optimizer, an initial learning rate of 1e-5, and a batch size of 4. Categorical cross-entropy loss was utilized, and the model was trained from scratch. We used a data-augmentation technique to train the classification model.

RESULTS

Lesion segmentation model

Our segmentation model for lesion segmentation in the training set achieved a Dice score of 0.891± 0.034. For the test set, our model resulted in a Dice score, precision, and recall of 0.843±0.009, 0.842±0.012, and 0.844±0.017, respectively. Fig. 4 shows some examples of lesion prediction results compared with assessments by neurologists. Our segmentation model accurately predicted extremely small lesions. Most cases of failure occurred when the lesions had very poor contrast in the diffusion images, as shown in Fig. 5.

Stroke subtype classification model

To underscore the benefits of leveraging segmentation data, we integrated an enhanced feature map into our stroke-subtype classification model. By applying an enhanced feature map informed by the segmentation results, the model obtains the spatial context that the raw images lack. This context allows the model to “see” beyond mere pixel intensity, recognizing patterns and structures pertinent to stroke subtype.

This strategic modification led to a notable improvement in the performance metrics. Before integrating the segmentation information, the average classification accuracy of the model is 71.1%. However, with the incorporation of an enhanced feature map, the accuracy significantly increased to 81.9%. This effect was evident across all subtypes, with accuracies of 81.6% for LAA, 86.8% for CE, 72.9% for SVO, and 86.3% for control. The enhanced feature map sharpens the model's ability to concentrate on lesion-specific areas, thereby refining the differentiation process between various stroke subtypes.

Fig. 6 shows the confusion matrix obtained using the subtype classification model. Our model showed lower accuracy for SVO than for other subtypes, indicating that the model confused SVO with control cases owing to its poor analysis performance for small lesions.

DISCUSSION

This is the first study to perform subtype classification of stroke mechanisms by analyzing the patterns of acute ischemic stroke lesions through deep learning based on a 3D-CNN using DWI and ADC in patients with acute ischemic stroke. The main findings of this study are as follows. First, the 3D-CNN-based segmentation accuracy for acute ischemic stroke lesions was 0.843 based on the Dice score. Second, in terms of subtyping to classify the cause of acute ischemic stroke, the predicted degree of cause classification according to the TOAST classification, which is the "ground truth," was confirmed to be 81.3% for LAA, 84.6% for SVO, and 73.0% for CE.

With technological advances, brain imaging plays a crucial role in diagnosing and identifying mechanisms underlying the development of acute ischemic stroke according to technological advances [23]. Among the various MR sequences, DWI and ADC maps are useful tools for the early detection of acute ischemic lesions and for differentiating between stroke mimics and acute ischemic stroke [24]. Several previous studies have attempted to segment the infarction volume in acute ischemic stroke using artificial intelligence (AI). Various imaging patterns of acute ischemic stroke in DWI lesions correlate with pathogenic mechanisms. In the case of cardiac embolic stroke, acute stroke lesions on DWI often show single cortical/subcortical lesions or occur multiple times in various vascular branches. Multiple unilateral lesions in the anterior circulation are characteristic findings of arteriogenic embolism. Meanwhile, small infarction (2–20 mm in diameter) lesions observed in the deep acute ischemic white matter, basal ganglia, thalamus, and pons were highly associated with SVO [8,25]. We attempted to apply a doctor’s diagnostic process to determine the cause of acute ischemic stroke based on the characteristic findings of brain MRI using AI.

However, there are many limitations in predicting the pathogenesis of acute ischemic stroke using only DWI/ADC maps. The TOAST classification system is the most widely used system for classifying acute ischemic stroke based on its pathogenesis. Clinical findings and the results of ancillary diagnostic studies, including brain imaging and cardiac evaluation, were used to classify patients’ acute ischemic stroke mechanism [1]. Although widely use and popular, its overall inter-rater agreement is moderate. Its reliability is notably lower for small-vessel occlusion and strokes of undetermined causes, especially when compared to LAA and CE [3]. To overcome this, an improved classification method that applies a new diagnostic technique was used; nevertheless, it still has limitations [4,5,26,27]. In particular, acute ischemic stroke with unknown mechanisms, such as LAA, CE, SVO, and stroke of other determined etiologies, is known as cryptogenic stroke. It is observed in approximately all the patients with acute ischemic stroke [28]. These cryptogenic strokes are often observed as embolic strokes, and are called embolic strokes of undetermined source (ESUS) [29]. There is a need to determine the mechanism of ESUS and provide proper treatment; however, a definitive method for achieving this objective remains elusive [30]. We conducted this study to diagnose acute ischemic stroke using a deep learning algorithm. To the best of our knowledge, this model is the first algorithm for identifying the mechanism of cerebral infarction in patients with ESUS. In the future, it will be necessary to create a multimodal algorithm that includes cerebrovascular imaging, laboratory data, and cardiac tests, such as transthoracic echocardiography, transesophageal echocardiography, and electrocardiography. We expect to improve the model in this study.

Our study introduces a novel approach to lesion segmentation and stroke subtype classification that significantly advances technology beyond previous methodologies [10-21]. Unlike traditional techniques [16,17] which process individual slides and therefore cannot utilize contextual information from adjacent slides, often resulting in diminished accuracy, our technique leverages a 3D CNN with a deep residual network architecture. This allowed for stable learning and improved recognition of complex patterns across multiple slides, culminating in a high Dice score of 0.845 in the test set.

Moreover, our subtype classification model exhibited an average accuracy of 81.9%, which is a notable improvement over the existing models. This is achieved through an attention mechanism that utilizes the lesion information predicted by the segmentation model, focusing on key areas for accurate prediction. This not only identified the stroke subtype but also highlighted the specific regions the model analyzed to arrive at its conclusion.

The integration of these advanced segmentation and classification models is expected to have a substantial impact on medical AI applications that rely on 3D volumetric data, such as CT and MRI scans. Our approach sets a new precedent for accuracy and reliability in medical diagnostics, offering a comprehensive solution that outperforms previous single-slice-based techniques.

This study has several limitations. First, the labeling of subtypes in the stroke prediction is unclear. Despite our meticulous process of employing the TOAST classification system and expert annotations by two experienced neurologists, the subjective nature of clinical diagnoses presents the potential for inconsistency. The difficulty in standardizing labels across different raters and cases is an inherent limitation not only in our study, but also in the broader context of machine learning applications in stroke subtype classification. This could result in variability, affecting the reliability of our model. Recognizing this limitation, we emphasize the need for continuous improvement in annotation methodologies and exploration of more objective measures in future studies to minimize such discrepancies. Second, this study lacks external validation. Therefore, there may be a bias in this model, and it is necessary to improve it by performing external validation in future studies. Third, this algorithm does not include images of cerebral infarction caused by causes other than LAA, CE, or SVO. To predict and diagnose these mechanisms, additional clinical data, such as cerebrovascular imaging, laboratory studies, and cardiological evaluation, are required in addition to DWI/ADC. In this study, the algorithm was only trained on three mechanisms that are known to be diagnosable or predictable by DWI patterns. Therefore, it is limited in classifying cerebral infarction caused by other mechanisms. In our follow-up study, we plan to improve the algorithm by including various types of clinical data. Fourth, this study is a case-control study. The study was based on a stroke database from a single center; hence, there is a possibility of selection bias in the selection of subjects. Therefore, it is necessary to overcome this limitation using multicenter data in subsequent studies. Finally, this study was conducted using data collected from a single cohort, which limited our consideration of variations in the MR parameters. Therefore, the generalizability of our findings to datasets with different MR parameters may be limited. Specifically, our normalization approach based on Window Center and WindowWidth may not be applicable to other datasets with varying imaging protocols. This limitation highlights the need for further research using diverse MRI datasets to validate and refine our methodology. Future studies should aim to incorporate data from multiple sources with varying MR parameters to ensure broader applicability and robustness of the findings.

In summary, this study aimed to predict the pathogenesis of cerebral infarction using only brain diffusion MRI and apply it clinically. Using only the initial diffusion MRI information, we present a feasible model that predicts the mechanism of occurrence by applying an algorithm based on a 3D-CNN through deep learning. The diffusion lesion volume measurement and stroke subtype classification using our proposed method showed a strong correlation with those performed by manual segmentation and subtype classification by professional neurologists. This study is significant because it is the first to predict the mechanism of acute ischemic stroke by using diffusion MRI alone. In future studies, it will be necessary to develop a multimodal algorithm that includes not only diffusion MRI, but also other brain imaging modalities and clinical data to predict the exact pathogenesis of cerebral infarction.

ARTICLE INFORMATION

Ethics statement

This single-center, retrospective case-control study was conducted in accordance with the Declaration of Helsinki. The study protocol was approved by the Institutional Review Board (No. 2023-10-017), which waived the requirement for informed consent.

Conflict of interest

Moon Ku Han and Jeong-Ho Hong are editorial board members of the journal, but they were not involved in the peer reviewer selection, evaluation, or decision process of this article. No other potential conflict of interest relevant to this article was reported.

Acknowledgments

This work was supported by funding from the Academic Research Program of the Chungbuk National University in 2022.

Author contributions

Conceptualization: all authors. Methodology: BKK, SP, DIL, KSY. Formal analysis: BKK, SP, DIL, KSY. Data curation: BKK, SP, DIL, KY. Visualization: BKK, SP, DIL, KSY. Project administration: BKK, SP, DIL, KSY. Funding acquisition: KSY. Writing–original draft: BKK, SP, DIL, KY. Writing–review & editing: all authors.

Fig. 1.

Study profile. LAA, large artery atherosclerosis; SVO, small vessel occlusion; CE, cardioembolism; DWI, diffusion-weighted imaging; ADC, apparent diffusion coefficient; AI, artificial intelligence.

Fig. 2.

Our network architecture for stroke lesion segmentation. Based on three-dimensional (3D) U-Net, the network learns the features based on a hierarchy framework starting from simple features such as edges and shapes to high-level features in the deeper levels. ADC, apparent diffusion coefficient; DWI, diffusion-weighted imaging; Conv, convolution; BN, batch normalization; ReLU, rectified linear unit; Up-conv, up-convolutional.

Fig. 3.

Our network architecture for stroke subtype classification. To guide the network towards the lesion areas, we adopted the attention mechanism using the lesion segmentation result. ADC, apparent diffusion coefficient; DWI, diffusion-weighted imaging; LAA, large artery atherosclerosis; SVO, small vessel occlusion; CE, cardioembolism.

Fig. 4.

Prediction outcomes using our lesion segmentation model. In each panel, the images in the first and second rows are diffusion-weighted imaging (DWI) and apparent diffusion coefficient (ADC) slices, respectively. The third-row images are the “ground truth” labels annotated by two neurologists, while the fourth-row images show lesion areas predicted by our model. AI, artificial intelligence.

Fig. 5.

Failure cases of our lesion segmentation model. Most cases have occurred when the lesions have extremely poor contrast. DWI, diffusion-weighted imaging; ADC, apparent diffusion coefficient. AI, artificial intelligence.

Fig. 6.

Confusion matrix of our subtype classification model. Values are presented as number (ratio). 3D, three-dimensional; CNN, convolutional neural network; LAA, large artery atherosclerosis; CE, cardioembolism; SVO, small vessel occlusion.

Table 1.

Baseline characteristics of the study population according to stroke mechanism

Variable	Total	LAA	SVO	CE
Number	1,396	601	358	437
Female	591 (42.3)	250 (41.6)	136 (38.0)	205 (46.9)
Male	805 (57.7)	351 (58.4)	222 (62.0)	232 (53.1)
Age (yr)	70±12	69±12	66±13	73±11
Delayed time (min)	101 (57–198)	118 (57–213)	152 (63.25–296)	85 (55–154)
Medical history
Stroke	276 (19.8)	113 (18.8)	64 (17.9)	99 (22.7)
Hypertension	895 (64.1)	416 (69.2)	214 (59.8)	265 (60.6)
Diabetes mellitus	475 (34.0)	222 (36.9)	131 (36.6)	122 (37.9)
Dyslipidemia	178 (12.8)	84 (14.0)	52 (14.5)	42 (9.6)
Atrial fibrillation	327 (23.4)	9 (1.5)	2 (0.6)	316 (72.3)
NIHSS	5.67±5.72	4.96±4.99	3.02±2.13	8.81±7.11

Values are presented as number (%), mean±standard deviation, or median (interquartile range).

LAA, large artery atherosclerosis; SVO, small vessel occlusion; CE, cardioembolism; NIHSS, National Institutes of Health Stroke Scale Rating.

REFERENCES

1. Adams HP Jr, Bendixen BH, Kappelle LJ, Biller J, Love BB, Gordon DL, et al. Classification of subtype of acute ischemic stroke. Definitions for use in a multicenter clinical trial. TOAST. Trial of Org 10172 in Acute Stroke Treatment. Stroke 1993;24:35-41.

2. Amarenco P, Bogousslavsky J, Caplan LR, Donnan GA, Hennerici MG. Classification of stroke subtypes. Cerebrovasc Dis 2009;27:493-501.

3. Meschia JF, Barrett KM, Chukwudelunzu F, Brown WM, Case LD, Kissela BM, et al. Interobserver agreement in the trial of org 10172 in acute stroke treatment classification of stroke based on retrospective medical record review. J Stroke Cerebrovasc Dis 2006;15:266-72.

4. Goldstein LB, Jones MR, Matchar DB, Edwards LJ, Hoff J, Chilukuri V, et al. Improving the reliability of stroke subgroup classification using the Trial of ORG 10172 in Acute Stroke Treatment (TOAST) criteria. Stroke 2001;32:1091-8.

5. Ko Y, Lee S, Chung JW, Han MK, Park JM, Kang K, et al. MRI-based algorithm for acute ischemic stroke subtype classification. J Stroke 2014;16:161-72.

6. Warach S, Gaa J, Siewert B, Wielopolski P, Edelman RR. Acute human stroke studied by whole brain echo planar diffusion-weighted magnetic resonance imaging. Ann Neurol 1995;37:231-41.

7. Lansberg MG, Thijs VN, O'Brien MW, Ali JO, de Crespigny AJ, Tong DC, et al. Evolution of apparent diffusion coefficient, diffusion-weighted, and T2-weighted signal intensity of acute stroke. AJNR Am J Neuroradiol 2001;22:637-44.

8. Wessels T, Wessels C, Ellsiepen A, Reuter I, Trittmacher S, Stolz E, et al. Contribution of diffusion-weighted imaging in determination of stroke etiology. AJNR Am J Neuroradiol 2006;27:35-9.

9. Kang DW, Chalela JA, Ezzeddine MA, Warach S. Association of ischemic lesion patterns on early diffusion-weighted imaging with TOAST stroke subtypes. Arch Neurol 2003;60:1730-4.

10. Wu O, Winzeck S, Giese AK, Hancock BL, Etherton MR, Bouts MJ, et al. Big data approaches to phenotyping acute ischemic stroke using automated lesion segmentation of multi-center magnetic resonance imaging data. Stroke 2019;50:1734-41.

11. Lee H, Lee EJ, Ham S, Lee HB, Lee JS, Kwon SU, et al. Machine learning approach to identify stroke within 4.5 hours. Stroke 2020;51:860-6.

12. Boldsen JK, Engedal TS, Pedraza S, Cho TH, Thomalla G, Nighoghossian N, et al. Better diffusion segmentation in acute ischemic stroke through automatic tree learning anomaly segmentation. Front Neuroinform 2018;12:21.

13. Yu Y, Xie Y, Thamm T, Gong E, Ouyang J, Huang C, et al. Use of deep learning to predict final ischemic stroke lesions from initial magnetic resonance imaging. JAMA Netw Open 2020;3:e200772.

14. Do LN, Baek BH, Kim SK, Yang HJ, Park I, Yoon W. Automatic assessment of ASPECTS using diffusion-weighted imaging in acute ischemic stroke using recurrent residual convolutional neural network. Diagnostics (Basel) 2020;10:803.

15. Woo I, Lee A, Jung SC, Lee H, Kim N, Cho SJ, et al. Fully automatic segmentation of acute ischemic lesions on diffusion-weighted imaging using convolutional neural networks: comparison with conventional algorithms. Korean J Radiol 2019;20:1275-84.

16. Kim YC, Lee JE, Yu I, Song HN, Baek IY, Seong JK, et al. Evaluation of diffusion lesion volume measurements in acute ischemic stroke using encoder-decoder convolutional network. Stroke 2019;50:1444-51.

17. Hui H, Zhang X, Li F, Mei X, Guo Y. A partitioning-stacking prediction fusion network based on an improved attention u-net for stroke lesion segmentation. IEEE Access 2020;8:47419-32.

18. Zhang R, Zhao L, Lou W, Abrigo JM, Mok VC, Chu WC, et al. Automatic segmentation of acute ischemic stroke from DWI using 3-D fully convolutional DenseNets. IEEE Trans Med Imaging 2018;37:2149-60.

19. Liu X, Yang H, Qi K, Dong P, Liu Q, Liu X, et al. MSDF-net: Multi-scale deep fusion network for stroke lesion segmentation. IEEE Access 2019;7:178486-95.

20. Zhang L, Song R, Wang Y, Zhu C, Liu J, Yang J, et al. Ischemic stroke lesion segmentation using multi-plane information fusion. IEEE Access 2020;8:45715-25.

21. Liu L, Wu FX, Wang J. Efficient multi-kernel dcnn with pixel dropout for stroke mri segmentation. Neurocomputing 2019;350:117-27.

22. Milletari F, Navab N, Ahmadi SA. fully convolutional neural networks for volumetric medical image segmentation. In: 2016 fourth international conference on 3D vision (3DV); 2016;Stanford, CA, USA.

23. Bang OY, Chung JW, Son JP, Ryu WS, Kim DE, Seo WK, et al. Multimodal MRI-based triage for acute stroke therapy: challenges and progress. Front Neurol 2018;9:586.

24. Fung SH, Roccatagliata L, Gonzalez RG, Schaefer PW. MR diffusion imaging in ischemic stroke. Neuroimaging Clin N Am 2011;21:345-77.

25. Lee LJ, Kidwell CS, Alger J, Starkman S, Saver JL. Impact on stroke subtype diagnosis of early diffusion-weighted magnetic resonance imaging and magnetic resonance angiography. Stroke 2000;31:1081-9.

26. Ay H, Furie KL, Singhal A, Smith WS, Sorensen AG, Koroshetz WJ. An evidence-based causative classification system for acute ischemic stroke. Ann Neurol 2005;58:688-97.

27. Ay H, Benner T, Arsava EM, Furie KL, Singhal AB, Jensen MB, et al. A computerized algorithm for etiologic classification of ischemic stroke: the Causative Classification of Stroke System. Stroke 2007;38:2979-84.

28. Hart RG, Catanese L, Perera KS, Ntaios G, Connolly SJ. Embolic stroke of undetermined source: a systematic review and clinical update. Stroke 2017;48:867-72.

29. Hart RG, Diener HC, Coutts SB, Easton JD, Granger CB, O'Donnell MJ, et al. Embolic strokes of undetermined source: the case for a new clinical construct. Lancet Neurol 2014;13:429-38.

30. Healey JS, Gladstone DJ, Swaminathan B, Eckstein J, Mundl H, Epstein AE, et al. Recurrent stroke with rivaroxaban compared with aspirin according to predictors of atrial fibrillation: secondary analysis of the NAVIGATE ESUS randomized clinical trial. JAMA Neurol 2019;76:764-73.