Keywords: Sleep - Cardiovascular & Metabolic consequences of sleep disorders, Cardiovascular, respiratory, and sleep devices - Smart systems, Cardiovascular, respiratory, and sleep devices - Sensors
Abstract: Driver drowsiness has been a vastly underestimated cause of traffic accidents due to the difficulty in establishing whether a driver involved in a crash was drowsy. A recent report from the AAA estimates that almost ten percent of vehicle accidents involve drowsy driving, while another survey from the German ADAC has found out that one fourth of car drivers and almost half of the truck drivers have fallen asleep during driving. The European Respiratory Society has established a Task Force to address the topic of sleep apnea, sleepiness and driving. The planned session will start dealing with the causes of driver drowsiness. Different techniques for real-time monitoring of fatigue-related parameters will be presented and it will be discussed whether the standard sources of information used to detect drowsiness can also be used to predict when a given drowsiness level will be reached. Finally, the session will discuss current approaches to alert drivers about their drowsiness while driving.

Keywords: Models and simulations of therapeutic devices and systems, Computer modeling for treatment planning, Therapeutic devices and systems - ablation systems and technologies
Abstract: Electromagnetic therapies rely on the targeted energy deposition as a medical treatment for a variety of pathological conditions in humans. Treatments include tumor-treating fields, electroconvulsive therapy, electrical neuro-stimulation, deep-brain stimulation, magnetic stimulation, and thermal therapies. In each of these applications, there is a desire to understand the ability of the therapeutic system to target the electrical energy to a specific site(s) within the human body. The safety of each treatment modality must also be considered since the interaction between applied electromagnetic energy and human tissue may cause tissue heating and potentially irreversible tissue damage. While benchtop experiments in phantoms and in vivo animal testing can provide some insights into the potential safety and efficacy of a therapeutic technology, these testing environments may differ considerably from the tissue anatomic and physiologic state in humans. For example, ex vivo tissue models typically do not account for the effects of blood perfusion, and animal models may not provide a suitable representation of the disease in humans. Meanwhile, testing of therapeutic systems in humans is often not possible without exposing test subjects to potential harm.

Computational modeling and simulation tools offer strong potential to serve as a platform for evaluating the safety of electromagnetic therapeutic systems with no patient harm, but model credibility remains a challenge. The processes of verification, validation, and uncertainty quantification (VVUQ) provide a methodology for establishing that a model is credible for decision-making. This mini-symposium will include presentations that review current VVUQ best practices, as well as academic, industrial, and clinical studies that include verification and/or validation approaches to establish model credibility for a variety of electromagnetic therapeutic systems.

Keywords: Data mining and big data methods - Machine learning and deep learning methods, Connectivity, Physiological systems modeling - Closed loop systems
Abstract: In the new era of Big Data in biology and medicine, there is a pressing need for cross-disciplinary collaborations between investigators in the Life/Medical Sciences, Data Science and Engineering. This is particularly true in Neuroscience, a field where novel signal and image processing, machine learning and statistics approach are critically needed to extract interpretable, reliable and generalizable information from highly complex brain datasets. Once appropriately translated, computational tools from fields, such as Computer Science and Engineering can be widerly applied to neurophysiological, neuroimaging and neurochemical data. At the level of the individual brain, these techniques may provide novel representations of the brain as a collection of hierarchically organized multi-layer networks, communities with a specific organization and/or as a dynamic closed-loop system. These tools can also provide a means for data-derived, unsupervised estimations of anatomical and functional features, which to date have been limited. At the cohort or collective brain level, machine learning and AI tools can identify low-dimensional manifolds and network patterns across connectomes with similar topologies and again identify common structure and organization principles in communities of brains.

This mini-symposium will bring together investigators in Computational Neuroscience, Computer Science and Engineering to present data science methods for novel investigations of large, highly heterogeneous neuroimaging and neurophysiological datasets. Novel graph signal processing approaches, such as signed and/or directional graph signal processing, machine learning techniques, such as Graph Convolutional Neural Networks and unsupervised classification methods, as well as tools/models from control theory (e.g., Stochastic Block Models) will be presented and their applications to large-scale brain data will be demonstrated. The goal of the session is to highlight the utility of these

Keywords: Systems modeling - Decision making, Computational modeling - Analysis of high-throughput systems biology data, Data-driven modeling
Abstract: Computational intelligence has emerged as a key field within the domain of biomedical engineering. Especially in the case of applied areas like healthcare, food and water safety, physiology, genetic engineering etc., the aspect of computational intelligence is highly prominent. Considering its relevance in the recent times, Mini Symposium is focused to highlighting the theoretical and practical aspects of computational intelligence, further reaching to machine learning, predictive modeling etc. to different fields of biomedical engineering. The primary goal of this Mini Symposium is to highlight the trending research in this field across the world. It would provide a global perspective of the state of the art. The secondary goal is to open a platform for discussion on the existing challenges in applying computational intelligence in biomedical engineering, which would lead to finding collaborative solutions to the existing problems in this field. Featuring invited talks from 6 speakers (having diverse backgrounds) followed by papers specifically directed to this domain, the Mini Symposium aims at fortifying the global discussion on computation intelligence in biomedical engineering.

Keywords: Image reconstruction and enhancement - Machine learning / Deep learning approaches, Image reconstruction and enhancement - Image synthesis, Electrical source imaging
Abstract: Analysing human gait from plantar pressure is critical for human health. The majority of works focus on classifying the healthy plantar pattern from unhealthy ones. Different from previous works, we adopt a generative adversarial network to produce healthy plantar pressure for individual patients. In this work, we do not have pairs of images for training thus we cast the problem as an unsupervised generative adversarial learning task. Our network benefits from multiple components: an encoder-decoder generator, a convolution-based discriminator, a convolution-based evaluation network, and a new term in the loss function to preserve the person gait style. Our method achieves high performance on the CAD WALK databases which has patients with hallux valgus disease.

Keywords: CT imaging applications, Image analysis and classification - Machine learning / Deep learning approaches, Multiscale image analysis
Abstract: Preoperative predicting histological grade of hepatocellular carcinoma (HCC) is a crucial issue for the evaluation of patient prognosis and determining clinical treatment strategies. Previous studies have shown the potential of preoperative medical imaging in HCC grading diagnosis, however, there still remain challenges. In this work, we proposed a multi-scale 2D dense connected convolutional neural network (MS-DenseNet) for the classification of grade. This architecture consisted of three CNN branches to extract features of CT image patches in different scale. Then the outputs for each CNN branch were concatenated to the final fully connected layer. Our network was developed and evaluated on 455 HCC patients from two different centers. For data augmentation, more than 2000 patches for each scale were cropped from transverse section 2D region of interest on these patients. Besides, three-channel inputs including original CT image, tumor region and peritumoral component provided complementary knowledge. Experimental results demonstrated that the proposed method achieved encouraging prediction performance with AUC of 0.798 in testing dataset.

Keywords: Image analysis and classification - Machine learning / Deep learning approaches, X-ray imaging applications
Abstract: Cobb angle is the most common quantification of the spine deformity called scoliosis. Recently, automatic Cobb angle estimation has become popular with either semantic segmentation networks or landmark detectors. However, such methods can not perform robustly when some vertebrae have ambiguous appearances in X-ray images. To alleviate the above problem, we propose a multi-task model that simultaneously output semantic masks and keypoints of vertebrae. When training this model, we propose a heterogeneous consistency loss function to enhance the consistency between keypoints and semantic masks. Extensive experiments on anterior-posterior (AP) X-ray images from AASCE MICCAI 2019 Challenge demonstrate that our method significantly reduces Cobb angle estimation errors and achieves state-of-the-art performances.

Keywords: Image segmentation, CT imaging, Machine learning / Deep learning approaches
Abstract: For COVID-19 prevention and treatment, it is essential to screen the pneumonia lesions in the lung region and analyze them in a qualitative and quantitative manner. Three-dimensional (3D) computed tomography (CT) volumes can provide sufficient information; however, extra boundaries of the lesions are also needed. The major challenge of automatic 3D segmentation of COVID-19 from CT volumes lies in the inadequacy of datasets and the wide variations of pneumonia lesions in their appearance, shape, and location. In this paper, we introduce a novel network called Comprehensive 3D UNet (C3D-UNet). Compared to 3D-UNet, an intact encoding (IE) strategy designed as residual dilated convolutional blocks with increased dilation rates is proposed to extract features from wider receptive fields. Moreover, a local attention (LA) mechanism is applied in skip connections for more robust and effective information fusion. We conduct five-fold cross-validation on a private dataset and independent offline evaluation on a public dataset. Experimental results demonstrate that our method outperforms other compared methods.

Keywords: Image analysis and classification - Machine learning / Deep learning approaches, Image classification, Image feature extraction
Abstract: Acupuncture therapy is one of the cornerstones in traditional Chinese medicine. It requires rich experiences from Chinese medicine practitioner. However, repeatability among different practitioners are low. Meanwhile, there is a large variety of skin conditions in terms of color, diseases, size, etc. In recent year, deep neural network for acupuncture point detection is proposed. However, it is difficult to localize multiple acupuncture points. In this paper, a high repeatability robot with a new approach of acupuncture points positioning is proposed which can be adaptive to variety skin conditions and achieve multiple acupuncture points’ localization.

Keywords: Image analysis and classification - Machine learning / Deep learning approaches, Image feature extraction, Image classification
Abstract: Inpatient falls are a serious safety issue in hospitals and healthcare facilities. Recent advances in video analytics for patient monitoring provide a non-intrusive avenue to reduce this risk through continuous activity monitoring. However, in-bed fall risk assessment systems have received less attention in the literature. The majority of prior studies have focused on fall event detection, and do not consider the circumstances that may indicate an imminent inpatient fall. Here, we propose a video-based system that can monitor the risk of a patient falling, and alert staff of unsafe behaviour to help prevent falls before they occur. We propose an approach that leverages recent advances in human localisation and skeleton pose estimation to extract spatial features from video frames recorded in a simulated environment. We demonstrate that body positions can be effectively recognised and provide useful evidence for fall risk assessment. This work highlights the benefits of video-based models for analysing behaviours of interest, and demonstrates how such a system could enable sufficient lead time for healthcare professionals to respond and address patient needs, which is necessary for the development of fall intervention programs.

Keywords: Image analysis and classification - Machine learning / Deep learning approaches, X-ray radiography
Abstract: Conventional methods for artificial age determination of skeletal bones have several problems, such as strong subjectivity, large random errors, complex evaluation processes, and long evaluation cycles. In this study, an automated age determination of skeletal bones was performed based on Deep Learning. Two methods were used to evaluate bone age, one based on examining all bones in the palm and another based on the deep convolutional neural network (CNN) method. Both methods were evaluated using the same test dataset. Moreover, we can extend the dataset and increase the generalisation ability of the network by data expansion. Consequently, a more accurate bone age can be obtained. This method can reduce the average error of the final bone age evaluation and lower the upper limit of the absolute value of the error of the single bone age. The experiments show the effectiveness of the proposed method, which can provide doctors and users with more stable, efficient and convenient diagnosis support and decision support.

Keywords: Deformable registration, Machine learning / Deep learning approaches, Brain imaging and image analysis
Abstract: Multi-modality magnetic resonance image (MRI) registration is an essential step in various MRI analysis tasks. However, it is challenging to have all required modalities in clinical practice, and thus the application of multi-modality registration is limited. This paper tackles such problem by proposing a novel unsupervised deep learning based multi-modality large deformation diffeomorphic metric mapping (LDDMM) framework which is capable of performing multi-modality registration only using single-modality MRIs. Specifically, an unsupervised image-to-image translation model is trained and used to synthesize the missing modality MRIs from the available ones. Multi-modality LDDMM is then performed in a multi-channel manner. Experimental results obtained on one publicly-accessible datasets confirm the superior performance of the proposed approach.

Keywords: Optical imaging, Image segmentation, Machine learning / Deep learning approaches
Abstract: Diabetic Retinopathy (DR) is a progressive chronic eye disease that leads to irreversible blindness. Detection of DR at an early stage of the disease is crucial and requires proper detection of minute DR pathologies. A novel Deeply-Supervised Multiscale Attention U-Net (Mult-Attn-U-Net) is proposed for segmentation of different DR pathologies viz. Microaneurysms (MA), Hemorrhages (HE), Soft and Hard Exudates (SE and EX). A publicly available dataset (IDRiD) is used to evaluate the performance. Comparative study with four state-of-the-art models establishes its superiority. The best segmentation accuracy obtained by the model for MA, HE, SE are 0.65, 0.70, 0.72, respectively.

Keywords: Ultrasound imaging - Other organs, Image classification, Machine learning / Deep learning approaches
Abstract: The global pandemic of the novel coronavirus disease 2019 (COVID-19) has put tremendous pressure on the medical system. Imaging plays a complementary role in the management of patients with COVID-19. Computed tomography (CT) and chest X-ray (CXR) are the two dominant screening tools. However, difficulty in eliminating the risk of disease transmission, radiation exposure and not being cost-effective are some of the challenges for CT and CXR imaging. This fact induces the implementation of lung ultrasound (LUS) for evaluating COVID-19 due to its practical advantages of noninvasiveness, repeatability, and sensitive bedside property. In this paper, we utilize a deep learning model to perform the classification of COVID-19 from LUS data, which could produce objective diagnostic information for clinicians. Specifically, all LUS images are processed to obtain their corresponding local phase filtered images and radial symmetry transformed images before fed into the multi-scale residual convolutional neural network (CNN). Secondly, image combination as the input of the network is used to explore rich and reliable features. Feature fusion strategy at different levels is adopted to investigate the relationship between the depth of feature aggregation and the classification accuracy. Our proposed method is evaluated on the point-of-care US (POCUS) dataset together with the Italian COVID-19 Lung US database (ICLUS-DB) and shows promising performance for COVID-19 prediction.

Keywords: Optical imaging and microscopy - Microscopy, Image segmentation, Image reconstruction and enhancement - Filtering
Abstract: With the use of computer-aided diagnostic systems, the automatic detection and segmentation of the cell nuclei have become essential in pathology due to cellular nuclei counting and nuclear pleomorphism analysis are critical for the classification and grading of breast cancer histopathology. This work describes a methodology for automatic detection and segmentation of cellular nuclei in breast cancer histopathology images obtained from the BreakHis database, the Standford tissue microarray database, and the Breast Cancer Cell Segmentation database. The proposed scheme is based on the characterization of Hematoxylin and Eosin (H&E) staining, size, and shape features. In addition, we use the information obtained from morphological transformations and adaptive intensity adjustments to detect and separate each cell nucleus detected in the image. The segmentation was carried out by testing the proposed methodology in a histological breast cancer database that provides the associated groundtruth segmentation. Subsequently, the Sørensen-Dice similarity coefficient was calculated to analyze the suitability of the results.

Clinical relevance— In this work, the detection and segmentation of cell nuclei in breast cancer histological images are carried out automatically. The method can identify cell nuclei regardless of variations in the level of staining and image magnification. Moreover, a granulometric analysis of the components allows identifying cell clumps and segment them into individual cell nuclei. Improved identification of cell nuclei under different image conditions was demonstrated to reach a sensitivity average of 0.76 ± 0.12. The results provide a base for further and complex processes such as cell counting, feature analysis, and nuclear pleomorphism, which are relevant tasks in the evaluation and diagnostic performed by the expert pathologist.

Keywords: Machine learning / Deep learning approaches, Image segmentation, CT imaging
Abstract: Abdominal aortic aneurysms (AAAs) are balloon-like dilations in the descending aorta that are associated with high mortality rates. Between 2009 and 2019, ruptured AAAs resulted in ~28,000 deaths while without mention of rupture AAAs led to ~15,000 deaths. Automating identification of the presence, 3D geometric structure, and precise location of AAAs can inform clinical risk of AAA rupture and timely interventions. We investigate the feasibility of automatic segmentation of AAAs, inclusive of the aorta, aneurysm sac, intra-luminal thrombus, and surrounding calcifications, using 30 patient-specific computed tomography angiograms (CTAs). Binary masks of the AAA region and their corresponding CTA images were used to train and test a 3D U-Net - a convolutional neural network (CNN) - model to automate AAA detection and studied model-specific convergence and overall segmentation accuracy via a loss-function developed based on the Dice Similarity Coefficient (DSC) for overlap between the predicted and actual segmentation masks. Further, we determined optimum probability thresholds (OPTs) for voxel-level probability outputs of a given model to optimize the DSC in our training set and utilized 3D volume rendering using the visualization tool kit (VTK) to validate the same and inform the parameter optimization exercise. Model-specific consistency with regard to improving accuracy by increasing training samples was examined by training the CNN with incrementally increasing training samples and examining trends in DSC and corresponding OPTs to determine AAA segmentations. Our final trained models consistently produced automatic segmentations that were visually accurate with train and test set losses in inference converging as our training sample size increased. Transfer learning led to improvements in DSC loss in inference, with the median OPT of both the training segmentations and testing segmentations approaching 0.5, as more training samples were utilized.

Keywords: Brain imaging and image analysis, Image registration, segmentation, compression and visualization - Volume rendering, Magnetic resonance imaging - MR neuroimaging
Abstract: In this paper, we proposed and validated a fully automatic pipeline for hippocampal surface generation via 3D U-net coupled with active shape modeling (ASM). Principally, the proposed pipeline consisted of three steps. In the beginning, for each magnetic resonance image, a 3D U-net was employed to obtain the automatic hippocampus segmentation at each hemisphere. Secondly, ASM was performed on a group of pre obtained template surfaces to generate mean shape and shape variation parameters through principal component analysis. Ultimately, hybrid particle swarm optimization was utilized to search for the optimal shape variation parameters that best match the segmentation. The hippocampal surface was then generated from the mean shape and the shape variation parameters. The proposed pipeline was observed to provide hippocampal surfaces at both hemispheres with high accuracy, correct anatomical topology, and sufficient smoothness.

Keywords: Image segmentation, Ultrasound imaging - Other organs, Image registration, segmentation, compression and visualization - Volume rendering
Abstract: Delineation of thyroid nodule boundaries is necessary for cancer risk assessment and accurate categorization of nodules. Clinicians often use manual or bounding-box approach for nodule assessment which leads to subjective results. Consequently, agreement in thyroid nodule categorization is poor even among experts. Computer-aided diagnosis systems could reduce this variability by minimizing the extent of user interaction and by providing precise nodule segmentations. In this study, we present a novel approach for effective thyroid nodule segmentation and tracking using a single user click on the region of interest. When a user clicks on an ultrasound sweep, our proposed model can predict nodule segmentation over the entire sequence of frames. Quantitative evaluations show that the proposed method out-performs the bounding box approach in terms of the dice score on a large dataset of 372 ultrasound images. The proposed approach saves expert time and reduces the potential variability in thyroid nodule assessment. The proposed one-click approach can save clinicians time required for annotating thyroid nodules within ultrasound images/sweeps. With minimal user interaction we would be able to identify the nodule boundary which can further be used for volumetric measurement and characterization of the nodule. This approach can also be extended for fast labeling of large thyroid imaging datasets suitable for training machine-learning based algorithms.

Keywords: Image classification, Brain imaging and image analysis, Image analysis and classification - Machine learning / Deep learning approaches
Abstract: Mild Cognitive Impairment (MCI) is the stage between the declining of normal brain function and the more serious decline of dementia. Alzheimer's disease (AD) is one of the leading forms of dementia. Although MCI does not always lead to AD, an early diagnosis of MCI may be helpful in finding those with early signs of AD. The Alzheimer's Disease Neuroimaging Initiative (ADNI) has utilized magnetic resonance imaging (MRI) for the diagnosis of MCI and AD. MCI can be separated into two types: Early MCI (EMCI) and Late MCI (LMCI). Furthermore, MRI results can be separated into three views of axial, coronal and sagittal planes. In this work, we perform binary classifications between healthy people and the two types of MCI based on limited MRI images using deep learning approaches. Specifically, we implement and compare two various convolutional neural network (CNN) architectures. The MRIs of 516 patients were used in this study: 172 control normal (CN), 172 EMCI patients and 172 LMCI patients. For this data set, 50% of the images were used for training, 20% for validation, and the remaining 30% for testing. The results showed that the best classification for one model was between CN and LMCI for the coronal view with an accuracy of 79.67%. In addition, we achieved 67.85% accuracy for the second proposed model for the same classification group.

Keywords: Image analysis and classification - Machine learning / Deep learning approaches, Brain imaging and image analysis
Abstract: Abstract—Background: In order to realize precision medicine, it is important to realize the detection of the individual atrophy of Alzheimer's disease (AD) patients. Our objective is to find individual brain regions of interest (ROIs) in AD patients via an unsupervised deep learning network. Methods: This study collected structural Magnetic Resonance Imaging (sMRI) scans with the 732 healthy control (HC) subjects and 202 AD patients from the Alzheimer’s disease Neuroimaging Initiative (ADNI), and the 105 HC subjects from Xuanwu Hospital. An unsupervised deep learning network based on Adversarial Autoencoders (AAE) was proposed to delineate individual atrophy of AD patients. In the proposed model, Variational Autoencoders (VAE) and Generative Adversarial Networks (GAN) were combined to learn the potential distribution and train a generator. In this step, the 530 HCs from ADNI were applied as the training dataset and the 105 HCs from Xuanwu Hospital were applied as an external validation dataset. The structural similarity (SSIM) was used to judge the robustness of the proposed model. Then, ROIs of the 202 AD patients were detected. In order to verify the clinical performance of these ROIs, other 202 HCs were selected from ADNI and a multilayer perceptron (MLP) was used to classify AD versus HC by 5 folder cross-validation. In the comparative experiments, we compared our model with three other previous models. Results: The SSIM reached 0.86 in both training and external validation datasets. Eventually, the classification accuracy of our model achieved 0.94±0.02. In the meanwhile, the classification accuracies were 0.89±0.01, 0.85±0.04 and 0.91±0.03 for the three previous methods. Conclusion: Our deep learning model could detect individual atrophy in AD patients. It may be a useful tool for AD diagnosis in clinics.

Keywords: Image feature extraction, Image analysis and classification - Machine learning / Deep learning approaches, Brain imaging and image analysis
Abstract: For survival prediction of brain tumor patients based on MRI scans, radiomic features have been a major research focus in the last years. However, radiomic features do not take the location of the lesion into account, which, in relation to the functional regions of the brain, could be a significant factor in predicting survival. An automatic and exact localization of the tumor in relation to specific functional areas is not straightforward, as typical brain parcellation methods fail in presence of large lesions. Here, we propose a model that replaces the tumorous region in 3D brain MRI scans with healthy tissue in order to improve the registration process towards a brain template. Further, we assemble a set of features for quantitative description of brain tumor location. On an openly available dataset, registration is strongly improved. The extracted location features also have better predictive performance when used after the proposed registration step and reach accuracies in survival prediction comparable to radiomic features.

Keywords: Magnetic resonance imaging - Diffusion tensor, diffusion weighted and diffusion spectrum imaging, Magnetic resonance imaging - MR neuroimaging, Brain imaging and image analysis
Abstract: We present an automatic algorithm for the group-wise parcellation of the cortical surface. The method is based on the structural connectivity obtained from representative brain fiber clusters, calculated via an inter-subject clustering scheme. Preliminary regions were defined from cluster-cortical mesh intersection points. The final parcellation was obtained using parcel probability maps to model and integrate the connectivity information of all subjects, and graphs to represent the overlap between parcels. Two inter-subject clustering schemes were tested, generating a total of 171 and 109 parcels, respectively. The resulting parcels were quantitatively compared with three state-of-the-art atlases. The best parcellation returned 69 parcels with a Dice similarity coefficient greater than 0.5. To the best of our knowledge, this is the first diffusion-based cortex parcellation method based on whole-brain inter-subject fiber clustering.

Keywords: Image analysis and classification - Machine learning / Deep learning approaches, Image feature extraction, Multimodal image fusion
Abstract: In this work, we compare the performance of six state-of-the-art deep neural networks in classification tasks when using only image features, to when these are combined with patient metadata. We utilise transfer learning from networks pretrained on ImageNet to extract image features from the ISIC HAM10000 dataset prior to classification. Using several classification performance metrics, we evaluate the effects of including metadata with the image features. Furthermore, we repeat our experiments with data augmentation. Our results show an overall enhancement in performance of each network as assessed by all metrics, only noting degradation in a vgg16 architecture. Our results indicate that this performance enhancement may be a general property of deep networks and should be explored in other areas. Moreover, these improvements come at a negligible additional cost in computation time, and therefore are a practical method for other applications.

Keywords: Image analysis and classification - Digital Pathology, Image analysis and classification - Machine learning / Deep learning approaches, Image feature extraction
Abstract: Deep learning enabled medical image analysis is heavily reliant on expert annotations which is costly. We present a simple yet effective automated annotation pipeline that uses autoencoder based heatmaps to exploit high level information that can be extracted from a histology viewer in an unobtrusive fashion. By predicting heatmaps on unseen images the model effectively acts like a robot annotator. The method is demonstrated in the context of coeliac disease histology images in this initial work, but the approach is task agnostic and may be used for other medical image annotation applications. The results are evaluated by a pathologist and also empirically using a deep network for coeliac disease classification. Initial results using this simple but effective approach are encouraging and merit further investigation, specially considering the possibility of scaling this up to a large number of users.

Keywords: Electrical source brain imaging, EEG imaging, Multimodal image fusion
Abstract: Interictal epileptiform discharges (IEDs) serve as sensitive but not specific biomarkers of epilepsy that can delineate the epileptogenic zone (EZ) in patients with drug resistant epilepsy (DRE) undergoing surgery. Intracranial EEG (icEEG) studies have shown that IEDs propagate in time across large areas of the brain. The onset of this propagation is regarded as a more spec ific biomarker of epilepsy than areas of spread. Yet, the limited spatial resolution of icEEG does not allow to identify the onset of this activity with high precision. Here, we propose a new method of mapping the spatiotemporal propagation of IEDs (and identify its onset) by using Electrical Source Imag-ing (ESI) on icEEG bypassing the spatial limitations of icEEG. We validated our method on icEEG recordings from 8 children with DRE who underwent surgery with good outcome (Engel score = 1). On each icEEG channel, we detected IEDs and identified the propagation onset using an automated algorithm. We localized the propagation of IEDs with dynamic Statistical Parametric Mapping (dSPM) using a time-sliding window approach. We defined two brain regions: the ESI-onset and ESI-spread zone. We estimated the overlap of these regions with resection volume (in percentage), which served as the gold standard of the EZ. We also estimated the mean distance of these regions from resection and clinically defined seizure onset zone (SOZ). We observed spatiotemporal propagation of IEDs in all patients across several channels (98 [85-102]) with a mean duration of 155 ms [96-186 ms]. A higher overlap with resection was seen for the ESI-onset zone compared to spread (73.3 % [ 47.4-100 %], 36.5 % [20.3-59.9 %], p = 0.008). The distance of the ESI-onset from resection was shorter compared to the ESI-spread zone and the same trend was observed for the distance from the SOZ. These findings show that our method can map the spatiotemporal propagation of IEDs and delineate its onset.

Keywords: Image segmentation, Machine learning / Deep learning approaches
Abstract: Surgical instrument segmentation is critical for the field of computer-aided surgery system. Most of deep-learning based algorithms only use either multi-scale information or multi-level information, which may lead to ambiguity of semantic information. In this paper, we propose a new neural network, which extracts both multi-scale and multi-level features based on the backbone of U-net. Specifically, the cascaded and double convolutional feature pyramid is input into the U-net. Then we propose a DFP (short for Dilation Feature-Pyramid) module for decoder which extracts multi-scale and multi-level information. The proposed algorithm is evaluated on two publicly available datasets, and extensive experiments prove that the five evaluation metrics by our algorithm are superior than other comparing methods.

Keywords: Image analysis and classification - Machine learning / Deep learning approaches, Image classification, Image feature extraction
Abstract: Brain surgery is complex and has evolved as a separate surgical specialty. Surgical procedures on the brain are performed using dedicated micro-instruments which are designed specifically for the requirements of operating with finesse in a confined space. The usage of these microsurgical tools in an operating environment defines the surgical skill of a surgeon. Video recordings of micro-surgical procedures are a rich source of information to develop automated surgical assessment tools that can offer continuous feedback for surgeons to improve their skills, effectively increase the outcome of the surgery, and make a positive impact on their patients. This work presents a novel deep learning system based on the Yolov5 algorithm to automatically detect, localize and characterize microsurgical tools from recorded intra-operative neurosurgical videos. The tool detection achieves a high 93.2% mean average precision. The detected tools are then characterized by their on-off time, motion trajectory and usage time. Tool characterization from neurosurgical videos offers useful insight into the surgical methods employed by a surgeon and can aid in their improvement. Additionally, a new dataset of annotated neurosurgical videos is used to develop the robust model and is made available for the research community.

Keywords: Image enhancement - Denoising, CT imaging, Image reconstruction and enhancement - Machine learning / Deep learning approaches
Abstract: X-ray Computed Tomography (CT) is an imaging modality where patients are exposed to potentially harmful ionizing radiation. To limit patient risk, reduced-dose protocols are desirable, which inherently lead to an increased noise level in the reconstructed CT scans. Consequently, noise reduction algorithms are indispensable in the reconstruction processing chain. In this paper, we propose to leverage a conditional Generative Adversarial Networks (cGAN) model, to translate CT images from low-to-routine dose. However, when aiming to produce realistic images, such generative models may alter critical image content. Therefore, we propose to employ a frequency-based separation of the input prior to applying the cGAN model, in order to limit the cGAN to high-frequency bands, while leaving low-frequency bands untouched. The results of the proposed method are compared to a state-of-the art model within the cGAN model as well as in a single-network setting. The proposed method generates visually superior results compared to the single-network model and the cGAN model in terms of quality of texture and preservation of fine structural details. It also appeared that the PSNR, SSIM and TV metrics are less important than a careful visual evaluation of the results. The obtained results demonstrate the relevance of defining and separating the input image into desired and undesired content, rather than blindly denoising entire images. This study shows promising results for further investigation of generative models towards finding a reliable deep learning-based noise reduction algorithm for low-dose CT acquisition.

Keywords: Optical imaging - Confocal microscopy, Image segmentation, Machine learning / Deep learning approaches
Abstract: Kidney biopsy interpretation is the gold standard for the diagnosis of and prognosis for kidney disease. Pathognomonic diagnosis hinges on the correct assessment of different structures within a biopsy that is manually visualized and interpreted by a renal pathologist. This laborious undertaking has spurred attempts to automate the process, offloading the consumption of temporal resources. Segmentation of kidney structures, specifically, the glomeruli, tubules, and interstitium, is a precursory step for disease classification problems. Translating renal disease decision making into a deep learning model for diagnostic and prognostic classification also relies on adequate segmentation of structures within the kidney biopsy. This study showcases a semi-automated segmentation technique where the user defines starting points for glomeruli in kidney biopsy images of both healthy normal and diabetic kidney disease stained with Nile Red that are subsequently partitioned into four areas: background, glomeruli, tubules and interstitium. Five of 30 biopsies that were segmented using the semi-automated method were randomly selected and the regions of interest were compared to the manual segmentation of the same images. Dice Similarity Coefficients (DSC) between the methods showed excellent agreement; Healthy (glomeruli: 0.92, tubules: 0.86, intersititium: 0.78) and diabetic nephropathy: (glomeruli: 0.94, tubules: 0.80, intersititium: 0.80). To our knowledge this is the first semi-automated segmentation algorithm performed with human renal biopsies stained with Nile Red. Utility of this methodology includes further image processing within structures across disease states based on biological morphological structures. It can also be used as input into a deep learning network to train semantic segmentation and input into a deep learning algorithm for classification of disease states.

Keywords: Image analysis and classification - Machine learning / Deep learning approaches, Image analysis and classification - Digital Pathology, Image segmentation
Abstract: Diabetic Retinopathy is a major cause of vision loss caused by retina lesions, including hard and soft exudates, microaneurysms, and hemorrhages. The development of a computational tool capable of detecting these lesions can assist in the early diagnosis of the most severe forms of the lesions and assist in the screening process and definition of the best treatment form. This paper proposes a computational model based on pre-trained convolutional neural networks capable of detecting fundus lesions to promote medical diagnosis support. The model was trained, adjusted, and evaluated using the DDR Diabetic Retinopathy dataset and implemented based on a YOLOv4 architecture and Darknet framework, reaching an mAP of 11.13% and a mIoU of 13.98%. The experimental results show that the proposed model presented results superior to those obtained in related works found in the literature.

Keywords: Novel imaging modalities, Photoacoustic, Optoacoustic, Thermoacoustic imaging
Abstract: The magnetic nanoparticles have been widely explored as an important kind of biomaterial for the treatment and diagnosis of cancer. Imaging of magnetic nanoparticles can greatly facili-tate treatment and diagnosis in both preclinical and clinical applications. The magnetoacoustic tomography is a non-invasive imaging modality for the distribution of the mag-netic nanoparticles. However, the traditional magnetoacoustic imaging system requires higher power and the large instanta-neous current that suffers cost and safety issues. In this paper, we propose a low-power magnetoacoustic tomography system, whose power amplifier only has 30 W peak power. The system used a pulse train of excitation to gain energy accumulation by resonance. And the reconstructed algorithm, i.e universal back-project was applied for imaging. To prove the feasibility and potential of the proposed system, we performed the imag-ing experiments with the gelatin phantom containing the mag-netic nanoparticles.

Keywords: X-ray imaging applications, Machine learning / Deep learning approaches
Abstract: Recent studies have shown that Dental Panoramic Radiograph (DPR) images have great potential for prescreening of osteoporosis given the high degree of correlation between the bone density and trabecular bone structure. Most of the research works in these area had used pretrained models for feature extraction and classification with good success. However, when the size of the data set is limited it becomes difficult to use these pretrained networks and gain high confidence scores. In this paper, we evaluated the diagnostic performance of deep convolutional neural networks (DCNN)-based computer-assisted diagnosis (CAD) system in the detection of osteoporosis on panoramic radiographs, through a comparison with diagnoses made by oral and maxillofacial radiologists. With the available labelled dataset of 70 images, results were reproduced for the preliminary study model. Furthermore, the model performance was enhanced using different computer vision techniques. Specifically, the age meta data available for each patient was leveraged to obtain more accurate predictions. Lastly, we tried to leverage these images, ages and osteoporotic labels to create a neural network based regression model and predict the Bone Mineral Density (BMD) value for each patient. Experimental results showed that the proposed CAD system was in high accord with experienced oral and maxillofacial radiologists in detecting osteoporosis and achieved 87.86% accuracy.

Keywords: Magnetic resonance imaging - Parallel MRI, Image reconstruction - Fast algorithms, Image reconstruction and enhancement - Machine learning / Deep learning approaches
Abstract: As an inverse problem, parallel magnetic resonance imaging (pMRI) reconstruction accelerates imaging speed by interpolating missing k-space data from a group of phased-array coils. Deep learning methods have been used for improving pMRI reconstruction quality in recent years. However, deep learning approaches need a large amount of training data that are acquired from different hardware configurations and anatomical areas. Data distributions may be different between training data and testing data, and a long-time training is needed. In this work, we proposed a broad learning system based parallel MRI reconstruction that exploits approximation capability of one-layer neural network through broadening network structure with expanded nodes. Experimental results show that the proposed method is able to suppress noise in compared to the conventional pMRI reconstruction.

Keywords: Ultrasound imaging - Other organs, Image reconstruction and enhancement - Parametric image reconstruction
Abstract: Ultrasound (US) imaging is becoming the routine modality for the diagnosis and prognosis of lung pathologies. Lung US imaging relies on artifacts from acoustic impedance (Z) mismatches to distinguish and interpret the normal and pathological lung conditions. The air-pleura interface of the normal lung displays specularity due to the huge Z mismatches. However, in the presence of pathologies, the interface alters exhibiting a diffuse behavior due to increased density and reduced spatial distribution of air in the sub-pleural space. The specular or the diffuse behavior influences the reflected acoustic intensity distribution. This study aims to understand the reflection pattern in a normal and pathological lung through a novel approach of determining pixel-level acoustic intensity vector field (IVF) at high frame rates. Detailed lung modeling procedures using k-Wave US toolbox under normal, edematous, and consolidated conditions are illustrated. The analysis of the IVF maps on the three lung models clearly shows the drifting of the air-pleura interface from specular to diffuse with the severity of the pathology.

Keywords: Image segmentation, Machine learning / Deep learning approaches
Abstract: Convolutional neural networks are increasingly used in the medical field for the automatic segmentation of several anatomical regions on diagnostic and non-diagnostic images. Such automatic algorithms allow to speed up time-consuming processes and to avoid the presence of expert personnel, reducing time and costs. The present work proposes the use of a convolutional neural network, the U-net architecture, for the segmentation of ear elements. The auricular elements segmentation process is a crucial step of a wider procedure, already automated by the authors, that has as final goal the realization of surgical guides designed to assist surgeons in the reconstruction of the external ear. The segmentation, performed on depth map images of 3D ear models, aims to define of the contour of the helix, antihelix, tragus-antitragus and concha. A dataset of 131 ear depth map was created;70% of the data are used as the training set, 15% composes the validation set, and the remaining 15% is used as testing set. The network showed excellent performance, achieving 97% accuracy on the validation test.

Keywords: Image segmentation, Machine learning / Deep learning approaches, Optical imaging and microscopy - Microscopy
Abstract: In this work we try to address if there is a better way to classify two distributions, rather than using histograms; and answer if we can make a deep learning network learn and classify distributions automatically. These improvements can have wide ranging applications in computer vision and medical image processing. More specifically, we propose a new vessel segmentation method based on pixel distribution learning under multiple scales. In particular, a spatial distribution descriptor named Random Permutation of Spatial Pixels (RPoSP) is derived from vessel images and used as the input to a convolutional neural network for distribution learning. Based on our preliminary experiments we currently believe that a wide network, rather than a deep one, is better for distribution learning. There is only one convolutional layer, one rectified linear layer and one fully connected layer followed by a softmax loss in our network. Furthermore, in order to improve the accuracy of the proposed approach, the RPoSP features are captured at multiple scales and combined together to form the input of the network. Evaluations using standard benchmark datasets demonstrate that the proposed approach achieves promising results compared to the state-of-the-art.

Keywords: Ultrasound imaging - Cardiac, Fetal and Pediatric Imaging, Cardiac imaging and image analysis
Abstract: Automatic analysis of fetal heart and related components in fetal echocardiography can help cardiologists to reach a diagnosis for Congenital Heart Disease (CHD). Previous studies mainly focused on cardiac chamber segmentation, while few researches deal with the cardiac component detection. In this paper, we tackle the task of simultaneous detection of the fetal heart and descending aorta in four-chamber view of fetal echocardiography, which is useful to analyze some kinds of CHD, such as left/right atrial isomerism, dextroversion of heart, etc. Several CNN-based object detection methods with different backbones are thoroughly evaluated, and finally, the Hybrid Task Cascade method with HRNet is selected as the detection method. Experiments on a fetal echocardiography dataset show that the method can achieve superior performance according to common-used evaluation metrics.

Keywords: Image classification, Image feature extraction, Novel imaging modalities
Abstract: Machine learning algorithms are progressively assuming important roles as computational tools to support clinical diagnosis, namely in the classification of pigmented skin lesions using RGB images. Most current classification methods rely on common 2D image features derived from shape, colour or texture, which does not always guarantee the best results. This work presents a contribution to this field, by exploiting the lesions' border line characteristics using a new dimension -- depth, which has not been thoroughly investigated so far. A selected group of features is extracted from the depth information of 3D images, which are then used for classification using a quadratic Support Vector Machine. Despite class imbalance often present in medical image datasets, the proposed algorithm achieves a top geometric mean of 94.87%, comprising 100.00% sensitivity and 90.00% specificity, using only depth information for the detection of Melanomas. Such results show that potential gains can be achieved by extracting information from this often overlooked dimension, which provides more balanced results in terms of sensitivity and specificity than other settings.

Keywords: Image analysis and classification - Digital Pathology, Machine learning / Deep learning approaches, Image segmentation
Abstract: Chagas disease is a widely spreaded illness caused by the parasite Trypanosoma cruzi (T. cruzi). Most cases go unnoticed until the accumulated myocardial damage affect the patient. The endomyocardium biopsy is a tool to evaluate sustained myocardial damage, but analyzing histopathological images takes a lot of time and its prone to human error, given its subjective nature. The following work presents a deep learning method to detect T. cruzi amastigotes on histopathological images taken from a endomyocardium biopsy during an experimental murine model. A U-Net convolutional neural network architecture was implemented and trained from the ground up. An accuracy of 99.19% and Jaccard index of 49.43% were achieved. The obtained results suggest that the proposed approach can be useful for amastigotes detection in histopathological images.

Keywords: Image analysis and classification - Machine learning / Deep learning approaches, Image segmentation, Image classification
Abstract: Tracking an individual’s food intake provides useful insight into their eating habits. Technological advancements in wearable sensors such as the automatic capture of food images from wearable cameras have made the tracking of food intake efficient and feasible. For accurate food intake monitoring, an automated food detection technique is needed to recognize foods from unstaged real-world images. This work presents a novel food detection and segmentation pipeline to detect the presence of food in images acquired from an egocentric wearable camera, and subsequently segment the food image. An ensemble of YOLOv5 detection networks is trained to detect and localize food items among other objects present in captured images. The model achieves an overall 80.6% mean average precision on four objects—Food, Beverage, Screen, and Person. Post object detection, the predicted food objects which are sufficiently sharp were considered for segmentation. The Normalized-Graph-Cut algorithm was used to segment the different parts of the food resulting in an average IoU of 82%.

Keywords: Optical imaging and microscopy - Fluorescence microscopy, Image enhancement
Abstract: This paper focuses on the analysis of image sequences acquired during fast photobleaching using a standard wide-field microscope. We show that the photobleaching rate estimated for each pixel is not constant for the whole field of view, but it provides a new spatially variant parametric image related to the cell structure and diffusion of fluorophores. We also provide an alternative way to estimate a pixel-wise photobleaching rate with significantly less computation time than exponential model fitting.

Keywords: Fetal and Pediatric Imaging, Brain imaging and image analysis, Magnetic resonance imaging - Perinatal
Abstract: In magnetic resonance imaging (MRI) studies of fetal brain development, structural brain atlases usually serve as essential references for the fetal population. Individual images are spatially normalized into a common or standard atlas space to extract regional information on volumetric or morphological brain variations. However, the existing fetal brain atlases are mostly based on MR images obtained from Caucasian populations and thus are not ideal for the characterization of the brains of the Chinese population due to neuroanatomical differences related to genetic factors. In this paper, we use an unbiased template construction algorithm to create a set of age-specific Chinese fetal atlases between 21-35 weeks of gestation from 115 normally developing fetal brains. Based on the 4D spatiotemporal atlas, the morphologically developmental patterns, e.g., cortical thickness, sulcal and gyral patterns, were quantified from in utero MRI. Additionally, after applying the Chinese fetal atlases and Caucasian fetal atlases to an independent Chinese pediatric dataset, we find that the Chinese fetal atlases result in significantly higher accuracy than the Caucasian fetal atlases in guiding brain tissue segmentation. These results suggest that the Chinese fetal brain atlases are necessary for quantitative analysis of the typical and atypical development of the Chinese fetal population in the future.

Keywords: Image segmentation, Image feature extraction
Abstract: The major breakthroughs in the fields of reverse engineering and additive manufacturing have dramatically changed medical practice in recent years, pushing for a modern clinical model in which each patient is considered unique. Among the wide spectrum of medical applications, reconstructive surgery is experiencing the most benefits from this new paradigm. In this scenario, the present paper focuses on the design and development of a tool able to support the surgeon in the reconstruction of the external ear in case of malformation or total absence of the anatomy. In particular, the paper describes an appositely devised software tool, named G-ear, which enables the semi-automatic modeling of intraoperative devices to guide the physician through ear reconstruction surgery. The devised system includes 3D image segmentation, semi-automated CAD modelling and 3D printing to manufacture a set of patient-specific surgical guides for ear reconstruction. Usability tests were carried out among the surgeons of the Meyer Children's Hospital to obtain an assessment of the software by the end user. The devised system proved to be fast and efficient in retrieving the optimal 3D geometry of the surgical guides and, at the same time, to be easy to use and intuitive, thus achieving high degrees of likability.

Keywords: Image analysis and classification - Machine learning / Deep learning approaches, Brain imaging and image analysis, Magnetic resonance imaging - Diffusion tensor, diffusion weighted and diffusion spectrum imaging
Abstract: Diffusion Tensor Imaging (DTI) is widely used to find brain biomarkers for various stages of brain structural and neuronal development. Processing DTI data requires a detailed Quality Assessment (QA) to detect artifactual volumes amongst a large pool of data. Since large cohorts of brain DTI data are often used in different studies, manual QA of such images is very labor-intensive. In this paper, a deep learning-based tool is developed for quick automatic QA of 3D raw diffusion MR images. We propose a 2-step framework to automate the process of binary (i.e., ‘good’ vs ‘poor’) quality classification of diffusion MR images. In the first step, using two separately trained 3D convolutional neural networks with different input sizes, quality labels for individual Regions of Interest (ROIs) sampled from whole DTI volumes are predicted. In the second step, two distinct novel voting systems are designed and fine-tuned to predict the quality label of whole brain DTI volumes using the individual ROI labels predicted in the previous step. Our results demonstrate the validity and practicality of our tool. Specifically, using a balanced dataset of 6,940 manually-labeled 3D DTI volumes from 85 unique subjects for training, validation, and testing, our model achieves 100% accuracy via one voting system, and 98% accuracy via another voting system on the same test set.

Keywords: Image analysis and classification - Machine learning / Deep learning approaches, Magnetic resonance imaging - MR neuroimaging
Abstract: In this work, we proposed and validated a hybrid learning pipeline for automated diagnosis of first-episode schizophrenia (FES) utilizing T1-weighted images. Amygdalar and hippocampal shape abnormalities in FES have been observed in previous studies. In this work, we jointly made use of two types of features, together with advanced machine learning techniques, for an automated discrimination of FES and healthy control (96 versus 102). Specifically, we first employed a ResNet34 model to extract convolutional neural network (CNN) features. We then combined these CNN features with shape features of the bilateral hippocampi and the bilateral amygdalas, before being inputted to advanced classification algorithms such as the Gradient Boosting Decision Tree (GBDT) for classifying between FES and healthy control. Shape features were represented using log Jacobian determinants, through a well-established statistical shape analysis pipeline. When combining CNN with hippocampal shape, the best results came from utilizing GBDT as the classifier, with an overall accuracy of 75.15%, a sensitivity of 69.35%, a specificity of 80.19%, an F1 of 72.16%, and an AUC of 79.68%. When combing CNN and amygdalar shape, the best results came from utilizing Bagging as the classifier, with an overall accuracy of 74.39%, a sensitivity of 67.93%, a specificity of 80%, an F1 of 71.11%, and an AUC of 80.98%. Compared with using each single set of features, either CNN or shape, significant improvements have been observed, in terms of FES discrimination. To the best of our knowledge, this is the first work that has tried to combine CNN features and hippocampal/amygdalar shape features for automated FES identification.

Keywords: Image segmentation, Machine learning / Deep learning approaches, Image reconstruction and enhancement - Machine learning / Deep learning approaches
Abstract: Thyroid ultrasound (US) image segmentation is of great significance for both doctors and patients. However, it is a challenging task because of the low image quality, low contrast and complex background in each US image. In recent years, some researchers have done thyroid nodule segmentation tasks, but the results achieved are not particularly satisfactory. In this paper, we have broadened the targets of interest and included both thyroid nodules and capsules into our research scope. We propose a method that implements a C-MMDetection to detect and extract the region of interest (ROI), and a modified salient object detection network U2-RNet to segment nodules and capsules respectively. Experiments show that our method segments nodules and capsules in US images more effectively than other networks, which is very helpful for doctors to diagnose central compartment lymph node metastasis (CLNM).

Keywords: Image segmentation, Machine learning / Deep learning approaches, Optical imaging and microscopy - Microscopy
Abstract: Karyotyping is an important process for finding chromosome abnormalities that could cause genetic disorders. This process first requires cytogeneticists to arrange each chromosome from the metaphase image to generate the karyogram. In this process, chromosome segmentation plays an important role and it is directly related to whether the karyotyping can be achieved. The key to achieving accurate chromosome segmentation is to effectively segment the multiple touching and overlapping chromosomes at the same time identify the isolated chromosomes. This paper proposes a method named Enhanced Rotated Mask R-CNN for automatic chromosome segmentation and classification. The Enhanced Rotated Mask R-CNN method can not only accurately segment and classify the isolated chromosomes in metaphase images but also effectively alleviate the problem of inaccurate segmentation for touching and overlapping chromosomes. Experiments show that the proposed approach achieves competitive performances with 49.52 AP on multi-class evaluation and 69.96 AP on binary-class evaluation for chromosome segmentation.

Keywords: Image analysis and classification - Machine learning / Deep learning approaches, Brain imaging and image analysis, Image feature extraction
Abstract: Millions of people around the world suffer from Parkinson’s disease, a neurodegenerative disorder with no remedy. Currently, the best response to interventions is achieved when the disease is diagnosed at an early stage. Supervised machine learning models are a common approach to assist early diagnosis from clinical data, but their performance is highly dependent on available example data and selected input features. In this study, we explore 23 single photon emission computed tomography (SPECT) image features for the early diagnosis of Parkinson’s disease on 646 subjects. We achieve 94 % balanced classification accuracy in independent test data using the full feature space and show that matching accuracy can be achieved with only eight features, including original features introduced in this study. All the presented features can be generated using a routinely available clinical software and are therefore straightforward to extract and apply.

Keywords: Image classification, Image feature extraction, Image analysis and classification - Machine learning / Deep learning approaches
Abstract: Knowing the type (i.e., the biochemical composition) of kidney stones is crucial to prevent relapses with an appropriate treatment. During ureteroscopies, kidney stones are fragmented, extracted from the urinary tract, and their composition is determined using a morpho-constitutional analysis. This procedure is time consuming (the morpho-constitutional analysis results are only available after some days) and tedious (the fragment extraction lasts up to an hour). Identifying the kidney stone type only with the in-vivo endoscopic images would allow for the dusting of the fragments, while the morpho-constitutional analysis could be avoided. Only few contributions dealing with the in vivo identification of kidney stones were published. This paper discusses and compares five classification methods including deep convolutional neural networks (DCNN)-based approaches and traditional (non DCNN-based) ones. Even if the best method is a DCCN approach with a precision and recall of 98% and 97% over four classes, this contribution shows that a XGBoost classifier exploiting well chosen feature vectors can closely approach the performances of DCNN classifiers for a medical application with a limited number of annotated data.

Keywords: CT imaging
Abstract: For the CT iterative reconstruction, choosing the parameters of different regularization terms has been a difficult problem. Transforming the reconstruction problem into constrained optimization can solve this problem, but determining the constraint range and accurately solving it remains a challenge. This paper proposes a CT reconstruction method based on constrained data fidelity term, which estimates the constrained range with Taylor expansion. We respectively use Douglas-Rachford splitting (DRS) and Projection-based primal-dual algorithm (PPD) to split the reconstruction problem and solve the data fidelity subproblem. This method can accurately estimate the constrained range of data fidelity terms to ensure reconstruction accuracy and apply different regularization terms to reconstruction without parameter adjustment. TV, L0TV, and BM3D, which are convex, nonconvex regularization terms and filtering operations, respectively, are applied to reconstruction experiments. Simulation results show that the proposed method can converge for different regularization terms, and its reconstruction quality is better than the filtered back-projection.

Keywords: Image classification, Machine learning / Deep learning approaches, Image analysis and classification - Machine learning / Deep learning approaches
Abstract: Ocular surface disorder is one of common and prevalence eye diseases and complex to be recognized accurately. This work presents automatic classification of ocular surface disorders in accordance with densely connected convolutional networks and smartphone imaging. We use various smartphone cameras to collect clinical images that contain normal and abnormal, and modify end-to-end densely connected convolutional networks that a the hybrid unit to learn more diverse features, significantly reducing the network depth, the total number of parameters and the float calculation. The validation results demonstrate that our proposed method provides a promising and effective strategy to accurately screen ocular surface disorders. In particular, our deeply learned smartphone photographs based classification method achieved an average automatic recognition accuracy of 90.6%, while it is conveniently used by patients and integrated into smartphone applications for automatic patient-self screening ocular surface diseases without seeing a doctor in person in a hospital.

Keywords: Image analysis and classification - Machine learning / Deep learning approaches, Machine learning / Deep learning approaches
Abstract: In this paper, we proposed and validated a multi-task based deep learning method for simultaneously segmenting the foveal avascular zone (FAZ) and classifying three ocular disease related states (normal, diabetic, and myopia) utilizing optical coherence tomography angiography (OCTA) images. The essential motivation of this work is that reliable predictions on disease states may be made based on features extracted from a segmentation network, by sharing a same encoder between the classification network and the segmentation network. In this study, a cotraining network structure was designed for simultaneous ocular disease discrimination and FAZ segmentation. Specifically, we made use of a classification head following a segmentation network's encoder, so that the classification branch used the feature information extracted in the segmentation branch to improve the classification results. The performance of our proposed network structure has been tested and validated on the FAZID dataset, with the best Dice and Jaccard being 0.9031pm0.0772 and 0.8302pm0.0990 for FAZ segmentation, and the best Accuracy and Kappa being 0.7533 and 0.6282 for classifying three ocular disease related states.

Keywords: X-ray imaging applications, Image analysis and classification - Machine learning / Deep learning approaches, Image classification
Abstract: Disease detection using chest X-ray (CXR) images is one of the most popular radiology methods to diagnose diseases through a visual inspection of abnormal symptoms in the lung region. A wide variety of diseases such as pneumonia, heart failure and lung cancer can be detected using CXRs. Although CXRs can show the symptoms of a variety of diseases, detecting and manually classifying those diseases can be difficult and time-consuming adding to clinicians' work burden. Research shows that nearly 90% of mistakes made in a lung cancer diagnosis involved chest radiography. A variety of algorithms and computer-assisted diagnosis tools (CAD) were proposed to assist radiologists in the interpretation of medical images to reduce diagnosis errors. In this work, we propose a deep learning approach to screen multiple diseases using more than 220,000 images from the CheXpert dataset. The proposed binary relevance approach using Deep Convolutional Neural Networks (CNNs) achieves high performance results and outperforms past published work in this area.

Keywords: Image analysis and classification - Machine learning / Deep learning approaches, Image segmentation, Machine learning / Deep learning approaches
Abstract: The vascular topology is of vital importance in building a chemotherapy model for the liver cancer in rats. And segmentation of vessels in the liver is an indispensable part of vessels' topological analysis. In this paper, we proposed and validated a novel pipeline for segmenting liver vessels and extracting their skeletons for topological analysis. We employed a dual-attention based U-Net trained in a generative adversarial network (GAN) fashion to obtain precise segmentations of vessels. For subsequent topological analysis, the vessels' skeletons are extracted and classified according to their lengths and bifurcation orders. Based on 40 samples with carefully-annotated ground truth labels, our experiments revealed consistent superiority in terms of both segmentation accuracy and topology correctness, demonstrating the robustness of the proposed pipeline.

Keywords: EEG imaging, Image analysis and classification - Machine learning / Deep learning approaches, Image classification
Abstract: A convolution neural network (CNN) architecture has been designed to classify epileptic seizures based on two-dimensional (2D) images constructed from decomposed mono-components of electroencephalogram (EEG) signals. For the decomposition of EEG, Hilbert vibration decomposition (HVD) has been employed. In this work, four brain rhythms – delta, theta, alpha, and beta have been utilized to obtain the mono-components. Certainly, the data-driven CNN model is most efficient for 2D image processing and recognition. Therefore, 2D images have been generated from one-dimensional (1D) decomposed mono-components by employing continuous wavelet transform (CWT). Next, simultaneous multiple input images in parallel have been directly fed into the CNN pipeline for feature extraction and classification. For evaluation, the EEG dataset provided by the Bonn University has been taken into consideration. Further, a 5-fold cross-validation technique has been applied to obtain generalized and robust classification performance. The average classification accuracy, sensitivity, and specificity reached up to 98.6%, 97.2%, and 100% respectively. The results show that the proposed idea is very much efficient in seizure classification. The proposed idea resourcefully combines the advantages of HVD and CNN to classify epileptic seizures from EEG signal.

Keywords: Optical imaging - Coherence tomography, Image classification, Image analysis and classification - Machine learning / Deep learning approaches
Abstract: Although automated pathology classification using deep learning (DL) has proved to be predictively efficient, DL methods are found to be data and compute cost intensive. In this work, we aim to reduce DL training costs by pre-training a Resnet feature extractor using SimCLR contrastive loss for latent encoding of OCT images. We propose a novel active learning framework that identifies a minimal sub-sampled dataset containing the most uncertain OCT image samples using label propagation on the SimCLR latent encodings. The pre-trained Resnet model is then fine-tuned with the labelled minimal sub-sampled data and the underlying pathological sites are visually explained. Our framework identifies upto 2% of OCT images to be most uncertain that need prioritized specialist attention and that can fine-tune a Resnet model to achieve upto 97% classification accuracy. The proposed method can be extended to other medical images to minimize prediction costs.

Keywords: Image analysis and classification - Machine learning / Deep learning approaches, Image feature extraction, Image classification
Abstract: Supervised machine learning methods are usually used to build a custom model for disease diagnosis and auxiliary prognosis in radiomics studies. A classical machine learning pipeline involves a series of steps and multiple algorithms, which leads to a great challenge to find an appropriate combination of algorithms and an optimal hyper-parameter set for radiomics model building. We developed a freely available software package for radiomics model building. It can be used to lesion labeling, feature extraction, feature selection, classifier training and statistic result visualization. This software provides a user-friendly graphic interface and flexible IOs for radiologists and researchers to automatically develop radiomics models. Moreover, this software can extract features from corresponding lesion regions in multi-modality images, which is labeled by semi-automatic or full-automatic segmentation algorithms. It is designed in a loosely coupled architecture, programmed with Qt, VTK, and Python. In order to evaluate the availability and effectiveness of the software, we utilized it to build a CT-based radiomics model containing peritumoral features for malignancy grading of cell renal cell carcinoma. The final model got a good performance of grading study with AUC=0.848 on independent validation dataset.

Keywords: Image visualization, Electrical source imaging, Image analysis and classification - Machine learning / Deep learning approaches
Abstract: Conventional electrocardiograms (ECG) are displayed in one dimension. Reading one-dimensional ECG waveform becomes challenging when one wants to visualize the heart rate variability with naked eye. Some ECG visualization techniques have been proposed. However, they rely on domain knowledge to comprehend the heart rate variability. To improve the readability for patients and non-experts, we introduce Star-ECG, a novel ECG visualization approach. Such approach projects ECG waveforms onto a two-dimensional plane in a circular form. We demonstrate that Star-ECG offers not only easily deciphered visualization of cardiac abnormalities and heart rate variability, but also the application of state-of-the-art arrhythmia classification with integrated deep neural networks. We also report positive user feedback from both experts and non-experts that Star-ECG can provide readable and helpful information to monitor cardiac activities.

Clinical relevance — A powerful and easy-to-read ECG visualization tool can critically improve healthcare environment and raise awareness of abnormal cardiovascular functioning.

Keywords: Infra-red imaging, Image visualization
Abstract: Complex Regional Pain Syndrome (CRPS) is a pain disorder that can be triggered by injury or trauma affecting most often limbs. Its complex pathophysiology makes its diagnosis and treatment a demanding task. To reduce pain, patients diagnosed with CRPS commonly undergo sympathetic blocks which involves the injection of a local anesthetic drug around the nerves. Currently, this procedure is guided by fluoroscopy which occasionally is considered as little accurate. For this reason, the use of infrared thermography as a technique of support has been considered. In this work, thermal images of feet soles in patients with lower limbs CRPS undergoing lumbar sympathetic blocks were recorded and evaluated. The images were analyzed by means of a computer-aided intuitive software tool developed using MATLAB. This tool provides the possibility of editing regions of interest, extracting the most important information of these regions and exporting the results data to an Excel file.

Keywords: CT imaging applications, Image segmentation, CT imaging
Abstract: Primary Live Cancer (PLC) is the sixth most common cancer worldwide and its occurrence predominates in patients with chronic liver diseases and other risk factors like hepatitis B and C. Treatment of PLC and malignant liver tumors depend both in tumor characteristics and the functional status of the organ, thus must be individualized for each patient. Liver segmentation and classification according to Couinaud’s classification is essential for computer-aided diagnosis and treatment planning, however, manual segmentation of the liver volume slice by slice can be a time-consuming and challenging task and it is highly dependent on the experience of the user. We propose an alternative automatic segmentation method that allows accuracy and time consumption amelioration. The procedure pursues a multi-atlas based classification for Couinaud segmentation. Our algorithm was implemented on 20 subjects from the IRCAD 3D data base in order to segment and classify the liver volume in its Couinaud segments, obtaining an average DICE coefficient of 0.94.

Keywords: Machine learning / Deep learning approaches, Image segmentation, Brain imaging and image analysis
Abstract: The earlier studies on brain vasculature semantic segmentation used classical image analysis methods to extract the vascular tree from within the images. Nowadays, deep learning methods are widely exploited for various image analysis tasks. One of the strong restrictions when dealing with neural networks in the framework of semantic segmentation is the need to dispose of a ground truth segmentation dataset, on which the task will be learned. It may be cumbersome to manually segment the arteries in a 3D volume (MRA-TOF typically). In this work, we aim to tackle the vascular tree segmentation from a new perspective. Our objective is to build an image dataset from mouse vasculatures acquired using CT-Scans, and enhance these vasculatures in such a way to precisely mimic the statistical properties of the human brain. The segmentation of mouse images is easily automatized thanks to their specific acquisition modality. Thus, such a framework allows to generate the data necessary for the training of a Convolutional Neural Network - i.e. the enhanced mouse images and there corresponding ground truth segmentation - without requiring any manual segmentation procedure. However, in order to generate an image dataset having consistent properties (strong resemblance with MRA images), we have to ensure that the statistical properties of the enhanced mouse images do match correctly the human MRA acquisitions. In this work, we evaluate at length the similarities between the human arteries as acquired on MRA-TOF and the ``humanized'' mouse arteries produced by our model. Finally, once the model duly validated, we experiment its applicability with a Convolutional Neural Network.

Keywords: Image reconstruction and enhancement - Image synthesis, Image reconstruction and enhancement - Machine learning / Deep learning approaches
Abstract: The diagnosis and treatment of eye diseases is heavily reliant on the availability of retinal imagining equipment. To increase accessibility, lower-cost ophthalmoscopes, such as the Arclight, have been developed. However, a common drawback of these devices is a limited field of view. The narrow-field-of-view images of the eye can be concatenated to replicate a wide field of view. However, it is likely that not all angles of the eye are captured, which creates gaps. This limits the usefulness of the images in teaching, wherefore, artist's impressions of retinal pathologies are used. Recent research in the field of computer vision explores the automatic completion of holes in images by leveraging the structural understanding of similar images gained by neural networks. Specifically, generative adversarial networks are explored, which consist of two neural networks playing a game against each other to facilitate learning. We demonstrate a proof of concept for the generative image inpainting of retinal images using generative adversarial networks. Our work is motivated by the aim of devising more realistic images for medical teaching purposes. We propose the use of a Wasserstein generative adversarial network with a semantic image inpainting algorithm, as it produces the most realistic images.

Keywords: Image segmentation
Abstract: Detection of lung contour on chest X-ray images (CXRs) is a necessary step for computer-aid medical imaging analysis. Because of the low-intensity contrast around lung boundary and large inter-subject variance, it is challenging to detect lung from structural CXR images accurately. To tackle this problem, we design an automatic and hybrid detection network containing two stages for lung contour detection on CXRs. In the first stage, an image preprocessing stage based on a deep learning model is used to automatically extract coarse lung contours. In the second stage, a refinement step is used to fine-tune the coarse segmentation results based on an improved principal curve-based method coupled with an improved machine learning method. The model is evaluated on several public datasets, and experiments demonstrate that the performance of the proposed method outperforms state-of-the-art methods.

Keywords: Image reconstruction and enhancement - Machine learning / Deep learning approaches, Image enhancement - Denoising, Machine learning / Deep learning approaches
Abstract: Artifacts and defects in Cone-beam Computed Tomography (CBCT) images have become a problem in radiotherapy and surgical assistance.Unsupervised learning-based image translation techniques have been studied to improve the image quality of head and neck CBCT images, but there have been few studies on improving the image quality of abdominal CBCT images, which are highly affected by organ deformation due to posture and breathing. In this study, we propose a method to improve the image quality of abdominal CBCT images by statistically translating them to the original CT values.This method preserves the anatomical structure through adversarial learning that corresponds the imaging regions between CBCT and CT images of the same case. The image translation model learned from 68 CT-CBCT datasets was applied to 8 test datasets, and the effectiveness of the proposed method in improving the image quality of CBCT images was confirmed.

Keywords: Ultrasound imaging - Elastography
Abstract: Accurately estimating all strain components in quasi-static ultrasound elastography is crucial for the full analysis of biological media. In this paper, 2D strain tensor imaging is investigated, using a partial differential equation (PDE)-based regularization method. More specifically, this method employs the tissue property of incompressibility to smooth the displacement fields and reduce the noise in the strain components. The performance of the method is assessed with phantoms and in vivo breast tissues. For all the media examined, the results showed a significant improvement in both lateral displacement and strain but also, to a lesser extent, in the shear strain. Moreover, axial displacement and strain were only slightly modified by the regularization, as expected. Finally, the easier detectability of the inclusion/lesion in the final lateral strain images is associated with higher elastographic contrast-to-noise ratios (CNRs), with values in the range [0.68 - 9.40] vs [0.09 - 0.38] before regularization.

Keywords: Image analysis and classification - Machine learning / Deep learning approaches, Machine learning / Deep learning approaches, Image analysis and classification - Digital Pathology
Abstract: Deep Neural Networks using histopathological images as an input currently embody one of the gold standards in automated lung cancer diagnostic solutions, with Deep Convolutional Neural Networks achieving the state of the art values for tissue type classification. One of the main reasons for such results is the increasing availability of voluminous amounts of data, acquired through the efforts employed by extensive projects like The Cancer Genome Atlas. Nonetheless, whole slide images remain weakly annotated, as most common pathologist annotations refer to the entirety of the image and not to individual regions of interest in the patient's tissue sample. Recent works have demonstrated Multiple Instance Learning as a successful approach in classification tasks entangled with this lack of annotation, by representing images as a bag of instances where a single label is available for the whole bag. Thus, we propose a bag/embedding-level lung tissue type classifier using Multiple Instance Learning, where the automated inspection of lung biopsy whole slide images determines the presence of cancer in a given patient. Furthermore, we use a post-model interpretability algorithm to validate our model's predictions and highlight the regions of interest for such predictions.

Keywords: Machine learning / Deep learning approaches, Image segmentation
Abstract: Lung segmentation represents a fundamental step in the development of computer-aided decision systems for the investigation of interstitial lung diseases. In a holistic lung analysis, eliminating background areas from Computed Tomography (CT) images is essential to avoid the inclusion of noise information and spend unnecessary computational resources on non-relevant data. However, the major challenge in this segmentation task relies on the ability of the models to deal with imaging manifestations associated with severe disease. Based on U-net, a general biomedical image segmentation architecture, we proposed a light-weight and faster architecture. In this 2D approach, experiments were conducted with a combination of two publicly available databases to improve the heterogeneity of the training data. Results showed that, when compared to the original U-net, the proposed architecture maintained performance levels, achieving 0.894 +/- 0.060, 4.493 +/- 0.633 and 4.457 +/- 0.628 for DSC, HD and HD-95 metrics, respectively, when using all patients from the ILD database for testing only. while allowing a more efficient computational usage. Quantitative and qualitative evaluations on the ability to cope with high-density lung patterns associated with severe disease were conducted, supporting the idea that more representative and diverse data are necessary to build robust and reliable segmentation tools.

Keywords: Brain imaging and image analysis, Image registration, segmentation, compression and visualization - Volume rendering, Multimodal imaging
Abstract: A significant challenge for brain histological data analysis is to precisely identify anatomical regions in order to perform accurate local quantifications and evaluate therapeutic solutions. Usually, this task is performed manually, becoming therefore tedious and subjective. Another option is to use automatic or semi-automatic methods, among which segmentation using digital atlases co-registration. However, most available atlases are 3D, whereas digitized histological data are 2D. Methods to perform such 2D-3D segmentation from an atlas are required. This paper proposes a strategy to automatically and accurately segment single 2D coronal slices within a 3D volume of atlas, using linear registration. We validated its robustness and performance using an exploratory approach at whole-brain scale.

Keywords: Electrical impedance imaging, Image reconstruction - Performance evaluation, Image reconstruction and enhancement - Tomographic reconstruction
Abstract: There is an urgent need to bring forth portable, low-cost, point-of-care diagnostic instruments to monitor patient health and wellbeing. This is elevated by the COVID-19 global pandemic in which the availability of proper lung imaging equipment has proven to be pivotal in the timely treatment of patients. Electrical impedance tomography (EIT) has long been studied and utilized as such a critical imaging device in hospitals especially for lung ventilation. Despite decades of research and development, many challenges remain with EIT in terms of 1) image reconstruction algorithms, 2) simulation and measurement protocols, 3) hardware imperfections, and 4) uncompensated tissue bioelectrical physiology. Due to the inter-connectivity of these challenges, singular solutions to improve EIT performance continue to fall short of the desired sensitivity and accuracy. Motivated to gain a better understanding and optimization of the EIT system, we report the development of a bioelectric facsimile simulator demonstrating the dynamic operations, sensitivity analysis, and reconstruction outcome prediction of the EIT sensor with stepwise visualization. By building a sandbox platform to incorporate full anatomical and bioelectrical properties of the tissue under study into the simulation, we created a tissue-mimicking phantom with adjustable EIT parameters to interpret bioelectrical interactions and to optimize image reconstruction accuracy through better hardware setup and sensing protocol selections.

Keywords: Image analysis and classification - Digital Pathology, Image classification
Abstract: Deep learning (DL) thrives on the availability of a large number of high quality images with reliable labels. Due to the large size of whole slide images in digital pathology, patches of manageable size are often mined for use in DL models. These patches are variable in quality, weakly supervised, individually less informative, and noisily labelled. To improve classification accuracy even with these noisy inputs and labels in histopathology, we propose a novel method for robust feature generation using an adversarial autoencoder (AAE). We utilize the likelihood of the features in the latent space of AAE as a criterion to weigh the training samples. We propose different weighting schemes for our framework and evaluate the effectiveness of our methods on the publically available BreakHis and BACH histopathology datasets. We observe consistent improvement in AUC scores using our methods, and conclude that robust supervision strategies should be further explored for computational pathology.

Keywords: Magnetic resonance imaging - MR neuroimaging, Machine learning / Deep learning approaches, Multimodal image fusion
Abstract: Alzheimer's disease (AD) is a progressive brain disorder that causes memory and functional impairments. The advances in machine learning and publicly available medical datasets initiated multiple studies in AD diagnosis. In this work, we utilize a multi-modal deep learning approach in classifying normal cognition, mild cognitive impairment and AD classes on the basis of structural MRI and diffusion tensor imaging (DTI) scans from the OASIS-3 dataset. In addition to a conventional multi-modal network, we also present an input agnostic architecture that allows diagnosis with either sMRI or DTI scan, which distinguishes our method from previous multi-modal machine learning-based methods. The results show that the input agnostic model achieves 0.96 accuracy when both structural MRI and DTI scans are provided as inputs.

Keywords: Photoacoustic, Optoacoustic, Thermoacoustic imaging, Ultrasound imaging - Photoacoustic/Optoacoustic/Thermoacoustic
Abstract: Minimum variance (MV) beamforming improves resolution and reduces sidelobes when compared to delay-and-sum (DAS) beamforming for photoacoustic imaging (PAI). However, some level of sidelobe signal and incoherent clutter persist degrading MV PAI quality. Here, an adaptive beamforming algorithm (PSAPMV) combining MV formulation and sub-aperture processing is proposed. In PSAPMV, the received channel data are split into two complementary nonoverlapping sub-apertures and beamformed using MV. A weighting matrix based on similarity between sub-aperture beamformed images was derived and multiplied with the full aperture MV image resulting in suppression of sidelobe and incoherent clutter in the PA image. Numerical simulation experiments with point targets, diffuse inclusions and microvasculature networks are used to validate PSAPMV. Quantitative evaluation was done in terms of main-lobe-to-side-lobe ratio, full width at half maximum (FWHM), contrast ratio (CR) and generalized contrast-to-noise ratio (gCNR). PSAPMV demonstrated improved beamforming performance both qualitatively and quantitatively. PSAPMV had higher resolution (FWHM =0.19 mm) than MV (0.21 mm) and DAS (0.22mm) in point target simulations, better target detectability (gCNR =0.99) than MV (0.89) and DAS (0.84) for diffuse inclusions and improved contrast (CR in microvasculature simulation, DAS = 15.38, MV = 22.42, PSAPMV = 51.74 dB).

Keywords: Ultrasound imaging - Elastography, Ultrasound imaging - Cardiac, Ultrasound imaging - High-frequency technology
Abstract: A cardiac strain imaging framework with adaptive Bayesian regularization (ABR) is proposed for in vivo assessment of murine cardiac function. The framework uses ultrasound (US) radio-frequency data collected with a high frequency (fc = 30MHz) imaging system and a multi-level block matching algorithm with ABR to derive inter-frame cardiac displacements. Lagrangian cardiac strain (radial, er and longitudinal, el) tensors were derived by segmenting the myocardial wall starting at the ECG R-wave and accumulating interframe deformations over a cardiac cycle. In vivo feasibility was investigated through a longitudinal study with two mice (one ischemia-perfusion (IR) injury and one sham) imaged at five sessions (pre-surgery (BL) and 1,2,7 and 14 days post-surgery). End-systole (ES) strain images and segmental strain curves were derived for quantitative evaluation. Both mice showed periodic variation of er and el strain at BL with segmental synchroneity. Infarcted regions of IR mouse at Day 14 were associated with reduced or sign reversed ES er and el values while the sham mouse had similar or higher strain than at BL. Infarcted regions identified in vivo were associated with increased collagen content confirmed with Masson’s Trichrome stained ex vivo heart sections.

Keywords: Image analysis and classification - Machine learning / Deep learning approaches, Cardiac imaging and image analysis, Image classification
Abstract: Heart failure (HF) is a serious syndrome, with high rates of mortality. Accurate classification of HF according to the left ventricular ejection faction (EF) plays an important role in the clinical treatment. Compared to echocardiography, cine cardiac magnetic resonance images (Cine-CMR) can estimate more accurate EF, whereas rare studies focus on the application of Cine-CMR. In this paper, a self-supervised learning framework for HF classification called SSLHF was proposed to automatically classify the HF patients into HF patients with preserved EF and HF patients with reduced EF based on Cine-CMR. In order to enable the classification network better learn the spatial and temporal information contained in the Cine-CMR, the SSLHF consists of two stages: self-supervised image restoration and HF classification. In the first stage, an image restoration proxy task was designed to help a U-Net like network mine the HF information in the spatial and temporal dimensions. In the second stage, a HF classification network whose weights were initialized by the encoder part of the U-Net like network was trained to complete the HF classification. Benefitting from the proxy task, the SSLHF achieved an AUC of 0.8505 and an ACC of 0.8208 in the 5-fold cross-validation.

Keywords: Image reconstruction and enhancement - Machine learning / Deep learning approaches, Magnetic resonance imaging - MR spectroscopy
Abstract: Common to most medical imaging techniques, the spatial resolution of Magnetic Resonance Spectroscopic Imaging (MRSI) is ultimately limited by the achievable SNR. This work presents a deep learning method for 1H-MRSI spatial resolution enhancement, based on the observation that multi-parametric MRI images provide relevant spatial priors for MRSI enhancement. A Multi-encoder Attention U-Net (MAU-Net) architecture was constructed to process a MRSI metabolic map and three different MRI modalities through separate encoding paths. Spatial attention modules were incorporated to automatically learn spatial weights that highlight salient features for each MRI modality. MAU-Net was trained based on in vivo brain imaging data from patients with high-grade gliomas, using a combined loss function consisting of pixel, structural and adversarial loss. Experimental results showed that the proposed method is able to reconstruct high-quality metabolic maps with a high-resolution of 64×64 from a low-resolution of 16×16, with better performance compared to several baseline methods.

Keywords: Image analysis and classification - Machine learning / Deep learning approaches, Image feature extraction, CT imaging
Abstract: Cancer is a major public health issue and takes the second-highest toll of deaths caused by non-communicable diseases worldwide. Automatically detecting lesions at an early stage is essential to increase the chance of a cure. This study proposes a novel dilated Faster R-CNN with modulated deformable convolution and modulated deformable positive-sensitive region of interest pooling to detect lesions in computer tomography images. A pre-trained VGG-16 is transferred as the backbone of Faster R-CNN, followed by a region proposal network and a region of interest pooling layer to achieve lesion detection. The modulated deformable convolutional layers are employed to learn deformable convolutional filters, while the modulated deformable positive-sensitive region of interest pooling provides an enhanced feature extraction on the feature maps. Moreover, dilated convolutions are combined with the modulated deformable convolutions to fine-tune the VGG-16 model with multi-scale receptive fields. In the experiments evaluated on the DeepLesion dataset, the modulated deformable positive-sensitive region of interest pooling model achieves the highest sensitivity score of 58.8 % on average with dilation of [4,4,4] and outperforms state-of-the-art models in the range of [2,8] average false positives per image. This research demonstrates the suitability of dilation modifications and the possibility of enhancing the performance using a modulated deformable positive-sensitive region of interest pooling layer for universal lesion detectors.

Keywords: CT imaging applications, Image registration, segmentation, compression and visualization - Volume rendering, Image feature extraction
Abstract: Screening of the gastrointestinal tract is imperative for the detection and treatment of physiological and pathological disorders in humans. Ingestible devices (e.g., magnetic capsule endoscopes) represent an alternative to conventional flexible endoscopy for reducing the invasiveness of the procedure and the related patient’s discomforts. However, to properly design localization and navigation strategies for capsule endoscopes, the knowledge of anatomical features is paramount. Therefore, authors developed a semi-automatic software for measuring the distance between the small bowel and the closest human external body surface, using CT colonography images. In this study, volumetric datasets of 30 patients were processed by gastro-intestinal endoscopists with the dedicated custom-made software and results showed an average distance of 79.29 ± 23.85 mm.

Keywords: Image reconstruction and enhancement - Machine learning / Deep learning approaches, Image reconstruction - Fast algorithms, Ultrasound imaging - Other organs
Abstract: Automatic learning algorithms for improving the image quality of diagnostic B-mode ultrasound (US) images have been gaining popularity in the recent past. In this work, a novel convolutional neural network (CNN) is trained using time of flight corrected in-vivo receiver data of plane wave transmit to produce corresponding high-quality minimum variance distortion less response (MVDR) beamformed image. A comprehensive performance comparison in terms of qualitative and quantitative measures for fully connected neural network (FCNN), the proposed CNN architecture, MVDR and Delay and Sum (DAS) using the dataset from Plane wave Imaging Challenge in Ultrasound (PICMUS) is also reported in this work. The CNN architecture can leverage the spatial information and will be more region adaptive during the beamforming process. This is evident from the improvement seen over the baseline FCNN approach and conventional MVDR beamformer, both in resolution and contrast with an improvement of 6 dB in CNR using only zero-angle transmission over the baseline. The observed reduction in the requirement of number of angles to produce similar image metrics can provide a possibility for higher frame rates.

Keywords: Machine learning / Deep learning approaches, Image analysis and classification - Machine learning / Deep learning approaches, CT imaging applications
Abstract: Aortic dissection (AD) is a rare but potentially fatal disease with high mortality. The aim of this study is to synthesize contrast enhanced computed tomography (CE-CT) images from non-contrast CT (NCE-CT) images for detecting aortic dissection. In this paper, a cascaded deep learning framework containing a 3D segmentation network and a synthetic network was proposed and evaluated. A 3D segmentation network was firstly used to segment aorta from NCE-CT images and CE-CT images. A conditional generative adversarial network (CGAN) was subsequently employed to map the NCE-CT images to the CE-CT images non-linearly for the region of aorta. The results of the experiment suggest that the cascaded deep learning framework can be used for detecting the AD and outperforms CGAN alone.

Keywords: Optical imaging and microscopy - Fluorescence microscopy, Image analysis and classification - Machine learning / Deep learning approaches
Abstract: Multi-scale architectures at a granular level are characterised by separating input features into groups and applying multi-scale feature extractions to the split input features, and thus the correlations among the input features as global information are no longer retained. Moreover, they usually require more input features due to the separation, and therefore, more complexity is introduced. To retain the global information while utilising the advantages of feature-level hierarchical multi-scale architectures, we propose a multi-scale aggregated-dilation architecture (MSAD) to perform hierarchical fusion of features at a layer level, with the integration of dilated convolutions to overcome these issues. To evaluate the model, we integrate it into ResNet, and apply it to a unique dataset, containing over 60,000 fluorescence lifetime endomicroscopic images (FLIM) collected on textit{ex-vivo} lung normal/cancerous tissues from 14 patients, by a custom fibre-based FLIM system. We evaluate the performance of our proposal we use accuracy, precision, recall, and AUC. We first compare our MSAD model with eight networks achieving a superiority over 6%. To illustrate the advantages and disadvantages of multi-scale architectures at layer and feature-level, we thoroughly compare our MSAD model with the state-of-the-art feature-level multi-scale network, namely Res2Net, in terms of parameters, scales, and effective convolutions.

Keywords: Magnetic resonance imaging - Diffusion tensor, diffusion weighted and diffusion spectrum imaging, Brain imaging and image analysis, Image analysis and classification - Machine learning / Deep learning approaches
Abstract: Alzheimer's disease (AD) is a typical neurodegenerative disease that is associated with cognitive decline, memory loss, and functional disconnection. Diffusion tensor imaging (DTI) has been widely used to investigate the integrity and degeneration of white matter in AD. In this study, with one of the world’s largest DTI biobanks (865 individuals), we aim to explore the diagnosis utility and stability of tract-based features (extracted by automated fiber quantification (AFQ) pipeline) in AD. First, we studied the clinical association of tract-based features by detecting AD-associated alterations of diffusion properties along with fiber bundles. Then, a binary classification experiment between AD and normal controls was performed using tract-based diffusion properties as features and support vector machine (SVM) as a classifier with an independent site cross-validation strategy. The average accuracy of 77.90% (the highest was 88.89%) showed that white matter properties as biomarkers had a relatively stable role in the clinical diagnosis of AD.

Keywords: Image feature extraction, Image segmentation, Ultrasound imaging - Other organs
Abstract: Feature matching is a crucial component of computer vision that has various applications. With the emergence of Computer-Aided Diagnosis (CAD), the need for feature matching has also emerged in the medical imaging field. In this paper, we proposed a novel algorithm using the Explainable Artificial Intelligence (XAI) approach to achieve feature detection for ultrasound images based on the Deep Unfolding Super-resolution Network (USRNET). Based on the experimental results, our method shows higher interpretability and robustness than existing traditional feature extraction and matching algorithms. The proposed method provides a new insight for medical image processing, and may achieve better performance in the future with advancements of deep neural networks.

Keywords: Machine learning / Deep learning approaches, Cardiac imaging and image analysis, CT imaging
Abstract: Left ventricular (LV) segmentation is an important process which can provide quantitative clinical measurements such as volume, wall thickness and ejection fraction. The development of an automatic LV segmentation procedure is a challenging and complicated task mainly due to the variation of the heart shape from patient to patient, especially for those with pathological and physiological changes. In this study, we focus on the implementation, evaluation and comparison of three different Deep Learning architectures of the U-Net family: the custom 2-D U-Net, the ResU-Net++ and the DenseU-Net, in order to segment the LV myocardial wall. Our approach was applied to cardiac CT datasets specifically derived from patients with hypertrophic cardiomyopathy. The results of the models demonstrated high performance in the segmentation process with minor losses. The model revealed a dice score for U-Net, Res-U-net++ and Dense U-Net, 0.81, 0.82 and 0.84, respectively.

Keywords: Image segmentation, Machine learning / Deep learning approaches
Abstract: In recent years, polyp segmentation plays an important role in the diagnosis and treatment of colorectal cancer. Accurate segmentation of polyps is very challenging due to different sizes, shapes, and unclear boundaries. Making full use of multi-scale contextual information to segment polyps may bring better results. In this paper, we propose an enhanced multi-scale network for accurate polyp segmentation. It is composed of a multi-scale connected baseline (U-Net+++), a multi-scale backbone (Res2Net), three Receptive Field Block (RFB) modules, and four Local Context Attention (LCA) modules. Specifically, the baseline's multi-scale skip connections can aggregate features in both low-level and high-level layers. We have evaluated our model on three publicly available and challenging datasets (EndoScene, CVC-ClinicDB, Kvasir-SEG). Compared with other methods, our model achieves SOTA performance. It is noteworthy that our model is the only network that has achieved over 0.900 mean Dice on EndoScene and CVC-ClinicDB.

Keywords: Image analysis and classification - Machine learning / Deep learning approaches, Image classification
Abstract: Fast detection and classification of bacteria species play a crucial role in modern clinical microbiology systems. These processes are often performed manually by medical biologists using different shapes and morphological characteristics of bacteria species. However, it is clear that the manual taxonomy of bacteria types from microscopy images takes time, effort and is a great challenge for even experienced experts. A new revolution has been inaugurating with the development of machine learning methods to identify bacteria automatically from digital electron microscopy. In this paper, we introduce an automated model of bacteria shape classification based on Depthwise Separable Convolution Neural Networks (DS-CNNs). This architecture has great advantages with lower computational cost and reliable recognition accuracy. The results of the experiment indicate that the proposed architecture after training with a total of 1669 images can reach 97% validation accuracy and work well for classifying three main shapes of bacteria.

Keywords: Optical imaging - Confocal microscopy, Optical imaging and microscopy - Fluorescence microscopy, Image visualization
Abstract: We present a cell tracking method for time-lapse confocal microscopy (3D) images that uses dynamic hierarchical data structures to assist cell and colony segmentation and tracking. During the segmentation, the cell and colony numbers and their geometric data are recorded for each 3D image set. In tracking, the colony correspondences between neighboring frames of time-lapse 3D images are first computed using the recorded colony centers. Then, cell correspondences in the correspondent colonies are computed using the recorded cell centers. The examples show the proposed cell tracking method can achieve high tracking accuracy for time-lapse 3D images of undifferentiated but self-renewing mouse embryonic stem (mES) cells where the number and mobility of ES cells in a cell colony may change suddenly by a colony merging or splitting, and cell proliferation or death. The geometric data in the hierarchical data structures also help the visualization and quantitation of the cell shapes and mobility.

Keywords: Image registration, segmentation, compression and visualization - Volume rendering, Machine learning / Deep learning approaches, CT imaging applications
Abstract: In this paper, machine learning approaches are proposed to support dental researchers and clinicians to study the shape and position of dental crowns and roots, by implementing a Patient Specific Classification and Prediction tool that includes RootCanalSeg and DentalModelSeg algorithms and then merges the output of these tools for intraoral scanning and volumetric dental imaging. RootCanalSeg combines image processing and machine learning approaches to automatically segment the root canals of the lower and upper jaws from large datasets, providing clinical information on tooth long axis for orthodontics, endodontics, prosthodontic and restorative dentistry procedures. DentalModelSeg includes segmenting the teeth from the crown shape to provide clinical information on each individual tooth. The merging algorithm then allows users to integrate dental models for quantitative assessments. Precision in dentistry has been mainly driven by dental crown surface characteristics, but information on tooth root morphology and position is important for successful root canal preparation, pulp regeneration, planning of orthodontic movement, restorative and implant dentistry. In this paper we propose a patient specific classification and prediction of dental root canal and crown shape analysis workflow that employs image processing and machine learning methods to analyze crown surfaces, obtained by intraoral scanners, and three-dimensional volumetric images of the jaws and teeth root canals, obtained by cone beam computed tomography (CBCT).

Keywords: Machine learning / Deep learning approaches, Image segmentation, CT imaging applications
Abstract: In order to diagnose TMJ pathologies, we developed and tested a novel algorithm, MandSeg, that combines image processing and machine learning approaches for automatically segmenting the mandibular condyles and ramus. A deep neural network based on the U-Net architecture was trained for this task, using 109 cone-beam computed tomography (CBCT) scans. The ground truth label maps were manually segmented by clinicians. The U-Net takes 2D slices that are extracted from the 3D volumetric images. All the 3D scans were cropped depending on their size in order to keep only the mandibular region of interest. The same anatomic cropping region was used for every scan in the dataset. The scans were acquired at different centers; therefore, every slice was interpolated linearly and resampled to 512x512 pixels. As a pre-processing step, contrast adjustment was performed, because the original scans were low contrast images. This helped the deep learning model to make a better prediction. After image pre-processing, 300-400 slices were extracted from each scan, and used to train the U-Net algorithm. For the cross-validation, the data set was divided into 10 folds. The training was performed with 60 epochs, a batch size of 8 and a learning rate of 210-5. The average performance of the models on the test set presented 0.95 ± 0.05 AUC, 0.93 ± 0.06 sensitivity, 0.9998 ± 0.0001 specificity, 0.9996 ± 0.0003 accuracy, and 0.91 ± 0.03 F1 score. This study findings suggest that fast and efficient CBCT image segmentation of the mandibular condyles and ramus from different clinical data sets and centers can be analyzed effectively. Future studies can now extract radiomic and imaging features as potentially relevant objective diagnostic criteria for TMJ pathologies, such as osteoarthritis (OA). This segmentation will allow large datasets to be analyzed more efficiently towards data sciences and machine learning approaches for disease classification.

Keywords: Image segmentation, CT imaging applications, X-ray imaging applications
Abstract: COVID-19, a new strain of coronavirus disease, has been one of the most serious and infectious disease in the world. Chest CT is essential in prognostication, diagnosing this disease, and assessing the complication. In this paper, a multi-class COVID-19 CT segmentation is proposed aiming at helping radiologists estimate the extent of effected lung volume. We utilized four augmented pyramid networks on an encoder-decoder segmentation framework. Quadruple Augmented Pyramid Network(QAP-Net) not only enable CNN capture features from variation size of CT images, but also act as spatial interconnections and down-sampling to transfer sufficient feature information for semantic segmentation. Experimental results achieve competitive performance in segmentation with the Dice of 0.8163, which outperforms other state-of-the-art methods, demonstrating the proposed framework can segment of consolidation as well as glass, ground area via COVID-19 chest CT efficiently and accurately.

Keywords: Image reconstruction and enhancement - Machine learning / Deep learning approaches, CT imaging applications
Abstract: Computed tomography and magnetic resonance imaging produce high-resolution images; however, during surgery or radiotherapy, only low-resolution cone-beam CT and low-dimensional X-ray images can be obtained. Furthermore, because the duodenum and stomach are filled with air, even in high-resolution CT images, it is hard to accurately segment their contours. In this paper, we propose a method that is based on a graph convolutional network (GCN) to reconstruct organs that are hard to detect in medical images. The method uses surrounding detectable-organ features to determine the shape and location of the target organ and learns mesh deformation parameters, which are applied to a target organ template. The role of the template is to establish an initial topological structure for the target organ. We conducted experiments with both single and multiple organ meshes to verify the performance of our proposed method.

Keywords: Image analysis and classification - Machine learning / Deep learning approaches, X-ray radiography
Abstract: Tuberculosis (TB) is a serious infectious disease that mainly affects the lungs. Drug resistance to the disease makes it more challenging to control. Early diagnosis of drug resistance can help with decision making resulting in appropriate and successful treatment. Chest X-rays (CXRs) have been pivotal to identifying tuberculosis and are widely available. In this work, we utilize CXRs to distinguish between drug-resistant and drug-sensitive tuberculosis. We incorporate Convolutional Neural Network (CNN) based models to discriminate the two types of TB, and employ standard and deep learning based data augmentation methods to improve the classification. Using labeled data from NIAID TB Portals and additional non-labeled sources, we were able to achieve an Area Under the ROC Curve (AUC) of up to 85% using a pretrained InceptionV3 network.

Keywords: Brain imaging and image analysis, Magnetic resonance imaging - MR neuroimaging
Abstract: Functional magnetic resonance imaging (fMRI) is a powerful tool that allows for analysis of neural activity via the measurement of blood-oxygenation-level-dependent (BOLD) signal. The BOLD fluctuations can exhibit different levels of complexity, depending upon the conditions under which they are measured. We examined the complexity of both resting-state and task-based fMRI using sample entropy (SampEn) as a surrogate for signal predictability. We found that within most tasks, regions of the brain that were deemed task-relevant displayed significantly low levels of SampEn, and there was a strong negative correlation between parcel entropy and amplitude.

Keywords: Image analysis and classification - Machine learning / Deep learning approaches, CT imaging, Image classification
Abstract: Abstract - Cone-Beam Computed Tomography (CBCT) imaging modality is used to acquire 3D volumetric image of the human body. CBCT plays a vital role in diagnosing dental diseases, especially cyst or tumour-like lesions. Current computer-aided detection and diagnostic systems have demonstrated diagnostic value in a range of diseases, however, the capability of such a deep learning method on transmissive lesions has not been investigated. In this study, we propose an automatic method for the detection of transmissive lesions of jawbones using CBCT images. We integrated a pre-trained DenseNet with pathological information to reduce the intra-class variation within a patient’s images in the 3D volume (stack) that may affect the performance of the model. Our proposed method separates each CBCT stacks into seven intervals based on their disease manifestation. To evaluate the performance of our method, we created a new dataset containing 353 patients’ CBCT data. A patient-wise image division strategy was employed to split the training and test sets. The overall lesion detection accuracy of 80.49% was achieved, outperforming the baseline DenseNet result of 77.18%. The result demonstrates the feasibility of our method for detecting transmissive lesions in CBCT images. Clinical relevance - The proposed strategy aims at providing automatic detection of the transmissive lesions of jawbones with the use of CBCT images that can reduce the workload of clinical radiologists, improve their diagnostic efficiency, and meet the preliminary requirement for the diagnosis of this kind of disease when there is a lack of radiologists.

Keywords: Ultrasound imaging - Other organs
Abstract: Osteoarthritis is a common disease that implies joint degeneration and that strongly affects the quality of life. Conventional radiography remains currently the most used diagnostic method, even if it allows only an indirect assessment of the articular cartilage and employ the use of ionizing radiations. A non-invasive, continuous and reliable diagnosis is crucial to detect impairements and to improve the treatment outcomes. Quantitative ultrasound techniques have proved to be very useful in providing an objective diagnosis of several soft tissues. In this study, we propose quantitative ultrasound parameters, based on the analysis of radiofrequency data derived from both healthy and osteoarthritis-mimicking (through chemical degradation) ex-vivo cartilage samples. Using a transmission frequency typically employed in the clinical practice (15 MHz) with an external ultrasound probe, we found results in terms of reflection at the cartilage surface and sample thickness comparable to those reported in the literature by exploiting arthroscopic transducers at high frequency (from 20 to 55 MHz). Moreover, for the first time, we introduce an objective metric based on the phase entropy calculation, able to discriminate the healthy cartilage from the degenerated one. Clinical Relevance— This preliminary study proposes a novel and quantitative method to discriminate healthy from degenerated cartilage. The obtained results pave the way to the use of quantitative ultrasound in the diagnosis and monitoring of knee osteoarthritis.

Keywords: Image analysis and classification - Machine learning / Deep learning approaches, Image analysis and classification - Digital Pathology
Abstract: The low number of annotated training images and class imbalance in the field of machine learning is a common problem that is faced in many applications. With this paper, we focus on a clinical dataset where cells were extracted in a previous research. Class imbalance can be experienced within this dataset since the normal cells are in a great majority in contrast to the abnormal ones. To address both problems, we present our idea of synthetic image generation using a custom variational autoencoder, that also enables the pretraining of the subsequent classifier network. Our method is compared with a performant solution, as well as presented with different modifications. We have experienced a performance increase of 4.52% regarding the classification of the abnormal cells.

We extract images from cervical smears, using digitized Pap test. When working with these kinds of smears, a single one can contain more than 10,000 cells. Examination of these is done manually by going over each cell individually. Our main goal is to make a system that can rank these samples by importance, thus making the process easier and more effective. The research that is described in this paper gets us a step closer to achieving our goal.

Keywords: Machine learning / Deep learning approaches, Image segmentation, Optical imaging and microscopy - Microscopy
Abstract: Cell individualization has a vital role in digital pathology image analysis. Deep Learning is considered as an efficient tool for instance segmentation tasks, including cell individualization. However, the precision of the Deep Learning model relies on massive unbiased dataset and manual pixel-level annotations, which is labor intensive. Moreover, most applications of Deep Learning have been developed for processing oncological data. To overcome these challenges, i) we established a pipeline to synthesize pixel-level labels with only point annotations provided; ii) we tested an ensemble Deep Learning algorithm to perform cell individualization on neurological data. Results suggest that the proposed method successfully segments neuronal cells in both object-level and pixel-level, with an average detection accuracy of 0.93.

Keywords: Ultrasound imaging - Photoacoustic/Optoacoustic/Thermoacoustic, Image reconstruction and enhancement - Machine learning / Deep learning approaches, Iterative image reconstruction
Abstract: Photoacoustic (PA) tomography is a relatively new medical imaging technique that combines traditional ultrasound imaging and optical imaging, which has great application prospects in recent years. To reveal the light absorption coefficient of biological tissues, the images are reconstructed from PA signals by reconstruction algorithms. However, traditional model-based reconstruction method requires a huge number of iterations to obtain relatively good experimental results, which is quite time-consuming. In this paper, we propose to use deep learning method to replace brute parameter adjustment in model-based reconstruction, and speed up the rate of convergence by building convolutional neural networks (CNN). The parameters we defined in our model can be learned automatically. Meanwhile, our method can optimize the increment of gradient in each step of iteration. The numerical experiment validates our method, showing that only three iterations are needed to obtain the satisfactory image quality, which normally requires 10 iterations for tradition method. It demonstrated that efficiency of photoacoustic reconstruction can be greatly improved by our proposed method, compared with traditional model-based methods.

Keywords: Multimodal image fusion, Deformable registration, Machine learning / Deep learning approaches
Abstract: TRUS-MR fusion guided biopsy highly depends on the quality of alignment between pre-operative Magnetic Resonance (MR) image and live trans-rectal ultrasound (TRUS) image during biopsy. Lot of factors influence the alignment of prostate during the biopsy like rigid motion due to patient movement and deformation of the prostate due to probe pressure. For MR-TRUS alignment during live procedure, the efficiency of the algorithm and accuracy plays an important role. In this paper, we have designed a comprehensive framework for fusion based biopsy using an end-to-end deep learning network for performing both rigid and deformation correction. Both rigid and deformation correction in one single network helps in reducing the computation time required for live TRUS-MR alignment. We have used 6500 images from 34 subjects for conducting this study. Our proposed registration pipeline provides Target Registration Error (TRE) of 2.51 mm after rigid and deformation correction on unseen patient dataset. In addition, with a total computation time of 70ms, we are able to achieve a rendering rate of 14 frames per second (FPS) that makes our network well suited for live procedures.

Keywords: Brain imaging and image analysis, Image analysis and classification - Machine learning / Deep learning approaches, Optical imaging and microscopy - Neuroimaging
Abstract: We developed Carignan, a real-time calcium imaging software that can automatically detect activity patterns of neurons. Carignan can activate an external device when synchronized neural activity is detected in calcium imaging obtained by a one-photon (1p) miniscope. Combined with optogenetics, our software enables closed-loop experiments for investigating functions of specific types of neurons in the brain. In addition to making existing pattern detection algorithms run in real-time seamlessly, we developed a new classification module that distinguishes neurons from false-positives using deep learning. We used a combination of convolutional and recurrent neural networks to incorporate both spatial and temporal features in activity patterns. Our method performed better than existing neuron detection methods for false-positive neuron detection in terms of the F1 score. Using Carignan, experimenters can activate or suppress a group of neurons when specific neural activity is observed. Because the system uses a 1p miniscope, it can be used on the brain of a freely-moving animal, making it applicable to a wide range of experimental paradigms.

Keywords: Optical imaging, Image registration, segmentation, compression and visualization - Volume rendering
Abstract: Fundus examination of the newborn is quite important, which needs to be done timely so as to avoid irreversible blindness. Ophthalmologists have to review at least five images of each eye during one examination, which is a time-consuming task. To improve the diagnosis efficiency, this paper proposed a stable and robust fundus image mosaic method based on improved Speeded Up Robust Features (SURF) with Shannon entropy and make real assessment when ophthalmologists used it clinically. Our method is characterized by avoiding the useless detection and extraction of the feature points in the non-overlapping region of the paired images during registration process. The experiments showed that the proposed method successfully registered 90.91% of 110 different field of view (FOV) image pairs from 22 eyes of 13 screening newborns and acquired 93.51% normalized correlation coefficient and 1.2557 normalized mutual information. Also, the total fusion success rate reached 86.36% and a subjective visual assessment approach was adopted to measure the fusion performance by three experts, which obtained 84.85% acceptance rate. The performance of our proposed method demonstrated its accuracy and effectiveness in the clinical application, which can help ophthalmologists a lot during their diagnosis.

Keywords: Magnetic resonance imaging - MR neuroimaging, Image analysis and classification - Machine learning / Deep learning approaches, Brain imaging and image analysis
Abstract: Alzheimer's disease (AD) is a non-treatable and non-reversible disease that affects about 6% of people who are 65 and older. Brain magnetic resonance imaging (MRI) is a pseudo-3D imaging modality that is widely used for AD diagnosis. Convolutional neural networks with 3D kernels (3D CNNs) are often the default choice for deep learning based MRI analysis. However, 3D CNNs are usually computationally costly and data-hungry. Such disadvantages post a barrier of using modern deep learning techniques in the medical imaging domain, in which the number of data can be used for training is usually limited. In this work, we propose three approaches that leverage 2D CNNs on 3D MRI data. We test the proposed methods on the Alzheimer's Disease Neuroimaging Initiative dataset across two popular 2D CNN architectures. The evaluation results show that the proposed method improves the model performance on AD diagnosis by 8.33% accuracy or 10.11% auROC, while significantly reduce the training time by over 89%. We also discuss the potential causes for performance improvement and the limitation. We believe this work can serve as a strong baseline for future researchers.

Keywords: Machine learning / Deep learning approaches, Fetal and Pediatric Imaging
Abstract: Preterm infants’ spontaneous motility is a valuable diagnostic and prognostic index of motor and cognitive impairments. Despite being recognized as crucial, preterm infant’s movement assessment is mostly based on clinicians’ visual inspection. The aim of this work is to present a 2D dense convolutional neural network (denseCNN) to detect preterm infant’s joints in depth images acquired in neonatal intensive care units. The denseCNN allows to improve the performance of our previous model in the detection of joints and joint connections, reaching a median recall value equal to 0.839. With a view to monitor preterm infants in a scenario where computational resources are scarce, we tested the architecture on a mid-range laptop. The prediction occurs in real-time (0.014 s per image), opening up the possibility of integrating such monitoring system in a domestic environment.

Keywords: Machine learning / Deep learning approaches, Image segmentation, Image analysis and classification - Machine learning / Deep learning approaches
Abstract: Blood vessels provide oxygen and nutrients to all tissues in the human body, and their incorrect organisation or dysfunction contributes to several diseases. Correct organisation of blood vessels is achieved through vascular patterning, a process that relies on endothelial cell polarization and migration against the blood flow direction. Unravelling the mechanisms governing endothelial cell polarity is essential to study the process of vascular patterning. Cell polarity is defined by a vector that goes from the nucleus centroid to the corresponding Golgi complex centroid, here defined as axial polarity. Currently, axial polarity is calculated manually, which is time-consuming and subjective. In this work, we used a deep learning approach to segment nuclei and Golgi in 3D fluorescence microscopy images of mouse retinas, and to assign nucleus-Golgi pairs. This approach predicts nuclei and Golgi segmentation masks but also a third mask corresponding to joint nuclei and Golgi segmentations. The joint segmentation mask is used to perform nucleus-Golgi pairing. We demonstrate that our deep learning approach using three masks successfully identifies nucleus-Golgi pairs, outperforming a pairing method based on a cost matrix. Our results pave the way for automated computation of axial polarity in 3D tissues and in vivo.

Keywords: Machine learning / Deep learning approaches, Fetal and Pediatric Imaging, Image segmentation
Abstract: Computer-assisted tools for preterm infants’ movement monitoring in neonatal intensive care unit (NICU) could support clinicians in highlighting preterm-birth complications. With such a view, in this work we propose a deep-learning framework for preterm infants’ pose estimation from depth videos acquired in the actual clinical practice.The pipeline consists of two consecutive convolutional neural networks (CNNs). The first CNN (inherited from our previous work) acts to roughly predict joints and joint-connections position, while the second CNN (Asy-regression CNN) refines such predictions to trace the limb pose. Asy-regression relies on asymmetric convolutions to temporally optimize both the training and predictions phase. Compared to its counterpart without asymmetric convolutions, Asy-regression experiences a reduction in training and prediction time of 66% , while keeping the root mean square error, computed against manual pose annotation, merely unchanged. Research mostly works to develop highly accurate models, few efforts have been invested to make the training and deployment of such models time effective. With a view to make these monitoring technologies sustainable, here we focused on the second aspect and addressed the problem of designing a framework as trade-off between reliability and efficiency.

Keywords: Image segmentation, Ultrasound imaging - Other organs, Machine learning / Deep learning approaches
Abstract: Carpal tunnel syndrome (CTS) is the most common entrapment neuropathy. Ultrasound imaging (US) may help to diagnose and assess CTS, through the evaluation of median nerve morphology. To support sonographers, this paper proposes a fully-automatic deep-learning approach to median nerve segmentation from US images. The approach relies on Mask R-CNN, a convolutional neural network that is trained end-to-end. The segmentation head of Mask R-CNN is here evaluated with three different configurations, with the goal of studying the effect of the segmentation-head output resolution on the overall Mask R-CNN segmentation performance. For this study, we collected and annotated a dataset of 151 images acquired in the actual clinical practice from 53 subjects with CTS. To our knowledge, this is the largest dataset in the field in terms of subjects. We achieved a median Dice similarity coefficient equal to 0.931 (IQR = 0.027), demonstrating the potentiality of the proposed approach. These results are a promising step towards providing an effective tool for CTS assessment in the actual clinical practice.

Keywords: Ultrasound imaging - Other organs, Image feature extraction, Image analysis and classification - Machine learning / Deep learning approaches
Abstract: Over the last few decades, Lung Ultrasound (LUS) has been increasingly used to diagnose and monitor different lung diseases in neonates. It is a noninvasive tool that allows a fast bedside examination while minimally handling the neonate. Acquiring a LUS scan is easy, but understanding the artifacts concerned with each respiratory disease is challenging. Mixed artifact patterns found in different respiratory diseases may limit LUS readability by the operator. While machine learning (ML), especially deep learning can assist in automated analysis, simply feeding the ultrasound images to an ML model for diagnosis is not enough to earn the trust of medical professionals. The algorithm should output LUS features that are familiar to the operator instead. Therefore, in this paper we present a unique approach for extracting seven meaningful LUS features that can be easily associated with a specific pathological lung condition: Normal pleura, irregular pleura, thick pleura, A-lines, Coalescent B-lines, Separate B-lines and Consolidations. These artifacts can lead to early prediction of infants developing later respiratory distress symptoms. A single multi-class region proposal-based object detection model faster-RCNN (fRCNN) was trained on lower posterior lung ultrasound videos to detect these LUS features which are further linked to four common neonatal diseases. Our results show that fRCNN surpasses single stage models such as RetinaNet and can successfully detect the aforementioned LUS features with a mean average precision of 86.4%. Instead of a fully automatic diagnosis from images without any interpretability, detection of such LUS features leave the ultimate control of diagnosis to the clinician, which can result in a more trustworthy intelligent system.

Keywords: Image segmentation, Machine learning / Deep learning approaches, Image feature extraction
Abstract: Deep learning techniques have been widely employed in semantic segmentation problems, especially in medical image analysis, for understanding image patterns. Skin cancer is a life-threatening problem, whereas timely detection can prevent and reduce the mortality rate. The aim is to segment the lesion area from the skin cancer image to help experts in the process of deeply understanding tissues and cancer cells' formation. Thus, we proposed an improved fully convolutional neural network (FCNN) architecture for lesion segmentation in dermoscopic skin cancer images. The FCNN network consists of multiple feature extraction layers forming a deep framework to obtain a larger vision for generating pixel labels. The novelty of the network lies in the way layers are stacked and the generation of customized weights in each convolutional layer to produce a full resolution feature map. The proposed model was compared with the top four winners of the International Skin Imaging Collaboration (ISIC) challenge using evaluation metrics such as accuracy, Jaccard index, and dice co-efficient. It outperformed the given state-of-the-art methods with higher values of the accuracy and Jaccard index.

Keywords: CT imaging, Image analysis and classification - Machine learning / Deep learning approaches
Abstract: Predicting response to treatment plays a key role to assist radiologists in hepato-cellular carcinoma (HCC) therapy planning. The most widely used treatment for unresectable HCC is the trans-arterial chemoembolization (TACE). A complete radiological response after the first TACE is a reliable predictor of treatment favourable outcome. However, visual inspection of contrast-enhanced CT scans is time-consuming, error prone and too operator-dependent. Thus, in this paper we propose TwinLiverNet: a deep neural network that is able to predict TACE treatment outcome through learning visual cue from CT scans. TwinLiverNet, specifically, integrates 3D convolutions and capsule networks and is designed to process simultaneously late arterial and delayed phases from contrast-enhanced CTs. Experimental results carried out on a dataset consisting of 126 HCC lesions show that TwinLiverNet reaches an average accuracy of 82% in predicting complete response to TACE treatment. Furthermore, combining multiple CT phases (specifically, late arterial and delayed ones) yields a performance increase of over 12 percent points. Finally, the introduction of capsule layers into the model avoids the model to overfit, while enhancing accuracy.

Clinical relevance — TwinLiverNet supports radiologists in visual inspection of CT scans to assess TACE treatment outcome, while reducing inter-operator variability.

Keywords: Image segmentation, Machine learning / Deep learning approaches, Magnetic resonance imaging - Other organs
Abstract: Joint effusion is a hallmark of osteoarthritis (OA) associated with stiffness, and may relate to pain, disability, and long-term outcomes. However, it is difficult to quantify accurately. We propose a new Deep Learning (DL) approach for automatic effusion assessment from Magnetic Resonance Imaging (MRI) using volumetric quantification measures (VQM). We developed a new multiplane ensemble convolutional neural network (CNN) approach for 1) localizing bony anatomy and 2) detecting effusion regions. CNNs were trained on femoral head and effusion regions manually segmented from 3856 images (63 patients). Upon validation on a non-overlapping set of 2040 images (34 patients) DL showed high agreement with ground-truth in terms of Dice score (0.85), sensitivity (0.86) and precision (0.83). Agreement of VQM per-patient was high for DL vs experts in term of Intraclass correlation coefficient (ICC)= 0.88[0.80,0.93]. We expect this technique to reduce inter-observer variability in effusion assessment, reducing expert time and potentially improving the quality of OA care

Keywords: MR molecular imaging, Magnetic resonance imaging - MR spectroscopy, Image reconstruction and enhancement - Compressed sensing / Sampling
Abstract: Molecular imaging has long been recognized as an important tool for diagnosis, characterization, and monitoring of treatment responses of brain tumors. Magnetic resonance spectroscopic imaging (MRSI) is a label-free molecular imaging technique capable of mapping metabolite distributions non-invasively. Several metabolites detectable by MRSI, including Choline, Lactate and N-Acetyl Aspartate, have been proved useful biomarkers for brain tumor characterization. However, clinical application of MRSI has been limited by poor resolution, small spatial coverage, low signal-to-noise ratio and long scan time. This work presents a novel MRSI method for fast, high-resolution metabolic imaging of brain tumor. This method synergistically integrates fast acquisition sequence, sparse sampling, subspace modeling and machine learning to enable 3D mapping of brain metabolites with a spatial resolution of 2.0×3.0×3.0 mm3 in a 7-minute scan. Experimental results obtained from patients with diagnosed brain tumor showed great promise for capturing small-size tumors and revealing intra-tumor metabolic heterogeneities.

Keywords: Image reconstruction and enhancement - Machine learning / Deep learning approaches, Image enhancement - Denoising, CT imaging
Abstract: CT machines can be tuned in order to reduce the radiation dose used for imaging, yet reducing the radiation dose results in noisy images which are not suitable in clinical practice. In order for low dose CT to be used effectively in practice this issue must be addressed. Generative Adversarial Networks (GAN) have been used widely in computer vision research and have proven themselves as a powerful tool for producing images with high perceptual quality. In this work we use a cascade of two neural networks, the first is a Generative Adversarial Network and the second is a Deep Convolutional Neural Network. The first network generates a denoised sample which is then fine-tuned by the second network via residue learning. We show that our cascaded method outperforms related works and more effectively reconstructs fine structural details in low contrast regions of the image.

Keywords: Magnetic resonance imaging - Parallel MRI, Image reconstruction and enhancement - Machine learning / Deep learning approaches, Iterative image reconstruction
Abstract: Parallel magnetic resonance imaging (pMRI) accelerates data acquisition by undersampling k-space through an array of receiver coils. Finding accurate relationships between acquired and missing k-space data determines the interpolation performance and reconstruction quality. Autocalibration signals (ACS) are generally used to learn the interpolation coefficients for reconstructing the missing k-space data. Based on the estimation-approximation error analysis in machine learning, increasing training data size can reduce estimation error and therefore enhance generalization ability of the interpolator, but scanning time will be longer if more ACS data are acquired. We propose to augment training data using unacquired and acquired data outside of ACS region through semi-supervised learning idea and autoregressive model. Local neighbor unacquired k-space data can be used for training tasks and reducing the generalization error. Experimental results show that the proposed method outperforms the conventional methods by suppressing noise and aliasing artifacts.

Keywords: Machine learning / Deep learning approaches, Image segmentation
Abstract: Instrument segmentation is a crucial and challenging task for robot-assisted surgery operations. Recent commonly-used models extract feature maps in multiple scales and combine them via simple but inferior feature fusion strategies. In this paper, we propose a hierarchical attentional feature fusion scheme, which is efficient and compatible with encoder-decoder architectures. Specifically, to better combine feature maps between adjacent scales, we introduce dense pixel-wise relative attentions learned from the segmentation model; to resolve specific failure modes in predicted masks, we integrate the above attentional feature fusion strategy based on position-channel-aware parallel attention into the decoder. Extensive experimental results evaluated on three datasets from MICCAI 2017 EndoVis Challenge demonstrate that our model outperforms other state-of-the-art counterparts by a large margin.

Keywords: Brain imaging and image analysis, Functional image analysis, Image feature extraction
Abstract: The study of brain network connectivity as a time-varying property began relatively recently and to date has remained primarily concerned with capturing a handful of discrete static states that characterize connectivity as measured on a timescale shorter than that of the full scan. Capturing group-level representations of temporally evolving patterns of connectivity is a challenging and important next step in fully leveraging the information available in large resting state functional magnetic resonance imaging (rs-fMRI) studies. We introduce a flexible, extensible data-driven framework for the identification of group-level multiframe (movie-style) dynamic functional network connectivity (dFNC) states. Our approach employs uniform manifold approximation and embedding (UMAP) to produce a planar embedding of the high-dimensional whole-brain connectivity dynamics that preserves important features, such as trajectory continuity, characterizing dynamics in the native high dimensional state space. The method is validated in application to a large rs-fMRI study of schizophrenia where it extracts naturalistic fluidly-varying connectivity motifs that differ between schizophrenia patients (SZs) and healthy controls (HC).

Keywords: Image enhancement, Image reconstruction and enhancement - Machine learning / Deep learning approaches, Machine learning / Deep learning approaches
Abstract: During endoscopic surgery, smoke removal is important and meaningful for increasing the visual quality of endoscopic images. However, unlike natural image dehaze, it is practical impossible to build a large paired endoscopic image training dataset with/without smoke. Therefore, in this paper, we propose a new approach, called Desmoke-CycleGAN, which combined detection and removal of smoke together, to improve the CycleGAN model for endoscopic image smoke removal. The detector can provide information about smoke locations and densities, which helps the GAN model to be more stable and efficient for smoke removal. Although some imperfections still exist, the experimental results have demonstrated that this method outperforms other state-of-the-art smoke removal approaches with unpaired real endoscopic images.

Keywords: Image analysis and classification - Digital Pathology, Image classification, Image analysis and classification - Machine learning / Deep learning approaches
Abstract: Melanoma is considered as one of the world's deadly cancers. This type of skin cancer will spread to other areas of the body if not detected at an early stage. Convolutional Neural Network (CNN) based classifiers are currently considered one of the most effective melanoma detection techniques. This study presents the use of recent deep CNN approaches to detect melanoma skin cancer and investigate suspicious lesions. Tests were conducted using a set of more than 36,000 images extracted from multiple datasets. The obtained results show that the best performing deep learning approach achieves high scores with an accuracy and Area Under Curve (AUC) above 99%.

Keywords: Image segmentation, CT imaging, Machine learning / Deep learning approaches
Abstract: Abstract— Automatic segmentation of the kidney and tumor from computed tomography (CT) images is an essential step in precision oncology and personalized treatment planning. Due to the irregular shapes and vague boundaries of kidney and tumor, this is a challenging task. Most of existing methods focused on local features without fully considering the associations between regions and contextual relationships between features. We propose a new segmentation method, CR-UNet, to extract, encode and adaptively integrate multiple layers of relevant features. Since the semantic features of different channels contribute differently to the segmentation of kidney and tumor, we introduce semantic attention mechanism of channels. The regional association attention mechanism is established to integrate the semantic and positional connections between different regions. Ablation studies demonstrate the contributions of semantic associations between deep learning channels, and regional relation modelling. Comparison results with state-of-the-art methods over public dataset demonstrated improved tumor and kidney segmentation performance.

Keywords: Image analysis and classification - Machine learning / Deep learning approaches, Image classification
Abstract: Epilepsy is a neurological disorder that causes sudden seizures due to abnormal excitation of neurons in the brain. Approximately 30 % of patients cannot control their seizures using medication. In addition, since seizures can occur anywhere and at any time, caregivers must always be with the patient. Various researchers have developed seizure detection methods using multichannel EEG to improve the quality of life of patients and caregivers. However, the large size of the measurement device impedes transportation. We believe that a portable measurement device with a small number of channels is suitable for detecting seizures in daily life. Therefore, we need a system that can detect seizures using a small number of channels. The purpose of this research is to develop a seizure detection algorithm using a single-channel frontal EEG and to confirm its basic performance. We used EEG signals from a single electrode position (Fp1-F7, Fp2-F8), which is a bipolar derivation of the frontal region. We segmented the EEG using a 2 s sliding window with 50 % overlap and converted the segments into images. After preprocessing, we fine-tuned ResNet18, pre-trained on ImageNet, and developed an ensemble classification method. In the experiments with 10 epileptic patients (3 – 19 years old) registered in the CHB-MIT scalp EEG database, the results showed that the average sensitivity was 88.73 %, the average specificity was 98.98 %, and the average detection latency time was 7.39 s. In conclusion, the developed algorithm was validated as sufficiently accurate to detect epileptic seizures.

Keywords: Image segmentation, Machine learning / Deep learning approaches
Abstract: Wireless capsule endoscopy is a non-invasive and painless procedure to detect anomalies from the gastrointestinal tract. Single examination results in up to 8 hrs of video and requires between 45 - 180 mins for diagnosis depending on the complexity. Image and video computational methods are needed to increase both efficiency and accuracy of the diagnosis. In this paper, a compact U-Net with lesser encoder-decoder pairs is presented, to detect and precisely segment bleeding and red lesions from endoscopy data. The proposed compact U-Net is compared with the original U-Net and also with other methods reported in the literature. The results show the proposed compact network performs on par with the original network but with faster training and lesser memory consumption. Also, the proposed model provided a dice score of 91% outperforming other methods reported on a blind tested WCE dataset with no images from this set used for training.

Keywords: Machine learning / Deep learning approaches, Fetal and Pediatric Imaging
Abstract: Bone age Assessment or the skeletal age is a general clinical practice to detect endocrine and metabolic disarrangement in child development. The bone age indicates his/her level of structural and biological growth better than the chronological age calculated from the birth date. The X-Ray of the wrist and hand is used in common to estimate the bone age of a person. The degree of agreement among the automated methods used to evaluate the X-rays is more than any other manual method. In this work, we propose a fully automated deep learning approach for bone age assessment. The dataset used is from the 2017 Pediatric Bone Age Challenge released by the Radiological Society of North America. Each X-Ray image in this dataset is an image of a left hand tagged with the age and gender of the patient. Transfer learning is employed by using pre-trained neural network architecture. InceptionV3 architecture is used in the present work, and the difference between the actual and predicted age is 5.921 months.

Keywords: Image segmentation, Machine learning / Deep learning approaches, Image reconstruction and enhancement - Tomographic reconstruction
Abstract: Abstract—Accurate root canal segmentation provides an important assistance for root canal therapy. The existing research such as level set method have made effective pro-gress in tooth and root canal segmentation. In the current situation, however, doctors are required to specify an initial area for the target root canal manually. In this paper, we propose a fully automatic and high precision root canal seg-mentation method based on deep learning and hybrid level set constraints. We set up the global image encoder and local region decoder for global localization and local segmenta-tion, and then combine the contour information generated by level set. Through using CLAHE algorithm and a combina-tion loss based on dice loss, we solve the class imbalance problem and improved recognition ability. More accurate and faster root canal segmentation is implemented under the framework of multi-task learning and evaluated by experi-ments on 78 Cone Beam CT images. The experimental results show that the proposed 3D U-Net had higher segmentation performance than state of the art algorithms. The average dice similarity coefficient (DSC) is 0.952. Clinical Relevance— We propose an end-to-end automatic root canal segmentation method, which had high accuracy and can reduce the workload of marking samples for dentists. It can also be used for root canal treatment planning and preoperative evaluation.

Keywords: Image classification, Image feature extraction, Multivariate image analysis
Abstract: Cerebellar ataxia (CA) is defined by disrupted coordination of movement suffering from disease of the cerebellum. It reflects fragmented movements of the eyes, vocal, upper limbs, balance, gait, and lower limbs. This study aims to use a motion sensor to form a simple yet effective CA quantitative assessment framework. We suggest a pendant device to use a single kinematic sensor attached to the wearer's chest to investigate the balance capability. Via a standard neurological test (Romberg's standing), the device may reveal an early symptom of Cerebellar Ataxia tailoring toward rehabilitation or therapeutic program. We adopt a transformed-image based approach to leverage the advantage of state-of-the-art deep learning models into diagnosis CA. Three transform techniques are employed including recurrence plot, melspectrogram, and Poincar'e plot. Experiment results show that melspectrogram transform technique performs best in implementation with MobileNetV2 to diagnose CA with an average validation accuracy of 89.99%.

Keywords: Image segmentation, Machine learning / Deep learning approaches, Optical imaging - Confocal microscopy
Abstract: The Interstitial Cells of Cajal (ICC) are specialized gastrointestinal (GI) pacemaker cells that generate and actively propagate electrophysiological events called slow waves. Slow waves regulate the GI motility necessary for digestion. Several functional GI motility disorders have been associated with depletion in the ICC. In this study, a validated Fast Random Forest (FRF) classification method using Trainable WEKA Segmentation for segmenting the networks of ICC was applied to confocal microscopy images of a whole mount tissue from the distal antrum of a mouse stomach (583 × 3,376 × 133 μm³, parcellated into 24 equal image stacks). The FRF model performance was compared to 6 manually segmented subfields and produced an area under the receiver-operating characteristic (AUROC) of 0.95. Structural variations of ICC network in the longitudinal muscle (ICC-LM) and myenteric plexus (ICC-MP) were quantified. The average volume of ICC-MP was significantly higher than ICC-LM at any point throughout the antral tissue sampled. There was a pronounced decline of up to 80% in ICC-LM (from 3,705 μm³ to 716 μm³) over a distance of 279.3 μm, that eventually diminished towards the distal antrum. However, an inverse relationship was observed in ICC-MP with an overall increase of up to 157% (from 59,100 μm³ to 151,830 μm³) over a distance of approximately 2 mm that proceeds towards the distal antrum.

Keywords: Image reconstruction and enhancement - Tomographic reconstruction, Image reconstruction and enhancement - Machine learning / Deep learning approaches, Image reconstruction and enhancement - Compressed sensing / Sampling
Abstract: Bioluminescence tomography (BLT) is an effective noninvasive molecular imaging modality for three dimensional visualization of in vivo tumor research in small animals. The approaches of deep learning have shown great potential in the field of optical molecular imaging in recent years. However, the common problem with these existing end-to-end networks is the black box technology, whose solving process is not theoretically proven. In this work, we proposed a novel Alternating Direction Method of Multipliers Network (ADMM-Net) to solve the poor interpretation problem of internal process. The ADMM-Net combines the framework of deep learning on the basis of traditional ADMM algorithm to dynamically learn various parameters of the algorithm in the form of network. To evaluate the performance of our proposed network, we implemented numerical simulation experiments. The results show that the ADMM-Net can accurately reconstruct the location of the source, and the morphological similarity with the real source is also higher.

Keywords: Image segmentation, Machine learning / Deep learning approaches, Image feature extraction
Abstract: Colorectal cancer has become the second leading cause of cancer-related death, attracting considerable interest for automatic polyp segmentation in polyp screening system. Accurate segmentation of polyps from colonoscopy is a challenging task as the polyps diverse in color, size and texture while the boundary between polyp and background is sometimes ambiguous. We propose a novel alternative prediction refinement network (APRNet) to more accurately segment polyps. Based on the UNet architecture, our APRNet aims at exploiting all-level features by alternatively leveraging features from encoder and decoder branch. Specifically, a series of prediction residual refinement modules (PRR) learn the residual and progressively refine the segmentation at various resolution. The proposed APRNet is evaluated on two benchmark datasets and achieves new state-of-the-art performance with a dice of 91.33% and an accuracy of 97.31% on the Kvasir-SEG dataset, and a dice of 86.33% and an accuracy of 97.12% on the EndoScene dataset.

Keywords: Ultrasound imaging - Other organs, Image segmentation, Machine learning / Deep learning approaches
Abstract: Thyroid cancer has a high prevalence all over the world. Accurate thyroid nodule diagnosis can lead to effective treatment and decrease the mortality rate. Ultrasound imaging is a safe, portable, and inexpensive tool for thyroid nodule monitoring. However, the widespread use of ultrasound has also resulted in over-diagnosis and over-treatment of nodules. There is also large variability in the assessment and characterization of nodules. Thyroid nodule classification requires precise delineation of the nodule boundary which is tedious and time-consuming. Automatic segmentation of nodule boundaries is highly desirable, however, it is challenging due to the wide range of nodule appearances, shapes, and sizes. In this study, we propose an end-to-end pipeline for nodule segmentation and classification. A residual dilated UNet (resDUnet) model is proposed for nodule segmentation. The output of resDUnet is fed to two rule-based classifiers to categorize the composition and echogenicity of the segmented nodule. We evaluate our segmentation method on a large dataset of 352 ultrasound images reviewed by a certified radiologist. When compared with ground-truth, resDUnet gives a higher Dice score than the standard UNet (82% vs. 81%). Our method requires minimal user interaction and it is robust to reasonable variations in the user-specified region-of-interest. We expect the proposed method to reduce variability in thyroid nodule assessment which results in more efficient and cost-effective monitoring of thyroid cancer.

Keywords: Ultrasound imaging - Cardiac, Image segmentation
Abstract: Accurate segmentation of cardiac chambers is helpful for the diagnosis of Congenital Heart Disease (CHD) in fetal echocardiography. Previous studies mainly focused on single cardiac chamber segmentation, which cannot provide sufficient information for the cardiologists. In this paper, we present an instance segmentation approach capable of segmenting four cardiac chambers accurately and simultaneously. A novel object proposal recovery strategy is further deployed to retrieve possible missing objects. To alleviate the shortage of medical data and further improve the segmentation performance, we utilize a rotation and distortion method for data augmentation. Experiments on a fetal echocardiography dataset of 319 fetuses demonstrate that the proposed approach can achieve superior performance according to common-used evaluation metrics.

Keywords: Magnetic resonance imaging - MR neuroimaging, Brain imaging and image analysis, Fetal and Pediatric Imaging
Abstract: Cortical asymmetry and functional lateralization form intriguing and fundamental features of human brain organization, and is complicated by individual differences and evolvement with age. While many studies have investigated neuroanatomical differences between hemispheres as well as functional lateralization of the brain for different age groups, few have looked into the associations between cortical asymmetry and development of cognitive functions in children. In this study, we aimed to identify relationships between hemispheric asymmetry in brain cortex measured by MRI and cognitive development in healthy young children evaluated by a comprehensive battery of neuropsychological tests. Structural MRI data were obtained from 71 children in the age range of 7.5 to 8.5 years. Structural lateralization index (SLI), a reflection of the brain asymmetry, was computed for each of the 3 cortical morphometry measurements: cortical thickness, surface area and gray matter volume. A total of 34 bilateral regions were studied for the whole brain cortex as defined by the Desikan atlas. Region-wise SLI was correlated with domain specific cognitive scores using partial correlation analysis controlled for the potential confounding effects of age and sex. Significant correlations were identified between test scores of multiple cognitive domains and SLI of several cortical regions. Specifically, SLI of total surface area of precuneus and insula significantly correlated with measures of executive function behavior; significant relationships were also found between SLI of mean cortical thickness of superior parietal cortex and memory and language tests scores; in addition, SLI of parahippocampal gyrus also showed significant correlations with language test scores for all 3 morphometry features. These findings revealed regional hemispheric asymmetries that may be linked to specific cognitive abilities in children.

Keywords: Functional image analysis, Brain imaging and image analysis, Magnetic resonance imaging - MR neuroimaging
Abstract: Ongoing large-scale human brain studies are generating complex neuroimaging data from thousands of individuals that can be leveraged to derive data-driven, anatomically accurate brain parcellations. However, despite their promise and many strengths, these data are highly heterogeneous, a characteristic that may affect the anatomical accuracy and generalization of the template but has received relatively little attention. Using multiple similarity measures and thresholding approaches, this study investigated the topological intra- and inter-individual variability of resting-state (rs) functional edge maps (often used for brain parcellation), estimated from rs-fMRI connectivity in n = 5878 children from the Adolescent Brain Cognitive Development (ABCD) study. Findings from this initial investigation indicate that choosing a subject- vs cohort-based threshold for estimating edge maps from connectivity matrices does not significantly impact the map topology. In contrast, the choice of similarity measure and non-linear relationship between similarity and edge map sparsity may have a significant impact on map classification and the generation of parcellation atlases. Multi-level classification revealed multiple clusters with a potentially complex mapping onto biological variables beyond simple demographics.

Clinical Relevance— Case-control neuroimaging studies should use domain-specific (e.g., demographics-specific) atlases for parcellating the brain, to improve accuracy and rigor of cohort comparisons. To be generalizable, such atlases need to be derived from large datasets, which are inherently heterogeneous. In a cohort of 5878 children (age ~9-10 years), this study systematically assessed the impact of heterogeneity and similarity of edge maps, which are derived from rs-fMRI connectivity and typically used to generate parcellation atlases.

Keywords: Brain imaging and image analysis, Image enhancement - Denoising, Functional image analysis
Abstract: Physiological fluctuations such as cardiac pulsations (heart rate) and respiratory rhythm (breathing) have been studied in the resting state functional magnetic resonance imaging (rs-fMRI) studies as the potential sources of confounds in functional connectivity. Independent component analysis (ICA) provides a data driven approach to investigate functional connectivity at the network level. However, the effect of physiological noise correction on the dynamic of ICA-derived networks has not yet been studied. The goal of this study was to investigate the effect of retrospective correction of cardiorespiratory artifacts on the time-varying aspects of functional network connectivity. Blood oxygenation-level dependent (BOLD) rs-fMRI data were collected from healthy subjects using a 3.0T MRI scanner. Whole-brain dynamic functional network connectivity (dFNC) was computed using sliding time window correlation, and k-means clustering of windowed correlation matrices. Results showed significant effects of physiological denoising on dFNC between several network pairs in particular the subcortical, and cognitive/attention networks (false discovery rate [FDR]-corrected p < 0.01). Meta-state dynamics further revealed significant changes in the number of unique windows for each subject, number of times each subject changes from one meta-state to other, and sum of L1 distances between successive meta-states. In conclusion, removal of artifacts is important for achieving reliable fMRI results, however a more cautious approach should be adapted in regressing such "noise" in ICA functional connectivity approach. More experiments are needed to investigate impact of denoising on dFNC especially across different datasets.

Keywords: Image registration, segmentation, compression and visualization - Volume rendering
Abstract: After Total Knee Arthroplasty (TKA), a global post-operative rehabilitation programme is commonly performed. However, this current program is not always adapted to every patient and it could be improved by deeply reinforcing weaker thigh muscles. To do this, a muscle volume estimation coupled with force evaluation is required to therefore adapt the rehabilitation and to be able to propose a specific patient exercise plan. In this paper, we presented an MRI protocol allowing the acquisition of the whole thigh as well as a semi-automated pipeline to segment two main groups of thigh muscles, i.e., the quadriceps femoris and the hamstrings muscles. The pipeline is based on a few cross-sections manually labelled and a 3D-spline interpolation using direct graphs corresponding points. The seven muscles of ten thighs (70 muscles in total) were segmented and reconstructed in 3D. To assess this pipeline, three types of metrics (volumetric similarity, surface distance, and classical measures) were employed. Furthermore, the inter-muscle overlapping was calculated as an additional metric. The results showed mean DICE was 99.6% (±0.1), Hausdorff Distance was 4.9 mm (±1.8) and Absolute Volume Difference was 2.97 cm3 (±1.94) in comparison to the manual ground truth. The average overlap was 2.05% (±0.54). This method is fast, accurate and possible to integrate in the clinical workflow of TKA.

Keywords: Brain imaging and image analysis, Image enhancement - Denoising, Functional image analysis
Abstract: Physiological processes such as cardiac pulsations and respiration can induce signal modulations in functional magnetic resonance imaging (fMRI) time series, and confound inferences made about neural processing from analyses of the blood oxygenation level-dependent (BOLD) signals. Retrospective image space correction of physiological noise (RETROICOR) is a widely used approach to reduce physiological signals in data. Independent component analysis (ICA) is a valuable blind source separation method for analyzing brain networks, referred to as intrinsic connectivity networks (ICNs). Previously, we showed that temporal properties of the ICA-derived networks such as spectral power and functional network connectivity could be impacted by RETROICOR corrections. The goal of this study is to investigate the effect of retrospective correction of physiological artifacts on the ICA dimensionality (model order) and intensities of ICN spatial maps. To this aim, brain BOLD fMRI, heartbeat, and respiration were measured in 22 healthy subjects during resting state. ICA dimensionality was estimated using minimum description length (MDL) based on i.i.d. data samples and smoothness FWHM kernel, and entropy-rate based order selection by finite memory length model (ER-FM) and autoregressive model (ER-AR). Differences in spatial maps between the raw and denoised data were compared using the paired t-test and false discovery rate (FDR) thresholding was used to correct for multiple comparisons. Results showed that ICA dimensionality was greater in the raw data compared to the denoised data. Significant differences were found in the intensities of spatial maps for three ICNs: basal ganglia, precuneus, and frontal network. These preliminary results indicate that the retrospective physiological noise correction can induce change in the resting state spatial map intensity related to the within-network connectivity.

Keywords: Cardiac imaging and image analysis, Image reconstruction and enhancement - Machine learning / Deep learning approaches, Iterative image reconstruction
Abstract: Noninvasive electrophysiological imaging plays an important role in the clinical diagnosis and treatment of heart diseases over recent years. Transmembrane potential (TMP) is one of the most important cardiac physiological signals, which can be used to diagnose heart disease such as premature beat and myocardial infarction. Considering the nonlocal self-similarity of TMP distribution and integrating traditional optimization strategy into deep learning, we proposed a novel global features based Fast Iterative Shrinkage/Thresholding network, named as GFISTA-Net. The proposed method has two main advantages over traditional methods, namely, the l1-norm regularization helps to avoid overfitting the model on high-dimensional but small-training data, and facilitates embedded the spatio-temporal correlation of TMP. Experiments demonstrate the power of our method.

Keywords: Magnetic resonance imaging - Dynamic contrast-enhanced MRI, Image analysis and classification - Machine learning / Deep learning approaches
Abstract: Multi-parametric MRI is part of the standard prostate cancer (PCa) diagnostic protocol. Recent imaging guidelines (PI-RADS v2) downgraded the value of Dynamic Contrast-Enhanced (DCE)-MRI in the diagnosis of PCa. A purely qualitative analysis of the DCE-MRI time series, as it is generally done by radiologists, might indeed overlook information on the microvascular architecture and function. In this study, we investigate the discriminative power of quantitative imaging features derived from texture and pharmacokinetic analysis of DCE-MRI. In 605 regions of interest (benign and malignant tissue) delineated in 80 patients, we found through independent cross-validation that a subset of quantitative spatial and temporal features extracted from DCE-MRI and incorporated in machine learning classifiers obtains a good diagnostic performance (AUC = 0.80-0.86) in distinguishing malignant from benign regions.

Keywords: Micro-CT imaging, Image reconstruction - Performance evaluation, Image reconstruction and enhancement - Tomographic reconstruction
Abstract: For a tomographic imaging system, image reconstruction quality is dependent on the accurate determination of coordinates for the true center of rotation (COR). A significant COR offset error may introduce ringing, streaking, or other artifacts, while smaller error in determining COR may blur the reconstructed image. Well known COR correction techniques including image registration, center of mass calculation, or reconstruction evaluation work well under certain conditions. However, many of these methods do not consider various real-world cases such as a tilted sensor or non-parallel projections. Furthermore, a limited number of projections introduces stripe artifacts into the image reconstruction that interfere with many of these classic COR correction techniques. In this paper, we propose a revised variance-based algorithm to find the correct COR position automatically prior to tomographic reconstruction. The algorithm was tested on both simulated phantoms and acquired datasets, and our results show improved reconstruction accuracy.

Keywords: Optical imaging
Abstract: Tissular blood perfusion is helpful to assess the health condition of a subject and even monitor superficial lesions. Current state of the art is focused on developing non-invasive, quantitative and accessible methods for blood flow monitoring in large areas. This paper presents an approach based on multispectral images on the VIS-NIR range to quantify blood perfusion. Our goal is to estimate the changes in deoxygenated hemoglobin. To do so, we employ principal component analysis followed by a linear regression model. The proposal was evaluated using in-vivo data from a vascular occlusion protocol, and the results were validated against photoplethysmography measurements. Although the number of subjects in the protocol was limited, our model made a prediction with an average similarity of 91.53% with a mean R-squared adjusted of 0.8104.

Keywords: Brain imaging and image analysis, Functional image analysis, Magnetic resonance imaging - MR neuroimaging
Abstract: Spatial smoothing is a common preprocessing step in the analysis of functional magnetic resonance imaging (fMRI) data. However, little is known about the effect of spatial smoothing kernel size on the temporal properties of functional brain networks. This study presents a pilot investigation on the influence of spatial smoothing using independent component analysis (ICA) as a data-driven technique to extract functional networks of brain in the form of intrinsic connectivity networks (ICNs). BOLD resting state fMRI data were collected from 22 healthy subjects on a 3.0 T MRI scanner. 3D spatial smoothing was applied using a Gaussian filter with full width at half maximum (FWHM) kernel sizes of 4 mm, 8 mm, and 12 mm in the preprocessing step. Group ICA with the Infomax algorithm was performed at 75-IC decomposition. Network temporal features including functional network connectivity (FNC) and BOLD power spectra were calculated and compared pairwise using a paired t-test with a false discovery rate (FDR) correction for multiple comparisons. Results revealed robust effects of smoothing kernel size on FNC measures of most ICNs, largely indicating a decrease in inter-network connectivity as the smoothing kernel size decreased. Power spectra analysis showed increased high-frequency power (0.15 - 0.25 Hz) but decreased low-frequency power (0.01 - 0.10 Hz) with a decrease in the smoothing kernel size (corrected p < 0.01). These findings provide a preliminary observation on the effect of spatial smoothing kernel size on the FNC and power spectra.

Keywords: Image feature extraction, Machine learning / Deep learning approaches, CT imaging
Abstract: Early detection of COVID-19 is vital to control its spread. Deep learning methods have been presented to detect suggestive signs of COVID-19 from chest CT images. However, due to the novelty of the disease, annotated volumetric data are scarce. Here we propose a method that does not require either large annotated datasets or backpropagation to estimate the filters of a convolutional neural network (CNN). For a few CT images, the user draws markers at representative normal and abnormal regions. The method generates a feature extractor composed of a sequence of convolutional layers, whose kernels are specialized in enhancing regions similar to the marked ones, and the decision layer of our CNN is a support vector machine. As we have no control over the CT image acquisition, we also propose an intensity standardization approach. Our method can achieve mean accuracy and kappa values of 0.97 and 0.93, respectively, on a dataset with 117 CT images extracted from different sites, surpassing its counterpart in all scenarios.

Keywords: Ultrasound imaging - Other organs, Fetal and Pediatric Imaging
Abstract: Hip displacement is a common orthopedic abnormality in children with cerebral palsy and is assessed on anteroposterior pelvic radiographs during surveillance. Repeated exposure to ionizing radiation is a major concern of cancer risks for children. Ultrasound (US) has been proposed to image the hips. The severity of hip displacement is measured by the Reimers’ migration percentage (MP), which is calculated by the ratio of the femoral head distance from the acetabulum to the width of the femoral head. Methods have been published to estimate MP from the US hip images in literature; however, validation for accuracy has not been reported. This study aimed to determine the accuracy of the 2D ultrasound techniques using two 3D printed hip phantoms with known MP values. The MPs estimated from the US images were compared with those measured from the X-ray images. Based on the experimental results, the US measurements had a maximum absolute discrepancy of 2.2% as compared to 9.8% from the X-ray measurements for the MP. The study on phantoms has showed the proposed US approach is promising with better accuracy and without ionizing radiation.

Keywords: Ultrasound imaging - Elastography
Abstract: This study focuses on the reconstruction of the shear modulus contrast in linear elastic and isotropic media, in quasi-static ultrasound elastography. The method proposed is based on the variational formulation of the equilibrium equations and on the choice of adapted discretization spaces to estimate the parameters of interest. Experimental results obtained with CIRS phantoms are presented, for which regions with different mechanical properties can be clearly identified in the stiffness contrast maps. Elastic modulus images collected with a shear-wave elastography technique from a clinical ultrasound scanner (Aixplorer) are also provided for comparison. Results show very similar values for the modulus ratios determined by the two elastography approaches.

Keywords: Machine learning / Deep learning approaches, Image feature extraction, Image analysis and classification - Machine learning / Deep learning approaches
Abstract: We propose a novel method for deriving ground truth labels for regression problems that considers the precision of annotators separately for each label. This method ensures that higher performing annotators contribute more to the final landmark position which is in contrast to conventional methods that assume all annotators are equally accurate in completing the set task. In addition to describing the novel method, a set of preliminary experimental results is also provided, comparing the performance of the precision method to that of the global mean.

Keywords: PET and SPECT Imaging applications
Abstract: Recently, deep neural network has been an effective tool. Owing to the fact that the traditional optimized algorithm, Iterative Shrinkage-Thresholding Algorithm (ISTA) or Alternating Direction Method of Multipliers (ADMM), could be presented by form of network, and network framework could overcome some shortcoming of traditional algorithm, which inspired us to introduce the structured deep network into PET timing calibration. In this paper, we introduce the ADMM-Net by reformulating an ADMM algorithm to a deep network for calibration, and combines the advantage of compatibility of consistency condition method. To verify the performance, several experiments of Monte Carlo simulation in GATE is performed.

Keywords: Brain imaging and image analysis, Functional image analysis, Image feature extraction
Abstract: The most common pipelines for studying time-varying network connectivity in resting state functional magnetic resonance imaging (rs-fMRI) operate at the whole brain level, capturing a small discrete set of “states” that best represent time-resolved joint measures of connectivity over all network pairs in the brain. This whole-brain hidden Markov model (HMM) approach “uniformizes” the dynamics over what is typically more than 1000 pairs of networks, forcing each time-resolved high-dimensional observation into its best-matched high-dimensional state. While straightforward and convenient, this HMM simplification obscures functional and temporal nonstationarities that could reveal systematic, informative features of resting state brain dynamics at a more granular scale. We introduce a framework for studying functionally localized dynamics that intrinsically embeds them within a whole-brain HMM frame of reference. The approach is validated in a large rs-fMRI schizophrenia study where it identifies group differences in localized patterns of entropy and dynamics that help explain consistently observed differences between schizophrenia patients and controls in occupancy of whole-brain dFNC states more mechanistically.

Keywords: Ultrasound imaging - Breast, Image segmentation, Machine learning / Deep learning approaches
Abstract: Automatic breast ultrasound image (BUS) segmentation is still a challenging task due to poor image quality and inherent speckle noise. In this paper, we propose a novel multi-scale fuzzy generative adversarial network (MSF-GAN) for breast ultrasound image segmentation. The proposed MSF-GAN consists of two networks: a generative network to generate segmentation maps for input BUS images, and a discriminative network that employs a multi-scale fuzzy (MSF) entropy module for discrimination. The major contribution of this paper is applying fuzzy logic and fuzzy entropy in the discriminative network which can distinguish the uncertainty of segmentation maps and groundtruth maps and forces the generative network to achieve better segmentation performance. We evaluate the performance of MSF-GAN on three BUS datasets and compare it with six state-of-the-art deep neural network-based methods in terms of five metrics. MSF-GAN achieves the highest mean IoU of 78.75%, 73.30%, and 71.12% on three datasets, respectively.

Keywords: Brain imaging and image analysis, Functional image analysis, Magnetic resonance imaging - MR neuroimaging
Abstract: In functional magnetic resonance imaging (fMRI), spatial smoothing procedure is generally a stable step in the preprocessing stream. Previous research (including ours) suggested dependency of the static functional connectivity on the size of the spatial smoothing kernel size. But its impact on the time-varying patterns of functional connectivity has not been investigated. Here, we sought to identify the effects of spatial smoothing on brain dynamics by performing dynamic functional network connectivity (dFNC) and meta-state analysis, a unique approach capable of examining a higher-dimensional temporal dynamism of whole-brain functional connectivity. Gaussian smoothing kernel with different widths at half of the maximum of the height of the Gaussian (4, 8, and 12 mm FWHM) were used during preprocessing prior to the group independent component analysis (ICA) with a relatively high model order of 75. dFNC was conducted using the sliding- time window approach and k-means clustering algorithm. Meta-state dynamics method was performed by reducing the number of windowed FNC correlations using principal components analysis (PCA), temporal and spatial ICA and k-means. Results revealed robust effects of spatial smoothing on the connectivity dynamics of several network pairs including a variety of cognitive/attention networks in a connectivity state with the highest occurrence (FDR corrected-p < 0.01). Meta-state analyses indicated significant changes in meta-state metrics including the number of meta-states, meta-state changes, meta-state span, and the total distance. These changes were particularly pronounced when we compared resting state data smoothed with 8 vs. 12 mm FWHM. Our preliminary findings give insights into the effects of spatial smoothing kernel size on the dynamics of functional connectivity and its consequences on meta-state parameters. It also provides further indication of the importance of evaluating variance associated with preprocessing steps.

Keywords: Machine learning / Deep learning approaches, Optical imaging and microscopy - Microscopy, Image feature extraction
Abstract: Cell segmentation is a common step in cell behavior analysis. Reliably and automatically segmenting cells in microscopy images remains challenging, especially in differential inference contrast microscopy images and phase-contrast microscopy images. In this paper, we propose a deep learning solution combining a Mask RCNN architecture with Shape-Aware Loss to produce cell instance segmentation. Our approach outperforms prior works in cell segmentation, achieving an IOU of 91.91% on the DIC-C2DH-HeLa dataset and an IOU of 94.93% on the PhC-C2DH-U373 dataset. Our framework can calculate cell instance segmentation masks from both types of microscopy images without any additional post-processing.

Keywords: Image segmentation, Machine learning / Deep learning approaches, Optical imaging and microscopy - Microscopy
Abstract: Nuclei segmentation in whole slide images (WSIs) stained with Hematoxylin and Eosin (H&E) dye, is a key step in computational pathology which aims to automate the laborious process of manual counting and segmentation. Nuclei segmentation is a challenging problem that involves challenges such as touching nuclei resolution, small-sized nuclei, size, and shape variations. With the advent of deep learning, convolution neural networks (CNNs) have shown a powerful ability to extract effective representations from microscopic H&E images. We propose a novel dual encoder Attention U-net (DEAU) deep learning architecture and pseudo hard attention gating mechanism, to enhance the attention to target instances. We added a new secondary encoder to the attention U-net to capture the best attention for a given input. Since H captures nuclei information, we propose a stain-separated H channel as input to the secondary encoder. The role of the secondary encoder is to transform attention prior to different spatial resolutions while learning significant attention information. The proposed DEAU performance was evaluated on three publicly available H&E data sets for nuclei segmentation from different research groups. Experimental results show that our approach outperforms other attention-based approaches for nuclei segmentation.

Keywords: Machine learning / Deep learning approaches, Image registration, segmentation, compression and visualization - Volume rendering
Abstract: Longitudinal follicle tracking is needed in clinical practice for diagnosis and management in assisted reproduction. Follicles are tracked over the in-vitro fertilization (IVF) cycle, and this analysis is usually performed manually by a medical practitioner. It is a challenging manual analysis and is prone to error as it is largely operator dependent. In this paper we propose a two-stage framework to address the clinical need for follicular growth tracking. The first stage comprises of an unsupervised deep learning network SFR-Net to automate registration of each and every follicle across the IVF cycle. SFR-Net is composed of the standard 3DUNet and Multi-Scale Residual Blocks (MSRB) in order to register follicles of varying sizes. In the second stage we use the registration result to track individual follicles across the IVF cycle. The 3D Transvaginal Ultrasound (3D TVUS) volumes were acquired from 26 subjects every 2-3 days, resulting in a total of 96 volume pairs for the registration and tracking task. On the test dataset we have achieved an average DICE score of 85.84% for the follicle registration task, and we are successfully able to track follicles above 4 mm. Ours is the novel attempt towards automated tracking of follicular growth.

Keywords: CT imaging, Image classification, Image registration, segmentation, compression and visualization - Volume rendering
Abstract: The wide spread of coronavirus pneumonia (COVID-19) has been a severe threat to global health since 2019. Apart from the nucleic acid detection, medical imaging examination is a vital diagnostic modality to confirm and treat the disease. Thus, implementing the automatic diagnosis of the COVID-19 bears particular significance. However, the limitations of data quality and size strongly hinder the classification and segmentation performance and it also result in high misdiagnosis rate. To this end, we propose a novel full scale attention mechanism (FUSA) to capture more contextual dependencies of features, which enables the model easier to classify positive cases and improve the sensitivity. Specifically, FUSA parallelly extracts the information of channel domain and spatial domain, and fuses them together. The experimental study shows FUSA can significantly improve the COVID-19 automated diagnosis performance and eliminate false negative cases compared with other state-of-the-art ones.

Keywords: Image segmentation, CT imaging, CT imaging applications
Abstract: Pulmonary cancer is one of the most commonly diagnosed and fatal cancers and is often diagnosed by incidental findings on computed tomography. Automated pulmonary nodule detection is an essential part of computer-aided diagnosis, which is still facing great challenges and difficulties to quickly and accurately locate the exact nodules' positions. This paper proposes a dual skip connection upsampling strategy based on a Dual-Path network in a U-Net structure generating multiscale feature maps, which aims to minimize the ratio of false positives and maximize the sensitivity for lesion detection of nodules. The results show that our new upsampling strategy improves the performance by having 85.3% sensitivity at 4 FROC per image compared to 84.2% for the regular upsampling strategy or 81.2% for VGG16-based Faster-R-CNN.

Keywords: Brain imaging and image analysis, Functional image analysis, Magnetic resonance imaging - MR neuroimaging
Abstract: Using a relatively high model order of independent component analysis (ICA with 75 ICs) of functional magnetic resonance imaging (fMRI) data, we have reported a clear effect of spatial smoothing Gaussian kernel size on spatiotemporal properties of intrinsic connectivity networks (ICNs). However, many if not the majority of ICA fMRI studies are usually performed at low model order, e.g., 20-IC decomposition, as such low order is generally enough to extract the few networks of interest such as the default-mode network (DMN). The aim of this study is to investigate if we can replicate the spatial smoothing effects on spatiotemporal features of ICNs at low ICA model order. Same resting state fMRI data that we used with 75-IC analysis were used here. Spatial smoothing using an isotropic Gaussian filter kernel with full width at half maximum (FWHM) of 4, 8, and 12mm was applied during preprocessing. ICNs were identified from 20-IC decomposition and evaluated in terms of three primary features: spatial map intensity, functional network connectivity (FNC), and power spectra. The results identified similar effects of spatial smoothing on spatial map intensities and power spectra at p < 0.01, false discovery rate (FDR) corrected for multiple comparisons. Reduced spatial smoothing kernel size resulted in decreased spatial map intensities as well as a generally decreased low-frequency power (0.01 - 0.10Hz) but increased high-frequency power (0.15 - 0.25Hz). FNC, however, did not show a uniform change in correlation values with the size of smoothing kernel. Notably, FNC between DMNs decreased but FNC between central executive and visual networks increased with an increase in smoothing kernel size. These preliminary findings confirm spatial smoothing influences ICN features regardless of model order. The discussion focuses on differences between observed changes at low and high ICA model orders.

Keywords: Optical imaging and microscopy - Optical vascular imaging, Image analysis and classification - Machine learning / Deep learning approaches
Abstract: Pseudo-label-based unsupervised domain adaptation (UDA) has increasingly gained interest in medical image analysis, aiming to solve the problem of performance degradation of deep neural networks when dealing with unseen data. Although it has achieved great success, it still faced two significant challenges: improving pseudo labels' precision and mitigating the effects caused by noisy pseudo labels. To solve these problems, we propose a novel UDA framework based on label distribution learning, where the problem is formulated as noise label correcting and can be solved by converting a fixed categorical value (pseudo labels on target data) to distribution and iteratively update both network parameters and label distribution to correct noisy pseudo labels, and then these labels are used to re-train the model. We have extensively evaluated our framework with vulnerable plaques detection between two IVOCT datasets. Experimental results show that our UDA framework is effective in improving the detection performance of unlabeled target images.

Keywords: Image analysis and classification - Digital Pathology, Machine learning / Deep learning approaches, Image classification
Abstract: Gleason grade stratification is the main histolo-gical standard to determine the severity and progression of prostate cancer. Nonetheless, there is a high variability on disease diagnosis among expert pathologists (kappa lower than 0.44). End-to-end deep representations have recently deal with the automatic classification of Gleason grades, where each grade is limited to namely code high-visual-variability sharing patterns among classes. Such limitation on models may be attributed to the relatively few labels to train the representation, as well as, to the natural imbalanced sets, available in clinical scenarios. To overcome such limitation, this work introduces a new embedding representation that learns intra and inter-Gleason relationships from more challenging class samples (grades tree and four). The proposed strategy implements a triplet loss scheme building a hidden embedding space that correctly differentiates close Gleason levels. The proposed approach shows promising results achieving an average accuracy of 74% to differentiate between degrees three and four. For classification of all degrees, the proposed approach achieves an average accuracy of 62%.

Keywords: Image analysis and classification - Machine learning / Deep learning approaches, Brain imaging and image analysis, Image classification
Abstract: Magnetic Resonance Imaging (MRI) technology has been widely applied to generate high-resolution images for brain tumor diagnosis. However, manual image reading is very time and labor consuming. Instead, automatic tumor detection based on deep learning models has emerged recently. Although existing models could well detect brain tumors from MR images, they seldom distinguished primary intracranial tumors from secondary ones. Therefore, in this paper, we propose an attention guided deep Convolution Neural Network (CNN) model for brain tumor diagnosis. Experimental results show that our model could effectively detect tumors from brain MR images with 99.18% average accuracy, and distinguish the primary and secondary intracranial tumors with 83.38% average accuracy, both under ten-fold cross-validation. Our model, outperforming existing works, is competitive to medical experts on brain tumor diagnosis.

Keywords: Optical imaging, Optical imaging and microscopy - Near infra-red spectroscopy, Image analysis and classification - Machine learning / Deep learning approaches
Abstract: Near infrared hyperspectral imaging (HSI) is an emerging optical imaging modality which boasts several advantages. Compared to conventional spectroscopy, HSI provides thousands of spectral samples with embedded spatial information in a single image. This allows for the collection of high quality and high volume spectral signals in a short time. In this paper, transmissive HSI combined with Partial Least Squares Regression (PLSR) was used to non-invasively predict aqueous glucose concentration. Aqueous glucose samples are prepared with concentration ranging from 0 - 1000 mg/dL at intervals of 100 mg/dL and 100 - 300 mg/dL at intervals of 20 mg/dL. Our results are validated using leave-one-concentration-out cross validation, and demonstrate the feasibility of the proposed aqueous glucose concentration detection method using the combination of HSI and PLSR.

Keywords: Magnetic resonance imaging - MR neuroimaging, Brain imaging and image analysis
Abstract: Psychiatric diagnoses based on clinical manifestations are prone to be inaccurate. Schizophrenia (SZ) and autism spectrum disorder (ASD) were historically considered as the same disorder, and they still have many overlaps of clinical symptoms in the current standard. Therefore, there is an urgent need to explore the potential biotypes for them using neuroimaging measures such as brain functional connectivity (FC). However, previous studies have not effectively leveraged FC in detecting biotypes. Considering that graph theory helps reveal the topological information in FC, in this paper, we propose a graph kernel-based clustering method to explore transdiagnostic biotypes using FC estimated from functional magnetic resonance imaging (fMRI) data. In our method, frequent subnetworks are identified from the whole-brain FCs of all subjects, and then the graph kernel similarity is computed to measure the relationship between subjects for clustering. Based on fMRI data of 137 SZ and 150 ASD subjects, we obtained meaningful biotypes using our method, which shows significant differences between the identified biotypes in FC. In brief, our graph kernel-based clustering method is promising for transdiagnostic biotype detection.

Keywords: Image analysis and classification - Machine learning / Deep learning approaches, Brain imaging and image analysis, Image registration, segmentation, compression and visualization - Volume rendering
Abstract: A number of machine learning (ML), and particularly in recent years, deep learning (DL) approaches have been proposed for automatic classification of Alzheimer’s disease (AD) using brain structural magnetic resonance imaging (MRI) data. However, the data available are limited in the case of this specific disease. Training a DL model with a large number of feature parameters on a small dataset of MRI scans will likely lead to overfitting. Overfitting reduces the generality and efficiency of the model. In this study, we show that a traditional nonlinear transformation from native space to template space, as a preprocessing stage, is effective in reducing overfitting through the reduction of spatial variations in the input data. To evaluate this effectiveness, we compare two different pre-processing approaches for DL-based AD classification task: (1) affine registration and (2) nonlinear diffeomorphic anatomical registration using exponentiated Lie algebra (DARTEL). The results show that the accuracy of the nonlinear registration based approach is much higher than the affine registration based approach. Furthermore, from the classification results obtained with noisy images, DARTEL is less susceptible to noise than affine registration. In summary, our experimental results suggest that nonlinear transformation is a preferable preprocessing step for training DL-based AD classification models on limited size datasets.

Keywords: Ultrasound imaging - Vascular imaging, Ultrasound imaging - High-frequency technology
Abstract: Abstract—Intravascular Ultrasound ultrasonic imaging (IVUS) can microscopically image blood vessels and reveal tissue layers from within the blood vessel lumen. It has high tissue penetration ability for lesion classification and can image through blood. Compared to optical techniques, however, IVUS has lower resolution arising from low acoustic bandwidths which cannot resolve sharp edges. The presented 100-V withstanding Analog-Front-End (AFE) was developed to enable a high resolution, low cost IVUS system using a high-bandwidth focused polymer transducer with 40-MHz center frequency. The fabricated AFE interfaced with the transducer with minimal insertion loss, could withstand and duplex 100-V high voltage pulses and echo signal, and had a total signal chain gain of 9.8 dB. The AFE achieved a signal-to-noise ratio (SNR) of 20.1 dB including the insertion loss of the high-impedance transducer. AFE SNR was limited by input impedance required for high-voltage pulse clamping circuitry, but was sufficient for IVUS echo reception.

Clinical Relevance— This work has the potential to enable much higher resolution, and potentially cheaper, IVUS imaging in blood vessels by integrating low-cost acoustic transducers with interface amplifiers directly on the catheter.

Keywords: Magnetic resonance imaging - Cardiac imaging, Image segmentation, Machine learning / Deep learning approaches
Abstract: Cardiovascular diseases (CVD) have been identified as one of the most common causes of death in the world. Advanced development of imaging techniques is allowing timely detection of CVD and helping physicians in providing correct treatment plans in saving lives. Segmentation and Identification of various substructures of the heart are very important in modeling a digital twin of the patient-specific heart. Manual delineation of various substructures of the heart is tedious and time-consuming. Here we have implemented Dense VNet for detecting substructures of the heart from both CT and MRI multimodality data. Due to the limited availability of data we have implemented an on-the-fly elastic deformation data augmentation technique. The result of the proposed has been shown to outperform other methods reported in the literature on both CT and MRI datasets.

Keywords: PET and SPECT imaging, Multivariate image analysis, Brain imaging and image analysis
Abstract: Metabolic connectivity is conventionally calculated in terms of correlation of static positron emission tomography (PET) measurements across subjects. There is increasing interest in deriving metabolic connectivity at the single-subject level from dynamic PET data, in a similar way to functional magnetic resonance imaging. However, the strong multicollinearity among region-wise PET time-activity curves (TACs), their non-Gaussian distribution, and the choice of the best strategy for TAC standardization before metabolic connectivity estimation, are non-trivial methodological issues to be tackled. In this work we test four different approaches to estimate sparse inverse covariance matrices, as well as three similarity-based methods to derive adjacency matrices. These approaches, combined with three different TAC standardization strategies, are employed to quantify metabolic connectivity from dynamic [18F]fluorodeoxyglucose ([18F]FDG) PET data in four healthy subjects.

Keywords: Brain imaging and image analysis, Magnetic resonance imaging - MR neuroimaging
Abstract: Independent component analysis (ICA) has been widely applied to estimate brain functional networks from functional magnetic resonance imaging (fMRI) data. ICA is a data-driven approach, however, the number of components must be prespecified. Indeed, it is difficult to estimate or determine an optimal number of components in fMRI analysis. In this paper, we propose a SMART (splitting-merging assisted reliable) ICA to overcome the problem. Our method first estimates group-level components using different settings and then yields reliable components by using a splitting and merging clustering approach. Subject-specific components are obtained using our previously proposed group information guided ICA (GIG-ICA) based on reliable group-level components to estimate individual-subject independent components. Simulations with unique components for subjects showed our method extracted components with high similarity to the ground truth spatial maps (SMs). For real fMRI data, the functional networks extracted by our method showed both similarity and specificity across subjects. To sum up, our method can effectively and accurately identify subject-specific brain functional networks without a need of parameter setting.

Keywords: Image analysis and classification - Machine learning / Deep learning approaches, Multimodal image fusion, Magnetic resonance imaging - MR neuroimaging
Abstract: Neuropsychiatric disorders such as schizophrenia are very heterogeneous in nature and typically diagnosed using self-reported symptoms. This makes it difficult to pose a confident prediction on the cases and does not provide insight into the underlying neural and biological mechanisms of these disorders. Combining neuroimaging and genomic data with a multi-modal ‘predictome’ paves the way for biologically informed markers and may improve prediction reliability. With that, we develop a multi-modal deep learning framework by fusing data from different modalities to capture the interaction between the latent features and evaluate their complementary information in characterizing schizophrenia. Our deep model uses structural MRI, functional MRI, and genome-wide polymorphism data to perform the classification task. It includes a multi-layer feed-forward network, an encoder, and a long short-term memory (LSTM) unit with attention to learn the latent features and adopt a joint training scheme capturing synergies between the modalities. The hybrid network also uses different regularizers for addressing the inherent overfitting and modality-specific bias in the multi-modal setup. Next, we run the network through a saliency model to analyze the learned features. Integrating modalities enhances the performance of the classifier, and our framework acquired 88% (P < 0.0001) accuracy on a dataset of 437 subjects. The trimodal accuracy is comparable to the state-of-the-art performance on a data collection of this size and outperforms the unimodal and bimodal baselines we compared. Model introspection was used to expose the salient neural features and genes/biological pathways associated with schizophrenia. To our best knowledge, this is the first approach that fuses genomic information with structural and functional MRI biomarkers for predicting schizophrenia. We believe this type of modality blending can explain the dynamics of the disorder better by adding cross-modal prospects.

Keywords: Optical imaging, Image segmentation, Machine learning / Deep learning approaches
Abstract: Accurate segmentation of optic disc (OD) and optic cup (OC) can assist the effective and efficient diagnosis of glaucoma. The domain shift caused by cross-domain data, however, affect the performance of a well-trained model on new datasets from different domain. In order to overcome this problem, we propose a domain adaption model based OD and OC segmentation called Meta enhanced Entropy-driven Adver- sarial Learning (MEAL). Our segmentation network consists of a meta-enhanced block (MEB) to enhance the adaptability of high-level features, and an attention-based multi-feature fusion (AMF) module for attentive integration of multi-level feature representations. For the optimization, an adversarial cost function driven by entropy map is used to improve the adaptability of the framework. Evaluations and ablation studies on two public fundus image datasets demonstrate the effectiveness of our model, and outstanding performance over other domain adaption methods in comparison.

Keywords: Optical imaging and microscopy - Optical vascular imaging
Abstract: Automatic retinal vessel segmentation in fundus image can assist effective and efficient diagnosis of retina disease. Microstructure estimation of capillaries is a prolonged challenging issue. To tackle this problem, we propose attentionaware multi-scale fusion network (AMF-Net). Our network is with dense convolutions to perceive microscopic capillaries. Additionally, multi-scale features are extracted and fused with adaptive weights by channel attention module to improve the segmentation performance. Finally, spatial attention is introduced by position attention modules to capture longdistance feature dependencies. The proposed model is evaluated using two public datasets including DRIVE and CHASE_DB1. Extensive experiments demonstrate that our model outperforms existing methods. Ablation study valid the effectiveness of the proposed components.

Keywords: Image analysis and classification - Machine learning / Deep learning approaches, Magnetic resonance imaging - MR neuroimaging
Abstract: Autism spectrum disorder (ASD) is one of the most serious mental disorder in children. Machine learning based computer aided diagnosis (CAD) on resting-state functional magnetic resonance imaging (rs-fMRI) for ASD has attracted widespread attention. In recent years, learning using privileged information (LUPI), a supervised transfer learning method, has been generally used on multi-modality cases, which can transfer knowledge from source domain to target domain in order to improve the prediction capability on the target domain. However, multi-modality data is difficult to collect in clinical cases. LUPI method without introducing additional imaging modality images is worth further study. Random vector function link network plus (RVFL+) is a LUPI diagnosis algorithm, which has been proven to be effective for classification tasks. In this work, we proposed a self-paced learning based cascaded multi-column RVFL+ algorithm (SPL-cmcRVFL+) for ASD diagnosis. Initial classification model is trained using RVFL on the single-modal data (e.g. rs-fMRI). The output of the initial layer is then sent as privileged information (PI) to train the next layer of classification model. During this process, samples are selected using self-paced learning (SPL), which can adaptively select simple to difficult samples according to the loss value. The procedure is repeated until all samples are included. Experimental results show that our proposed method can accurately identify ASD and normal control, and outperforms other methods by a relatively higher classification accuracy.

Keywords: Machine learning / Deep learning approaches, Image feature extraction, CT imaging
Abstract: Lung cancer treatments that are accurate and effective are urgently needed. The diagnosis of advanced stage patients accounts for the majority of the cases, being essential to provide a specialized course of treatment. One the emerging course of treatment relies on target therapy through the testing of biomarkers, such as the Epidermal Growth Factor Receptor (EGFR) gene. Such testing can be obtained from invasive methods, namely through biopsy, which may be avoided by applying machine learning techniques to the imaging phenotypes extracted from Computerized Tomography (CT). This study aims to explore the contribution of ensemble methods when applied to the prediction of EGFR mutation status. The obtained results translate in a direct correlation between the semantic predictive model and the outcome of the combined ensemble methods, showing that the utilized features do not have a positive contribution to the predictive developed models.

Keywords: Image reconstruction and enhancement - Machine learning / Deep learning approaches, Magnetic resonance imaging - Parallel MRI
Abstract: Machine learning and artificial intelligence have shown remarkable performance in accelerated magnetic resonance imaging (MRI). Cloud computing technologies have great advantages in building an easily accessible platform to deploy advanced algorithms. In this work, we develop an open-access, easy-to-use and high-performance medical intelligence cloud computing platform (XCloud-pFISTA) to reconstruct MRI images from undersampled k-space data. Two state-of-the-art approaches of the Projected Fast Iterative Soft-Thresholding Algorithm (pFISTA) family have been successfully implemented on the cloud. This work can be considered as a good example of cloud-based medical image reconstruction and may benefit the future development of integrated reconstruction and online diagnosis system.

Keywords: Ultrasound imaging - Elastography, Ultrasound imaging - Doppler, Ultrasound imaging - Other organs
Abstract: Some studies suggested a correlation between tissue elasticity and diseases, such as Adhesive Capsulitis (AC) of the shoulder. One category of method to measure elasticity is by utilizing Doppler imaging. This paper discusses color Doppler shear wave elastography methods and demonstrated an experiment with biological tissue mimicking phantom. A simulation with binary pattern color Doppler shear wave elastography shows that wavelength of a shear wave with suggested magnitude is equal to four multiple of pitch strip in a color flow image. However, the larger amplitude changes the duty ratio and frequency of the pattern. An experiment with biological tissue mimicking Polyvinyl Alcohol (PVA) phantoms has shown that the binary pattern color Doppler method has successfully recovered shear wave velocity map and calculate the elasticity.

Keywords: CT imaging applications, Image analysis and classification - Machine learning / Deep learning approaches, Image analysis and classification - Digital Pathology
Abstract: COVID-19 is an acute severe respiratory disease caused by a novel coronavirus SARS-CoV-2. After its first appearance in Wuhan (China), it spread rapidly across the world and became a pandemic. It had a devastating effect on everyday life, public health, and the world economy. The use of advanced artificial intelligence (AI) techniques combined with radiological imaging can be helpful in speeding-up the detection of this disease. In this study, we propose the development of recent deep learning models for automatic COVID-19 detection using computed tomography (CT) images. The proposed models are fine-tuned an optimized to provide accurate results for multiclass classification of COVID-19 vs. Community Acquired Pneumonia (CAP) vs. Normal cases. Tests were conducted both at the image and patient-level and show that the proposed algorithms achieve very high scores. In addition, an explainability algorithm was developed to help visualize the symptoms of the disease detected by the best performing deep model.

Keywords: Image visualization, X-ray radiography, Image feature extraction
Abstract: Femur fractures due to traumatic forces often require surgical intervention. Such surgeries require alignment of the femur in the presence of large muscular forces up to 500 N. Currently, orthopedic surgeons perform this alignment manually before fixation, leading to extra soft tissue damage and inaccurate alignment. One of the limitations of femoral fracture surgery is the limited vision and two-dimensional nature of X-ray images, which typically guide the surgeon in diagnosing the position of the femur. Other limitations include the lack of precise intraoperative planning and the process of trial-and-error alignment. To alleviate the issues discussed, we develop a marker-based approach for detecting the position of femur fragments using two X-ray images. The relative spatial position of the femur fragments plays a key role in guiding an innovative robotic system, named Robossis, for femur fracture alignment surgeries. Using the derived three-dimensional data, we simulate pre-programmed movements to visualize the proposed steps of the alignment method, while the bone fragments are attached to the robot. Ultimately, Robossis aims to improve the accuracy of femur alignment, which results in improved patient outcomes.

Keywords: Magnetic resonance imaging - Dynamic contrast-enhanced MRI, Image segmentation, Machine learning / Deep learning approaches
Abstract: Colorectal cancer (CRC) has the second-highest tumor incidence and is a leading cause of death by cancer. Nearly 20% of patients with CRC will have metastases (mts) at the time of diagnosis, and more than 50% of patients with CRC develop metastases during their disease. Unfortunately, only 45% of patients after a chemotherapy will respond to treatment. The aim of this study is to develop and validate a machine learning algorithm to predict response of individual liver mts, using CT scans. Understanding which mts will respond or not will help clinicians in providing a more efficient per-lesion treatment based on patient specific response and not only following a standard treatment. A group of 92 patients was enrolled from two Italian institutions. CT scans were collected, and the portal venous phase was manually segmented by an expert radiologist. Then, 75 radiomics features were extracted both from 7x7 ROIs that moved across the image and from the whole 3D mts. Feature selection was performed using a genetic algorithm. Results are presented as a comparison of the two different approaches of features extraction and different classification algorithms. Accuracy (ACC), sensitivity (SE), specificity (SP), negative and positive predictive values (NPV and PPV) were evaluated for all lesions (per-lesion analysis) and patients (per-patient analysis) in the construction and validation sets. Best results were obtained in the per-lesion analysis from the 3D approach using a Support Vector Machine as classifier. We reached on the training set an ACC of 81%, while on test set, we obtained SE of 76%, SP of 67%, PPV of 69% and NPV of 75%. On the validation set a SE of 61%, SP of 60%, PPV of 57% and NPV of 64% were reached. The promising results obtained in the validation dataset should be extended to a larger cohort of patient to further validate our method.

Keywords: Image segmentation, CT imaging
Abstract: Convolutional neural networks have become popular in medical image segmentation, and one of their most notable achievements is their ability to learn discriminative features using large labeled datasets. Two-dimensional (2D) networks are accustomed to extracting multiscale features with deep convolutional neural network extractors, i.e., ResNet-101. However, 2D networks are inefficient in extracting spatial features from volumetric images. Although most of the 2D segmentation networks can be extended to three-dimensional (3D) networks, extended 3D methods are resource and time intensive. In this paper, we propose an efficient and accurate network for fully automatic 3D segmentation. We designed a 3D multiple-contextual extractor (MCE) to simulate multiscale feature extraction and feature fusion to capture rich global contextual dependencies from different feature levels. We also designed a light 3D ResU-Net for efficient volumetric image segmentation. The proposed multiple-contextual extractor and light 3D ResU-Net constituted a complete segmentation network. By feeding the multiple-contextual features to the light 3D ResU-Net, we realized 3D medical image segmentation with high efficiency and accuracy. To validate the 3D segmentation performance of our proposed method, we evaluated the proposed network in the context of semantic segmentation on a private spleen dataset and public liver dataset. The spleen dataset contains 50 patients' CT scans, and the liver dataset contains 131 patients' CT scans.

Keywords: Magnetic resonance imaging - MRI RF coil technology, Magnetic resonance imaging - Parallel MRI
Abstract: The recently developed rotating radiofrequency coil (RRFC) technique has been proven to be an alternative solution to phased-array coils for magnetic resonance imaging (MRI). However, most of the image reconstruction methods for the RRFC requires detailed knowledge of coil sensitivity to yield optimal results. In this work, a novel reconstruction algorithm based on Robust Principal Component Analysis (RPCA) with the k-t (k-space-time) sparse bin reformation method (or rotating k-t bin method) has been presented to restore images without using dedicated coil sensitivity information. The proposed algorithm recovers images by iteratively removing the artefacts in both temporal and frequency domains caused by the Fourier invariant violation from coil rotation. The data sampling scheme consists of the golden angle (GA) radial k-space and the stepping-mode coil rotation. Simulation results demonstrate the effectiveness of the proposed imaging method for the RRFC-based MR scan.