A Review of Computer Vision Methods for Purpose on Computer-Aided Diagnosis

Abstract

In the field of Radiology, the Computer Aided Diagnosis is the technology which gives valuable information for surgical purpose. For its importance, several computer vison methods are processed to obtain useful information of images acquired from the imaging devices such as X-ray, Magnetic Resonance Imaging (MRI) and Computed Tomography (CT). These methods, called pattern recognition, extract features from images and feed them to some machine learning algorithm to find out meaningful patterns. Then the learned machine is then used for exploring patterns from unseen images. The radiologist can therefore easily find the information used for surgical planning or diagnosis of a patient through the Computer Aided Diagnosis. In this paper, we present a review on three widely-used methods applied to Computer Aided Diagnosis. The first one is the image processing methods which enhance meaningful information such as edge and remove the noise. Based on the improved image quality, we explain the second method called segmentation which separates the image into a set of regions. The separated regions such as bone, tissue, organs are then delivered to machine learning algorithms to extract representative information. We expect that this paper gives readers basic knowledges of the Computer Aided Diagnosis and intuition about computer vision methods applied in this area.

Introduction

Medical imaging devices such as X-ray, CT and MRI provide information such as disease, abnormality and anatomic landmark of inside of human body which cannot be seen directly. This information can be used for diagnosis or surgical planning. Since there are hundreds of slices to examine for CT and MRI data, it is time-consuming to examine all slices. Also, some noises inherent in these devices disturb the diagnosis process. For an efficient and clear Computer Aided Diagnosis, it is important to process the images in order to remove the degradation and to extract meaningful information from slices. This is the reason why the computer vision algorithms are essential for the Computer Aided Diagnosis.

In the Computer Aided Diagnosis, extracting surgical information from image is processed through pattern recognition algorithms. In the pattern recognition procedure, image processing method is applied to eliminate possible noises from the imaging device and to enhance the image quality such as image contrast and sharpness. After that, the structural information of vessel, bone, organs is extracted via a segmentation method. Segmentation is the process which separates an image into segments based on its pixel information. Each segment should represent a region of human body on the image. Finally, the pattern inherent in the segmented image is categorized into several groups through machine learning algorithms. The features used for the machine learning depend on the knowledge of the problem domain. One can, for example, use the edge information of the segmented image to find the nose tip of X-ray image. The trained machine can be used for an arbitrary input in its domain.

In this paper, we present a review on the above techniques. The remainder of this paper is organized as follows. Section II introduces the image processing methods to improve the quality of images. Section III is about segmentation methods which are categorized into region growing, graph-based, thresholding-based and variational methods. Section IV presents Random forest, SVM (Supported Vector Machine) and Deep learning which are easily the most popular machine learning algorithms nowadays. In the last section, we discuss how these methods are applied and improved for Computer Aided Diagnosis and then present a conclusion for this paper.

 

Image Processing Methods

In this Section, we will briefly review how image processing methods are exploited in the field of medicine. These techniques are developed in two ways, image enhancement and de-noising. One thing in common is that these techniques make image look better and they try to preserve original data of medical images. The followings are the brief explanations about these two techniques.

Image Enhancement

This method is a technique that exaggerates the desired region so that surgeon can easily find organs which they want to see. By using image enhancement, it helps to find certain organs and diagnosis diseases using the information from medical images.

In many cases, medical images are represented in grayscale. So contrast enhancement techniques are widely used in medicine area. Lu (1) computed image gradient to extract edge information, and enhanced contrast by gradient stretching or upscaling. This method is simple and effective, so it is widely used for a long time.

Also, image enhancement progresses by using transformation technique which convert original image to frequency domain. And then, some specific sub-band is emphasized for enhancement. In conventional computer vision area, when image processing techniques are used, people usually get multiresolution image by using multiscale method (2). And then, this image is decomposed to various channels. A typical examples is Wavelet transform. This method decomposes an original image to various low frequency channels, but we can get only a few channels in high frequency band.

To solve this problem, Yang (3) used various techniques such as Wavelet transform, Harr transform, and non-linear histogram equalization. First, high frequency sub-band of wavelet transform coefficients are further decomposed by using Harr transform. Decomposed coefficients are enhanced by using histogram equalization to emphasize high frequency details. After that, we can get sharper images than before.

Image De-noising

This method is technique that minimizes image distortion during the conventional medical image shooting process. The most important issue in de-noising is that we have to preserve the structural information while we have to eliminate the component which distracts our perception of image. For example, if we use Gaussian filter to remove high frequency noise, noise is reduced, but image edge information is also removed. Therefore, the image looks blurry. To solve this problem, there are many filtering techniques.

Coupe (4) proposed non-local mean filtering for de-noising. Conventional non-local mean filter is used in 2D image, but the author used this filter for 3D magnetic resonance image. To do this, the author adopted automatic tuning of the smoothing parameter, selection of most relevant voxels, and block-wise implementation. Also, the author insisted that this method used parallelized computation, so it can save computation time. This non-local mean filtering is widely used in medical area.

In addition to filtering method, there are techniques to denoise by using frequency domain transformation. Starck (5) proposed a new form of transform, called curvelet transform to de-noise medical image. There are methods using Bayesian rule for noise removal. Achim (6) proposed speckle suppression method for medical ultrasound image. First, logarithmic transform is applied to original image. After that, image is decomposed by multiscale wavelet transform. We use the statistical characteristics of sub-band histogram by using alpha-stable model. Bayesian estimator use this distribution to estimate noise and remove it. The paper said that the proposed method produced a better performance than those of the already existed method like soft and hard thresholding.

 

Segmentation

Segmentation is one of the must-taken procedures to understand a medical image. Medical image from CT, MRI is in grayscale, so we cannot know the location of vessel, organs, and bones exactly. Using segmentation method, we can obtain the exact locations of them. In this sections, we provide segmentation methods which are widely used for medical images. We categorize these into 4 groups: Region growing, Thresholding method, Variational method, and Graph-based method. A brief introduction of each method is given below.

Region growing

Region growing expands the connected region which we want to segment from a seed point. It is very simple to expand the connected region. Region growing is pixel-based image segmentation where it calculates the differences between the adjacent pixels and the merged region which has similar pixel values. The advantage of this method is that we can obtain clear edges because this method expands small region to large region. Therefore, it is very important to find a suitable seed point to obtain a desired region. The disadvantage of this method is long computation time and sensitivity to noise. Pohle (7) suggested upgraded region growing that learns its homogeneity criterion automatically from characteristics of the region to be segmented for medical image segmentation. Their method is based on a model that describes the homogeneity and simple shape properties of the region.

Thresholding Method

Threshold segmentation is wildly used in many fields because of its simplicity and efficiency. This method is also useful for medical image segmentation. Park (8) suggested threshold method for finding abnormality of infant’s skull automatically. Infant’s skull is divided into 6 pieces and when they grow up the pieces adhere in 1 piece. But, in deformed child’s case, some of pieces already adhere, so when they grow up, they have abnormal shapes of skull. To distinguish this abnormality, doctors usually see medical image with their naked eyes. Therefore, Park developed thresholding method automatically finding abnormality.

Besides simple thresholding methods, there are methods which were used for automatically determine threshold by iterations. In this paper, Ostu thresholding and Water shedding are presented.

Ostu thresholding

Ostu’s method is used to automatically perform clustering-based image thresholding. This algorithm assumes that the image contains two classes of pixels and it finds the optimal threshold in an image by maximizing the between-class variance in an image. However, traditional Ostu’s method for medical image segmentation is time-consuming computation, which becomes an obstacle in real time application systems.

Bindu (9) suggested an efficient conventional Otsu’s method. He suggested a way of medical image segmentation using optimized Otsu’s method based on an improved thresholding algorithm, called Two Stage Multi threshold Otsu’s method (TSMO). The idea of the TSMO method is quite simple and straightforward: to greatly reduce the iterations required for calculating the zeroth- and first-order moments of a class. They obtain better results using the TSMO method than traditional Ostu’s method according to Fig. 1 and Table 1.

Fig .1.Original image (left), Segmented image (right).9)

Table 1.Computational result of medical image9)

Water shedding

It is easy to understand the Watershed transform approaching abstract concept (10). Consider an image f as a topographic surface and define the catchment basins of f and the watershed lines by means of a flooding process. Imagine that we pierce each minimum m(f) of the topographic surface S and that we plunge this surface into a lake with a constant vertical speed. During the flooding, two or more floods coming from different minima may merge. We want to avoid this event and we build a dam on the points of the surface S where the floods would merge. At the end of the process, only the dams emerge. These dams define the watershed of the function f.

Watershed algorithm is useful for many different image segmentation applications because it is simple and intuitive, can be parallelized, and always produces a complete division of the image. However, when applied to medical image analysis, it has drawbacks such as over-segmentation and sensitivity to noise.

Grau (11) suggested an improved watershed transform for medical image segmentation. The authors proposed the utilization of a set of lower cost functions, one for each of the objects to be detected in the image.

Where fk (p,q) is the function that quantifies the probability of having an edge between the pixels p and q, given that pixel p has previously received label k. With this new definition, for example, if a bright object has to be segmented from a darker background, the function fk can be selected so that it detects a big decrease in the pixel value when we travel from the inside to the outside of the object, but not the reverse way.

Variational Method

Variational segmentation category is one of the most popular methods in this field. The main feature of all variational methods is to seek for contours which are optimal solutions of some pre-defined criteria. Some most popular methods in this category are Mumford-Shah, snake-active contour and level set. All three define the cost functions which specify the desired properties of the contours and then minimize them to get the final optimal solutions.

Mumford-Shah method

The first typical method in Variational segmentation category is the Mumford-Shah model. This method is the inspiration to the Chan-Vese one described later. Mumford and Shah approximate the image f by a piecewise-smooth function u as the solution of the minimization problem

Where C is an edge set curve where u is allowed to be discontinuous. The first term ensures regularity of C, the second term encourages u to be close to f, and the third term ensures that u is differentiable on Ω/C. The Mumford-Shah approximation suggests selecting this edge set C as the segmentation boundary. The Mumford-Shah solution is computed by using the reduced Ambrosio-Tortorelli method of Vese and Chan (12).

Snake

Snake is another method in Variational segmentation category. A snake is an energy minimizing, deformable spline influenced by constraint and image forces that pull it towards object contours and internal forces that resist deformation. Snakes do not try to solve the entire problem of finding salient image contours (13). They rely on other mechanisms such as interaction with a user, interaction with some higher level image understanding process or information from image data adjacent in time or space to place them somewhere near the desired contour. Fig. 2 shows an example of a snake initialized from some user-defined landmarks and then it shrinks to wrap the part which the user wants to segment.

Fig. 2.Snake contour at initialization (left) and final formation of the curve (right).

A simple elastic snake is defined by a set of n points vi where i=1,...,n-1, the internal elastic energy term Einternal and the external edge-based energy term Eextemal. The purpose of the internal energy term is to control the deformations made to the snake, and the purpose of the external energy term is to control the fitting of the contour onto the image. The external energy is usually a combination of the forces due to the image itself Eimage and the constraint forces introduced by the user Econ. The energy function of the snake is the sum of its external and internal energy:

Level set

The third method in this category is level set method. Unlike snake, in this method, the curve is represented implicitly by the zero level set of a function Φ(x)=0. A special case of the implicit representation is the signed distance function |Φ(x)| which gives the shortest distance from x to the boundary Φ(x)=0. The evolution of the curve is affected by some geometric variables such as the normal vector and the curvature of it. Fig. 3 shows an example of level set method in medical image segmentation from initialization to finalization (14).

Fig. 3.A: Level set method in initialization15). B: Level set method in deformation and evolution and finalization.15)

Classically, the curve is propagated by the velocity field by solving the following equation

where t indicates time index, v=Fn, F is a speed function, and . Substitute these into below equation we obtain the level set equation

The speed function F depends on the mean curvature of the contour and the gradient information from edge-detector function where J▽u0 is the convolution between the gradient map of the image and a Gaussian kernel. Finally, the curve evolves as

where v≥0. The curvier the contour is, the faster it moves and it stops evolving when it reaches the areas having high gradient.

One popular modern-day approach is the method by Chan and Vese. In this method, edge information is ignored completely (15). Rather, the method optimally fits a two-phase piecewise constant model to the given image. The segmentation boundary is represented implicitly with a level set function as described above, which allows the segmentation to handle topological changes more easily than explicit snake methods.

Graph-based method

One of the representatives of this category is Graph cuts. In this method, an image is represented by a graph whose nodes are pixels or small groups of pixels. The goal is to partition the vertices into disjoint sets so that the similarity within each set is high and across different sets is low.

The problem formation is as follows (16). Let G=(V, E) be a graph where V denote vertices and E stands for edges. Each edge (u,v) between two nodes u and v has a weight w(u,v) depicting the similarity between u and v. Graph G can be broken into two disjoint graphs with node sets, for example, F (foreground) and B (background) (also called terminal nodes) by removing edges that connect these sets. Such a separation is called a cut denoted as

One way to segment G is to find the minimum cut, that is, the one that minimizes the cost:

Several optimization methods proposed to solve the minimization problem can be named such as the Ford-Fulkerson style (Augmenting path algorithm) and Goldberg-Tarjan style (Push-Relabel algorithm). Also, a more robust algorithm is proposed by Boykov and Komologrov and it is widely used nowadays.

 

Machine Learning in Medical Diagnosis

In this Section, we will briefly review machine learning methods used in the field of medical. The machine learning methods are used to diagnose some diseases and classify something in medical process. In this field, the mostly used techniques are random forest, support vector machine, and deep neural network. The followings are the brief explanations about these three techniques.

Random Forest

Random forest is a group of several trees that make up the whole forest and is used as an ensemble learning method for classification and regression (17). Random forest can be seen as a random set of decision trees because it creates multiple decision trees, which is called bagging. With Bagging, bias in the decision tree is maintained and variance gets narrowed because the sampling data evaluation is obtained with a random number of times. This shows that it may act more accurately compared to other classifiers. In addition, the structure of decision tree tends to become deep in the sense that it may have high decision. However, having a deep structure can also result in overfitting. This problem can be solved by using random forest. Random forest generates multiple trees so as to reduce errors by standardizing overfittings from several decisions. With existing medical images, this method brings a more accurate automatic diagnosis.

Representatively, it is a classification method; it enables the organ detection in medical images. According to the journal written by Shotton and Bucciarelli (18), it is proposed that the anatomical structure detection used random forest of 3d CT scan images. In this way, it is more accurate, faster, and simpler to implement than existing classification algorithms such as SVM (Support Vector Machines) or AdaBoost. Focusing on Organ detection, the proposed algorithm is applied to the localizing tasks of nine anatomical structures (head, heart, left eye, right eye, left kidney, right kidney, left lung, right lung, and liver) in CT volumes with varying resolution, patients, and scanner types. In Antonio (19)’s journal, he used regression forest for more accurate organ detection. Since the ensemble of many randomly trained decision trees (a random forest) yields much better generalization while maintaining the pros of conventional decision trees, it is applied to the automatic diagnosis system, replacing the conventionally used classifiers.

Support Vector Machine (SVM)

Support vector machine (SVM) is supervised learning models with associated learning algorithms that analyze data used for classification and regression analysis (20). Given a set of training examples, each belonging to one of two categories, the SVM training algorithm builds a model that assigns new examples into one category or the other, making it a non-probabilistic binary linear classifier. In addition to performing linear classification, the SVM can efficiently perform a non-linear classification using what is called the kernel trick which maps their inputs into high-dimensional feature spaces. Furthermore, the accuracy of SVM is relatively high, so the SVM is used as classifier or regressor in many fields.

Support vector machine is also used in medical systems. In fault diagnosis, the SVM has been developed as a tool for maintenance routine (21). Based on the input data vectors that consist of representation of fault in machine, SVM will recognize these patterns. SVM’s main task is to recognize and classify these patterns as accurately as possible. Also, the SVM is used in machines for condition monitoring. The condition monitoring such as bearings, induction motors, pumps, compressors, HVAC machines, Stamping machines and reciprocating engine use the SVM solver or SVM-combined solver (22). SVM is also used in classification and validation of cancer tissue samples (23). DNA microarray experiments which generate thousands of gene expression measurements are being used to gather information from tissue and cell samples. Then, the SVM classifies tissue samples and explore the data for mis-labeled or questionable tissue results.

Deep learning

Recently, Deep learning shows a state-of-the-art performance in data classification and regression like image recognition, object detection, speech recognition and so on. Many people study deep learning in various areas (24). There are many studies of deep learning in medical, auto diagnosis area.

Deep Learning is a model consisting of multiple-layer neural network that is modeled the human neural system. Deep learning model can learn the data representation automatically from the input data. Learned model can distort the data space and can change from solving a non-linear problem to solving a linear problem which is easier than the non-linear one (Fig. 4).

Fig. 4.Distort the input space.24)

Existing Alzheimer’s disease method (AD) and Mild Cognitive Impairment (MCI) diagnosis take hand-crafted feature from Resonance Imaging (MRI) and Position Emission Tomography (PET). But LeCun (24) proposed a method which extracts feature representation automatically using deep learning model. That method showed the exceptional performance compared those of other methods. Suk (25) defined the health state and development model of deep learning that can classify defined health state from multi sensor data. The proposed method has a good performance compared with four existing diagnosis techniques (26). The Fakoor (27) used the unlabeled gene expression data when they trained the deep learning model. They extracted features to detect cancer and analyze cancer type. He outperformed other cancer detection and diagnosis techniques.

 

Discussion

In general, the computer vision methods used for Computer Aided Diagnosis consist of the procedures which learn some specific features in images taken by medical imaging devices. The methods applied to computer vision vary according to the clinical purpose. For example, Park (8) used thresholding to segment skull structure of patient. Based on the centroid position of each segment, it determined whether the patient had Craniosynostosis or not. Another example of interest of Computer Aided Diagnosis application is vessel tracking. Lee (28) combined the Unscented Kalman filter and active contour snake technique for vessel tracking in order to segment the cardiovascular structure of chest CT.

 

Conclusion

With the development of medical imaging devices such as X-ray, MRI and CT, there is much research on the Computer Aided Diagnosis which utilizes the state of the art computer vision methods. In this paper, we presented several essential computer vison methods and their applications in the Computer Aided Diagnosis. Though there are many methods not presented here, those in this paper are the most general and essential techniques used in many problems. We believe that this paper provides some intuition about computer vision methods applied to the Computer Aided Diagnosis.