Patent for “Method to analyse medical treatment data using deep learning and intelligence analyzer”

Artificial Intelligence – Kuan Chen, Infervision Medical Technology Co Ltd

Search Patent for “Method to analyse medical treatment data using deep learning and intelligence analyzer”

Abstract for “Method to analyse medical treatment data using deep learning and intelligence analyzer”

A method to analyse medical treatment data using deep learning via an intelligent analyzer thereof. This effectively relieves work stress for doctors at a hospital and medical researchers. It performs scientific analysis of large amounts of medical treatment or medical data and returns an analysis result that matches the data. The method’s core content is to create a model on a computer by using deep convolution neuron algorithms in deep learning. The model aids doctors in making correct judgements and effective decisions regarding large amounts of medical treatment data. It uses mass medical treatment data selection and optimizes model parameters through?training? The model will automatically learn the pathology analysis process of doctors and medical researchers, then assist them in processing large amounts of medical treatment data.

Background for “Method to analyse medical treatment data using deep learning and intelligence analyzer”

“Field of Invention”

“The invention is a smart device that analyses medical treatment data. More specifically, it’s an intelligent analyzer that automatically summarises large amounts of medical treatment data.

“Description of Related Arts.”

“Generally, researchers and doctors at hospitals or medical research institutes need to do a lot of work each day.” A doctor working in a hospital’s clinical department must conduct analysis, research and make decisions every day on medical treatment data.

The following presentation is of data from medical treatment taken at a Beijing tertiary hospital. It includes 1162 CT (Computer Tomography), 1361 X-ray cases and 325 NMR (Nuclear Magnetic Resonance) cases. Each CT case contains 2 two-dimensional photos on average, 50 CT cases have two-dimensional images on average, and 100 NMR cases have 100 two-dimensional photographs in average.

“All of these detection data should have been drafted into reports radiologists. There are only twenty radiologists in this hospital, including inexperienced young radiologists and older radiologists who can type slowly. The related doctors and researchers are therefore under immense work pressure due to the high volume of daily work. This results in a rapid decline in energy, low work efficiency, and even high error rates when analysing conclusions.

“The invention solves the technique problem by providing a method to analyse medical treatment data using deep learning and an intelligent analyzer thereof. This can be used to effectively relieve work stress for doctors in hospitals or medical researchers. It can also perform scientific analysis on large amounts of medical data or medical data and produce an analysis result that matches it.”

“The present invention solves the problem of deep learning by providing a method to analyse medical treatment data using deep learning. It includes the following steps:

“1. Collecting and storing large amounts of registered medical treatment and medical diagnosis rawdata that are matched and of equal type with the medical treatments raw data as medical training data at a computer via an input device.

“2. Linking the variation value in the textual and image data that is not less than 2D in the medical training data with space and time with the corresponding data.

“3) Unifying the medical treatment data and the variation values of each subject into one unit data during the step of collecting large amounts of medical training data.

“4) transforming or formatting the medical treatment training data into computer-understandable structured data matrix by segmentation, correlation or text data mining methods, and extracting features from each unit data;”

“5) Inputting the structured data matrix of medical treatment into a deep learning model of the computer at a storage module;”

“6) Optimizing deep learning models via the computer device. The optimization method consists of the following steps:

“a. Building a primary deep-learning frame to establish data models that include an input layer, at most a hidden layer, and an output layer according to features of medical treatment training data. The input layer contains a plurality nodes with a plurality feature of the medical data raw data and the output layer contains a plurality nodes with a plurality feature of the medical diagnosis data. Each hidden layer includes a plurality nodes mapping with an output from a previous layer.

“b. Constructing data models for each node using a mathematical formula. The inputs to nodes of the input layers are features from the medical treatment data. The inputs to each hidden layer and output layer are outputs from a previous layer. Each node at each layer’s output is calculated according to the mathematical formula.

“c. Initializing the parameter Ai, and comparing each output at each node in the output layer with medical treatment diagnostic data stored at that respective node to modify Ai at the note. This will ultimately produce a parameter Aiat the respective note which allows the output at the respective note to be partial-maximally comparable to the features of medical treatment diagnosis information at the corresponding node.”

“7) Input the medical data to be analyzed, as structured matrix data, into a deep learning model to perform a medical pathological analysis.

“8. Outputting a medical pathology analysis result that matches the to-be-analyzed data by the deep-learning model through an out device.”

Unsupervised learning is the best way to optimize parameter Ai. Unsupervised learning is achieved by using a Restricted Boltzmann machine or Denoising Autoencoder to self-learn.

The supervised learning method is used to optimize parameter Ai. A parametric or non-parametric mathematical formula is possible. The parametric formula could be a linear model formula, neuron formula formula or convolution operation. The non-parametric formula can also be extreme. This is how the mathematic model looks like:

“y=g(X)=fn?fn?1?fn?2? . . . ?f1 (X), where y represents the features of the output layer’s medical treatment diagnosis data. The dimension of Mn is the dimension of X. The f1 to fi are the formulas at each layer. The f1 transforms the X in M0-dimension into the M1-dimensioned output Z1, Z1 being the input to the formula at F2 and is propagated in a forward-propagation fashion. Each layer’s model fi has an Ai array that is matched with it.

“The medical raw data” refers to the records that relate to patient diagnosis, detection, and treatment by medical technicians and clinical doctors.

“The medical treatment data consist of the records related to initial diagnosis, discharge result, and disease treatment effect by medical doctors and medical technicians, as well as the textual visiting and follow-up data provided by doctors.”

“The data features include the variation value for the medical treatment training data with space and time, and the varying mathematics statistics values of data themselves, i.e. the decreasing or increasing trend in the data with respect to the time.”

“The structured data from the to-be-analyzed data medical data and the medical analyses data matched therewith will be fed back to deep-learning model as new training data.”

“The present invention is an intelligent analyzer that can analyse medical treatment data using deep learning. It includes:

“A means to input the medical treatment data and the to be analyzed medical data into a computer device.

“A storage module that stores the data for medical treatment training and the data to be analyzed separately or together;”

“A deep learning module for calling the medical training data stored at storage models for self-learning;

“An output means to outputting the medical-pathological analysis results matched with the to be-analyzed medical data; and

“a CPU and/or GPU processor, wherein the medical treatment training data comprise a medical raw data and a medical treatment diagnosis data matched therewith and the medical treatment training data and the to-be-analyzed medical data are computer-understandable structured data matrix, wherein the self-learning of deep learning model module uses the parametrical mathematical formulas including linear model equation, neuron model equation, convolution equation, and/or extreme formulas, wherein the inputting means comprises computer devices provided at hospitals or medical research institutions, and stationary computing output terminals and portable intelligent terminals networked with the computer devices, wherein the output means comprises stationary computing output terminals and portable intelligent terminals provided at hospitals and medical research institutions and networked with the inputting means.”

“The intelligent analyzer also includes a network linking module that allows you to link to the internet, Ethernet or WIFI model using wire, WIFI module or GPRS modules.”

“The core content and intelligent analyzer of the method is the establishment in a computer of a model using deep convolution neuron algorithms in deep learning. The model aids doctors in making correct judgements and effective decisions regarding large amounts of medical treatment data. It is used for mass medical data selection and optimization model parameters. The model can automatically learn the pathology analysis process used by doctors and medical researchers, which then aids in the processing of large amounts of medical data. This invention can significantly reduce work stress and increase efficiency for doctors and medical researchers. This invention allows doctors and medical researchers to focus their efforts on more important tasks, rather than heavy analysis of medical treatment data.

“But further objects and benefits will become evident from a consideration the ensuing description, drawings.”

These and other features and benefits of the invention will be apparent in the detailed description, accompanying drawings and the attached claims.

“The following description has been given to allow any person skilled in art to make and utilize the present invention. The following description contains preferred embodiments. However, the details are only examples. Modifications will be obvious to those who are skilled in the art. These general principles can be applied to all embodiments, alternative, modifications, equivalents and applications, without departing from its spirit and scope.

Referring to FIG. “Referring to FIG. The model will automatically learn the pathology analysis process of doctors and medical researchers, which then aids them in processing large amounts of medical treatment data.

“Medical treatment data intelligence analysis system, which is generally one of the most important areas within medical technology, i.e. The analysing system for medical treatment data intelligence analysis system, where a lot of effort is centralized at the Pulmonary CT Nodule Analysis. It mainly comprises two technique modules: image segmentation, and intelligence detection. Image segmentation serves two purposes: it assists imaging doctors and clinical doctors to analyze the lung structure. It also prepares the surgeon for surgery by segmenting important parts of the lung such as the trachea and lung lobes. There are many mature algorithms and technologies in image segmentation. However, the majority of them are traditional model algorithms like cascade model. This is not able to compete with the intelligent analyser. The image segmentation analysis system only accounts for a small percentage of medical treatment data processing and is therefore not very important to doctors.

“Deep Learning Technology” is a groundbreaking technology in artificial intelligence. It has successfully brought many breakthrough applications, such as Google image analysis, Google driverless car, Google book and Google Brain.

“However the most commonly used methods in the medical treatment data analysis area are traditional classification method such Support Vector Machine, rather than the most advanced technology available in artificial intelligence field. The patent number is CN201110376737 X. This method was adopted by a patent No. CN201110376737 X refers to Gradient Boosting, which was the most popular machine learning method between 1995 and 2005 but is not able to provide the most advanced technology in the artificial intelligence field.

“Recently, it was recognized that the Deep Neural Network is the most advanced algorithm for 2-D and 3-D image detection (see Bengio-2009, Yoshua Bengio, Learning Deep Architectures to AI?, Foundations, and in Machine Learning 2 (1) ).).). Some applications, such as traffic light recognition and hand-writing number recognition, may have a machine that outperforms human-identification with higher correct rates.

“The invention uses the most advanced deep-learning algorithms in medical treatment data analytics to construct a medical data analysis system. It uses large amounts of data to create models. This is able to significantly relieve the doctor’s work pressure and increase their efficiency.”

The medical treatment data analysis system is composed mainly of a module fine-tuning and a module per-training module. The model pre-training module is designed to determine the best mathematical representation of the features of the medical treatment analysis process using the medical treatment training data. An application module is the main module of the intelligence analyser system. It is designed to input the to-be-analyzed data for medical treatment into the pre-training module. This module then generates an automatic medical pathological analysis result that matches the to-be analyzed data.

“The following are the detailed descriptions for the present invention.”

“The present invention consists of the following steps.”

“The first step is to gather and store huge registered medical treatment raw and medical diagnosis data that is of the same type as the medical treatment data as medical training data at a computing device via an input device. Medical treatment pre-training is designed to allow the computer to automatically determine the corresponding medical diagnosis analysis data based on the medical treatment raw data.

The medical treatment raw data includes the records on patient diagnosis, treatment and treatment by medical technicians and clinical doctors. The medical treatment diagnosis data includes the records that relate to the initial diagnosis, discharge result, and treatment effect of the disease by medical technicians and clinical doctors, as well as the text visiting records, and any follow-up data provided by doctors.

“In clinical aspects, i.e. “In clinical aspects, i.e.,. The medical treatment diagnosis data (also known as target data) includes the records on initial diagnosis, discharge results and treatment effect.

“Clinical Examples”

The medical treatment data analysis system can provide an analysis of disease identification and treatment advice. After entering relevant information about the patient, such as their age, gender, past illness history, and treatment plan, the system will then summarize the data. The following data is provided for a particular patient: 65 year old male, cough, chest suppress, recent weight loss and long-term smoking history.

“In medical technology aspects, such as pathology, laboratory radiology and nuclear medicine, the medical treatment raw information comprises original image data, pathological data, disease-related detection and data, specific location and with or without dissemination. The medical treatment diagnosis data include the doctor’s follow-up and visiting records in text format.

“One example of medical technology in radiology is the ability for the intelligence analyser to diagnose and analyze the disease. This is possible by training the analysis and use of different image detection methods and training the analysis of the original images on different body parts. The intelligence analyser can search all images within a short time span to find the location, size and inner density of the lesion. It also shows the position of any other parts in the image.

“The second step involves associating the variation value of text data and image data that is not less than 2D of the medical training data with the appropriate time and space. The second step is to associate the medical treatment raw data and the medical treatment diagnosis data for the same case.

“The third step is unifying the medical training data and the variation data of each subject into one data unit. This is the step of collecting large amounts of medical training data. It is the unification of the medical training data of one subject or related to a number of diseases and the variation data into one data unit.

“The fourth step is to transform or format the medical treatment training data into computer-understandable structured data matrix by segmentation, correlation or text data mining methods, and to extract features from each unit data, wherein the data features comprise the variation values of the medical treatment training data with time and space, and the varying mathematical statistics values of the data.”

“In particular, data features include the variation in the medical training data with time-going, i.e. the decreasing or increasing trend data with respect to time-going, and the variation in the medical training data with space-changing. The relationship between one image data and the other. Data features also include the various mathematical statistics values of data, such the comparison values between individual data and others. All the data features will be formatted as vectors, matrix, or array that is, transformed into a computer-understandable data structure.”

The steps of initial data processing and image processing are part of the step of data collection. The first step in image processing to determine image data features is to extract the image content that is related to medical treatment diagnosis data.

“The TF-IDF (term-inverse document frequency), can be used as a quantized data retrieval method and text mining technique in the textual document processing step. These text and image data processing techniques would greatly assist in data feature collection.

“The fifth step involves importing the structured data matrix of medical treatment into a deep learning model for the computer device stored in a storage module.”

“The sixth step involves optimizing the deep learning model via a computer device. The optimization method is described below.”

“The first step in model optimization is to create a primary deep-learning frame and to establish a data modeling consisting of an input layer, at most a hidden layer, and an output layer with regard to the data features for the medical training data. The input layer contains a plurality nodes with a plurality data features of medical treatment raw data. The output layer comprises an output layer that includes a plurality nodes with features related to medical treatment diagnosis data. Each hidden layer has a plurality nodes mapping to an output of the layer.

The second step is to create a data model for each of the nodes using a mathematical formula. Here, the relevant parameters are either manually set or randomly generated automatically. The inputs to nodes in the input layer represent the data features of medical treatment raw data. The inputs of nodes in the hidden layers or the output layer represent the outputs from the previous layers, with each node of each layer being calculated according to the mathematical formula.

The third step is to initialize the parameter Ai. Next, compare the output at each node in the output layer with the medical diagnosis data stored at that node to modify Ai at that node. This will allow the output at the note to be partial-maximally identical to the features of the medical diagnosis data at the corresponding Node.

“The methods to optimize parameter Ai include supervised and unsupervised learning methods, where the unsupervised learning uses a Denoising Autoencoder (or Restricted Boltzmann Machine) to self-learn.”

“More specifically, the mathematical equation can be parametric or non-parametric. The parametric formula can include linear, neuron, or convolution model formulas, while the non-parametric formula can include extreme formulas. This is how the mathematic model looks like:

“y=g(X)=fn?fn?1?fn?2? . . . ?f1 (X), where y represents the features of the medical diagnosis data at the output layers and the dimension thereof are Mn, and X is the raw medical treatment training data. The f1 is used to transform the X in M0-dimensioned output Z1 into the M1 dimensioned X1. Z1 is then the input to the formula off2 and is propagated in a forward-propagation fashion. Each layer’s model fi has an Ai array that is matched with it.

“For instance, the logical mathematics formula can be found by:

“y = exp (? ) M? M? ? x m ? x m? M? M? ? x m ? a m)

“Moreover, the linear formula for modeling is given by:

“y = ? “y =???? m?? M? ? x m ? A m?, where Xm represents the input to the formula, y the output, and am the primary parameter.

The parameters (from A1 to An), and depth of the deep-learning model can be decided arbitrarily. Alternately, the initialization parameters for deep learning models (from A1 to An), can be set according to a specific method.

“Detailed Description of Operation Method”

“The heart of the model operation’s supervised deep-learning method is revolutionary in the field artificial intelligence and machine-learning. The DNN algorithm (Deep Neural Network), which is a novel technology in artificial intelligence and machine learning, allows for the creation of spatial and temporal variations of lesion scans and also takes into account 3-D imaging regulations to improve the likelihood of recognising them. The factor of doctors’ human judgment can also be added to the deep learning model. This allows for the purely machine-judgment factor to be combined with professional doctor-judgment factors in order construct the model and predict the lesion probabilities.

“The artificial intelligence technology is the present intention. The ultimate goal is to?train? The model is designed to aid doctors in diagnosing and treating lesion by automatically recognising the images and then indicating the likelihood. The model’s architecture and large data functions serve as teaching materials. They also abstract the data according to the algorithm. The intelligence data operation is dependent on the use of massive data and intelligence algorithms. These two elements will be discussed in more detail in the next section.

“1. “1.

“a”) The medical treatment training data must not be subjective and cannot contain any make-up information. It must have been generated during the actual hospital diagnosis and treatment. The training effect will be greater if the training data are closer to the actual medical treatment diagnosis data. Both medical treatment raw data as well as medical treatment diagnosis data are essential to the creation of this patent. The data that can be used to teach must also meet the requirements. All original medical treatment data (i.e. Before entering the training model, all the original medical treatment data, i.e. The MM (Magnetic Reflectance Imaging) image can be converted into a 3-dimensional matrix with a two-dimensional grayscale and one-dimensional crosssection, or a four-dimensional matrix that includes a three-dimensional color index, a three-color index, and a one-dimensional crosssection. You can extract all medical treatment data and convert it into a matrix that is easily readable.

“Unsupervised learning methods use such medical treatment raw matrix as the foundation data for model construction. However, supervised learning methods require analysis goals to match the images for further modeling. An analysis goal can be as simple or as complex as binary information such as the lesion measurement and lesion probability detection. You can also add more complex medical information, such as the type and treatment effect, as well as the exact location of the lesion to the analysis goals. For a more complex medical analysis system such as a prediction system for lesion development, it is worth noting that the historical results of a patient’s physical examination can be matched up to allow the algorithm to learn how to predict the development and progression of medical phenomena.

“b” The simulation data that is used to train medical treatment personnel can also be called simulation data. This data is created by simulation or computer processing and serves as training data for model building.

“One of the most well-known examples of this is the?Xbox Kinect, in which all the fundamental data are completely generated by 3-D modeling during the development phrase for hand-geture recognition model construction.”

“In the present invention, simulation data can be created by deforming, distortioning, and noise-superimposing original medical treatment data. The simulation data can be used for two reasons. First, the addition of deformed data makes it easier for the algorithms to recognize core varying regulation in medical treatment data more accurately. The DNN model must calculate millions of parameters, so it is susceptible to over-fitting phenomena if the data is small. The model learns from historical data and cannot be summarized or abstracted. To solve data over-fitting issues, simulate deformation data can be added to the training process.

“2. Machine Learning Algorithm Model

“The machine learning algorithm model is a primary mathematical frame for information summarizing and abstraction, and the main purpose of which is to interpret the pattern recognition process into a computer-understandable mathematic structure. Training is the process of estimating parameters. The?training? phase of the model involves estimating its parameters. After that, the model becomes the core of the method according to the invention. The machine learning algorithm can be divided into unsupervised and supervised learning depending on its purpose. Both of these types are covered by the present invention.

“a) Supervised learning”

“The supervised-learning algorithms model attempts to find a target regulation predetermined by human. The supervised learning algorithm requires additional data beyond the original image matrix data. Analysis result data that matches with the original data (e.g., medical diagnosis data) are also required.

“The patent covers supervised learning algorithms.

“i. Deep Neural Network”

The DNN algorithm’s fundamental idea is to mimic human-brain identification. The NDD algorithm can automatically complete the analysis by importing original medical treatment data as well as historical analysis from the doctors. The DNN can be derived as f(x)=y. Here x is the original data for medical treatment, y is the result from the intelligence analysing software, and f is the mathematical mapping relationship between these two. [Note: This is only one type of the DNN models. The convolution layer is the top layer, while the max-pooling layer is in a cycling fashion.

The structure of DNN can be broken down into multiple layers. Each layer performs different mathematical operations. The model has multiple neural layers, with the first layer performing a multitude of inner product calculations simultaneously. Convolution algorithm is the most popular algorithm in the first layer. This slides a new algorithm at the original input data, and then outputs the inner product from the new formula and original data. The DNN algorithm creates multiple convolution blocks that each contain a 3D matrix. The convolution block’s x and y axes cover a formula for spatial variation and the z of its axis covers a formula for variation of an image in space. Each convolution block matrix must be slid with the respective data dimension in order to calculate the inner products between the respective data dimension and convolution blocks. The inner product is loosely defined as the similarity of the data dimension to the convolution block. The outputs at the respective dimensions are the inputs to each successive neural layer. The convolution matrix represents a particular morphology. The inner product calculation of convolution matrix determines if different parts of the image data match that morphology.

“As shown at FIG. “As shown in FIG. 5, the second layer of the DNN model is often a layer to perform Max Pooling operation (MP). The MP operation uses dimension information to create a larger-range block. Each block is used for maximum calculation. The MP operation is based on the characteristics of active neurons in the visual neuron network. Within a given range of information frames, only the most active unit of information is entered into each layer. The MP operation is graphical in that it retains the same calculation results regardless of data rotation. The MP operation reduces the dimension of an input representation. Combining with the first neural layers operation, the region that has a low similarity with the convolution block of the first convolution layer’s first convolution layer is deleted to reduce the amount of invalid information in each area.

“The structure of DNN is generally the combination and the repetition of the convolution layer and the max polling layer, which is aimed to extract features related to the medical treatment-or-diagnosis-related data. As shown in FIG. 4 The algorithm for a medium layer is a nonlinear combination between the first and second layers features. This allows the creation of an abstracted frame. According to the patent, it is possible to construct a multitude of neural layers.

Theoretically, the more repetitions of a combination are, the better the model training will be. This is true even if massive data can be obtained for training. While the brain’s operation system is still not fully understood, it is known that there are deep neural networks in the brain. Therefore, the more complex the neural model, the better it would be. A deeper neural model requires more parameters, which can be more difficult to estimate and train. This would lead to issues such as vanishing gradients and over-fitting.

“After undergoing many convolution and Max pooling operations, the left information is entered into a full-interconnected multi-layer-perceptron. The MLP structure is made up of two layers of logical operations that attempt to assign different abstract elements to the final judging result. The MLP operation’s ultimate output is the medical treatment analysis. FIG. FIG. 6 shows that the MLP is often a hidden-perceptron layers. In this case, the variables at successive layers are fully-interconnected with the variables at the previous. The output logically calculated at previous layers are the inputs to the subsequent layer for computing the outputs.

“b) Unsupervised Learning

The concept of neural networks is well-known for many years. The vanishing gradient problem is very serious due to the limitations of data availability and processor operation. Therefore, the DNN cannot solve this practical problem. The basis for model parameter optimization is also determined by the difference in the outputs of the model to predicate the true value. An overly-deep neural structure cannot back-propagate parameter optimization information to its bottom layer. This means that the information at the top layer cannot be transferred downwardly into the deep network structure layer-by-layer, which creates great difficulties in model construction, particularly in the area of medical treatment data analysis. Medical treatment data is typically very large, making it more difficult than any other field. It is therefore impossible to perform a comprehensive parameter optimization search in medical treatment data analysis. The present invention optimizes the parameters of the model using the unsupervised learning method. This provides a better starting condition for parameter optimization. It also allows the local minimum parameter to be determined more quickly during the parameter optimization process.

“It is well-known that denoising Autoencoders, (dAE), and Restricted Boltzmann Machines (BRM) are the best unsupervised learning methods.

“i. Denoising Autoencoders (DAE)”

“The principle behind the autoencoder’s is to determine an effective latent variable that can be used with a data set. Referring to FIG. FIG. 7 shows the complete operation of the autoencoder. The autoencoder searches the data for a latent variable mapping W, using a new Z data source. This data is based on the original medical treatment data. The ultimate goal of the autoencoder’s is to find the parameter W that minimizes the difference between the two data variables. The autoencoder’s goal is to find a parameter that can fully represent all data variables with limited information. DAE adds a lot of noise to the autoencoder’s operation. The large amount of noise is, intuitively speaking, used to force the model’s search for more valuable potential regulation without being affected in any way by invalid regulations. To speed up parameter optimization, the parameters that are ultimately trained would be used as the starting points for supervised learning.

“ii. Restricted Boltzmann Machine (BRM)”

Summary for “Method to analyse medical treatment data using deep learning and intelligence analyzer”