Invented by Lvwei WANG, BOE Technology Group Co Ltd
The BOE Technology Group Co Ltd invention works as followsA system for presenting medical images is comprised of a memory, and a processor. The memory contains computer program instructions. While loading the program instructions, the processor performs the following: acquiring 2D images; extracting features from the medical images; transforming them into image vectors; and then transferring the vectors to a pre-established vector space. A training method and presentation generating method are also provided.
Background for The presentation generating system, the training method and the method of generating presentations are all part of a new invention.
In order for the accuracy of diagnosis of diseases to be improved, different technologies are emerging to represent human images, including computed tomography, X-ray scans, and B-scan Ultrasonography. These technologies allow us to get some key information and improve accuracy in disease diagnosis. The reading of medical images is becoming increasingly difficult as the amount of data grows. Since the amount of data in medical images is increasing faster than the number of radiology doctors, every reading physician must read about one thousand medical pictures each day. “Overburdening work increases the likelihood of misdiagnosis by a significant amount.
According to one embodiment, a system for presenting medical images comprises a memory that stores computer program instructions, and a processor. The processor executes the following instructions while loading the instructions: acquiring 2D images, extracting the features from the images, and transforming them into image vectors, and then transferring them to a pre-established vector space.
For example determining and displaying semantic feature vectors corresponding the image vectors according a correspondence between the image vectors within the first pre-established vector space and matching semantic vectors located in the second pre-established vector space includes: after determining the image vectors pre-established with locations that are identical or similar to those image vectors in first vector, determining the semantic vectors determined to correspond to the predetermined image vectors by comparing the correspondence between the image vectors inside the first pre-established vector space
For example, transforming semantic feature-vectors that match image feature-vectors into natural language corresponds to: transforming, and outputting by a decoder semantic feature-vectors that match image feature feature vectors in natural language corresponds to:
The processor can, for example, execute the following instructions while loading the program: scaling and trimming the medical images, color enhancement and/or duplicate.
The embodiments of the present disclosure further provide a method of generating a presentation for the above-mentioned presentation generating system, which comprises: acquiring 2D images; extracting features from the images, and transforming them into image vectors, and then transferring the vectors to a pre-established vector space. The method also includes: determining and displaying semantic feature corresponding vectors to the image vectors, according to correspondence between the image vectors in the first vector area and the semantic feature corresponding in the second space, and the based on the first vectors, and the first vectors, and a second vector space, and the first vectors, and the corresponding image vectors, and outputting the vectors, according to the vectors, and vectors in the corresponding vectors in the first space, and corresponding vectors in the first space, and determining the corresponding vectors in the first space, and the corresponding in the corresponding in the a second vectors in the a match,
In a possible embodiment, in accordance with the embodiments of the present disclosure, the step determining and displaying semantic vectors corresponding the image vectors is performed by: after determining image vectors pre-established in the first space that are identical or similar, determining the semantic vectors determined to be corresponding the predetermined image vectors, according to correspondence between the image vectors pre-established in the first space and matching semantic vectors within the second space as the semantic vectors corresponding the image vectors, and determining
In a possible embodiment, a presentation-generating method of the embodiment of the present disclosure includes: subjecting acquired medical images for scaling and trimming; color enhancement and/or copying.
The embodiments of the present disclosure provide a training method that includes: supplying a plurality 2D medical pictures and a presentation document with semantic features matching those images into the presentation generation system; extracting medical image features from the medical images; generating medical feature vectors; and then transferring them to an established first vector space. After pre-processing input presentation documents to the system by the latter, extracting semantic information, generating semantic feature vectors, and transferring them to an already-established second vectorspace; extracting a
In a possible embodiment of the training method described above, in the embodiments in this disclosure, pre-processing of the input images by the system that generates the presentations includes at least one of: scaling and trimming by the system; color enhancement by the system; and duplication by the system.
In a possible embodiment of the training method described above, the preprocessing of the input documents by the presentation generation system includes: word segmentation by the system.
In a possible embodiment of the training method described above, the step of adjusting mapping parameters by the presentation-generating system between image feature and semantic feature according to image feature and semantic feature includes: adjusting mapping parameters by the system according to a degree of loss between matching semantic and semantic feature, determined by the last adjusted mapping parameter, until the degree of loss between matching semantic and semantic feature determined by the current adjusted mapping parameter is within a predetermined range.
In a possible embodiment of the training method described above, the step of determining loss between matching semantic feature vectors and the matching feature vectors determined using the last-adjusted mapping parameters includes: Calculating the loss degree L (S, Y), between the matching feature vectors S and those determined by the last-adjusted mapping parameters Y according to the formula:
The following is an example: “in which N denotes how many sub-units are contained in the recurrent network unit of the presentation-generating system. Yt denotes a semantic feature that was determined by the sub-unit t (th) with the most recent adjusted mapping parameters. St denotes a semantic feature that matches the image. RNN(St=Yt), means that the t (th) feature that matches the image.
The present disclosure includes a number of elements that increase the reading efficiency and improve the reading quality, while reducing the likelihood of misdiagnosis.
Below, with the help of accompanying drawings, we will provide a detailed description of implementations of a system for generating medical images presentations and training methods. Note that the embodiments described are not all of the disclosure. “All other embodiments that can be obtained by a person of ordinary skill based on embodiments in the present disclosure, without any creative effort, fall within the scope
It is important to note that the system for generating medical images in embodiments of this disclosure can only be used to process 2D (two-dimensional) images, such as X ray films, and not 3D (3-dimensional) images. The following description is mainly based on the processing of 2D medical image with the presentation system of medical images in embodiments provided by the present disclosure.
As shown in FIG. The first unit comprises a medical image capture unit 101, as well as a convolutional network unit, a recurrent network unit, and a presentation unit.
As shown in FIG. The presentation generating system for medical images can include a processor 700 and a memory. The memory 600 contains computer program instructions. While processing the program instructions described above, the processor 700 performs the functions of the convolutional network unit, 102, recurrent network unit, 103, and presentation output unit, 104. The medical image capture unit 101 can also include a camera or a pick up head. It may also include a module that implements medical image acquisition.
The medical image acquisition unit 101 can be configured to capture 2D medical images.Click here to view the patent on Google Patents.