3D Printing – Seong-Jae Lim, Bon-Woo HWANG, Kap-Kee KIM, Seung-Uk Yoon, Hye-Ryeong JUN, Jin-Sung Choi, Bon-Ki Koo, Electronics and Telecommunications Research Institute ETRI
Abstract for “Apparatus for creating three-dimensional personalized figures”
The disclosed apparatus and method allow for the creation of a 3D personalized figure that can be printed by automatically detecting features and areas from face data obtained by heterogeneous sensors. This is done by optimizing global/local translation and detecting features. The 3D personalized model creation apparatus takes face data from a user and extracts feature points for each region. Based on these feature points, it creates unique 3D figures of the user using the unique 3D model and any previously stored facial expression and body models. It then verifies that each 3D model corresponds to 3D printing by correcting and editing the 3D model.
Background for “Apparatus for creating three-dimensional personalized figures”
“1. “1.
The present invention relates to an apparatus for creating a 3D figure. More specifically, it relates to an apparatus for creating a 3D personalized 3D figure that can be printed by automatically detecting the face area and features of each region from heterogeneous sensors. This is done by optimizing global/local translation.
“2. “2.
Methods of creating 3D faces models include scanning static information about an entity with an active sensor like a laser or pattern light, creating 3D faces models using stereo or multiview images, depending on whether lighting conditions, sensor geometric information and viewpoints have been previously set or using depth sensors or the like, and creating 3D models manually by a skilled expert.
“For example, U.S. Patent Application Publication No. 2011-0148868 entitled?Apparatus, Method and Device for Reconstructing Three Dimensional Face Avatars through Stereo Vision and Facial Detection? This article describes a method for creating the face of a 3D avatar by using stereo vision and a face detector.
“Further Korean Patent Application Publication No. 2012-0102275 entitled “Virtual Service system for hair and method thereof, and device supporting the same?” This technology allows for the creation of a 3D model using 3D rendering on the head and face of the user and then applying a specific hair object in the 3D model.
“This is why, of all the traditional methods for creating a 3D facial model, scanning requires a post-processing process by an expert. The existing sensor-based method is also disadvantageous because it can only be used in a pre-set condition. Multiple cameras are required to be used. This creates problems such as color matching, synchronization, and geometric correction. The problem is that the appearance data generated by scanning the entire head, including the face, or by only capturing a portion of it, is made up of many unrefined meshes with holes or overlaps, or only part of the face data. It is therefore not possible to directly apply the data to 3D printing.
“Accordingly, this invention was made in consideration of the above problems in the prior art. An object of the present invention are an apparatus and a method to create a 3D personalized figure suitable 3D printing. This is done by detecting the face area and features in respective regions using heterogeneous sensors, and optimizing global/local translation.
According to an aspect of the invention, it is possible to create a 3D personalized figure. The method involves acquiring facial data from a target, extracting feature points for each region, creating unique 3D models of the user’s faces based upon the extracted feature points, and creating 3D figures based the unique 3D models, previously stored facial expression models, and body/adornment model models. Finally, verify that each 3D model corresponds to 3D printing by correcting and editing the 3D model based the results.
“Reconstructing unique 3D models for the user’s faces may involve creating a front picture based upon the face data, detecting a region from the face area, detecting feature point for that area, matching the appearance information from the 3D model with the face points for the respective regions, and then generating a facial texture map using the transform appearance information from the 3D model and the front photo. Finally, the user can create unique 3D models using the face texture maps.
“Creating unique 3D models for the user’s faces may involve creating unique 3D models based on a process of matching a color to a captured area using the face texture map.
“Creating 3D figures may involve creating facial expressions that match the input facial expression of the user based upon the unique 3D model and the previously stored body/adornment modeling; creating adorned unique 3D designs by combining the body/adornment models with unique 3D; and choosing a facial expression from the created facial expressions or from the adorned unique 3D modelling; and then creating 3D figures based on these results.
“The face data could correspond to a 3D mesh model that is unrefined and not refined.”
“According to another aspect of the invention, an apparatus is provided for creating a 3D personalized body. It includes an information acquisition unit to acquire face data of a user that corresponds to a reconstruction target. A face reconstruction unit to extract feature points from respective regions of the face data and reconstruct unique 3D figures of the user’s faces based upon the extracted feature points. A model creation unit to create 3D model of the user’s facial expression models and body/adornment model. Finally, a model verification unit to verify that each 3D print-compatible with 3D printing.
“The information acquisition unit could acquire the face data from the user that corresponds to the reconstruction target with heterogeneous sensors.”
“The face data could correspond to a 3D mesh model that is unrefined and not refined.”
The face reconstruction unit can include a unit that generates a front-image based upon the face data, a unit that detects a facial area from the front photo, a unit that detects feature points; a unit that matches and transforms the appearance information from the 3D Standard Model to determine 3D corresponding points; a unit that generates a texture map using the face texture information and transformed appearance information; and a unit that creates unique 3D models of the user’s faces using the unique
“The unique 3D modeling unit can create unique 3D models for the user’s faces based on a process of matching a color of the captured face area using the face texture map.
The model creation unit can include a facial expression generator unit to generate facial expressions that match the input facial expression of a user. It may also include a body/adornment modeling unit for creating unique 3D models. This unit combines the body/adornment models with the unique 3D model. A selection unit is used to select a facial expression and body/adornment models that most closely match the facial expressions created and those from the adorned unique 3D model.
“The model creation unit can be configured so that a storage unit for facial expression models, including facial expression models, and a storage unit for body/adornment modeling units including body/adornment mods are used in conjunction.”
Summary for “Apparatus for creating three-dimensional personalized figures”
“1. “1.
The present invention relates to an apparatus for creating a 3D figure. More specifically, it relates to an apparatus for creating a 3D personalized 3D figure that can be printed by automatically detecting the face area and features of each region from heterogeneous sensors. This is done by optimizing global/local translation.
“2. “2.
Methods of creating 3D faces models include scanning static information about an entity with an active sensor like a laser or pattern light, creating 3D faces models using stereo or multiview images, depending on whether lighting conditions, sensor geometric information and viewpoints have been previously set or using depth sensors or the like, and creating 3D models manually by a skilled expert.
“For example, U.S. Patent Application Publication No. 2011-0148868 entitled?Apparatus, Method and Device for Reconstructing Three Dimensional Face Avatars through Stereo Vision and Facial Detection? This article describes a method for creating the face of a 3D avatar by using stereo vision and a face detector.
“Further Korean Patent Application Publication No. 2012-0102275 entitled “Virtual Service system for hair and method thereof, and device supporting the same?” This technology allows for the creation of a 3D model using 3D rendering on the head and face of the user and then applying a specific hair object in the 3D model.
“This is why, of all the traditional methods for creating a 3D facial model, scanning requires a post-processing process by an expert. The existing sensor-based method is also disadvantageous because it can only be used in a pre-set condition. Multiple cameras are required to be used. This creates problems such as color matching, synchronization, and geometric correction. The problem is that the appearance data generated by scanning the entire head, including the face, or by only capturing a portion of it, is made up of many unrefined meshes with holes or overlaps, or only part of the face data. It is therefore not possible to directly apply the data to 3D printing.
“Accordingly, this invention was made in consideration of the above problems in the prior art. An object of the present invention are an apparatus and a method to create a 3D personalized figure suitable 3D printing. This is done by detecting the face area and features in respective regions using heterogeneous sensors, and optimizing global/local translation.
According to an aspect of the invention, it is possible to create a 3D personalized figure. The method involves acquiring facial data from a target, extracting feature points for each region, creating unique 3D models of the user’s faces based upon the extracted feature points, and creating 3D figures based the unique 3D models, previously stored facial expression models, and body/adornment model models. Finally, verify that each 3D model corresponds to 3D printing by correcting and editing the 3D model based the results.
“Reconstructing unique 3D models for the user’s faces may involve creating a front picture based upon the face data, detecting a region from the face area, detecting feature point for that area, matching the appearance information from the 3D model with the face points for the respective regions, and then generating a facial texture map using the transform appearance information from the 3D model and the front photo. Finally, the user can create unique 3D models using the face texture maps.
“Creating unique 3D models for the user’s faces may involve creating unique 3D models based on a process of matching a color to a captured area using the face texture map.
“Creating 3D figures may involve creating facial expressions that match the input facial expression of the user based upon the unique 3D model and the previously stored body/adornment modeling; creating adorned unique 3D designs by combining the body/adornment models with unique 3D; and choosing a facial expression from the created facial expressions or from the adorned unique 3D modelling; and then creating 3D figures based on these results.
“The face data could correspond to a 3D mesh model that is unrefined and not refined.”
“According to another aspect of the invention, an apparatus is provided for creating a 3D personalized body. It includes an information acquisition unit to acquire face data of a user that corresponds to a reconstruction target. A face reconstruction unit to extract feature points from respective regions of the face data and reconstruct unique 3D figures of the user’s faces based upon the extracted feature points. A model creation unit to create 3D model of the user’s facial expression models and body/adornment model. Finally, a model verification unit to verify that each 3D print-compatible with 3D printing.
“The information acquisition unit could acquire the face data from the user that corresponds to the reconstruction target with heterogeneous sensors.”
“The face data could correspond to a 3D mesh model that is unrefined and not refined.”
The face reconstruction unit can include a unit that generates a front-image based upon the face data, a unit that detects a facial area from the front photo, a unit that detects feature points; a unit that matches and transforms the appearance information from the 3D Standard Model to determine 3D corresponding points; a unit that generates a texture map using the face texture information and transformed appearance information; and a unit that creates unique 3D models of the user’s faces using the unique
“The unique 3D modeling unit can create unique 3D models for the user’s faces based on a process of matching a color of the captured face area using the face texture map.
The model creation unit can include a facial expression generator unit to generate facial expressions that match the input facial expression of a user. It may also include a body/adornment modeling unit for creating unique 3D models. This unit combines the body/adornment models with the unique 3D model. A selection unit is used to select a facial expression and body/adornment models that most closely match the facial expressions created and those from the adorned unique 3D model.
“The model creation unit can be configured so that a storage unit for facial expression models, including facial expression models, and a storage unit for body/adornment modeling units including body/adornment mods are used in conjunction.”
Click here to view the patent on Google Patents.