Software Technology for “Methods and apparatus for performing voice command in an electronic device”

Software – Subhojit CHAKLADAR, Sang-Hoon Lee, Hee-Woon Kim, Samsung Electronics Co Ltd

Abstract for “Methods and apparatus for performing voice command in an electronic device”

“An apparatus and a method of executing a voice command within an electronic device. An exemplary embodiment detects a voice signal and recognizes the speech. A voice command mode is activated when the voice signal detected contains a wakeup call. The signal that contains at least part of the voice signal is sent to a server. The server generates either a control signal, or a result signal that corresponds to the voice command and sends it to the electronic device. The control signal or result signal is received by the device, processed and woken it. Voice commands can be used to control the device without the user having to touch it.

Background for “Methods and apparatus for performing voice command in an electronic device”

Electronic devices that can perform multiple functions have become more common in recent years, thanks to the advancements in multimedia technology. Multi-function devices can include mobile terminals like smart phones, tablet PCs and smart cameras as well as fixed home-based electronic devices such as electronic devices that are integrated with kitchen appliances. Most electronic devices include a convergence function that combines several functions.

“Portable terminal designers aim to achieve advanced performance as also the convergence function as well as slimming and aesthetic designs of all devices.” Terminal manufacturers must compete to offer substantially the same or better performance, and to create new models that are smaller than previous designs.

A recently released device offers voice recognition with high accuracy. This voice recognition function recognizes the voice of the user and allows them to quickly execute the corresponding function without touching a button, key, or touchscreen.

“For example, voice recognition allows users to call or send a text message using a mobile terminal like a smart phone. They can also set up various functions, such as an alarm, route planner, Internet search, or alarm.

“To perform the voice recognition function the related art drives the corresponding voice recognition app, activates it, and then performs that function.”

The voice recognition application must be started when a touch input command is received on the touchscreen or a separate key. This is against the voice recognition’s unique function to facilitate data input without touching. In some cases, it may prove difficult or time-consuming to launch the voice recognition app because you must locate it on a screen that includes various application objects.

“Embodiments and methods for executing an electronic voice command are disclosed. An exemplary embodiment detects a voice signal and recognizes the speech. A voice command mode is activated when the voice signal detected contains a wakeup call. The signal that contains at least part of the voice signal is sent to a server. The server generates either a control signal, or a result signal that corresponds to the voice command and sends it back to the device. The control signal or result signal is received by the device and processed. Voice commands can be used to control the device without the user having to touch it.

“In various embodiments:”

“The voice signal could include the wakeup command, followed by the voice instruction.”

“The wakeup command could also include the voice command.”

“A silence duration can be established between the wakeup and voice commands.”

“Processing either the control signal, or the result signal could include executing a specific application of the electronic devices.”

“Processing the control signal or result signal may include displaying data that correspond to the result signal.”

“Once the voice commands mode is activated, an object can be activated on a display that indicates the activation of the voice command modes.”

“A screen that is locked before it recognizes the wakeup command in speech may not be responsive to the recognized wakeup call.

“The speech may only be recognized if the speaker of the voice signal is identified as containing a predetermined wakeup call. If the speaker of the predetermined voice signal is recognized, the wakeup command can be automatically detected.

“Alternatively, the wakeup signal may be detected when the speaker’s voice is recognized. Another embodiment of a method of executing a voice order in an electronic device includes: receiving a result signal from the server that indicates the server has detected the wakeup command in the vocal signal; awakening the device by processing the control signal, or the result signal, corresponding with the voice command.

“In one embodiment, a method for supporting an electronic voice command in a server includes: receiving a transmitted voice signal that contains at least one voice command from the electronic devices; generating a control or result signal corresponding with the voice command by recognizing the voice command and analysing it; and then sending the control signal, or the result signal, to the electronic devices.

“In one embodiment, an electronic device includes: one or multiple processors; memory; and one, or more, programs stored in the memory that can be executed by the processors. The program contains instructions for detecting voice signals and recognizing speech; activating voice command modes when speech is detected to contain a wakeup signal; transmitting a transmit message containing at most a portion the detected voice signal to server; and processing and receiving a control signal, or a result signal, generated by the server in response a voicecommand

“Other advantages, salient features, and other aspects of the invention will be apparent to those skilled with the art from this detailed description. This is taken in conjunction the annexed illustrations, which discloses exemplary embodiments.

The following description, with reference to the accompanying illustrations, is intended to provide a complete understanding of the exemplary embodiments as described in the claims and the equivalents. These specific details are intended to aid in understanding the invention, but they should not be considered as exhaustive. Those of ordinary skill in art will be able to recognize that modifications and variations can be made to the embodiments without departing from their scope and spirit. For clarity and simplicity, it is possible to omit descriptions of well-known constructions and functions.

The inventor uses the terms and phrases in the claims and description to help with a consistent and clear understanding of the invention. It should be obvious to all those skilled in the arts that the following description and claims of exemplary embodiments are provided only for illustration purposes and does not limit the invention as defined by the appended Claims and their equivalents.

“It is important to understand that the singular forms?”a? ?an,? ?an,? If the context requires otherwise, plural referents should be used. For example, a reference to “a component surface?” This includes a reference to any one or more of these surfaces.”

“Substantially” is a synonym for “substantially”. It is intended that the recited characteristic or parameter or value does not have to be exactly achieved. However, deviations or variations (including tolerances, measurement error and measurement accuracy limitations) may not prevent the characteristic from having the desired effect.

“Exemplary embodiments” of the invention include an apparatus and a method to execute a voice command in an electronics device and server.

“FIG. “FIG. Any one of many fixed or portable electronic devices 100 is possible. A portable device could be a mobile terminal, portable pad, mobile terminal, mobile tablet, media player or smart phone. It can also include a personal digital assistant (PDA), smart camera, notebook computer and a desktop computer. An electronic device can be portable and combine two or more functions from other devices. An electronic display device that is attached to a home appliance, such as a kitchen oven, would be an example of a fixed device.

“Electronic device 100” can include a controller 110 and a speaker/microphone 112. A camera 120, a Global Positioning System receiver 130, an Radio Frequency unit 140, a radio frequency (RF) unit 140, and a sensor module 150. It also includes a touchscreen 160, touch screen controller 165, as well as an external memory 170.

“Briefly: according to embodiments, device 100 detects voice signals and recognizes speech in them. Device 100 activates the voice command mode when it is confirmed that the speech contains a wakeup call. Device 100 can respond to voice commands in the voice command mode. The device 100 can respond to voice commands even if it is locked or in an idle state. A speech recognition process is used to detect if the wakeup command was uttered. The device 100 will then be able to perform the operation associated with a voice command if it is recognized.

“In one embodiment, the device 100 performs both the wakeup command detection as well as the voice command recognition. Another embodiment performs the wakeup command detection at the device 100 while the voice command recognition takes place at the server. This is after a portion is transmitted from the device 100 to it. Another embodiment performs both the wakeup command detection as well as the voice command detection at the server.

“The controller 100 may include an interface 101, one, or more processors, 102, 103, and an inner memory 104. The processor may also be called the controller 110 in some cases. The interface 101, application processor 102 and communication processor 103 can all be separated components. Or, they can all be integrated into one or more integrated circuits.

“The application processor102 executes several functions for the electronic device. It runs various software programs, and the communication processor103 processes and controls data communication and voice communication. The processors 102, 103 execute the typical functions and also execute the software module (instructions set) stored in the internal memory 104 or the external memory 170 and perform specific functions that correspond to the module. The processors 102, 103 execute the method according to the invention with the software modules stored in the internal memory 104 or the external memory 170.

“Accordingly to an exemplary embodiment (corresponding to FIG. 2), the application processor102 receives a voice command including a wakeup call and a subsequent voice order from a user via the microphone 110 and performs speech recognition on that voice signal to detect the existence of the wakeup commands. The application processor 102 might detect a silence between the wakeup call and the next voice command in the voice message when the wakeup command has been detected. The application processor 102 then determines whether the voice signal that corresponds to the voice command starts. If it does, the application process 102 sends the appropriate portion of the audio signal to a server. The application processor 102 then receives the server’s voice recognition result and performs the corresponding operation.

“Accordingly to an alternative exemplary embodiment (corresponding to FIG. 4. The application processor 102 transmits all of the voice signal to the server, including the wakeup commands portion. It then receives a speaker verification result from the server corresponding to the wakeup order. The application processor 102 activates it when the result indicates that the wakeup commands have been detected. The application processor 102 then receives a voice recognition result that corresponds to the voice command. It performs the operation based upon the voice recognition result.

“Accordingly to another example embodiment of the invention (corresponding with the method in FIG. 6. The application processor 102 receives the voice message from the user via the microphone 110 and performs the wakeup commands detection using speech recognition. The application processor 102 detects the wakeup command and uses a voice recognition algorithm. It then performs the appropriate operation based upon the voice command.

One or more of the voice recognition processors or a speaker verification process can be included in the application processor 102 or provided separately. The speaker verification processor and the voice recognition processor can be combined and may include multiple processors that perform different functions depending on their implementation. Interface 101 connects the touch screen controller (165) of the electronic device to the external or internal memory 170, 104.

“The interface 101 is coupled with the sensor module 150 to perform various functions. To detect motion or detect light from outside, an optical sensor or motion sensor can be connected to the interface 101. Other sensors, such as a temperature sensor or position determining system can also be connected to the interface 101 in order to perform relevant functions.

“The camera 120 can be coupled to the sensor module 150 via the interface 101 in order to perform a camera function like photo and video clip record.”

The RF unit 140 is a communication device that may contain at least one processor. The RF unit 140, for example, converts an RF signals to a baseband signal, and then provides that signal to the communication process 103. Or converts a baseband output from the communication process 103 into an RF signal, and transmits it through an antenna ANT. The baseband signal is processed by the communication processor 103 according to different communication schemes. The communication scheme could include, among others, a Global System for Mobile Communication (GSM), an Enhanced Data GSM Environment(EDGE) communication system, a Code Division Multiple Access communication strategy, a W?CDMA communication schema, a Long Term Evolution communication program, an Orthogonal Frequency Division Multiple Access communication plan, a Wireless Fidelity communication protocol, a WiMax communication method, and/or a Bluetooth communications scheme.

The speaker/microphone 110 is capable of generating and converting audio signals such as one for voice recognition (used in a training process to teach device 100 to recognize a specific speaker/wakeup command/voice command), voice reproduction and digital recording, and a telephone function. The speaker/microphone 110 converts the audio signal to an electrical signal or the electric signal into the voice signal. An external port can be used to connect an attachable, detachable headphone, earphone or headset (not illustrated) to the electronic device.

The touch screen controller 160 can be coupled with the touch screen controller 165. The touch screen 160 can be coupled to the touch screen controller 160.

“The touch screen 160 acts as an input/output interface between electronic devices and users. The touch screen 160 transmits the user’s touch inputs to electronic device 100. The user can also view the output of device 100 on the touch screen 160. The touch screen 160 also presents a visual output for the user. The visual output can be in the form of text, graphics, video or a combination thereof.

“The touch screen 160 can use a variety of displays. Examples include Liquid Crystal Displays (LCD), Light Emitting Diode(LED), Light emitting polymer Display (LPD), OrganicLED (OLED), Active Matrix (AMOLED), Flexible LED (FLED), and Light Emitting Polymer Displays (LED).

“The GPS receiver 130 converts an artificial signal from an?artificial? satellite. The GPS receiver 130 converts a signal received from an?artificial? satellite to information such as speed, location, and time. A distance between the satellite 130 and the GPS receiver 130 can be calculated by multiplying speed of light with signal arrival time. This calculates the location of electronic device by using triangulation, which determines the exact positions and distances from three satellites.

“The internal memory 170 and the external memory 104 can contain fast random access memory, such as one or two magnetic disk storage devices and/or nonvolatile memory and/or one or more optical storage device and/or flash memory (e.g. NAND and/or NOR).

Software is stored in the external memory 170 and internal memory 104. Software components can include an operating system, communication, graphic, user interface, MPEG, camera, and one or several application software modules. The module, being the software component, can also be referred to by the term “instruction set”. A program may also be used to refer to the module.

“The operating system software contains various software components that control general system operations. Control of general system operations can include memory management and control as well as storage hardware (devices) control and management. Power control and management are also included. “The operating system software can process normal communication between different hardware devices and software components (modules).

“The communication module allows communication between other electronic devices, such as computer, server and/or portable terminals, via the RF unit 140. The protocol architecture for the communication software module is used to configure it.

“The graphic software module contains various software components that allow you to display graphics on touch screen 160. Graphics is a broad term. The term “graphics” can be used to describe text, image, website, icon, digital image or animation.

“The user interface software modules include various software components that relate to a user interface. The status of the user interface status changes and the condition of the status of the user interface status changes are both controlled by the user interface software module.

The camera software module contains camera-related software components that allow camera-related processes and functions. An application module contains a browser, an email, an instant message and keyboard emulation. It also includes a widget, Digital Right Management, voice recognition, voice reproduction, voice reproduction, position determining functions, location-based services, and other software components. Additional modules (instructions), can be added to the memory 170 and 104. You can also choose to not use some of the instructions (if necessary).

“Herein, an instruction is provided for performing a speaker recognition function, a speech recognition function, and a voice command execution operation function. These instructions correspond to the ones for performing the operations shown in FIGS. 2, 4, and 6”.

“The different functions of electronic device 100, as described above and to be explained can be executed in hardware or software and/or their combination including one/more signal processing (ASICs) and/or Application Specific Integrated Circuits.

“FIG. “FIG. The system 195 comprises the portable terminal 100 that communicates with a server (190) through a network 180. Server 190 could be, for example, a home network server or a remote server that is accessed via a large network like the Internet. Server 190, on the other hand, can be used by a third-party portable electronic device that performs speech/voice/speaker recognition or analysis of voice signals sent thereto. Server 190 has at least one processor (192) and a memory 194 for a variety of operations. We will show you how to use the server 190 with electronic device 100.

“FIG. “FIG.

“At step 101, the electronic device 100 detects the voice signal that may include the wakeup call and the voice command of the user through microphone 110. The system activates a voice-command mode when the wakeup command is received. In this mode, no touch contact with the touchscreen or key is necessary to receive and analyze the voice command. The device 100 can be set to idle or lockscreen mode before it receives the voice signal. In certain embodiments, the device 100 may be in an application execution mode prior to receiving the voice command. This means that the device 100 is not listening to any voice commands and no operations are performed in response to them.

“In the following description it will be assumed the wakeup command is usually independent of a voice instruction that temporally follows it. Some?speaker-dependent’? embodiments may be different. As discussed below, any speech signal that is detected as being spoken by a speaker can be used to wake up the system. Other embodiments, speaker dependent or speaker independent, include an inherent voice command in the wakeup command. The wakeup command activates the voice command mode, and acts as a catalyst for device 100 to complete additional tasks, such running an application that the user has predefined.

“For instance, the wakeup command could instruct the screen to change to a mode that allows voice commands (?voice command mode) to be entered. The screen can be locked and/or unlocked. The electronic device 100 provides various functions that can be executed by voice commands. Voice commands can be used to dial, photograph, play MP3s, and other functions. The voice command can ask the server 190 to plan a route and search for a map in a variety of implementations.

“In step 200, the electronic device 100 performs speech detection on the voice signal in order to determine if the signal contains a predetermined wakeup call. The speech recognition can be speaker dependent or speaker independent. Other schemes can be used when a variety of wakeup commands have been predetermined. One or more commands is a speaker dependent wakeup command, while one or several other commands is an independent speaker command.

“A speaker dependent recognition scheme requires that a speaker or user train a recognizer using his/her voice in advance. The speech recognizer will only recognize the spoken words of the trained voice. A speaker independent recognition scheme is capable of recognizing speech from any speaker voice. Speaker independent recognition schemes extracts and stores information about hundreds of thousands of voices and allows any user to use it without the need for additional training.

“Using speaker dependent recognition, in certain embodiments, the speaker may be verified using the voice commands portion of the voice signals (which could include the whole voice signal). There is no need for a separate wakeup call. If the speaker can be verified by using the unique voice characteristics of the user to verify their identity, the separate wakeup command is not necessary. These embodiments allow the voice command to also function as the wakeup command. In steps 202 and204, the voice of the user is identified and the wakeup command is detected automatically via this recognition.

“Alternatively, the speaker dependent recognition scheme uses information from a voice signal that corresponds to a predefined word or words. This can be set up by the user through voice training. You can train the device 100 by entering your voice to match a predefined text. To do this, the user must input the wakeup call. The predefined text can either be inputted directly by the user, or converted using multiple voice-inputs. The electronic device 100 and the server 190 convert voice into text.

“When the wakeup command has been detected in step 204, the method proceeds to step 206. It returns to step 201 otherwise.

“Although FIG. 2. When the wakeup command has been successfully detected, an object for recognizing the voice command will be activated. (see FIG. 8A). This object may be a virtual mic and indicates activation for the voice command mode. The object can be displayed first, or in an emphasised manner if it was visible in the locked screen. An interactive Graphical User Interface (GUI), relating to speech/voice recognition, can also be displayed during this time. Alternativly, the GUI for voice recognition can be displayed immediately in the unlocked screen after the wakeup command has been detected.

“In an embodiment variant, the object for recognizing the wakeup command in idle mode and the screen is locked are shown together. The GUI relating to voice recognition and the object for recognizing it are both displayed together. If the screen isn’t locked, the object to recognize the voice command can be displayed with the GUI relating the voice recognition.

“In step 206, the electronic device 100 detects the silence duration (if any) between the first portion of the detected vocal signal (hereafter, ‘first voice signal?). The wakeup command and the second portion of the detected vocal signal (hereafter, the?second voice signals?) The voice command. This assumes that the voice commands are distinct from the wakeup commands (as previously mentioned, embodiments where the wakeup command and voice command can be combined is possible). As an example, suppose that the wakeup command for?Hi Galaxy is?Call Hong Gil-dong? The voice command is called?Call Hong Gildong?. When the user pronounces?Hi Galaxy consecutively, it will be called?Call Hong Gildong? ?Call Hong Gildong? will create a silence between?Hi Galaxy and?Call Hong Gildong? ?Call Hong Gildong?

A short pause can be used to determine the beginning of the voice command by detecting the first two words in the detected speech. An embodiment allows for the blocking of an extraneous part of the detected voice signal that follows the wakeup command from being sent to server along with the following voice command. A Voice Activity Detection technique (VAD), can be used to do this. A voice signal, for example, has more energy than background noise signals that include the?silence? period. If background noise is low, it’s possible to identify unique characteristics of the human vocal chords. The most common way to identify the unique characteristics in the human voice is to observe energy distribution at different frequencies. The characteristics of the human voice include noiseless but not noisy. The VAD technique is able to distinguish speech from background noise and silence. In one embodiment, instead transmitting an audio signal to the server including all sounds detected after the wakeup commands, the device 100 waits for speech to be detected and then transmits only the sound signals that begin with the wakeup commands. The 200 method avoids transmitting noises that are only silent after a wakeup command detection.

“In step 208, device 100 determines if the second voice signal corresponding with the voice command starts. Device 100, for example, checks the start position of the voice signal that corresponds to?Call Hong Gildong?. Device 100 sends the voice signal to the server at step 210 when the second voice signal starts. The server. The transmit signal is the portion of the voice signal that is transmitted to the server. The voice signal that corresponds to the voice command must not start at 208. Otherwise, the flow will return to 206. Device 100 can transmit the voice command to the server and is thus freed from the CPU intensive task of recognising the speech.

“In step 212, device 100 receives the voice recognition result that corresponds to the voice command received from the server. The server may analyze the voice command “Call Hong Gildong?” and send a control signal that corresponds to “Call Hong Gildong?” Sends the search result of the route planning or map search request to device 100.”

“In step214, device 100 performs a corresponding operation based upon the voice recognition result corresponding with the voice command or displays a result that corresponds to the voice recognition. Example: When receiving the control signal that corresponds to “Call Hong Gildong?” Device 100 searches the phonebook to find a number in Hong Gil-dong. It then attempts to connect the call to the number. Device 100 displays the map/route request search result. The process is over.

“FIG. “FIG. This method can be used in conjunction with the operation of the previously-described method 200, which is carried out in device 100. This embodiment sends the server the transmit signal. From the electronic device (e.g. transmitted at FIG. 2) in step 301

“Next, the server analyzes voice signals corresponding to voice commands using a voice recognition algorithm (equivalently?speech recognition?) algorithm) in step 302. The server then analyzes the voice signal to detect a voice command. In step 304, the server determines if the result corresponding with the voice recognition is a command signal. The server then sends the control signal that corresponds to the voice recognition to 100 in step 306 if so. After recognizing “Call Hong Gildong?”, the server sends the control signal to device 100, instructing device 100 to dial Hong Gildong at the associated number from its phone book storage.

“When the voice recognition result is not the control signal, then the server sends the result that corresponds to the voice recognition to step 308. The server can also send image content that contains the route planning request and map search request search results to the electronic device in step 308.

“Accordingly in the exemplary embodiments 200 and 300, both the electronic device and the server perform the wakeup command detection. Another embodiment of the present invention demonstrates that the server performs both the wakeup command detection as well as the voice recognition of the voice commands.

“FIG. “FIG. Device 100 is in an idle or locked screen mode. It receives a voice signal that includes the wakeup command. This voice signal is followed by the voice command of the user via the microphone 110 in step 411. The system activates when the wakeup command is recognized. The wakeup command, for example, can be used to instruct the system to change to the mode to input the voice command or to unlock the screen. The voice command can be used to control various functions of the electronic device 100. The voice command can be used to dial, photograph, play MP3s, and other functions.

“In step 402, device 100 transmits the entire voice signal, including the wakeup and voice commands, to the server as the transmit message. Next, the server receives a voice verification result that corresponds to the wakeup call (step 404). The server will send the recognition result to device 100 if it detects that the transmit message contains the wakeup command. Device 100 activates the system by receiving the recognition result at step 406 if device 100 has received the recognition result at the step 404. System activation unlocks the screen and switches between the active mode or voice command mode. Device 100 can detect voice commands in new voice signals by activating the system.

“Next, device 100 will receive the voice recognition result corresponding with the voice command in steps 408, and performs or displays the operation based upon the voice recognition result in step 410. Example: When receiving the control signal that corresponds to?Call Hong Gildong? Device 100 connects the call to the number searched by the server and searches the phonebook for Hong Gil-dong’s phone number. Device 100 displays the route/route search results or the route planning request in the map/route example. The process is over. Device 100 may then receive new voice signals at step 401, and forward them to the server for processing. The server may respond to subsequent voice commands by sending control signals or results. The steps 401 to 410 can be repeated only for voice commands. However, the server may continue processing the wakeup command because the device 100 has been awakened.

“FIG. “FIG. This method can be used in conjunction with the previously described method 400, which is found in device 100.

“At step 502, the server receives the audio signal, which includes the wakeup and voice commands from the electronic device 501 (i.e. the transmit signal received at step 402). The server uses the voice recognition algorithm to analyze the voice signal that corresponds to the wakeup call in step 502. The server analyses the voice signal that corresponds to the wakeup call and determines if it is detected. This operation could be identical to that described in FIG. 2. performed by device 100 according to that embodiment. You may perform a speaker-dependent or speaker-independent recognition operation, such as the FIG. 2. If multiple predetermined wakeup commands need to be considered, both types of recognition systems may be used.

“In step 504, a server gives a speech verification result for the electronic device. (Note: Step 504 could be skipped in other implementations.

“When the wakeup call is detected in step 506 by speech recognition processing, the server analyzes the voice signal corresponding the voice command using voice recognition algorithm step 508. The server recognizes the speech that corresponds to the voice command and generates a response message corresponding to the action to be taken by device 100. The flow will return to step 501 if the wakeup command does not get detected in the voice signal. The server may transmit a signal to device 100 informing it that there has been no wakeup command detected. Device 100 can then continue to transmit new voice signals to step 501. The server may request and only receive the first voice signal that corresponds to the wakeup command in various implementations.

“Though not illustrated, the server is able to detect a silence between the voice signal corresponding with the wakeup command (or the voice signal that corresponds to the voice commands) and distinguish the wakeup and voice commands.”

“In step510, the server notifies both the wakeup command and voice recognition results (the response signal to the electronic device). The server, for example, determines whether the wakeup call is detected by analysing the speech containing the phrase “Hi Galaxy”, and then sends the control signal that corresponds to “Call Hong Gildong”. to the electronic device 100.”

“Next, the server completes this process and may be configured for listening to any subsequent voice signal transmissions received from device 100, as in step 501. It is possible to design suitable signaling between server 190 and device 100 to notify server 190 when device 100 goes into an idle or lock screen mode. The server will treat any voice message that was received after the initial one is received as one that might contain the wakeup command. The server will listen for a new voice order if it is not.

“FIG. “FIG. The electronic device performs both the wakeup command detection as well as the voice recognition for voice commands in this embodiment.

“Steps 601, 602, 604, 606 and 608 could be identical to steps 201, 202-4, 206, 208 and 208 of FIG. 2. This description of steps 601-608 reaffirms some concepts discussed in connection to steps 201-208.

“At step 601, the electronic device 100 is in the idle mode, or locked screen. The voice signal includes the wakeup command. It is received by the microphone 110. The system is activated by the wakeup command. The wakeup command may instruct the system to change to the voice command mode or unlock the screen. The electronic device 100 provides many functions that can be executed by the voice command. The voice command can be used to dial, photograph, play MP3s, and other functions.

“In step 602, device 100 analyzes a voice signal using a speech recognition algorithm. (voice recognition algorithm) To determine if the voice signal contains the wakeup commands. If speaker-dependent recognition is used, the operation may merely detect that the voice matches a particular voice or that the speech matches a certain wakeup command. A speaker-independent recognition scheme could also be used. If the wakeup command has been recognized in step 604, it will be passed to step 606 and if not, it will return to step 600.

“In step 606, device 100 detects how long the silence lasts between the voice signal portion that corresponds to the wakeup call and the voice signal portions that correspond to the voice commands. If the wakeup command is “Hi Galaxy”, then this would be an example. The voice command is called?Call Hong Gildong?. When the user pronounces?Hi Galaxy consecutively, it will be called?Hi Galaxy? The silence duration is between?Hi Galaxy and?Call Hong Gildong? ?Call Hong Gildong?

“In step 608, an electronic device 100 determines if the voice signal corresponding with the voice command starts. The electronic device 100, for example, checks the point at which the voice signal corresponds to?Call Hong Gildong? ”

“When the voice signals corresponding with the voice commands begin in step 608, an electronic device 100 analyzes them using the voice recognition algorithm of step 610.”

“In step 612 the electronic device 100 performs a corresponding operation based upon the recognized voice command. If the voice command is “Call Hong Gildong?”, for example, the electronic device 100 will search the phonebook to find the number of Hong Gildong and attempt to connect the call to the number. The process is over.

“FIG. “FIG.7” shows an example voice signal that includes a wakeup call and a voice order. This can be used to analyze the embodiments above. Input to device 100 might include a wakeup and voice commands in succession. The voice signal input to device 100 may contain a 700-word wakeup command and a 720-word voice command. These are sequentially input to the electronic devices. Between the 700-word wakeup and the 720-word voice command portions, there is a silence duration of 720.

“FIGS. 8A,8B, and 8C show screenshots of a dialing operation that uses the voice signal and includes the wakeup command according to an example embodiment of the invention. FIG. FIG. 8A shows that an icon object 800 is activated to recognize the voice command according to the wakeup command’s voice signal portion 700. The voice command (Call Hong Gil-dong? The voice command (?Call Hong Gil-dong?) corresponds to the voice command section 720 of the voice message. FIG. 8B and the operation proceeds according to the voice command. The phone number Hong Gil-dong can be searched in the phonebook, and the call connection automatically begins with the search phone number, as shown in FIG. 8C.”

“FIGS. 9A and 9B show screenshots of screens that are unlocked by speech recognition and control operations in accordance with an exemplary embodiment the present invention. FIG. FIG. 9A shows an example lock screen. FIG. 9A shows an example lock screen; FIG. The process of unlocking the lockedscreen to create the unlocked screen is illustrated in FIGS. 9A and 9B are possible using any of the methods described in FIGS. 2, 4, and 6 (e.g. steps 214, 406,410, or 612).

“In response to detection that the wakeup signal portion 700 matches a predetermined wakeup order or matches a particular user?s voice, the lockedscreen of FIG. 9A switches to the unlocked FIG. screen 9B. 9B. After the screen has been unlocked, follow the voice signal 700 that corresponds to the wakeup call.

“In the exemplary embodiments described above, the wakeup and voice commands are separate. Alternately, both the speaker verification as well as the voice command can be performed using the same voice signal that corresponds to the voice command. The voice signal that corresponds to the voice command is used to verify the speaker. If the speaker verification passes, the function of the electronic device that corresponds to the voice command can be operated or controlled.

“The methods described in the present disclosure may be implemented as hardware or software, or both.”

Software can be stored on a computer-readable storage media that contains one or more programs (software module). A number of programs can be stored on the computer-readable storage media and are designed to execute one or more processors on the electronic device or server. Instructions that instruct the electronic device or server to execute the methods described in the claims, specifications and/or disclosure of this disclosure are included in one or more programs.

These programs (software module and software) can be stored in a random access memory or a non-volatile storage memory that includes a flash memory. Alternately, programs can be stored in a memory that combines all or part of the recording media. You may have multiple memories.

The programs can be saved to an attachment storage device attached to the electronic device, and/or the server accessible over a communication network, such as Internet, Intranet or Local Area Network (LAN), Wide LAN/WLAN or Storage Area Networks (SAN), or a combination of networks. The external port allows the storage device to access the electronic device or the server.

Summary for “Methods and apparatus for performing voice command in an electronic device”

Electronic devices that can perform multiple functions have become more common in recent years, thanks to the advancements in multimedia technology. Multi-function devices can include mobile terminals like smart phones, tablet PCs and smart cameras as well as fixed home-based electronic devices such as electronic devices that are integrated with kitchen appliances. Most electronic devices include a convergence function that combines several functions.

“Portable terminal designers aim to achieve advanced performance as also the convergence function as well as slimming and aesthetic designs of all devices.” Terminal manufacturers must compete to offer substantially the same or better performance, and to create new models that are smaller than previous designs.

A recently released device offers voice recognition with high accuracy. This voice recognition function recognizes the voice of the user and allows them to quickly execute the corresponding function without touching a button, key, or touchscreen.

“For example, voice recognition allows users to call or send a text message using a mobile terminal like a smart phone. They can also set up various functions, such as an alarm, route planner, Internet search, or alarm.

“To perform the voice recognition function the related art drives the corresponding voice recognition app, activates it, and then performs that function.”

The voice recognition application must be started when a touch input command is received on the touchscreen or a separate key. This is against the voice recognition’s unique function to facilitate data input without touching. In some cases, it may prove difficult or time-consuming to launch the voice recognition app because you must locate it on a screen that includes various application objects.

“Embodiments and methods for executing an electronic voice command are disclosed. An exemplary embodiment detects a voice signal and recognizes the speech. A voice command mode is activated when the voice signal detected contains a wakeup call. The signal that contains at least part of the voice signal is sent to a server. The server generates either a control signal, or a result signal that corresponds to the voice command and sends it back to the device. The control signal or result signal is received by the device and processed. Voice commands can be used to control the device without the user having to touch it.

“In various embodiments:”

“The voice signal could include the wakeup command, followed by the voice instruction.”

“The wakeup command could also include the voice command.”

“A silence duration can be established between the wakeup and voice commands.”

“Processing either the control signal, or the result signal could include executing a specific application of the electronic devices.”

“Processing the control signal or result signal may include displaying data that correspond to the result signal.”

“Once the voice commands mode is activated, an object can be activated on a display that indicates the activation of the voice command modes.”

“A screen that is locked before it recognizes the wakeup command in speech may not be responsive to the recognized wakeup call.

“The speech may only be recognized if the speaker of the voice signal is identified as containing a predetermined wakeup call. If the speaker of the predetermined voice signal is recognized, the wakeup command can be automatically detected.

“Alternatively, the wakeup signal may be detected when the speaker’s voice is recognized. Another embodiment of a method of executing a voice order in an electronic device includes: receiving a result signal from the server that indicates the server has detected the wakeup command in the vocal signal; awakening the device by processing the control signal, or the result signal, corresponding with the voice command.

“In one embodiment, a method for supporting an electronic voice command in a server includes: receiving a transmitted voice signal that contains at least one voice command from the electronic devices; generating a control or result signal corresponding with the voice command by recognizing the voice command and analysing it; and then sending the control signal, or the result signal, to the electronic devices.

“In one embodiment, an electronic device includes: one or multiple processors; memory; and one, or more, programs stored in the memory that can be executed by the processors. The program contains instructions for detecting voice signals and recognizing speech; activating voice command modes when speech is detected to contain a wakeup signal; transmitting a transmit message containing at most a portion the detected voice signal to server; and processing and receiving a control signal, or a result signal, generated by the server in response a voicecommand

“Other advantages, salient features, and other aspects of the invention will be apparent to those skilled with the art from this detailed description. This is taken in conjunction the annexed illustrations, which discloses exemplary embodiments.

The following description, with reference to the accompanying illustrations, is intended to provide a complete understanding of the exemplary embodiments as described in the claims and the equivalents. These specific details are intended to aid in understanding the invention, but they should not be considered as exhaustive. Those of ordinary skill in art will be able to recognize that modifications and variations can be made to the embodiments without departing from their scope and spirit. For clarity and simplicity, it is possible to omit descriptions of well-known constructions and functions.

The inventor uses the terms and phrases in the claims and description to help with a consistent and clear understanding of the invention. It should be obvious to all those skilled in the arts that the following description and claims of exemplary embodiments are provided only for illustration purposes and does not limit the invention as defined by the appended Claims and their equivalents.

“It is important to understand that the singular forms?”a? ?an,? ?an,? If the context requires otherwise, plural referents should be used. For example, a reference to “a component surface?” This includes a reference to any one or more of these surfaces.”

“Substantially” is a synonym for “substantially”. It is intended that the recited characteristic or parameter or value does not have to be exactly achieved. However, deviations or variations (including tolerances, measurement error and measurement accuracy limitations) may not prevent the characteristic from having the desired effect.

“Exemplary embodiments” of the invention include an apparatus and a method to execute a voice command in an electronics device and server.

“FIG. “FIG. Any one of many fixed or portable electronic devices 100 is possible. A portable device could be a mobile terminal, portable pad, mobile terminal, mobile tablet, media player or smart phone. It can also include a personal digital assistant (PDA), smart camera, notebook computer and a desktop computer. An electronic device can be portable and combine two or more functions from other devices. An electronic display device that is attached to a home appliance, such as a kitchen oven, would be an example of a fixed device.

“Electronic device 100” can include a controller 110 and a speaker/microphone 112. A camera 120, a Global Positioning System receiver 130, an Radio Frequency unit 140, a radio frequency (RF) unit 140, and a sensor module 150. It also includes a touchscreen 160, touch screen controller 165, as well as an external memory 170.

“Briefly: according to embodiments, device 100 detects voice signals and recognizes speech in them. Device 100 activates the voice command mode when it is confirmed that the speech contains a wakeup call. Device 100 can respond to voice commands in the voice command mode. The device 100 can respond to voice commands even if it is locked or in an idle state. A speech recognition process is used to detect if the wakeup command was uttered. The device 100 will then be able to perform the operation associated with a voice command if it is recognized.

“In one embodiment, the device 100 performs both the wakeup command detection as well as the voice command recognition. Another embodiment performs the wakeup command detection at the device 100 while the voice command recognition takes place at the server. This is after a portion is transmitted from the device 100 to it. Another embodiment performs both the wakeup command detection as well as the voice command detection at the server.

“The controller 100 may include an interface 101, one, or more processors, 102, 103, and an inner memory 104. The processor may also be called the controller 110 in some cases. The interface 101, application processor 102 and communication processor 103 can all be separated components. Or, they can all be integrated into one or more integrated circuits.

“The application processor102 executes several functions for the electronic device. It runs various software programs, and the communication processor103 processes and controls data communication and voice communication. The processors 102, 103 execute the typical functions and also execute the software module (instructions set) stored in the internal memory 104 or the external memory 170 and perform specific functions that correspond to the module. The processors 102, 103 execute the method according to the invention with the software modules stored in the internal memory 104 or the external memory 170.

“Accordingly to an exemplary embodiment (corresponding to FIG. 2), the application processor102 receives a voice command including a wakeup call and a subsequent voice order from a user via the microphone 110 and performs speech recognition on that voice signal to detect the existence of the wakeup commands. The application processor 102 might detect a silence between the wakeup call and the next voice command in the voice message when the wakeup command has been detected. The application processor 102 then determines whether the voice signal that corresponds to the voice command starts. If it does, the application process 102 sends the appropriate portion of the audio signal to a server. The application processor 102 then receives the server’s voice recognition result and performs the corresponding operation.

“Accordingly to an alternative exemplary embodiment (corresponding to FIG. 4. The application processor 102 transmits all of the voice signal to the server, including the wakeup commands portion. It then receives a speaker verification result from the server corresponding to the wakeup order. The application processor 102 activates it when the result indicates that the wakeup commands have been detected. The application processor 102 then receives a voice recognition result that corresponds to the voice command. It performs the operation based upon the voice recognition result.

“Accordingly to another example embodiment of the invention (corresponding with the method in FIG. 6. The application processor 102 receives the voice message from the user via the microphone 110 and performs the wakeup commands detection using speech recognition. The application processor 102 detects the wakeup command and uses a voice recognition algorithm. It then performs the appropriate operation based upon the voice command.

One or more of the voice recognition processors or a speaker verification process can be included in the application processor 102 or provided separately. The speaker verification processor and the voice recognition processor can be combined and may include multiple processors that perform different functions depending on their implementation. Interface 101 connects the touch screen controller (165) of the electronic device to the external or internal memory 170, 104.

“The interface 101 is coupled with the sensor module 150 to perform various functions. To detect motion or detect light from outside, an optical sensor or motion sensor can be connected to the interface 101. Other sensors, such as a temperature sensor or position determining system can also be connected to the interface 101 in order to perform relevant functions.

“The camera 120 can be coupled to the sensor module 150 via the interface 101 in order to perform a camera function like photo and video clip record.”

The RF unit 140 is a communication device that may contain at least one processor. The RF unit 140, for example, converts an RF signals to a baseband signal, and then provides that signal to the communication process 103. Or converts a baseband output from the communication process 103 into an RF signal, and transmits it through an antenna ANT. The baseband signal is processed by the communication processor 103 according to different communication schemes. The communication scheme could include, among others, a Global System for Mobile Communication (GSM), an Enhanced Data GSM Environment(EDGE) communication system, a Code Division Multiple Access communication strategy, a W?CDMA communication schema, a Long Term Evolution communication program, an Orthogonal Frequency Division Multiple Access communication plan, a Wireless Fidelity communication protocol, a WiMax communication method, and/or a Bluetooth communications scheme.

The speaker/microphone 110 is capable of generating and converting audio signals such as one for voice recognition (used in a training process to teach device 100 to recognize a specific speaker/wakeup command/voice command), voice reproduction and digital recording, and a telephone function. The speaker/microphone 110 converts the audio signal to an electrical signal or the electric signal into the voice signal. An external port can be used to connect an attachable, detachable headphone, earphone or headset (not illustrated) to the electronic device.

The touch screen controller 160 can be coupled with the touch screen controller 165. The touch screen 160 can be coupled to the touch screen controller 160.

“The touch screen 160 acts as an input/output interface between electronic devices and users. The touch screen 160 transmits the user’s touch inputs to electronic device 100. The user can also view the output of device 100 on the touch screen 160. The touch screen 160 also presents a visual output for the user. The visual output can be in the form of text, graphics, video or a combination thereof.

“The touch screen 160 can use a variety of displays. Examples include Liquid Crystal Displays (LCD), Light Emitting Diode(LED), Light emitting polymer Display (LPD), OrganicLED (OLED), Active Matrix (AMOLED), Flexible LED (FLED), and Light Emitting Polymer Displays (LED).

“The GPS receiver 130 converts an artificial signal from an?artificial? satellite. The GPS receiver 130 converts a signal received from an?artificial? satellite to information such as speed, location, and time. A distance between the satellite 130 and the GPS receiver 130 can be calculated by multiplying speed of light with signal arrival time. This calculates the location of electronic device by using triangulation, which determines the exact positions and distances from three satellites.

“The internal memory 170 and the external memory 104 can contain fast random access memory, such as one or two magnetic disk storage devices and/or nonvolatile memory and/or one or more optical storage device and/or flash memory (e.g. NAND and/or NOR).

Software is stored in the external memory 170 and internal memory 104. Software components can include an operating system, communication, graphic, user interface, MPEG, camera, and one or several application software modules. The module, being the software component, can also be referred to by the term “instruction set”. A program may also be used to refer to the module.

“The operating system software contains various software components that control general system operations. Control of general system operations can include memory management and control as well as storage hardware (devices) control and management. Power control and management are also included. “The operating system software can process normal communication between different hardware devices and software components (modules).

“The communication module allows communication between other electronic devices, such as computer, server and/or portable terminals, via the RF unit 140. The protocol architecture for the communication software module is used to configure it.

“The graphic software module contains various software components that allow you to display graphics on touch screen 160. Graphics is a broad term. The term “graphics” can be used to describe text, image, website, icon, digital image or animation.

“The user interface software modules include various software components that relate to a user interface. The status of the user interface status changes and the condition of the status of the user interface status changes are both controlled by the user interface software module.

The camera software module contains camera-related software components that allow camera-related processes and functions. An application module contains a browser, an email, an instant message and keyboard emulation. It also includes a widget, Digital Right Management, voice recognition, voice reproduction, voice reproduction, position determining functions, location-based services, and other software components. Additional modules (instructions), can be added to the memory 170 and 104. You can also choose to not use some of the instructions (if necessary).

“Herein, an instruction is provided for performing a speaker recognition function, a speech recognition function, and a voice command execution operation function. These instructions correspond to the ones for performing the operations shown in FIGS. 2, 4, and 6”.

“The different functions of electronic device 100, as described above and to be explained can be executed in hardware or software and/or their combination including one/more signal processing (ASICs) and/or Application Specific Integrated Circuits.

“FIG. “FIG. The system 195 comprises the portable terminal 100 that communicates with a server (190) through a network 180. Server 190 could be, for example, a home network server or a remote server that is accessed via a large network like the Internet. Server 190, on the other hand, can be used by a third-party portable electronic device that performs speech/voice/speaker recognition or analysis of voice signals sent thereto. Server 190 has at least one processor (192) and a memory 194 for a variety of operations. We will show you how to use the server 190 with electronic device 100.

“FIG. “FIG.

“At step 101, the electronic device 100 detects the voice signal that may include the wakeup call and the voice command of the user through microphone 110. The system activates a voice-command mode when the wakeup command is received. In this mode, no touch contact with the touchscreen or key is necessary to receive and analyze the voice command. The device 100 can be set to idle or lockscreen mode before it receives the voice signal. In certain embodiments, the device 100 may be in an application execution mode prior to receiving the voice command. This means that the device 100 is not listening to any voice commands and no operations are performed in response to them.

“In the following description it will be assumed the wakeup command is usually independent of a voice instruction that temporally follows it. Some?speaker-dependent’? embodiments may be different. As discussed below, any speech signal that is detected as being spoken by a speaker can be used to wake up the system. Other embodiments, speaker dependent or speaker independent, include an inherent voice command in the wakeup command. The wakeup command activates the voice command mode, and acts as a catalyst for device 100 to complete additional tasks, such running an application that the user has predefined.

“For instance, the wakeup command could instruct the screen to change to a mode that allows voice commands (?voice command mode) to be entered. The screen can be locked and/or unlocked. The electronic device 100 provides various functions that can be executed by voice commands. Voice commands can be used to dial, photograph, play MP3s, and other functions. The voice command can ask the server 190 to plan a route and search for a map in a variety of implementations.

“In step 200, the electronic device 100 performs speech detection on the voice signal in order to determine if the signal contains a predetermined wakeup call. The speech recognition can be speaker dependent or speaker independent. Other schemes can be used when a variety of wakeup commands have been predetermined. One or more commands is a speaker dependent wakeup command, while one or several other commands is an independent speaker command.

“A speaker dependent recognition scheme requires that a speaker or user train a recognizer using his/her voice in advance. The speech recognizer will only recognize the spoken words of the trained voice. A speaker independent recognition scheme is capable of recognizing speech from any speaker voice. Speaker independent recognition schemes extracts and stores information about hundreds of thousands of voices and allows any user to use it without the need for additional training.

“Using speaker dependent recognition, in certain embodiments, the speaker may be verified using the voice commands portion of the voice signals (which could include the whole voice signal). There is no need for a separate wakeup call. If the speaker can be verified by using the unique voice characteristics of the user to verify their identity, the separate wakeup command is not necessary. These embodiments allow the voice command to also function as the wakeup command. In steps 202 and204, the voice of the user is identified and the wakeup command is detected automatically via this recognition.

“Alternatively, the speaker dependent recognition scheme uses information from a voice signal that corresponds to a predefined word or words. This can be set up by the user through voice training. You can train the device 100 by entering your voice to match a predefined text. To do this, the user must input the wakeup call. The predefined text can either be inputted directly by the user, or converted using multiple voice-inputs. The electronic device 100 and the server 190 convert voice into text.

“When the wakeup command has been detected in step 204, the method proceeds to step 206. It returns to step 201 otherwise.

“Although FIG. 2. When the wakeup command has been successfully detected, an object for recognizing the voice command will be activated. (see FIG. 8A). This object may be a virtual mic and indicates activation for the voice command mode. The object can be displayed first, or in an emphasised manner if it was visible in the locked screen. An interactive Graphical User Interface (GUI), relating to speech/voice recognition, can also be displayed during this time. Alternativly, the GUI for voice recognition can be displayed immediately in the unlocked screen after the wakeup command has been detected.

“In an embodiment variant, the object for recognizing the wakeup command in idle mode and the screen is locked are shown together. The GUI relating to voice recognition and the object for recognizing it are both displayed together. If the screen isn’t locked, the object to recognize the voice command can be displayed with the GUI relating the voice recognition.

“In step 206, the electronic device 100 detects the silence duration (if any) between the first portion of the detected vocal signal (hereafter, ‘first voice signal?). The wakeup command and the second portion of the detected vocal signal (hereafter, the?second voice signals?) The voice command. This assumes that the voice commands are distinct from the wakeup commands (as previously mentioned, embodiments where the wakeup command and voice command can be combined is possible). As an example, suppose that the wakeup command for?Hi Galaxy is?Call Hong Gil-dong? The voice command is called?Call Hong Gildong?. When the user pronounces?Hi Galaxy consecutively, it will be called?Call Hong Gildong? ?Call Hong Gildong? will create a silence between?Hi Galaxy and?Call Hong Gildong? ?Call Hong Gildong?

A short pause can be used to determine the beginning of the voice command by detecting the first two words in the detected speech. An embodiment allows for the blocking of an extraneous part of the detected voice signal that follows the wakeup command from being sent to server along with the following voice command. A Voice Activity Detection technique (VAD), can be used to do this. A voice signal, for example, has more energy than background noise signals that include the?silence? period. If background noise is low, it’s possible to identify unique characteristics of the human vocal chords. The most common way to identify the unique characteristics in the human voice is to observe energy distribution at different frequencies. The characteristics of the human voice include noiseless but not noisy. The VAD technique is able to distinguish speech from background noise and silence. In one embodiment, instead transmitting an audio signal to the server including all sounds detected after the wakeup commands, the device 100 waits for speech to be detected and then transmits only the sound signals that begin with the wakeup commands. The 200 method avoids transmitting noises that are only silent after a wakeup command detection.

“In step 208, device 100 determines if the second voice signal corresponding with the voice command starts. Device 100, for example, checks the start position of the voice signal that corresponds to?Call Hong Gildong?. Device 100 sends the voice signal to the server at step 210 when the second voice signal starts. The server. The transmit signal is the portion of the voice signal that is transmitted to the server. The voice signal that corresponds to the voice command must not start at 208. Otherwise, the flow will return to 206. Device 100 can transmit the voice command to the server and is thus freed from the CPU intensive task of recognising the speech.

“In step 212, device 100 receives the voice recognition result that corresponds to the voice command received from the server. The server may analyze the voice command “Call Hong Gildong?” and send a control signal that corresponds to “Call Hong Gildong?” Sends the search result of the route planning or map search request to device 100.”

“In step214, device 100 performs a corresponding operation based upon the voice recognition result corresponding with the voice command or displays a result that corresponds to the voice recognition. Example: When receiving the control signal that corresponds to “Call Hong Gildong?” Device 100 searches the phonebook to find a number in Hong Gil-dong. It then attempts to connect the call to the number. Device 100 displays the map/route request search result. The process is over.

“FIG. “FIG. This method can be used in conjunction with the operation of the previously-described method 200, which is carried out in device 100. This embodiment sends the server the transmit signal. From the electronic device (e.g. transmitted at FIG. 2) in step 301

“Next, the server analyzes voice signals corresponding to voice commands using a voice recognition algorithm (equivalently?speech recognition?) algorithm) in step 302. The server then analyzes the voice signal to detect a voice command. In step 304, the server determines if the result corresponding with the voice recognition is a command signal. The server then sends the control signal that corresponds to the voice recognition to 100 in step 306 if so. After recognizing “Call Hong Gildong?”, the server sends the control signal to device 100, instructing device 100 to dial Hong Gildong at the associated number from its phone book storage.

“When the voice recognition result is not the control signal, then the server sends the result that corresponds to the voice recognition to step 308. The server can also send image content that contains the route planning request and map search request search results to the electronic device in step 308.

“Accordingly in the exemplary embodiments 200 and 300, both the electronic device and the server perform the wakeup command detection. Another embodiment of the present invention demonstrates that the server performs both the wakeup command detection as well as the voice recognition of the voice commands.

“FIG. “FIG. Device 100 is in an idle or locked screen mode. It receives a voice signal that includes the wakeup command. This voice signal is followed by the voice command of the user via the microphone 110 in step 411. The system activates when the wakeup command is recognized. The wakeup command, for example, can be used to instruct the system to change to the mode to input the voice command or to unlock the screen. The voice command can be used to control various functions of the electronic device 100. The voice command can be used to dial, photograph, play MP3s, and other functions.

“In step 402, device 100 transmits the entire voice signal, including the wakeup and voice commands, to the server as the transmit message. Next, the server receives a voice verification result that corresponds to the wakeup call (step 404). The server will send the recognition result to device 100 if it detects that the transmit message contains the wakeup command. Device 100 activates the system by receiving the recognition result at step 406 if device 100 has received the recognition result at the step 404. System activation unlocks the screen and switches between the active mode or voice command mode. Device 100 can detect voice commands in new voice signals by activating the system.

“Next, device 100 will receive the voice recognition result corresponding with the voice command in steps 408, and performs or displays the operation based upon the voice recognition result in step 410. Example: When receiving the control signal that corresponds to?Call Hong Gildong? Device 100 connects the call to the number searched by the server and searches the phonebook for Hong Gil-dong’s phone number. Device 100 displays the route/route search results or the route planning request in the map/route example. The process is over. Device 100 may then receive new voice signals at step 401, and forward them to the server for processing. The server may respond to subsequent voice commands by sending control signals or results. The steps 401 to 410 can be repeated only for voice commands. However, the server may continue processing the wakeup command because the device 100 has been awakened.

“FIG. “FIG. This method can be used in conjunction with the previously described method 400, which is found in device 100.

“At step 502, the server receives the audio signal, which includes the wakeup and voice commands from the electronic device 501 (i.e. the transmit signal received at step 402). The server uses the voice recognition algorithm to analyze the voice signal that corresponds to the wakeup call in step 502. The server analyses the voice signal that corresponds to the wakeup call and determines if it is detected. This operation could be identical to that described in FIG. 2. performed by device 100 according to that embodiment. You may perform a speaker-dependent or speaker-independent recognition operation, such as the FIG. 2. If multiple predetermined wakeup commands need to be considered, both types of recognition systems may be used.

“In step 504, a server gives a speech verification result for the electronic device. (Note: Step 504 could be skipped in other implementations.

“When the wakeup call is detected in step 506 by speech recognition processing, the server analyzes the voice signal corresponding the voice command using voice recognition algorithm step 508. The server recognizes the speech that corresponds to the voice command and generates a response message corresponding to the action to be taken by device 100. The flow will return to step 501 if the wakeup command does not get detected in the voice signal. The server may transmit a signal to device 100 informing it that there has been no wakeup command detected. Device 100 can then continue to transmit new voice signals to step 501. The server may request and only receive the first voice signal that corresponds to the wakeup command in various implementations.

“Though not illustrated, the server is able to detect a silence between the voice signal corresponding with the wakeup command (or the voice signal that corresponds to the voice commands) and distinguish the wakeup and voice commands.”

“In step510, the server notifies both the wakeup command and voice recognition results (the response signal to the electronic device). The server, for example, determines whether the wakeup call is detected by analysing the speech containing the phrase “Hi Galaxy”, and then sends the control signal that corresponds to “Call Hong Gildong”. to the electronic device 100.”

“Next, the server completes this process and may be configured for listening to any subsequent voice signal transmissions received from device 100, as in step 501. It is possible to design suitable signaling between server 190 and device 100 to notify server 190 when device 100 goes into an idle or lock screen mode. The server will treat any voice message that was received after the initial one is received as one that might contain the wakeup command. The server will listen for a new voice order if it is not.

“FIG. “FIG. The electronic device performs both the wakeup command detection as well as the voice recognition for voice commands in this embodiment.

“Steps 601, 602, 604, 606 and 608 could be identical to steps 201, 202-4, 206, 208 and 208 of FIG. 2. This description of steps 601-608 reaffirms some concepts discussed in connection to steps 201-208.

“At step 601, the electronic device 100 is in the idle mode, or locked screen. The voice signal includes the wakeup command. It is received by the microphone 110. The system is activated by the wakeup command. The wakeup command may instruct the system to change to the voice command mode or unlock the screen. The electronic device 100 provides many functions that can be executed by the voice command. The voice command can be used to dial, photograph, play MP3s, and other functions.

“In step 602, device 100 analyzes a voice signal using a speech recognition algorithm. (voice recognition algorithm) To determine if the voice signal contains the wakeup commands. If speaker-dependent recognition is used, the operation may merely detect that the voice matches a particular voice or that the speech matches a certain wakeup command. A speaker-independent recognition scheme could also be used. If the wakeup command has been recognized in step 604, it will be passed to step 606 and if not, it will return to step 600.

“In step 606, device 100 detects how long the silence lasts between the voice signal portion that corresponds to the wakeup call and the voice signal portions that correspond to the voice commands. If the wakeup command is “Hi Galaxy”, then this would be an example. The voice command is called?Call Hong Gildong?. When the user pronounces?Hi Galaxy consecutively, it will be called?Hi Galaxy? The silence duration is between?Hi Galaxy and?Call Hong Gildong? ?Call Hong Gildong?

“In step 608, an electronic device 100 determines if the voice signal corresponding with the voice command starts. The electronic device 100, for example, checks the point at which the voice signal corresponds to?Call Hong Gildong? ”

“When the voice signals corresponding with the voice commands begin in step 608, an electronic device 100 analyzes them using the voice recognition algorithm of step 610.”

“In step 612 the electronic device 100 performs a corresponding operation based upon the recognized voice command. If the voice command is “Call Hong Gildong?”, for example, the electronic device 100 will search the phonebook to find the number of Hong Gildong and attempt to connect the call to the number. The process is over.

“FIG. “FIG.7” shows an example voice signal that includes a wakeup call and a voice order. This can be used to analyze the embodiments above. Input to device 100 might include a wakeup and voice commands in succession. The voice signal input to device 100 may contain a 700-word wakeup command and a 720-word voice command. These are sequentially input to the electronic devices. Between the 700-word wakeup and the 720-word voice command portions, there is a silence duration of 720.

“FIGS. 8A,8B, and 8C show screenshots of a dialing operation that uses the voice signal and includes the wakeup command according to an example embodiment of the invention. FIG. FIG. 8A shows that an icon object 800 is activated to recognize the voice command according to the wakeup command’s voice signal portion 700. The voice command (Call Hong Gil-dong? The voice command (?Call Hong Gil-dong?) corresponds to the voice command section 720 of the voice message. FIG. 8B and the operation proceeds according to the voice command. The phone number Hong Gil-dong can be searched in the phonebook, and the call connection automatically begins with the search phone number, as shown in FIG. 8C.”

“FIGS. 9A and 9B show screenshots of screens that are unlocked by speech recognition and control operations in accordance with an exemplary embodiment the present invention. FIG. FIG. 9A shows an example lock screen. FIG. 9A shows an example lock screen; FIG. The process of unlocking the lockedscreen to create the unlocked screen is illustrated in FIGS. 9A and 9B are possible using any of the methods described in FIGS. 2, 4, and 6 (e.g. steps 214, 406,410, or 612).

“In response to detection that the wakeup signal portion 700 matches a predetermined wakeup order or matches a particular user?s voice, the lockedscreen of FIG. 9A switches to the unlocked FIG. screen 9B. 9B. After the screen has been unlocked, follow the voice signal 700 that corresponds to the wakeup call.

“In the exemplary embodiments described above, the wakeup and voice commands are separate. Alternately, both the speaker verification as well as the voice command can be performed using the same voice signal that corresponds to the voice command. The voice signal that corresponds to the voice command is used to verify the speaker. If the speaker verification passes, the function of the electronic device that corresponds to the voice command can be operated or controlled.

“The methods described in the present disclosure may be implemented as hardware or software, or both.”

Software can be stored on a computer-readable storage media that contains one or more programs (software module). A number of programs can be stored on the computer-readable storage media and are designed to execute one or more processors on the electronic device or server. Instructions that instruct the electronic device or server to execute the methods described in the claims, specifications and/or disclosure of this disclosure are included in one or more programs.

These programs (software module and software) can be stored in a random access memory or a non-volatile storage memory that includes a flash memory. Alternately, programs can be stored in a memory that combines all or part of the recording media. You may have multiple memories.

The programs can be saved to an attachment storage device attached to the electronic device, and/or the server accessible over a communication network, such as Internet, Intranet or Local Area Network (LAN), Wide LAN/WLAN or Storage Area Networks (SAN), or a combination of networks. The external port allows the storage device to access the electronic device or the server.

Click here to view the patent on Google Patents.