Alphabet – Erick Tseng, Nickolas Jay Sears, Google LLC

Abstract for “Search initiation”

“A computer-implemented user input technique involves sensing user manipulation of a mobile phone to present a keyboard and activating it to input into the search area.

Background for “Search initiation”

Internet search continues to grow and is a huge business. Because people are looking for relevant information to answer their questions, search engines do a better job of sorting out the wheat from the chaff.

People are increasingly using mobile computing devices’ search functions. Mobile devices can be particularly helpful in providing local information overlayed on maps. This allows a user to find a nearby restaurant or store easily. It is possible for users to search local on their devices, such as searching for a friend’s name from a long list of contacts.

This document describes the systems and techniques that can be used to interact directly with users of computing devices like mobile phones. The systems can detect when the user opens a slider or flips open the clamshell cell phone to reveal a keyboard. These techniques may respond in particular ways to the opening of a keyboard mechanism. For example, they might provide immediate search functionality to such a user. The keyboard may be exposed when a user first uses it. This could lead to the device assuming that the user intends to use the device to enter search queries. In some implementations, a search input mechanism, such as a searchbox, can be presented by opening a virtual keyboard on the screen of a device or by the user typing in a specific context such as typing while a desktop with icons appears. You can search for web, local, app-specific, library, and other software functionality using the search functionality.

“In some implementations, such systems or techniques may offer one or more benefits. A device that displays a search screen when a user engages a keyboard can help ensure that the user is able to quickly find information about the active application. If the search area is not automatically presented, or any other query entry mechanism is disabled, users may need to navigate to the search area by hand rather than just typing the desired query into the keyboard.

“In one implementation, the computer-implemented user interface method is disclosed. The method involves sensing user manipulation on a mobile device to present an input area. In response to this manipulation, the keyboard activates to allow input into the search area. A user manipulation could include sliding a keyboard under the front of the mobile device and into an open position. The user manipulation may also include rotating the device from a portrait to a landscape orientation.

“In some aspects, the active cursor may be placed in the automatically presented search area after user input.” Some aspects of the method include the determination that an active program is responsive or not, and the presentation of the search area only if it is.

In some cases, the search area may be displayed as a pop-up box above the active application. This can be done by receiving a search query using the search area, providing search results in response to that query to the active program, and changing the display of the active app to reflect those search results. If the active program is not responding to search input in certain aspects, the method may include opening a separate search entry area from the active app, receiving a query with the search area, and providing search result responses to the query in an alternative display to the active application. Search results can be obtained by using a search set type that corresponds to the application type of an active application. You can choose from any group, including local search, map search and web search.

“A computer-implemented user input method is disclosed in another implementation. The system includes a concealable keyboard with two modes. One mode is in which the keyboard cannot be accessed by a user. A second mode allows the keyboard to be accessed by a user. A keyboard monitor is used to determine when the keyboard switches from one mode to another. A search application is programmed to automatically generate a search area when the keyboard monitor detects that the keyboard has changed from one mode to the other. Text can be automatically entered into the search area if the keyboard switches from one mode to another.

“The system can also include, in certain implementations, a mechanical QWERTY slide out keyboard. The keyboard monitor may also include a switch in slide-out mechanism. A program that receives search results from the search area may also be part of the system. After presenting the search query, the system’s search application can automatically place an active cursor in search area.

“In another implementation, the disclosure includes a computer-implemented user interface system. The system includes a concealable keyboard with two modes. One mode is in which the keyboard cannot be accessed by a user. A second mode is in which it can be accessed by a user. There is also a keyboard monitor that generates a signal when the concealable keypad changes from one mode to another. Finally, the system provides a way to present a search area to a user to allow them to enter a search query using their keyboard.

“DESCRIPTION of Drawings”

“FIG. “FIG.

“FIG. 2A is an example of screen shots from a mobile device that displays a search box when activated using an open keyboard.

“FIG. 2B is an example of screen shots taken from a mobile device whose keyboard was opened during an operation.

“FIG. “FIG.

“FIG. “FIG.

“FIG. “FIG.

“FIG. “FIG.

“FIG. FIG. 6. A block diagram showing the internal architecture of FIG. 5.”

“FIG. FIG. 7 is a block diagram that illustrates the components of the operating systems used in FIG. 5.”

“FIG. “FIG. 7.”

“FIG. “FIG.

“Like reference symbols on the different drawings indicate like elements.”

“This document describes the systems and techniques that mobile devices can interact with their users. A mobile device can include a keyboard or touchscreen, a touch screen, a rollerball, a switch or any other input device that allows for tactile feedback. You can attach the input devices to or place them at different locations on your mobile device. You can connect input devices to the outside of your mobile device (e.g. a USB keyboard), attach them permanently to the device (e.g. a button or switch), or hide them within the device (e.g. a slideable keyboard or clamshell keypad). To perform different actions, a user can access any one or more input devices. One example is that a user might reveal a hidden keyboard on a smartphone by manipulating the device in one of these ways: turning, flipping, sliding, twisting, flipping or turning one or more components.

Hide or enclose one or more input devices within a mobile phone can offer the benefit of securely storing hardware to prevent damage. Some implementations allow the user to hide an input device (such as a keyboard) in order to make it easier to only use a small portion of their mobile device. If the user chooses to use voice dialing functionality on their mobile device, it may not be necessary to input a keyboard to complete that portion. It may be more convenient to save the keyboard for use in other applications or tasks.

A keyboard is the best way to accomplish certain tasks on mobile devices. A keyboard can be used to compose email or text messages, for example. Mobile device manufacturers may provide an accessible keyboard on the outside of the phone in some cases. Mobile device manufacturers may also provide hidden keyboards that can be activated by users. To reveal a keyboard on a mobile device, the user could activate a slide mechanism or switch.

“In this case, revealing a keyboard on the device could trigger multiple applications to be displayed to the user. Uncovering the keyboard could trigger an active search area such as a search box to be displayed on the device’s display screen. Search engine capabilities may be provided by the search box or another search entry area, which could include application-specific searching and menu selections. The keyboard unveiling could provide mobile device users with instant access to a Google search page. The keyboard that is unveiled may also be a virtual or physical keyboard. You can also uncover a physical keyboard and make it accessible to users by sliding it from behind another device or using other mechanisms. A virtual keyboard, on the other hand, may be revealed by a variety user inputs, such as tapping a screen or turning a phone’s orientation from portrait to landscape.

The search box may be located within an active application or as the active app. The search box can be placed next to an active application, such as a bar at top of a graphic display. Users can focus on the search box while they wait for input from the keyboard. To search for a specific topic, the user can enter a search query. The search engine box can be used in some cases to search files on a mobile device or other resources or networks.

When keyboard functionality is enabled, a user can immediately focus their search query data on the search box to find web content, contacts, maps, program information and other searchable data. The search box may be context-dependent in some cases. A context-dependent search box takes into account the context of any one or more active applications or the contents of the data entered to the box to determine the best strategy to use (e.g. by selecting the corpus to search). If the user is using a mapping app and slides the phone to activate the keyboard, the search box may be set up to search for addresses, businesses, and other locations. The search results displayed may include a map application if the address is entered into a general search box. If a user enters a lot of text, it may be entered into an application like an e-mail or word processor. This assumes that the user didn’t intend to search for a long entry.

“In some cases, the search boxes may not be contextually dependent and perform a search of the internet, or a large number of items locally on the device (e.g. all user files) when a search query is entered in the search box. If the device’s screen is displaying a desktop, but the user is not logged in to another app, the user can slide their phone to activate the keyboard. A context-independent search box is possible. In certain situations, the search area may be invoked by data entered by the user, such as through a search box. If a user starts typing on a keyboard while in an area that does not support textual entry (e.g., on a desktop or home page of a device) it could be assumed that they intend to have their entered data sent to a search engine. This data can be provided automatically, without the user having to change the display of their device. Or, a search area might be displayed and filled in with the characters that the user enters. To submit any search query, the user can select an appropriate control (e.g., an enter key) to do so.

“FIG. 1. A mobile device 100 displays a search box when it is asked for a keyboard. The device 100 currently runs a mapping app, such as GOOGLEMAPS, which displays a map in a graphical display 102. A user can use the mapping app to search for a specific location by using a rollerball control 104.

“The user may want to enter a new location via the mapping app at some point. To do this, slide the display 102 up to reveal a keyboard. The user might be looking at a map or another content in a first view (107a) and can switch to a second view (107b) by moving the keyboard 106 under the front of the mobile device 100. The first view 107a shows the mobile device 100 hidden while the second view107b shows the 100 open (or partially) mode.

A user can invoke the transition from hidden mode (e.g. view 107a) to open mode for the keyboard. A mobile device user can, for example, trigger a switch (or other mechanism) that can mechanically change the device 100 between the first mode (e.g. view 107a) and the second mode (e.g. view 107b). Mobile device 100 can detect user manipulations (e.g. by a switch near a mechanical linkage that allows sliding of the screen) and present a search box in display 102 along with activating keyboard 106. A map search box 108 might appear near or over the map displayed in display 102 to allow the user to enter an address or other relevant information. The context-dependent map search box 108 can be used to locate maps, addresses, and locations related to the mapping application. The map search box 108 can search a map database or a GPS database or an address database. However, it may not be able to search other databases like email, the internet, a phone directory or any other database.

“The context used to create a context-dependent search field may come from other sources than the applications running on the device. An accelerometer on a device might sense motion and match it to an activity signature. A bobbing motion that repeats may indicate that the user is walking. A small vibratory motion could indicate that the user is riding on a bus, train or driving their car. In the first example, the keyboard may be exposed and the search for local information (e.g. movies, restaurants, etc.) will be performed automatically. In the second example, however, the search could be automatically made to a mapping app. You can also program searches that will be associated with specific contexts. For example, a user may search their personal files while the device is resting (e.g. because they are working at their desk).

“In some cases, applications can be configured to offer a generic and simple search box when the user exposes the keyboard. A generic search box can be used to search for a specific set of files, such as all files on mobile device 100, internet database databases, or any other data store. The generic search box can be used to search the internet using a GOOGLE search tool. FIG. 1 shows how the user can use a generic search box in order to locate a nearby restaurant before mapping the address using the mapping program.

The search box 108 can be displayed over an active program or on a blank desktop. The search box 110 may be displayed in an active cursor 110. It is ready for the user to type a query about an active application or generic search. To submit the search, the user can enter a query and click the search button 112.

“In some cases, the search box may not appear upon activation of keyboard 106. One example is if an application that runs in the background doesn’t generally support a search tool, it may not be displayed. Mobile phone games, calculator apps, and configuration applications are all examples of applications that might not support search functionality. A user can optionally configure automatic provision of a search engine in some cases. A user might want to perform a search every time they open the keyboard. In this case, the user can choose to have a context-dependent or context-independent search box displayed on device 100. You can also set up searches that will be performed for contexts matching all applications. The search results for each application might be displayed as the user enters the relevant app.

“In some cases, the display may be delayed until a specific event occurs or after a keyboard has been opened. A keyboard may not be displayed if it is opened or revealed after another action, such as the launch of a new app. The search interface will appear as soon as the user starts typing on the keyboard. It may also be filled with data that the user has entered as they type.

“In such a delayed scenario and other, the type search results, including data that is resident in a device or specific to a person (such as being stored as part a user account on server),” A user may not see the search interface if they are at their desktop or home screen on a mobile device. In such situations, the search user interface may not display at first because the user might select an icon from a touchscreen to move it around and then use the keyboard. The search interface for the desktop or home page may be displayed as soon as the user begins typing. It will likely contain the information the user has entered from the keyboard.

“In certain implementations, a nonmechanical mechanism can be used for activating or unmasking a keyboard. A user might tap a keyboard icon on a mobile device to provide tactile feedback. When the keyboard is selected, the search box or search results may be displayed near the touchscreen keyboard.

“FIG. 2A is an example of screen shots from a mobile device that displays a search box when activated using an open keyboard. In general, FIG. 2A shows one example of how a search results box might be presented to a user. At two different times, the illustrations 202a and 202b are displayed on the same mobile device 200. For example, the illustrations 202a and 202b can be displayed after opening the mobile device 200.

“The first illustration 202 shows the general GOOGLE Search Box 204, which is located on screen 206 above the desktop of device 200. The general search box (204) may contain any search engine that a user can select or which is automatically configured within the mobile device. If there is no context for displaying a specific search box, the general search box 204 can be displayed. If the device 200 is initially turned off, or in hibernation mode, and then activated by sliding open the screen206 to reveal a keyboard, 208, it may be that the user has not been actively using the 200. The general search box (204) may be displayed to let the user search the internet or their local device (e.g. via desktop search) for information related to the entered keywords. It is not possible to search for application-specific information. Similar to the above, the 200 mobile device may not have any active applications and therefore, no context would allow for the provision of a specific search box. The user is shown the general GOOGLE search screen in this example.

“Although the default search box 204 displays, the user can choose to search for specific content, such as a map search or a local search. A book search, blog search, product search, image search, web search, and other search options. The general search box can be configured to display a specific search site, search strategy, or search genre. Some implementations may also allow for other restrictions to the search process.

“As shown at FIG. 2A: A user activates the keyboard 208 on device 200, and enters the search terms “Ben, Jerry, Ice cream?” The keyboard 208. The 200 automatically received the textual input from the keyboard 208. To submit a search query, the user can select the search button. You may also use other keystrokes and button combinations to submit your search query. The search engine associated to the search box 204 can execute the search in a standard fashion after receiving the query.

Screen 206 displays the search results for 214. The search results 214 contain links that are related to keywords (e.g. history of icecream link, Ben & Jerry’s Homemade ice cream link and other ice-cream related links). Screen 206 displays general search results because the user did not submit the search query through a specific application. Contrary to this, if the search was performed in a mapping app, the search results might have provided directions to the nearest Ben & Jerry’s icecream shop or other similar icecream venue, as well as address information and telephone numbers.

“FIG. 2B contains screen shots of an example mobile device 200, whose keyboard 202 was opened during an operation. Three different illustrations of the same 200-pound mobile device are shown at three different times. The display 206 shows an initial illustration 220 that shows a music app playing Coldplay music. A typical user wouldn’t need a keyboard to play music. Illustration 220 shows the keyboard 208 in concealed mode. Illustration 220 shows the mobile device 200 in portrait mode. The keyboard 208 can be flipped open by the user. The keyboard slide can also cause the display 206’s rotation from a portrait to a landscape orientation. This is illustrated in the transition between illustration 220 and illustration 225. The keyboard slide can also trigger the display of a specific type of search box 226, The search box 226 in this example is a GOOGLE desktop-search box. It covers only a small portion of the display 206. However, you can present other types and sizes (e.g., a search box that restricts search results to music-related information like an analog to the Internet Move Database, imDB).

Illustration 225 shows the search box 226, which is located on the top of the display 206. This caused other content to advance down so that the search bar 226 takes up some of the screen but doesn’t cover it. If the user scrolls down, they can still see the entire display. Some implementations of the search box 226, may be extended to cover display 206, rather than replacing the content. The search box can be made transparent to not completely obscure the content beneath in such cases.

“The music app shown on the device 200 can receive search results when a query is submitted from the search box 226, The application can register with the operating systems on device 200 so that the appropriate search context is provided to a user when they engage a search mechanism within the application.

“In illustration 225, user opened (e.g. triggered) the keyboard immediately after which he began to type the search term ‘coldplay? Use keyboard 208. The active cursor was automatically placed by the mobile device 200 and the user-entered content into the search field 226. The mobile device 200 concluded that the user might want to search because he triggered the keyboard. The device 200 launched a search app and placed the cursor in the appropriate search box 226, accordingly.

“As you can see, the 200 mobile device is running a music app which could determine how a search is conducted if the keyboard is opened. The search box 226 might present results related to music content on the internet or within the music app. The device 200 can initiate the search and submit the query to a search engine that is adapted to find music related content. Some information can be automatically entered into the search box from the app (e.g. the band name Coldplay). The user can choose to either submit the information or augment or replace it (e.g.?coldplay Steve carrell movie ).

“A third illustration, 230, depicts search results within Display 206 after the query for ‘coldplay? The search query for?coldplay? was submitted. The search results were provided within the active application. This changed the display 206 to reflect those search results. The search results for Coldplay include the albums. You can choose any of the albums displayed to purchase or play the content. Some implementations allow the user to select another type of search using search bar 232, to broaden their search to include other areas such as the web and local databases.

“FIG. “FIG. Box 302 is where the user activates a mobile device. The 300 process begins. The user might power on the mobile phone 100 and then slide open the keyboard 106. The keyboard opening can be detected at box 304. Device 100, for example, may have a hardware switch which is activated when the keyboard 106 is opened. To?activate?, the user can also press a key on their keyboard. The keyboard may also be activated by pressing a key on the keyboard. This is because the user might not be using their mobile device actively and there may not be any context or application to help them determine the type of search box they want to see when they engage the keyboard. Other implementations allow the active application to be used for setting parameters for search queries.

“At box 306, you will see a search box over the application or desktop of your mobile device. FIG. 1 shows an example of a device 100. 1. A search box 110 is displayed over a mobile device mapping app. The user can now enter search terms in the search box using the keyboard 106 The search box may appear on a small portion of the mobile device’s desktop, such as at one edge, rather than the whole desktop. The search box may be displayed in a pop-up or sidebar. The unveiled keyboard can be used to enter search terms if the user wants to use the search box.

“At box 308, the 100 mobile device may receive the user’s search terms and submit it to the search engine. The search box may automatically enter the user-entered characters without the need to expose the keyboard. The search query is submitted by the user. 100 can receive box 310 results and indicator result types from the search engine. These result type indicators can help determine which applications will present search results to users. The search engine will determine the best way to present search results to users. You may also find code that causes search results to be displayed, such as markup code for displaying a map showing search results in a browser.

The search engine can display results depending on the results type indicator at box 312. Search results can be displayed in the GUI of music applications if the user has accessed the search box from within a music app. Some implementations allow search queries to be restricted to specific applications that are receiving search results. Search queries in music applications may only be limited to music lyrics, titles, or bands.

“At box 314, the Search Engine can prompt the user 200 to launch a specific application or return to their desktop after presenting search results. If the user is accessing an app before opening the keyboard, the application might be displayed upon entering the search query, or after exiting the search box.

“FIG. “FIG. The process 318 can be used to generate a specific search box or provide a contextually dependent answer to a user. For example, the context could be related to one of several applications currently open on the mobile device 100. The context could be related to applications used to read emails, surf the web, or listen to music on the device 100. A user might wish to search for items that relate to the open applications. The 100 mobile device can identify which application is active (e.g. in use) and provide a search field related to that application.

“The process 318 begins at box 320 where an application is displayed on a mobile device. A mobile device 100 might present a music app that allows users to listen to music and then search for new music.

“At box 322, process 318 senses the keyboard opening. The mobile device 100 can detect a user manipulating the keyboard to trigger the opening of the keyboard. The process 318 will sense the keyboard opening and determine, at box 324, if the active application is search-associated. Programs that respond to search input are called search-associated apps. A mapping application, for example, may be search-associated because it can perform a search for information about an address. A calculator application, on the other hand, may not be search-associated because it only performs calculations and does not provide any searchable information. Another example is a music app that can be searched for artists, songs, playlists, artwork and other related items.

“The process 318 is able to determine if an application has been search-associated in different ways. The mobile device 100 may require that each application register with the operating systems running on the device 100. Registration can establish a flag to determine if an application is search-associated. If so, additional registration information can be used for determining which search screens and other interactions can be presented.

“In some cases, a separate value may also be stored in order to determine the type of search that will be performed. For example, mapping search for a related application. Another example is that the mobile device 100 could query every active application to determine if a search box for that application should be opened when the keyboard is visible. The mobile device 100 might determine that an application is search-associated for multiple types of search, and query the application to find out which resources are being used. The mobile device 100 may present a combination search box that allows the user to search the open library as well as the web. For instance, if the user is playing music with the application, it can display a library screen. The mobile device 100 may present a context-sensitive search box to allow the user to search music in the user’s library archives if they are listening to music.

“In some cases, an application program interface can be used to allow third parties to develop applications and give such information directly to the operating system. An API can include routines, protocols and other tools that allow integration within a specific search environment. One example is that the API could allow software applications to identify themselves to the operating system to ensure search application integration.

If process 318 determines that an active application does not have search-associated capabilities, a generic search screen may be presented at box 326. A separate search box may be displayed outside the active application. You can now enter a search in the generic search screen. The process 218 will then receive your search request at box 312.

“At box 333, the process 318 submits your received request to a generalized search engines. The mobile device 100, for example, submits the search query. A search engine may provide a list of matches web pages, files or documents, and display the results in box 332. Search results may be displayed in an additional display if the active program is not responding to search input.

“If process 318 finds that an active application has been search-associated at box 324, a cursor may be displayed in an application-specific search window at box 334. FIG. FIG. 1 shows a cursor 110 within a mapping searchbox 108. This is because the active mapping program application has the mapping searchbox 108. The cursor 110 is displayed in the search box 108 for receiving queries about the mapping application. The user can now enter a search in the application-specific search window. Box 336 can receive the request.

“At box 338, the process 318 submits the request to a specialized search engines. The mobile device 100, for example, submits the search query at a search engine that is adapted to find content specific to the active application. A mapping application, which provides address lookup functionality, is one example of an application-specific search query function. The search engine can give possible addresses matches to users’ address requests.

“In general, an active application that is responsive to search input can present search boxes and search results or other search-related content. The search results can also be displayed within an active application if they are obtained from a search set type that is similar to the active app type. If the search set type is map search, the corresponding search results can be displayed in the mapping application. Another example is that search results for a local search may be displayed within a text file or a generic search window. Similar to general web searches, book search, blog searches, or produce searches, they can all be displayed in a web browser.

“In some cases, the mobile device 100 might not be an internet-enabled phone. Or, a user may choose to not purchase internet service.” Search queries can be used to retrieve media files, applications and other information that is stored on the mobile device 100, rather than via the internet.

“FIG. 4. This is a schematic diagram for a mobile device system 400 that provides contextual search interactivity to users. A mobile device, such as the device 402, could be used to implement the system 400. Device 402 has various input and out mechanisms, such as touch screen display 404 or roller ball 406. Device 402 could include components that provide search functionality for applications displayed 404. For example, the concealable keyboard 408.

“One such component may be a keyboard switch410. This could trigger a search box for presentation 404 when keyboard 408 is opened. The keyboard 408 can be seen as a slide-out QWERTY keyboard that is connected via linkage 409. You can choose from a wide variety of linkages. The four-bar linkage connects to the keyboard monitor 412 via linkage 409. This mechanism could allow the display to slide higher upwards 410 than with other arrangements. However, the display 404 is not fully extended in this example to enable the implementation of a full keyboard 408.

A keyboard monitor 412 is a module of an operating system that tracks the position of the keyboard 408. It may also announce an event to an event manager. The keyboard monitor 412 can detect whether the keyboard is in a first or second mode. It may also report changes to the state of switch 410. The keyboard monitor 412 reports events to the operating systems, such as keyboard opening or closing, text being entered or search performed.

The input manager 414 interacts with the keyboard 412 to manage inputs. The input manager 414 is responsible for translating commands from device 402 users. The input manager 414 may translate commands provided by a user of device 402. For example, the input manager 414 can determine where commands are being received and for what application they are being displayed on the screen. The input manager 414 can also interpret keyboard openings to indicate that the user wants to search for content after opening the keyboard. These inputs may be reported to the event manager 414 (not shown), who then reports them to appropriate modules and applications.

“The display manager 416 is also part of the mobile device 402, and may be responsible for rendering content to display 404. Display manager can receive graphic-related content and decide how to present it to the user. The mobile device 402 may display mapping content in landscape mode because the keyboard 408 can be opened and used to enter the content while looking at screen 404. Another implementation allows the keyboard to be closed so that the user can position their phone vertically while viewing content on screen 404 in portrait mode. (e.g., FIG. 2B). Another example is that there may be multiple windows for different applications on the device 418. The display manager 416 can determine which window to display and which to hide. It may also decide what to display when there are overlaps between different graphical objects.

“A variety 418 of applications may run on the same microprocessor 402. Applications 418 can take many forms, including email, messaging, mapping, and web browser applications. They also may include extensions to web browsers or applications that run within a browser. A mobile device 402 can send a signal to the keyboard switch 410, which will allow a search app 420 to search for content contextually or not. The keyboard monitor 412 can detect if the keyboard has changed from concealed to open mode and the search application 420 will generate a search box. The search application 420 also has the ability to receive input from users and submit that input to remote search engines to return search results. You can invoke the search application by opening a web browser, and then directing it to a URL such as www.google.com.

The search application 420 can also give the results to other applications running on the device 402. The search application 420, for example, can determine (via the operating system of device 402) which database, network or website should be used in order to complete the search. The search application 420, for example, can determine what search result type is relevant for an active application so that it generates the correct type of search box and submits the query to the appropriate search engine.

The search application 420 is able to determine the type of screen or search box to use within an application, and the way and where results will be presented to the user. If the mobile phone 400 runs a text editor app such as SMS, MMS texting or email tasks, the search software 420 might pop up or provide a search screen that includes an address search, dictionary, instant messenger list, and other options. Another example is that the mobile device 400 may be running a camera app. The search application 420 might provide a search tool for memory, photos, camera configuration menus and other information. Another example is that the search app 420 might be used to search for the next available appointment or calendar entry. Another example is that the search application420 could be used to search for songs, artists, albums and ringtones on the internet or in local or networked directories. Another example is that the search app 420 might be used to search for addresses, driving directions and local maps.

“A wireless interface 422 allows for communication with a wireless network. This may include a data network that also supports voice communications. The wireless interface 422 can operate in the same way as the examples below. It may allow communication between the device 402 and a messaging service like text messaging or voice mail messaging. The wireless interface 422 can also be used to upload and download content and computer code via a wireless network.

“There are many options for persistent storage, including fixed disk drives and solid state memory devices. Here are two examples. Map data 424 may include data representing one or several geographic maps, or GPS data. This data can be displayed on device 402. Map data 424 can contain a small number of maps. For example, a single map may be stored in memory that is shared with many components of the system 400. The device 402 accesses online maps via a web browser. To reduce the number of map downloads, you can load multiple maps covering a large geographic area into the map data store 424. This will allow device 402 to use the map data stored 424 while it is being used.

“Other storage” includes user data 426, which can be stored on the same media with map data 424. User data 426 contains various parameters that describe a user of the device. The user data 426 may contain data such as user names, passwords and preferences. User data 426 may also contain configuration data about each user’s preferred search engines, search types, result presentations, button assignments, and other usability preferences.

The keyboard monitor 410 is able to generate a signal when the keyboard 408 switches from conceal mode to open mode. The input manager 414 will then direct the user’s text to the search box. The search application 420 can be programmed to position the cursor automatically in the search box after it presents the search box. The input manager 414 is able to automatically place any content entered by the keyboard into the search box.

The device 402 might respond to a keyboard being slid into active use by using the components shown and other parts that have been omitted for clarity. The device 402 might respond differently to keyboard opening depending on the active applications in screen 404. The device 402 can also determine how to search for an application and present search results based on that active app.

The mobile device 400 is capable of generating and displaying a search box in many ways. A pop-up search box that displays a search box may be used to display it over an active app. In some cases, the popup box can be used to direct the user to the active application. Other implementations allow the pop-up to be directed to the internet or another external network.

“In another example, a search field can be generated on a full screen (e.g. as shown in FIG. 2A). The search box can be a standard search engine that searches for information based on search queries. You can place the search box centered, sidebarred, bannered or corner rising over an active application. The search box can be found in some applications as a drop-down or menu box that appears upon mouse-over. The search box might be transparent on the screen until the user uses the roller ball 406, as an example.

“In some cases, the user may pre-configure one or more search controls within an application. Similar to the generalized search box, the user can also pre-configure a desktop displayed search box. Standard search engine technology can be combined into other applications so that a standard search box is displayed within the appropriate applications.

“Using the aforementioned mechanisms can allow a user with a mobile device search faster and more seamlessly through web content and application content without having to navigate to search engines or buried search functions. They can quickly check if the application has search functionality, and then perform a quick search to find the desired information.

Referring to FIG. 5 shows the exterior appearance for an exemplary 500 device that implements the described user interface features. The device 500 is a processor that displays notifications about events on the device 500 and allows a user to?pull down’ the notifications. Detail about the events related to the notifications can be viewed in an extended view.

“The hardware environment of the device 500 is described in detail. It includes a display 505 for text, images and video; a keyboard 502 to enter text data and user commands into 500; a pointing device 504. This device can point, select, and adjust objects on the display 501; antenna 505 and network connection 506; camera 507, microphone 509, and speaker 510. The device 500 has an external antenna 505, but the internal antenna is hidden from the user.

The display 501 displays video, graphics and images as well as text. It also contains the user interface for software applications 500 and operating system programs 500. A new mail indicator 511 alerts users to the existence of a message. An active call indicator 512 indicates that a call has been received, placed or is occurring. A data standard indicator 514 indicates the data standard 500 is currently being used to transmit and receive data. A signal strength indicator 515 measures the strength of a signal received via the antenna 505. A battery life indicator 516 indicates how much battery life is left. Or a clock 517 which outputs the current time.

“The display 501 might also display application icons that represent various applications, such as a web browser icon 519, a telephone application icon 520 and a search app icon 521. A contacts application icon 522. A mapping application icon 524. An email application icon 525. Or other application icons. One example implementation is the display 501, which is a quarter-video graphics array (QVGA), thin film transistor (TFT), liquid crystal display (LCD), and capable of 16 bit or better color.

“A user uses the keyboard or?keypad?” 502 is used to enter commands and data in order to control and operate the operating system. It also includes applications that respond to alerts and messages, and can be used to touch-screen. The keyboard 502 contains standard keyboard buttons and keys that correspond with alphanumeric characters. Keys 526 and 527 are examples of keys that correspond with the alphanumeric characters “Q?” ?W? When used alone or in conjunction with the characters??? When used alone, or in combination with the characters?*???? and?1? When pressed together with key 529, it will return?1? A single key can also be associated to special characters or functions depending on the state of the operating systems or the applications invoked by it. If an application requires the input of a numerical character, the selection of key 527 may result in a?1? ”

The keyboard 502 includes the usual function keys associated with an alphanumeric keyboard. These include an establish call key530 that answers a received call or initiates a new call. A terminate call key 531 causes an active call to be terminated. A drop down menu key 532 causes a menu to appear within display 501. A backward navigation key 534 causes a previously accessed address to be accessed again. A favorites key 535 causes an active website to be saved to a bookmarks or folder; a home page 536 that allows an application to navigate to a specified network address.

The pointing device 504 is used by the user to adjust and select graphics and text objects on the display 501. This interaction and control with the device 500 as well as the applications that are invoked on it 500 is called “Pointing device 504”. Any type of pointing device 504 can be used. It may include a joystick or trackball, a touchscreen device that is combined with the display 501, or another input device.

An antenna 505, which may be either an external antenna or internal antenna, can transmit and receive radiofrequency signals. It is used to determine location, establish point-to-point radio communications, and wireless local area networks (LAN) communication. The antenna 505 can facilitate point-to?point radio communication using Specialized Mobile Radio (SMR), Cellular, or Personal Communication Service(PCS) frequency bands. It may also implement data transmission using any number of data standards. The antenna 505 can allow data to be sent between the base station 500 and the device 500 using various technologies, such as Wireless Broadband, Worldwide Interoperability of Microwave ACCess, 5GPP Long Term Evolution, LTE, Ultra Mobile Broadband, High Performance Radio Metropolitan Network, Hiperman, High Speed Packet Access, HSPA, HSPA+, High-Speed Upload Packet Access, HSPA Evolution, HSPA+. W-CDMA, HSUPA and GPRS communications may be possible, for example, with a QUALCOMM MSM7200A chipset and a QUALCOMM MTR6285 transceiver.

The wired or wireless computer network connection 506 could be a modem connection or a local-area connection (LAN), including Ethernet, or a broadband wide area network (WAN) connection such a digital subscriber line or DSL, cable high-speed internet connection or dial-up connection. It can also include a T-1, T-3, fiber optic connection, satellite connection, and dial-up connection. The network connection 506 can connect to a LAN, corporate or government network, the Internet, or another network. The network connection 506 can be wired or wireless. Examples of wireless connectors are, for instance, an INFRARED DATA Association (IrDA), a Wi-Fi connector or an optical wireless connection, an INSTITUTE ELECTRICAL & ELECTRONICS ENGINEERS Standard 802.11 wireless connect, a BLUETOOTH connector (such as a BLUETOOTH version 1.2 connector or 5.0 connector), a near-field communications (NFC), an orthogonal frequency division multiplexing (TM-UWB), a time-modulated ultrawide band (TM-UWB), TM-UWB), a time-modulated wireless connector or another wireless connect. Examples of wired connectors are, for instance, an IEEE-1394 FIREWIRE connector and a Universal Serial Bus connector (including a miniature-B USB interface connector), as well as a serial port connector, parallel port connector or any other wired connector. Another implementation combines the functions of the antenna 505 and the network connection 506 into one component.

The camera 507 can capture digital images. It may also be a scanner or a digital still camera. The camera 507, a 5-megapixel (MP) camera that uses a complementary metal oxide semiconductor (CMOS), is an example of such an implementation.

The microphone 509 allows the device 500 capture sound. It can be an omnidirectional microphone or unidirectional microphone. The microphone 509 can be used to record sound that is generated by users, such as when they are speaking to each other via telephone using the device 500. The speaker 510, on the other hand, allows the device convert an electrical signal to sound. This includes a voice from another user or a ringtone generated from a phone application program. The FIG. 500 illustrates the device 500. FIG. 5 shows the device 500 as a handheld device. However, further implementations of the device 500 could be a laptop or a workstation, a middlerange computer, an embedded system and telephone.

“FIG. 6. This block diagram illustrates the internal architecture 600 for the device 500. A central processing unit (CPU), 601, is responsible for the processing of computer instructions. The architecture includes a central processing unit (CPU) 601 that processes computer instructions. A pointing device interface 605 provides a communication interface to the antenna 505. An antenna connection interface 607 provides a communication link to the network connection 506. A non-volatile memory device 612 or another suitable type of memory stores invariant low-level systems code or data for basic system functions such as startup, input and output (I/O), I/O, or receipt of keystrokes from the keyboard 502; telephony subsystem 612. The CPU 601 and its constituent devices communicate over a bus 621.

“The CPU 601 may be one of many computer processors. The computer CPU 601 can be used in more than one arrangement. RAM 610 connects to the computer bus 621 to provide RAM storage for the CPU 601 in a quick manner during execution of software programs like the operating system applications programs and device drivers. The CPU 601 loads computer executable process steps from other media or storage mediums into the RAM 610 field to execute software programs. The RAM 610 stores data, which is accessed during execution by the CPU 601. One example configuration of the device 500 contains at least 128 MB RAM and 256MB flash memory.

The storage medium 612 may contain a number physical drive units such as a redundant array independent disks (RAID), flash memory, a USB flash memory, an external hard drive drive or Blu-Ray optical disk drive. An external mini-dual inline memory module (DIMM), synchronous dynamic random acces memory (SDRAM), and an external micro-DIMMSDRAM. These computer-readable media enable the device 500 access to computer-executable processes, applications programs, and the like. The device 500 can also offload data from 500 or upload data to 500.

“A computer program product is tangible embodied within storage medium 612, which is machine-readable storage media. Instructions in the computer program product can be read by a machine to cause a mobile device’s data processing apparatus to store image information. In certain embodiments, the computer product contains instructions that generate notifications about new messages arriving on the device.

Summary for “Search initiation”

Internet search continues to grow and is a huge business. Because people are looking for relevant information to answer their questions, search engines do a better job of sorting out the wheat from the chaff.

People are increasingly using mobile computing devices’ search functions. Mobile devices can be particularly helpful in providing local information overlayed on maps. This allows a user to find a nearby restaurant or store easily. It is possible for users to search local on their devices, such as searching for a friend’s name from a long list of contacts.

This document describes the systems and techniques that can be used to interact directly with users of computing devices like mobile phones. The systems can detect when the user opens a slider or flips open the clamshell cell phone to reveal a keyboard. These techniques may respond in particular ways to the opening of a keyboard mechanism. For example, they might provide immediate search functionality to such a user. The keyboard may be exposed when a user first uses it. This could lead to the device assuming that the user intends to use the device to enter search queries. In some implementations, a search input mechanism, such as a searchbox, can be presented by opening a virtual keyboard on the screen of a device or by the user typing in a specific context such as typing while a desktop with icons appears. You can search for web, local, app-specific, library, and other software functionality using the search functionality.

“In some implementations, such systems or techniques may offer one or more benefits. A device that displays a search screen when a user engages a keyboard can help ensure that the user is able to quickly find information about the active application. If the search area is not automatically presented, or any other query entry mechanism is disabled, users may need to navigate to the search area by hand rather than just typing the desired query into the keyboard.

“In one implementation, the computer-implemented user interface method is disclosed. The method involves sensing user manipulation on a mobile device to present an input area. In response to this manipulation, the keyboard activates to allow input into the search area. A user manipulation could include sliding a keyboard under the front of the mobile device and into an open position. The user manipulation may also include rotating the device from a portrait to a landscape orientation.

“In some aspects, the active cursor may be placed in the automatically presented search area after user input.” Some aspects of the method include the determination that an active program is responsive or not, and the presentation of the search area only if it is.

In some cases, the search area may be displayed as a pop-up box above the active application. This can be done by receiving a search query using the search area, providing search results in response to that query to the active program, and changing the display of the active app to reflect those search results. If the active program is not responding to search input in certain aspects, the method may include opening a separate search entry area from the active app, receiving a query with the search area, and providing search result responses to the query in an alternative display to the active application. Search results can be obtained by using a search set type that corresponds to the application type of an active application. You can choose from any group, including local search, map search and web search.

“A computer-implemented user input method is disclosed in another implementation. The system includes a concealable keyboard with two modes. One mode is in which the keyboard cannot be accessed by a user. A second mode allows the keyboard to be accessed by a user. A keyboard monitor is used to determine when the keyboard switches from one mode to another. A search application is programmed to automatically generate a search area when the keyboard monitor detects that the keyboard has changed from one mode to the other. Text can be automatically entered into the search area if the keyboard switches from one mode to another.

“The system can also include, in certain implementations, a mechanical QWERTY slide out keyboard. The keyboard monitor may also include a switch in slide-out mechanism. A program that receives search results from the search area may also be part of the system. After presenting the search query, the system’s search application can automatically place an active cursor in search area.

“In another implementation, the disclosure includes a computer-implemented user interface system. The system includes a concealable keyboard with two modes. One mode is in which the keyboard cannot be accessed by a user. A second mode is in which it can be accessed by a user. There is also a keyboard monitor that generates a signal when the concealable keypad changes from one mode to another. Finally, the system provides a way to present a search area to a user to allow them to enter a search query using their keyboard.

“DESCRIPTION of Drawings”

“FIG. “FIG.

“FIG. 2A is an example of screen shots from a mobile device that displays a search box when activated using an open keyboard.

“FIG. 2B is an example of screen shots taken from a mobile device whose keyboard was opened during an operation.

“FIG. “FIG.

“FIG. “FIG.

“FIG. “FIG.

“FIG. “FIG.

“FIG. FIG. 6. A block diagram showing the internal architecture of FIG. 5.”

“FIG. FIG. 7 is a block diagram that illustrates the components of the operating systems used in FIG. 5.”

“FIG. “FIG. 7.”

“FIG. “FIG.

“Like reference symbols on the different drawings indicate like elements.”

“This document describes the systems and techniques that mobile devices can interact with their users. A mobile device can include a keyboard or touchscreen, a touch screen, a rollerball, a switch or any other input device that allows for tactile feedback. You can attach the input devices to or place them at different locations on your mobile device. You can connect input devices to the outside of your mobile device (e.g. a USB keyboard), attach them permanently to the device (e.g. a button or switch), or hide them within the device (e.g. a slideable keyboard or clamshell keypad). To perform different actions, a user can access any one or more input devices. One example is that a user might reveal a hidden keyboard on a smartphone by manipulating the device in one of these ways: turning, flipping, sliding, twisting, flipping or turning one or more components.

Hide or enclose one or more input devices within a mobile phone can offer the benefit of securely storing hardware to prevent damage. Some implementations allow the user to hide an input device (such as a keyboard) in order to make it easier to only use a small portion of their mobile device. If the user chooses to use voice dialing functionality on their mobile device, it may not be necessary to input a keyboard to complete that portion. It may be more convenient to save the keyboard for use in other applications or tasks.

A keyboard is the best way to accomplish certain tasks on mobile devices. A keyboard can be used to compose email or text messages, for example. Mobile device manufacturers may provide an accessible keyboard on the outside of the phone in some cases. Mobile device manufacturers may also provide hidden keyboards that can be activated by users. To reveal a keyboard on a mobile device, the user could activate a slide mechanism or switch.

“In this case, revealing a keyboard on the device could trigger multiple applications to be displayed to the user. Uncovering the keyboard could trigger an active search area such as a search box to be displayed on the device’s display screen. Search engine capabilities may be provided by the search box or another search entry area, which could include application-specific searching and menu selections. The keyboard unveiling could provide mobile device users with instant access to a Google search page. The keyboard that is unveiled may also be a virtual or physical keyboard. You can also uncover a physical keyboard and make it accessible to users by sliding it from behind another device or using other mechanisms. A virtual keyboard, on the other hand, may be revealed by a variety user inputs, such as tapping a screen or turning a phone’s orientation from portrait to landscape.

The search box may be located within an active application or as the active app. The search box can be placed next to an active application, such as a bar at top of a graphic display. Users can focus on the search box while they wait for input from the keyboard. To search for a specific topic, the user can enter a search query. The search engine box can be used in some cases to search files on a mobile device or other resources or networks.

When keyboard functionality is enabled, a user can immediately focus their search query data on the search box to find web content, contacts, maps, program information and other searchable data. The search box may be context-dependent in some cases. A context-dependent search box takes into account the context of any one or more active applications or the contents of the data entered to the box to determine the best strategy to use (e.g. by selecting the corpus to search). If the user is using a mapping app and slides the phone to activate the keyboard, the search box may be set up to search for addresses, businesses, and other locations. The search results displayed may include a map application if the address is entered into a general search box. If a user enters a lot of text, it may be entered into an application like an e-mail or word processor. This assumes that the user didn’t intend to search for a long entry.

“In some cases, the search boxes may not be contextually dependent and perform a search of the internet, or a large number of items locally on the device (e.g. all user files) when a search query is entered in the search box. If the device’s screen is displaying a desktop, but the user is not logged in to another app, the user can slide their phone to activate the keyboard. A context-independent search box is possible. In certain situations, the search area may be invoked by data entered by the user, such as through a search box. If a user starts typing on a keyboard while in an area that does not support textual entry (e.g., on a desktop or home page of a device) it could be assumed that they intend to have their entered data sent to a search engine. This data can be provided automatically, without the user having to change the display of their device. Or, a search area might be displayed and filled in with the characters that the user enters. To submit any search query, the user can select an appropriate control (e.g., an enter key) to do so.

“FIG. 1. A mobile device 100 displays a search box when it is asked for a keyboard. The device 100 currently runs a mapping app, such as GOOGLEMAPS, which displays a map in a graphical display 102. A user can use the mapping app to search for a specific location by using a rollerball control 104.

“The user may want to enter a new location via the mapping app at some point. To do this, slide the display 102 up to reveal a keyboard. The user might be looking at a map or another content in a first view (107a) and can switch to a second view (107b) by moving the keyboard 106 under the front of the mobile device 100. The first view 107a shows the mobile device 100 hidden while the second view107b shows the 100 open (or partially) mode.

A user can invoke the transition from hidden mode (e.g. view 107a) to open mode for the keyboard. A mobile device user can, for example, trigger a switch (or other mechanism) that can mechanically change the device 100 between the first mode (e.g. view 107a) and the second mode (e.g. view 107b). Mobile device 100 can detect user manipulations (e.g. by a switch near a mechanical linkage that allows sliding of the screen) and present a search box in display 102 along with activating keyboard 106. A map search box 108 might appear near or over the map displayed in display 102 to allow the user to enter an address or other relevant information. The context-dependent map search box 108 can be used to locate maps, addresses, and locations related to the mapping application. The map search box 108 can search a map database or a GPS database or an address database. However, it may not be able to search other databases like email, the internet, a phone directory or any other database.

“The context used to create a context-dependent search field may come from other sources than the applications running on the device. An accelerometer on a device might sense motion and match it to an activity signature. A bobbing motion that repeats may indicate that the user is walking. A small vibratory motion could indicate that the user is riding on a bus, train or driving their car. In the first example, the keyboard may be exposed and the search for local information (e.g. movies, restaurants, etc.) will be performed automatically. In the second example, however, the search could be automatically made to a mapping app. You can also program searches that will be associated with specific contexts. For example, a user may search their personal files while the device is resting (e.g. because they are working at their desk).

“In some cases, applications can be configured to offer a generic and simple search box when the user exposes the keyboard. A generic search box can be used to search for a specific set of files, such as all files on mobile device 100, internet database databases, or any other data store. The generic search box can be used to search the internet using a GOOGLE search tool. FIG. 1 shows how the user can use a generic search box in order to locate a nearby restaurant before mapping the address using the mapping program.

The search box 108 can be displayed over an active program or on a blank desktop. The search box 110 may be displayed in an active cursor 110. It is ready for the user to type a query about an active application or generic search. To submit the search, the user can enter a query and click the search button 112.

“In some cases, the search box may not appear upon activation of keyboard 106. One example is if an application that runs in the background doesn’t generally support a search tool, it may not be displayed. Mobile phone games, calculator apps, and configuration applications are all examples of applications that might not support search functionality. A user can optionally configure automatic provision of a search engine in some cases. A user might want to perform a search every time they open the keyboard. In this case, the user can choose to have a context-dependent or context-independent search box displayed on device 100. You can also set up searches that will be performed for contexts matching all applications. The search results for each application might be displayed as the user enters the relevant app.

“In some cases, the display may be delayed until a specific event occurs or after a keyboard has been opened. A keyboard may not be displayed if it is opened or revealed after another action, such as the launch of a new app. The search interface will appear as soon as the user starts typing on the keyboard. It may also be filled with data that the user has entered as they type.

“In such a delayed scenario and other, the type search results, including data that is resident in a device or specific to a person (such as being stored as part a user account on server),” A user may not see the search interface if they are at their desktop or home screen on a mobile device. In such situations, the search user interface may not display at first because the user might select an icon from a touchscreen to move it around and then use the keyboard. The search interface for the desktop or home page may be displayed as soon as the user begins typing. It will likely contain the information the user has entered from the keyboard.

“In certain implementations, a nonmechanical mechanism can be used for activating or unmasking a keyboard. A user might tap a keyboard icon on a mobile device to provide tactile feedback. When the keyboard is selected, the search box or search results may be displayed near the touchscreen keyboard.

“FIG. 2A is an example of screen shots from a mobile device that displays a search box when activated using an open keyboard. In general, FIG. 2A shows one example of how a search results box might be presented to a user. At two different times, the illustrations 202a and 202b are displayed on the same mobile device 200. For example, the illustrations 202a and 202b can be displayed after opening the mobile device 200.

“The first illustration 202 shows the general GOOGLE Search Box 204, which is located on screen 206 above the desktop of device 200. The general search box (204) may contain any search engine that a user can select or which is automatically configured within the mobile device. If there is no context for displaying a specific search box, the general search box 204 can be displayed. If the device 200 is initially turned off, or in hibernation mode, and then activated by sliding open the screen206 to reveal a keyboard, 208, it may be that the user has not been actively using the 200. The general search box (204) may be displayed to let the user search the internet or their local device (e.g. via desktop search) for information related to the entered keywords. It is not possible to search for application-specific information. Similar to the above, the 200 mobile device may not have any active applications and therefore, no context would allow for the provision of a specific search box. The user is shown the general GOOGLE search screen in this example.

“Although the default search box 204 displays, the user can choose to search for specific content, such as a map search or a local search. A book search, blog search, product search, image search, web search, and other search options. The general search box can be configured to display a specific search site, search strategy, or search genre. Some implementations may also allow for other restrictions to the search process.

“As shown at FIG. 2A: A user activates the keyboard 208 on device 200, and enters the search terms “Ben, Jerry, Ice cream?” The keyboard 208. The 200 automatically received the textual input from the keyboard 208. To submit a search query, the user can select the search button. You may also use other keystrokes and button combinations to submit your search query. The search engine associated to the search box 204 can execute the search in a standard fashion after receiving the query.

Screen 206 displays the search results for 214. The search results 214 contain links that are related to keywords (e.g. history of icecream link, Ben & Jerry’s Homemade ice cream link and other ice-cream related links). Screen 206 displays general search results because the user did not submit the search query through a specific application. Contrary to this, if the search was performed in a mapping app, the search results might have provided directions to the nearest Ben & Jerry’s icecream shop or other similar icecream venue, as well as address information and telephone numbers.

“FIG. 2B contains screen shots of an example mobile device 200, whose keyboard 202 was opened during an operation. Three different illustrations of the same 200-pound mobile device are shown at three different times. The display 206 shows an initial illustration 220 that shows a music app playing Coldplay music. A typical user wouldn’t need a keyboard to play music. Illustration 220 shows the keyboard 208 in concealed mode. Illustration 220 shows the mobile device 200 in portrait mode. The keyboard 208 can be flipped open by the user. The keyboard slide can also cause the display 206’s rotation from a portrait to a landscape orientation. This is illustrated in the transition between illustration 220 and illustration 225. The keyboard slide can also trigger the display of a specific type of search box 226, The search box 226 in this example is a GOOGLE desktop-search box. It covers only a small portion of the display 206. However, you can present other types and sizes (e.g., a search box that restricts search results to music-related information like an analog to the Internet Move Database, imDB).

Illustration 225 shows the search box 226, which is located on the top of the display 206. This caused other content to advance down so that the search bar 226 takes up some of the screen but doesn’t cover it. If the user scrolls down, they can still see the entire display. Some implementations of the search box 226, may be extended to cover display 206, rather than replacing the content. The search box can be made transparent to not completely obscure the content beneath in such cases.

“The music app shown on the device 200 can receive search results when a query is submitted from the search box 226, The application can register with the operating systems on device 200 so that the appropriate search context is provided to a user when they engage a search mechanism within the application.

“In illustration 225, user opened (e.g. triggered) the keyboard immediately after which he began to type the search term ‘coldplay? Use keyboard 208. The active cursor was automatically placed by the mobile device 200 and the user-entered content into the search field 226. The mobile device 200 concluded that the user might want to search because he triggered the keyboard. The device 200 launched a search app and placed the cursor in the appropriate search box 226, accordingly.

“As you can see, the 200 mobile device is running a music app which could determine how a search is conducted if the keyboard is opened. The search box 226 might present results related to music content on the internet or within the music app. The device 200 can initiate the search and submit the query to a search engine that is adapted to find music related content. Some information can be automatically entered into the search box from the app (e.g. the band name Coldplay). The user can choose to either submit the information or augment or replace it (e.g.?coldplay Steve carrell movie ).

“A third illustration, 230, depicts search results within Display 206 after the query for ‘coldplay? The search query for?coldplay? was submitted. The search results were provided within the active application. This changed the display 206 to reflect those search results. The search results for Coldplay include the albums. You can choose any of the albums displayed to purchase or play the content. Some implementations allow the user to select another type of search using search bar 232, to broaden their search to include other areas such as the web and local databases.

“FIG. “FIG. Box 302 is where the user activates a mobile device. The 300 process begins. The user might power on the mobile phone 100 and then slide open the keyboard 106. The keyboard opening can be detected at box 304. Device 100, for example, may have a hardware switch which is activated when the keyboard 106 is opened. To?activate?, the user can also press a key on their keyboard. The keyboard may also be activated by pressing a key on the keyboard. This is because the user might not be using their mobile device actively and there may not be any context or application to help them determine the type of search box they want to see when they engage the keyboard. Other implementations allow the active application to be used for setting parameters for search queries.

“At box 306, you will see a search box over the application or desktop of your mobile device. FIG. 1 shows an example of a device 100. 1. A search box 110 is displayed over a mobile device mapping app. The user can now enter search terms in the search box using the keyboard 106 The search box may appear on a small portion of the mobile device’s desktop, such as at one edge, rather than the whole desktop. The search box may be displayed in a pop-up or sidebar. The unveiled keyboard can be used to enter search terms if the user wants to use the search box.

“At box 308, the 100 mobile device may receive the user’s search terms and submit it to the search engine. The search box may automatically enter the user-entered characters without the need to expose the keyboard. The search query is submitted by the user. 100 can receive box 310 results and indicator result types from the search engine. These result type indicators can help determine which applications will present search results to users. The search engine will determine the best way to present search results to users. You may also find code that causes search results to be displayed, such as markup code for displaying a map showing search results in a browser.

The search engine can display results depending on the results type indicator at box 312. Search results can be displayed in the GUI of music applications if the user has accessed the search box from within a music app. Some implementations allow search queries to be restricted to specific applications that are receiving search results. Search queries in music applications may only be limited to music lyrics, titles, or bands.

“At box 314, the Search Engine can prompt the user 200 to launch a specific application or return to their desktop after presenting search results. If the user is accessing an app before opening the keyboard, the application might be displayed upon entering the search query, or after exiting the search box.

“FIG. “FIG. The process 318 can be used to generate a specific search box or provide a contextually dependent answer to a user. For example, the context could be related to one of several applications currently open on the mobile device 100. The context could be related to applications used to read emails, surf the web, or listen to music on the device 100. A user might wish to search for items that relate to the open applications. The 100 mobile device can identify which application is active (e.g. in use) and provide a search field related to that application.

“The process 318 begins at box 320 where an application is displayed on a mobile device. A mobile device 100 might present a music app that allows users to listen to music and then search for new music.

“At box 322, process 318 senses the keyboard opening. The mobile device 100 can detect a user manipulating the keyboard to trigger the opening of the keyboard. The process 318 will sense the keyboard opening and determine, at box 324, if the active application is search-associated. Programs that respond to search input are called search-associated apps. A mapping application, for example, may be search-associated because it can perform a search for information about an address. A calculator application, on the other hand, may not be search-associated because it only performs calculations and does not provide any searchable information. Another example is a music app that can be searched for artists, songs, playlists, artwork and other related items.

“The process 318 is able to determine if an application has been search-associated in different ways. The mobile device 100 may require that each application register with the operating systems running on the device 100. Registration can establish a flag to determine if an application is search-associated. If so, additional registration information can be used for determining which search screens and other interactions can be presented.

“In some cases, a separate value may also be stored in order to determine the type of search that will be performed. For example, mapping search for a related application. Another example is that the mobile device 100 could query every active application to determine if a search box for that application should be opened when the keyboard is visible. The mobile device 100 might determine that an application is search-associated for multiple types of search, and query the application to find out which resources are being used. The mobile device 100 may present a combination search box that allows the user to search the open library as well as the web. For instance, if the user is playing music with the application, it can display a library screen. The mobile device 100 may present a context-sensitive search box to allow the user to search music in the user’s library archives if they are listening to music.

“In some cases, an application program interface can be used to allow third parties to develop applications and give such information directly to the operating system. An API can include routines, protocols and other tools that allow integration within a specific search environment. One example is that the API could allow software applications to identify themselves to the operating system to ensure search application integration.

If process 318 determines that an active application does not have search-associated capabilities, a generic search screen may be presented at box 326. A separate search box may be displayed outside the active application. You can now enter a search in the generic search screen. The process 218 will then receive your search request at box 312.

“At box 333, the process 318 submits your received request to a generalized search engines. The mobile device 100, for example, submits the search query. A search engine may provide a list of matches web pages, files or documents, and display the results in box 332. Search results may be displayed in an additional display if the active program is not responding to search input.

“If process 318 finds that an active application has been search-associated at box 324, a cursor may be displayed in an application-specific search window at box 334. FIG. FIG. 1 shows a cursor 110 within a mapping searchbox 108. This is because the active mapping program application has the mapping searchbox 108. The cursor 110 is displayed in the search box 108 for receiving queries about the mapping application. The user can now enter a search in the application-specific search window. Box 336 can receive the request.

“At box 338, the process 318 submits the request to a specialized search engines. The mobile device 100, for example, submits the search query at a search engine that is adapted to find content specific to the active application. A mapping application, which provides address lookup functionality, is one example of an application-specific search query function. The search engine can give possible addresses matches to users’ address requests.

“In general, an active application that is responsive to search input can present search boxes and search results or other search-related content. The search results can also be displayed within an active application if they are obtained from a search set type that is similar to the active app type. If the search set type is map search, the corresponding search results can be displayed in the mapping application. Another example is that search results for a local search may be displayed within a text file or a generic search window. Similar to general web searches, book search, blog searches, or produce searches, they can all be displayed in a web browser.

“In some cases, the mobile device 100 might not be an internet-enabled phone. Or, a user may choose to not purchase internet service.” Search queries can be used to retrieve media files, applications and other information that is stored on the mobile device 100, rather than via the internet.

“FIG. 4. This is a schematic diagram for a mobile device system 400 that provides contextual search interactivity to users. A mobile device, such as the device 402, could be used to implement the system 400. Device 402 has various input and out mechanisms, such as touch screen display 404 or roller ball 406. Device 402 could include components that provide search functionality for applications displayed 404. For example, the concealable keyboard 408.

“One such component may be a keyboard switch410. This could trigger a search box for presentation 404 when keyboard 408 is opened. The keyboard 408 can be seen as a slide-out QWERTY keyboard that is connected via linkage 409. You can choose from a wide variety of linkages. The four-bar linkage connects to the keyboard monitor 412 via linkage 409. This mechanism could allow the display to slide higher upwards 410 than with other arrangements. However, the display 404 is not fully extended in this example to enable the implementation of a full keyboard 408.

A keyboard monitor 412 is a module of an operating system that tracks the position of the keyboard 408. It may also announce an event to an event manager. The keyboard monitor 412 can detect whether the keyboard is in a first or second mode. It may also report changes to the state of switch 410. The keyboard monitor 412 reports events to the operating systems, such as keyboard opening or closing, text being entered or search performed.

The input manager 414 interacts with the keyboard 412 to manage inputs. The input manager 414 is responsible for translating commands from device 402 users. The input manager 414 may translate commands provided by a user of device 402. For example, the input manager 414 can determine where commands are being received and for what application they are being displayed on the screen. The input manager 414 can also interpret keyboard openings to indicate that the user wants to search for content after opening the keyboard. These inputs may be reported to the event manager 414 (not shown), who then reports them to appropriate modules and applications.

“The display manager 416 is also part of the mobile device 402, and may be responsible for rendering content to display 404. Display manager can receive graphic-related content and decide how to present it to the user. The mobile device 402 may display mapping content in landscape mode because the keyboard 408 can be opened and used to enter the content while looking at screen 404. Another implementation allows the keyboard to be closed so that the user can position their phone vertically while viewing content on screen 404 in portrait mode. (e.g., FIG. 2B). Another example is that there may be multiple windows for different applications on the device 418. The display manager 416 can determine which window to display and which to hide. It may also decide what to display when there are overlaps between different graphical objects.

“A variety 418 of applications may run on the same microprocessor 402. Applications 418 can take many forms, including email, messaging, mapping, and web browser applications. They also may include extensions to web browsers or applications that run within a browser. A mobile device 402 can send a signal to the keyboard switch 410, which will allow a search app 420 to search for content contextually or not. The keyboard monitor 412 can detect if the keyboard has changed from concealed to open mode and the search application 420 will generate a search box. The search application 420 also has the ability to receive input from users and submit that input to remote search engines to return search results. You can invoke the search application by opening a web browser, and then directing it to a URL such as www.google.com.

The search application 420 can also give the results to other applications running on the device 402. The search application 420, for example, can determine (via the operating system of device 402) which database, network or website should be used in order to complete the search. The search application 420, for example, can determine what search result type is relevant for an active application so that it generates the correct type of search box and submits the query to the appropriate search engine.

The search application 420 is able to determine the type of screen or search box to use within an application, and the way and where results will be presented to the user. If the mobile phone 400 runs a text editor app such as SMS, MMS texting or email tasks, the search software 420 might pop up or provide a search screen that includes an address search, dictionary, instant messenger list, and other options. Another example is that the mobile device 400 may be running a camera app. The search application 420 might provide a search tool for memory, photos, camera configuration menus and other information. Another example is that the search app 420 might be used to search for the next available appointment or calendar entry. Another example is that the search application420 could be used to search for songs, artists, albums and ringtones on the internet or in local or networked directories. Another example is that the search app 420 might be used to search for addresses, driving directions and local maps.

“A wireless interface 422 allows for communication with a wireless network. This may include a data network that also supports voice communications. The wireless interface 422 can operate in the same way as the examples below. It may allow communication between the device 402 and a messaging service like text messaging or voice mail messaging. The wireless interface 422 can also be used to upload and download content and computer code via a wireless network.

“There are many options for persistent storage, including fixed disk drives and solid state memory devices. Here are two examples. Map data 424 may include data representing one or several geographic maps, or GPS data. This data can be displayed on device 402. Map data 424 can contain a small number of maps. For example, a single map may be stored in memory that is shared with many components of the system 400. The device 402 accesses online maps via a web browser. To reduce the number of map downloads, you can load multiple maps covering a large geographic area into the map data store 424. This will allow device 402 to use the map data stored 424 while it is being used.

“Other storage” includes user data 426, which can be stored on the same media with map data 424. User data 426 contains various parameters that describe a user of the device. The user data 426 may contain data such as user names, passwords and preferences. User data 426 may also contain configuration data about each user’s preferred search engines, search types, result presentations, button assignments, and other usability preferences.

The keyboard monitor 410 is able to generate a signal when the keyboard 408 switches from conceal mode to open mode. The input manager 414 will then direct the user’s text to the search box. The search application 420 can be programmed to position the cursor automatically in the search box after it presents the search box. The input manager 414 is able to automatically place any content entered by the keyboard into the search box.

The device 402 might respond to a keyboard being slid into active use by using the components shown and other parts that have been omitted for clarity. The device 402 might respond differently to keyboard opening depending on the active applications in screen 404. The device 402 can also determine how to search for an application and present search results based on that active app.

The mobile device 400 is capable of generating and displaying a search box in many ways. A pop-up search box that displays a search box may be used to display it over an active app. In some cases, the popup box can be used to direct the user to the active application. Other implementations allow the pop-up to be directed to the internet or another external network.

“In another example, a search field can be generated on a full screen (e.g. as shown in FIG. 2A). The search box can be a standard search engine that searches for information based on search queries. You can place the search box centered, sidebarred, bannered or corner rising over an active application. The search box can be found in some applications as a drop-down or menu box that appears upon mouse-over. The search box might be transparent on the screen until the user uses the roller ball 406, as an example.

“In some cases, the user may pre-configure one or more search controls within an application. Similar to the generalized search box, the user can also pre-configure a desktop displayed search box. Standard search engine technology can be combined into other applications so that a standard search box is displayed within the appropriate applications.

“Using the aforementioned mechanisms can allow a user with a mobile device search faster and more seamlessly through web content and application content without having to navigate to search engines or buried search functions. They can quickly check if the application has search functionality, and then perform a quick search to find the desired information.

Referring to FIG. 5 shows the exterior appearance for an exemplary 500 device that implements the described user interface features. The device 500 is a processor that displays notifications about events on the device 500 and allows a user to?pull down’ the notifications. Detail about the events related to the notifications can be viewed in an extended view.

“The hardware environment of the device 500 is described in detail. It includes a display 505 for text, images and video; a keyboard 502 to enter text data and user commands into 500; a pointing device 504. This device can point, select, and adjust objects on the display 501; antenna 505 and network connection 506; camera 507, microphone 509, and speaker 510. The device 500 has an external antenna 505, but the internal antenna is hidden from the user.

The display 501 displays video, graphics and images as well as text. It also contains the user interface for software applications 500 and operating system programs 500. A new mail indicator 511 alerts users to the existence of a message. An active call indicator 512 indicates that a call has been received, placed or is occurring. A data standard indicator 514 indicates the data standard 500 is currently being used to transmit and receive data. A signal strength indicator 515 measures the strength of a signal received via the antenna 505. A battery life indicator 516 indicates how much battery life is left. Or a clock 517 which outputs the current time.

“The display 501 might also display application icons that represent various applications, such as a web browser icon 519, a telephone application icon 520 and a search app icon 521. A contacts application icon 522. A mapping application icon 524. An email application icon 525. Or other application icons. One example implementation is the display 501, which is a quarter-video graphics array (QVGA), thin film transistor (TFT), liquid crystal display (LCD), and capable of 16 bit or better color.

“A user uses the keyboard or?keypad?” 502 is used to enter commands and data in order to control and operate the operating system. It also includes applications that respond to alerts and messages, and can be used to touch-screen. The keyboard 502 contains standard keyboard buttons and keys that correspond with alphanumeric characters. Keys 526 and 527 are examples of keys that correspond with the alphanumeric characters “Q?” ?W? When used alone or in conjunction with the characters??? When used alone, or in combination with the characters?*???? and?1? When pressed together with key 529, it will return?1? A single key can also be associated to special characters or functions depending on the state of the operating systems or the applications invoked by it. If an application requires the input of a numerical character, the selection of key 527 may result in a?1? ”

The keyboard 502 includes the usual function keys associated with an alphanumeric keyboard. These include an establish call key530 that answers a received call or initiates a new call. A terminate call key 531 causes an active call to be terminated. A drop down menu key 532 causes a menu to appear within display 501. A backward navigation key 534 causes a previously accessed address to be accessed again. A favorites key 535 causes an active website to be saved to a bookmarks or folder; a home page 536 that allows an application to navigate to a specified network address.

The pointing device 504 is used by the user to adjust and select graphics and text objects on the display 501. This interaction and control with the device 500 as well as the applications that are invoked on it 500 is called “Pointing device 504”. Any type of pointing device 504 can be used. It may include a joystick or trackball, a touchscreen device that is combined with the display 501, or another input device.

An antenna 505, which may be either an external antenna or internal antenna, can transmit and receive radiofrequency signals. It is used to determine location, establish point-to-point radio communications, and wireless local area networks (LAN) communication. The antenna 505 can facilitate point-to?point radio communication using Specialized Mobile Radio (SMR), Cellular, or Personal Communication Service(PCS) frequency bands. It may also implement data transmission using any number of data standards. The antenna 505 can allow data to be sent between the base station 500 and the device 500 using various technologies, such as Wireless Broadband, Worldwide Interoperability of Microwave ACCess, 5GPP Long Term Evolution, LTE, Ultra Mobile Broadband, High Performance Radio Metropolitan Network, Hiperman, High Speed Packet Access, HSPA, HSPA+, High-Speed Upload Packet Access, HSPA Evolution, HSPA+. W-CDMA, HSUPA and GPRS communications may be possible, for example, with a QUALCOMM MSM7200A chipset and a QUALCOMM MTR6285 transceiver.

The wired or wireless computer network connection 506 could be a modem connection or a local-area connection (LAN), including Ethernet, or a broadband wide area network (WAN) connection such a digital subscriber line or DSL, cable high-speed internet connection or dial-up connection. It can also include a T-1, T-3, fiber optic connection, satellite connection, and dial-up connection. The network connection 506 can connect to a LAN, corporate or government network, the Internet, or another network. The network connection 506 can be wired or wireless. Examples of wireless connectors are, for instance, an INFRARED DATA Association (IrDA), a Wi-Fi connector or an optical wireless connection, an INSTITUTE ELECTRICAL & ELECTRONICS ENGINEERS Standard 802.11 wireless connect, a BLUETOOTH connector (such as a BLUETOOTH version 1.2 connector or 5.0 connector), a near-field communications (NFC), an orthogonal frequency division multiplexing (TM-UWB), a time-modulated ultrawide band (TM-UWB), TM-UWB), a time-modulated wireless connector or another wireless connect. Examples of wired connectors are, for instance, an IEEE-1394 FIREWIRE connector and a Universal Serial Bus connector (including a miniature-B USB interface connector), as well as a serial port connector, parallel port connector or any other wired connector. Another implementation combines the functions of the antenna 505 and the network connection 506 into one component.

The camera 507 can capture digital images. It may also be a scanner or a digital still camera. The camera 507, a 5-megapixel (MP) camera that uses a complementary metal oxide semiconductor (CMOS), is an example of such an implementation.

The microphone 509 allows the device 500 capture sound. It can be an omnidirectional microphone or unidirectional microphone. The microphone 509 can be used to record sound that is generated by users, such as when they are speaking to each other via telephone using the device 500. The speaker 510, on the other hand, allows the device convert an electrical signal to sound. This includes a voice from another user or a ringtone generated from a phone application program. The FIG. 500 illustrates the device 500. FIG. 5 shows the device 500 as a handheld device. However, further implementations of the device 500 could be a laptop or a workstation, a middlerange computer, an embedded system and telephone.

“FIG. 6. This block diagram illustrates the internal architecture 600 for the device 500. A central processing unit (CPU), 601, is responsible for the processing of computer instructions. The architecture includes a central processing unit (CPU) 601 that processes computer instructions. A pointing device interface 605 provides a communication interface to the antenna 505. An antenna connection interface 607 provides a communication link to the network connection 506. A non-volatile memory device 612 or another suitable type of memory stores invariant low-level systems code or data for basic system functions such as startup, input and output (I/O), I/O, or receipt of keystrokes from the keyboard 502; telephony subsystem 612. The CPU 601 and its constituent devices communicate over a bus 621.

“The CPU 601 may be one of many computer processors. The computer CPU 601 can be used in more than one arrangement. RAM 610 connects to the computer bus 621 to provide RAM storage for the CPU 601 in a quick manner during execution of software programs like the operating system applications programs and device drivers. The CPU 601 loads computer executable process steps from other media or storage mediums into the RAM 610 field to execute software programs. The RAM 610 stores data, which is accessed during execution by the CPU 601. One example configuration of the device 500 contains at least 128 MB RAM and 256MB flash memory.

The storage medium 612 may contain a number physical drive units such as a redundant array independent disks (RAID), flash memory, a USB flash memory, an external hard drive drive or Blu-Ray optical disk drive. An external mini-dual inline memory module (DIMM), synchronous dynamic random acces memory (SDRAM), and an external micro-DIMMSDRAM. These computer-readable media enable the device 500 access to computer-executable processes, applications programs, and the like. The device 500 can also offload data from 500 or upload data to 500.

“A computer program product is tangible embodied within storage medium 612, which is machine-readable storage media. Instructions in the computer program product can be read by a machine to cause a mobile device’s data processing apparatus to store image information. In certain embodiments, the computer product contains instructions that generate notifications about new messages arriving on the device.

Click here to view the patent on Google Patents.