Software Technology for Multi-channel depth estimation using census transforms

Facebook Technologies, LLC (Menlo Park, CA)

A system for estimating depth can be described as a method of determining depth information using two images from two cameras. One camera captures one picture while the other camera captures a second. Both images include several light channels. The system calculates a Census Transformation for each pixel within both images. In a second light channel of the numerous light channels, the system calculates a census transform for each one of the first images and an equivalent census transform for every pixel in the second image. The system generates a depth map that is based on census transforms for every pixel of the first image as well as the second image of the first light channel, and in the second channel.

This disclosure generally refers to depth estimation derived from image data, and more specifically , to multi-channel depth estimation using census transforms.

A variety of headsets used in artificial reality systems typically have depth estimation systems. They can be used to determine the depth information of artificial real-world environments. The most common methods of getting depth information are based on stereo algorithms that establish a stereo correspondence between two images by considering visible light only using one luminance (luma) channel. But, stereo algorithms that are built on a luma channel are limited in instances where the lightchannel may have low contrast, e.g. when you are in the outdoors in a saturated environment with light or in a low-light environment.

The present disclosure is about how to determine depth information using a stereo pair images from two cameras attached to headset. The method involves taking one image at a first camera, and another at another camera (e.g. on a headset), the first and second images containing an environment, which is captured in multiple of light channels (e.g. some combination of red and green blue, infrared and red). The method also computes the census transform using at most two light channels of the plurality light channels for each pixels in the first image as well as each of the pixel in the second image. A depth map is created using the comparison of the census transformations in at least two lightchannels. Stereo algorithms are able to be utilized to generate the depth map. These algorithms calculate the stereo correlation between the first and second image by analyzing the census transforms that are in two lightchannels at minimum. A headset in an artificial reality environment that implements the method can generate virtual content that is displayed by an electronic display that displays the created depth map.

Click here to view the patent on USPTO website.

Get Patents with PatentPC

What is a patent?

The government grants patents to protect an invention, a patent provides the inventor with the exclusive right to develop, utilize and sell the invention?society benefits when a new technology is introduced to the market. The benefits may be directly, in that it allows people to do previously impossible things. Or indirectly by the economic benefits (business growth and employment) that innovation provides.

Patent protection is sought by many pharmaceutical companies and university researchers to protect their research and development. Patents may cover a physical or abstract product or process, or even an approach or composition of material that are new to the field. Patent protection is granted to any invention that is beneficial unique, innovative, and not already known by others in the same area.

Patents recognize and reward inventors for their commercially profitable inventions. They act as a motivator for inventors to invent. Patents permit inventors and small companies to know that there’s an excellent chance that they will be paid back on their time, effort, and money invested in technology development. They can earn a living from their work.

Patents play essential roles in firms and can be used to:

Make sure you protect your unique products and services

Improve the visibility and the value of your product’s presence on the market

Your business and your products should be distinguished from others;

Find out about business and technical information.

Avoid accidentally using third-party content or loosing valuable information, original outputs, or any other outputs that are creative.

Patents transform inventor’s knowledge into a marketable asset, which creates new opportunities to create jobs by licensing joint ventures and joint ventures.

Investors in the commercialization and development of technology may find small-scale businesses that have patent protection appealing.

Patenting can generate new ideas and new inventions. These information may be eligible for protection under patents.

Patents are a way to stop untrustworthy third-party companies from earning from the invention’s efforts.

Patent-protected technology revenues that are commercially viable can be used to finance technology-related research and development (R&D) which increases the likelihood of a better technology in the near future.

Intellectual property ownership is a way to convince lenders and investors that there are genuine opportunities to market your product. Sometimes, one patent can open the door to numerous financing opportunities. Patents as well as other IP assets are able to be utilized as collateral or security for debt financing. Investors can also see your patent assets in order to increase their company valuation. Forbes and other publications have stated that each patent can increase the value of your company by as much as $500,000 to $1 million.

A properly-constructed business plan is essential for new businesses. It should be built on IP and show what your service or product is distinctive. Investors will also be impressed if you have IP rights are secure or in the process to becoming secure, and if they agree with your business plan.

It is crucial to protect an invention before applying for patent protection. Making an invention public before filing it can often destroy its novelty and render it patent-infringing. Pre-filing disclosures, such as for investors, test marketing, or any other business partners, is best done after signing a confidentiality contract.

There are many kinds of patents and knowing them is essential to protecting your invention. Patents for utility cover processes and machine creations. Design patents cover ornamental designs. Utility patents are the best as they protect the owner from copycats as well as other competition. Patents for utility are usually granted to enhance or modify existing inventions. Patents issued under utility can be used to enhance or alter existing inventions. For example, a process patent will be able to cover actions or methods for performing an action, while chemical compositions will comprise a mixture of ingredients.

What is the average length of the patent? While utility patents last for 20 years from the date of the initial filing, they can be extended through delays at the patent office.

Are you planning to patent your ideas? Patents are granted only for applicants who are first to file, you need to file quickly – call an attorney for patents at PatentPC to patent your idea today!

When you’re writing an application for patents, you should do a patent search. the search will give you some insight into other people’s concepts. This can help you limit the extent of your invention. Also, you can discover the current state of the art within your field of invention. You’ll have a better idea of what your idea should be, and you’ll be more prepared to submit your patent application.

How to Search for Patents

The first step to get your patent is to perform an internet search for patents. You can do a google patent search or do a USPTO search. After the patent application has been submitted, the product that is covered by the application can be called patent-pending, and you can locate the patent application on a public pair. When the patent office has endorsed the patent application, you are able to perform a patent search to find the issued patent and your product will now be patented. In addition to the USPTO search engine, you can also utilize other search engines, such as espacenet as described below. For assistance, consult a patent lawyer. In the US patents are granted through the US patent and trademark office, or the United States patent and trademark office, which also examines trademark applications.

Are you interested in similar patents? These are the steps you should follow:

1. Brainstorm terms to describe your invention according to its function composition, use, or purpose.

Start by writing down a brief, precise description of your idea. Be sure to avoid using terms that are generic such as “device,” “process,” and “system.” Consider synonyms for the terms you chose initially. Then, note crucial technical terms and key words.

To help you recognize terms and keywords, you can use the following questions.

• What’s the purpose of this invention? Is it a utilitarian device or an ornamental design?
• Is the invention a way of making something or performing a function? Is it a product or process?
• What is the composition and function of the invention? What is the physical structure of the invention?
• What’s the purpose of the invention
• What are the terms in the technical field and keywords used to define the nature of an invention? A technical dictionary can help you identify the correct phrases.

2. These terms will enable you to look up relevant Cooperative Patent Classifications on the Classification Search Tool. If you’re not able to determine the correct classification to describe your invention, go through the classification’s Schemas of classes (class schedules). You may want to consider substituting the terms you use to describe your invention if you don’t get any results from your Classification Text Search with synonyms similar to the words you used in the first step.

3. Check 3. Check the CPC Classification Definition for confirmation of the CPC classification you have found. The link to the CPC classification definition will be given if the chosen classification title has a blue box that includes “D” to the left. CPC classification definitions can be used to determine the scope of the classification which is why you can be sure to select the most relevant. The definitions could also contain search tips or other suggestions that could be helpful for further research.

4. The Patents Full-Text Database and the Image Database allow you to retrieve patent documents that include the CPC classification. By focusing on the abstracts and representative drawings it is possible to narrow your search to find the relevant patent documents.

5. Use this selection of the most pertinent patent documents to examine each one thoroughly for similarity to your invention. Take note of the claims and specifications. You may find additional patents through contacting the patent examiner as well as the applicant.

6. Search for patent applications that have been published using the CPC classification you chose in Step 3 from the Applications Full-Text and Image Database. You can use the same strategy of searching as in Step 4. You can narrow your results down to the most relevant patent application through the abstract and representative drawings that appear on each page. Then, you must carefully review the patent applications that have been published and pay particular attention to the claims as well as additional drawings.

7. Find other US patent publications using keywords searching in the PatFT and AppFT databases, searching for classification of non-U.S. patents according to below, and searching non-patent patent disclosures in the literature of inventions using internet search engines. For instance:

• Add keywords to your search. Keyword searches may turn up documents that are not well-categorized or have missed classifications during Step 2. For example, US patent examiners often supplement their classification searches with keyword searches. Think about the use of technical engineering terminology rather than everyday words.
• Search for foreign patents using the CPC classification. Then, re-run the search using international patent office search engines such as Espacenet, the European Patent Office’s worldwide patent publication database of over 130 million patent publications. Other national databases include:
• European Patent Office (EPO) provides esp@cenet to access a network of Europe’s patent databases with access to machine translation of European patents. 
• Japan Patent Office (JPO) – with access to machine translations of Japanese patents.
• World Intellectual Property Organization (WIPO) offers PATENTSCOPE with a full-text search of published international patent applications and machine translations for some documents, as well as a list of international patent databases.
• Korean Intellectual Property Rights Information Service (KIPRIS)
• State Intellectual Property Office (SIPO) with machine translation of Chinese patents.
• Other International Intellectual Property Offices with online patent databases include Australia, Canada, Denmark, Finland, France, Germany, Great Britain, India, Israel, Netherlands, Norway, Sweden, Switzerland, and Taiwan.
• Search non-patent literature. Inventions can be made public in many non-patent publications. It is recommended that you search journals, books, websites, technical catalogs, conference proceedings, and other print and electronic publications.

To review your search, you can hire a registered patent attorney to assist. A preliminary search will help one better prepare to talk about their invention and other related inventions with a professional patent attorney. In addition, the attorney will not spend too much time or money on patenting basics.