Alphabet – Johannes Ritter
Abstract for Multivariate optimization method for testing
An advertisement optimization approach is described that uses multivariate testing to determine if characteristics of an advertisement correlate with favorable responses in an environment where the advertisement was placed. A text-based advertisement on a webpage may be optimized for different response metrics, such as click-through rates and conversion rates. There are other ways advertisements can be optimized across multiple media formats.
Background for Multivariate optimization method for testing
When creating an advertisement or marketing campaign, marketers and advertisers are often faced with many considerations. The quality or type of responses generated by exposure to an advertisement in a specific environment can be affected by the content and layout of the advertisement. This article describes a method that uses multivariate testing to guide the creation of ads that are optimized for specific responses. A text advertisement on an Internet website is optimized using multivariate testing. Multivariate testing can also be applied to different media formats.
Multivariate testing is a method for improving an advertising campaign or marketing campaign. Multivariate testing can include one or more of these methods: Optimal Design (Discrete Choice), Optimal Design (Taguchi Method), and the Monte Carlo Method.
“A multivariate testing approach to internet or web-based advertising is used in the non-limiting example. It should be noted that these methods can be used to improve advertising and/or marketing campaigns in a variety media formats, such as: Internet page design, email banner advertisements, Internet text links advertisements, text messaging, direct and courier mail, internet marketing, signage, radio and television, personal digital assistants (PDA), cell phones, global positioning system (GPS), in-vehicle displays and telephonic marketing via mobile phones, pagers and telephones or voice over Internet protocol(VOIP), and other media forms, 1A. FIG. FIG. 1A illustrates how a multivariate test approach indicated at 219 may be applied initially to advertising and/or market campaigns indicated at 180 to improve advertising and/or market campaigns indicated at 194.
The advent of online advertising has created a highly competitive market for advertising. Advertisers find internet-based or webpage-based text ads, such as the ones sold under the brand ADWORDS from GOOGLE, both extremely accessible and highly efficient. Businesses that are already established on the Internet market face some disadvantages due to this accessibility and efficiency. New competitors may be able to more easily access the Internet market through web-based advertising, which can reduce the market share for some businesses.
An example of Internet software sales will be shown in order to illustrate the different approaches to improving advertising campaigns. FIG. FIG. 1B illustrates an example computer network system 180. A web server 160 can communicate via the Internet 150 with a number of clients (in this case, 170-178). This is an example of a wide-area network. One client device can be included in each of the network clients 170-178. This could include a mobile phone, a portable digital assistant (PDA), or any other device. These devices can include a graphical display to display web pages to clients. In this example, web server 160 also includes memory 162, which can store computer-readable instructions that web server 160 and clients 170-178 can execute. Web server 160 can also be configured to display advertisements on the different web pages that are displayed by clients.
In this example, a company is advertising a software product that can automatically archive office documents at $20 through an internet campaign. The business began running online advertisements several months ago. There were no competitors, and one single advertisement was enough to draw customers. The cost-per-click (CPC), at that time, was only 5 cents. Each visitor to the website earned the business an average of 80c. This led to a profit margin that was 75 cents per visitor. The business experienced a sharp drop in sales one year later. The business first assumed that the advertisement was not correct. An examination of the online advertisement was therefore conducted. A GOOGLE search query was run as part of the examination. It turned up four advertisements, three from competitors, in the result. While GOOGLE is referred as an example search engine or advertising source throughout, the methods, systems and acts described herein can be applied to other search engines, mail marketing and/or advertising media. In this case, not only was there more competition but also the search rank of the advertisement for the business was lower, from the first to the third position. Each of these new ads was examined by the business and revealed that they were selling substantially the same software product. The business was left with the dilemma of how to fix the situation after concluding that the sales decline was at least partly due to increased competition.
Similar scenarios to this one can occur with almost any product and in any media. Sometimes, it may be difficult for businesses to break even due to the high cost-per-click associated with certain Internet search companies. Companies with large advertising budgets are able to afford to pay more per click to get a higher rank. This makes it harder for smaller companies to compete in this market.
Internet marketers are faced with the challenge of increasing marketing returns at minimal or no cost. An alternative approach is to increase marketing return. Advertising costs can be directly linked to product profit margins. The software company in the above example had a profit margin of 75c per visitor to its website over the past months. The business lost a significant market share to its competitors one year later. The business’s advertisement fell to the third position after it was discovered that competitors were paying higher maximum costs per click for each keyword.
One way to improve their marketing campaign is to increase the advertising budget. This could be done by raising the maximum cost per click up to 50cs. This may allow the business to compete with those of higher rankings. If the company can spend more advertising and has a higher profit margin then they may be able improve their marketing position through an increase in their search rank. The business might not be able to spend enough to attain a higher ranking. The business might decide to raise the cost-per click to 75c, which is the maximum amount that could be afforded while not making a loss. However, this might still not be enough to achieve the highest ranking advertising position. This scenario may lead to the conclusion that a competitor with a higher ranking may be more successful at converting customer traffic into customers. This is how escalating advertising costs and increased competition can threaten a business’s success. While it may work in certain cases to increase advertising spending by increasing profit margins and outbidding competitors, it is not always the best option.
A business might improve its profit margins to be ahead of their competitors by optimizing or optimising the marketing campaign. Businesses would normally allocate a portion of their budget to develop a new, improved marketing campaign that is based on focus groups surveys. Focus group surveys can sometimes provide inaccurate or misleading data, which could lead to a business launching a wrong-directed marketing campaign or advertising campaign.
Advertising campaigns can be more effective if they are based on actual customer responses in real market conditions. Multivariate (MVT), or multivariable, testing can be used to reduce the number and cost of experiments, which in turn will result in a more effective advertisement. This will allow the business to compete for higher search rankings by improving performance metrics like conversion rate. A conversion rate of 4 percent is, as an example, four out of 100 visitors to the website that purchased a product.
Customers can choose from a variety of advertisements to decide if they want to respond to an advertisement. This can give a better indicator of the effectiveness or ineffectiveness of advertising than traditional focus group surveys. This approach can also be applied to web-based advertising media such as landing pages (i.e. The web page to which a user is directed after clicking on an advertisement online or via the internet.
“The methods described in this document can be divided into four operations. FIG. FIG. 2A is an example flowchart that briefly describes these four operations. A more detailed description will be given below, with reference to FIGS. 2B, 10, 17 and 24. FIG. The method could begin with the creation of a hypothesis at 110. The brainstorming session may involve discussing factors such as test requirements, test factors and possible threats to test robustness. The experiment can then be set up at 120. This may include the initialization of test, applying the statistical methodology, transformation of selected variables into advertisements and placement of the advertisement. The experiment can be completed at 130. This may include documenting and monitoring the data and statistics, as well as observing and tracking any noise factors that could impact the experiment. The experiment results can be analysed and interpreted to determine their validity. They may then be implemented at 140 to create an improved “super?” Advertisement, which confirms the predictions made during the experiment.
“As discussed above with reference to FIG. 2A. A hypothesis can be created before the experiment is designed, performed, and analysed. FIG. FIG. 2B illustrates, at 110, a flow diagram that explains in more detail how to develop a hypothesis. One or more performance metrics may be chosen that can be improved or optimized starting at 210. For web-based advertising, performance metrics may include click-through rates (CTR), (e.g. The rate at which clients click on or select the advertisement, or the conversion rate (CR), (e.g. The rate or proportion of clients who purchase the advertised product/engage in the offer presented by an advertisement. An average visitor’s (e.g. a client’s) value, response rate, etc. The click through rate, or CTR, may be used as a performance metric to determine the success of an online advertisement campaign. CTR can be calculated by multiplying the number users who clicked on an advertisement placed on a website by the number times that the advertisements were delivered (i.e. impressions). If the advertisement was sent 100 times (i.e. 100 impressions were delivered), and one person clicked on the advertisement (i.e. clicks recorded), then the CTR would be 1%. Clicks recorded), the resulting CTR is 1%. The conversion rate can also refer to how many visitors purchased products from a webpage, based on the total number of visitors to that webpage. A conversion rate of 10% means that 1/10 visitors to a webpage make a purchase.
While the example above is for Internet advertising, CTR/CR metrics may be similar or different in other media formats. Direct mail marketing metrics may include metrics like response rate, which measures the number of recipients who respond to a specific direct mail advertising campaign.
A business might want to improve their landing page and/or web advertisement conversion rates. It may be beneficial for a business to determine their past and/or present conversion rate to compare with future conversion rates due to adjustments to their advertising campaign. A business might want to improve multiple performance metrics. In this case, they would identify and choose the performance metrics to be improved. This allows you to compare the performance of the previous advertising campaign with the new one.
“At 212 potential test factors may be identified and their levels determined. Before we go into detail about 212, it is helpful to look at some background information about multivariate testing. Multiple terms will be used to refer to the various parameters of the multivariate testing. As used herein, a factor may refer to one entity or a portion of an advertisement. This could potentially impact the output of multivariate testing. A factor may also include media type or format. One factor could be, for example, the amount or magnitude of the price included in an advertisement. A factor could also include a graphic or photograph. A body of text may also be included in a factor.
“Further, there could be more than one level within a factor. The specification of the factor may be included in the levels. An advertisement that uses a price factor like price may have several levels. These levels could include the quantity of the price, such as $100 or $200. Levels may also include information about the image type, copy, content, and text size. The media format is a factor. This means that the different levels can include multiple formats, such as billboard ads, direct mail, television advertisements, direct mail, and television advertisement. The non-limiting examples and approaches discussed herein may not be limited to one media format. They may also include multiple media formats.
“FIGS. The 3rd and 4th versions of the example Internet advertisement show the first and second versions, respectively. They also include a variety of factors and their respective levels. FIG. FIG. 3 shows an Internet advertisement in its first version 310. It includes a title 320 and an abstract with a first and second line 330, 340 and 350. Similarly, FIG. FIG. 4 shows the second version 410, which includes a title 420 and an abstract with a first and second line 430, respectively, as well as a URL address of 450. One example is that the title could be a first factor with a level 1 shown at 320. This would be called?automatic backup?. A second level is shown at 420, which can be referred to as “software makes life easier?”. The URL address, however, may only have one level. For example, it could include?carsonspage.com. 350 to 450 Some factors can include multiple levels while others may only include one level.
“Returning back to FIG. 2B at 212, the potential factors and their corresponding levels may be selected. The selection of testing levels and potential factors may be an important step in the optimization process. These levels and the factors may help to define the test structure. One example is that levels can be chosen to target specific groups of potential customers. Internet advertisements might target consumers more likely to use the Internet. Television advertisements may target viewers who watch television. Direct mail advertisements could target consumers who don’t necessarily have access to the Internet or television.
It may therefore be beneficial to gather a team of competent people for a brainstorming session in order to identify potential factors and levels. People with experience selling the product might be more suited to selecting potential factors and levels that could directly affect the results of the test. The brainstorming session could include sales staff, accounting, customer service, and marketing personnel. In some cases, however, even those with a minimal marketing background may come up with some of the most innovative ideas. It may be beneficial for those who are selecting the possible factors and levels to be open-minded and willing to test new slogans and ideas in order to obtain substantial results. It may be beneficial for at least one person in the brainstorming group to become familiar with specific advertising requirements before they begin.
“The performance metric(s), as defined at 210, may determine the factors and levels that are to be tested. It may be possible to choose factors or levels that could affect click-through rate or response rates, for example, if you are trying to improve a performance metric like click-through rate or response rate. Contradictory levels or factors can be eliminated or reduced in one approach. FIG. 5 shows an example. FIG. 5 illustrates an advertisement 510 that has factors 520 and540 which contradict each other. It may be possible to alter at least one factor or level so that they do not contradict each other.
“It may be possible to use substantially different levels within the same factor in another approach. FIGS. FIGS. 6 and 7 depict two very similar price levels, 620 and 720. A multivariate approach can help you create a better advertisement. It allows you to test substantially different price levels, such as $15 or $50. This is in contrast to the $15 and $16 advertisements. Another approach is to test the most relevant and/or stronger factors. If it is clear that certain factors or levels are not affecting the chosen performance metric(s), it might be possible to reduce/eliminate these factors/or levels from testing. You should remember that not all levels and factors can be tested. It may be possible to test the effect of national gross domestic products (GDPs) on the price level, but it may not be possible to test for its influence on religion, unemployment, gender, age, or sexual preference. Although some of these variables may have an impact on the test’s output, some factors or levels may not be possible to test due to information overload, insufficient data or otherwise unfeasible.
“Now that potential factors or levels have been selected at 212, it is possible to examine the selected factors and/or level in light of the guidelines above. At 214, it can be determined whether any levels or factors are inconsistent. Some levels might not be compatible with certain media formats, for example. If yes, you can select other factors or levels at 212. It can be determined whether the levels selected are significantly different from one another at 216. It may prove more beneficial to test different media formats, such as direct mail and the Internet, than traditional media formats like newspaper advertisements and magazine ads. For some situations, more levels can provide more useful and relevant results. Other levels can be chosen at 212 if the answer is “no”. It can be determined whether the levels and factors are relevant to the performance metrics at 210. If no, then other factors or levels can be chosen at 212. It can be determined whether the levels and/or factors are possible to test at 220. If no, then other factors or levels can be chosen at 212.
“The number of testing levels and factors at 222 can be determined based on the selections of potential factors and/or level. The extent of the test can be determined, for example, by selecting one or more potential levels and factors during brainstorming sessions or another method. The longer and more complex the experiment, the more factors and/or level that must be tested. Multivariate testing aims to reduce the number and complexity of tests to obtain comparable statistical information. However, it is possible to reduce the number and complexity of the factors or levels being tested. This may allow for a compromise between realistic time frames and the amount of knowledge about the impact of the factors on selected performance metrics.
“From 224 it can be determined whether the selected factors or levels are suitable for the testing timeframe and statistical information desired. If no, then the number of factors and/or levels can be increased or decreased. If multiple media formats are being used, one or more may be removed, which will reduce the number of levels associated to that factor.
An orthogonal array can be used to set up tests for different combinations of factors/levels. FIG. 8 shows an example of such an orthogonal array. FIG. 8 is a table that shows some examples of orthogonal arrays. For example, let’s say you select title, price and display URL address. Each factor can be at two levels (e.g. If you choose $100 as the price and $200 for display URL address, then the?L4′ option will be available. array could be chosen. The subscript number is?4. The subscript number?4? is located behind the capital?L? The capital?L? indicates the number of advertisements used (e.g. During multivariate testing, the actual test was performed. The test may include four different web ads if the L4 array was selected. An example of this is where four advertisements could be placed online via an Internet search site like GOOGLE. The first person to visit the website where the ads were placed will see the first of four advertisements. A second advertisement may be displayed to the next visitor. It could have a different combination of one or more factors. It is possible to observe how people react to advertisements by observing the reactions of different groups (e.g. Multivariate testing allows you to infer statistically the impact of the variables and/or limits on selected performance metrics.
“Some experiments can be used to compare the landing page and advertisement. This allows you to select factors and/or levels to determine how performance metrics may be affected by variations like text font, color, pictures or special offers. The L32 array would allow 31 factors to be evaluated for their potential impact on the desired performance metrics. This could result in 32 different landing page and advertisement combinations. Each person who visits the advertisement or landing page could see one of 32 combinations. You can observe the behavior of the person (e.g. When they are presented with different impulses (e.g. different advertisements, different landing pages, etc.) The influential factors and/or levels of influence may be determined.
“Multivariate testing may be used in some cases to determine the best combination of factors or levels that results in the most favorable (e.g. The performance metric(s), which are selected, may be the highest or lowest. Alternately, the multivariate approach could not be applied to tests involving 31 factors with 2 levels each. In such cases, 2,147,483,648 total or 231 advertisements could be tested. This may make it impossible to test all combinations. Multivariate testing allows for testing of far fewer combinations. For example, 32 combinations can be tested using the multivariate approach. Multivariate testing uses a mathematical and statistical approach to create orthogonal arrays. This allows for the testing of fewer combinations, which can reduce any potential loss of information. Other approaches can be used to identify trends at certain levels and factors using fewer tests.
Referring to FIG. 8 are mixed level arrays. The two arrays marked * with an asterisk (*). These arrays can be useful when some factors are tested with a number of levels, and others require testing with a number of levels that is different from the initial number. The L18 array, for example, has one factor with two levels and seven factors with three levels. The table in FIG. FIG. 8.8 shows some of the most commonly used arrays. However, it is important to remember that other arrays can be used depending upon the number of factors or corresponding levels.
“Returning back to FIG. “Returning to FIG. 2B, at 226, at minimum one array could be selected correspondingly to the number or levels of factors that were selected. If seven factors were brainstormed and each factor was assigned two levels, the L8 array might be chosen.
“At 228, selected factors and/or limitations may be compared with the requirements specific for the media format being tested. A web-based marketing campaign that uses a search engine like GOOGLE might have specific requirements for the advertisement. These guidelines may include style, editorial, and character limits. If they are not followed, the advertisement might not be approved. These guidelines prevent testing of levels or factors that are impossible to test.
“FIG. “FIG. 3. It is important to remember that different media formats might have different, more or less requirements than the one shown here. An example of this is the title or headline, which may limit an Internet advertisement to 25 characters. However, there may not be a limit for direct mail media formats like direct mailing. The maximum characters count for the description lines, destination URL, and display URL may be different. To comply with GOOGLE’s advertising policy, the text of the advertisement must be clear and concise, communicate with the audience, be easy to read and compelling without being misleading.
“Other requirements and recommendations may also be related to spelling, punctuation or grammar, grammar, capitalization. repetition, inappropriate language. unacceptable phrases, superlatives, prices, claims, discounts offers.
“Regarding spelling: Proper spelling is crucial to the credibility and clarity of an advertisement. The words should be correctly spelled in the advertisement. This is not true for commonly misspelled words and spelling variations. It may be acceptable if the word is found in an online dictionary. Customers and users should all be able to recognize the spelling error. In some instances, however, you may use any spelling.
“Regarding spacing: Adverts should be spaced between words and after punctuation. For example, C-h-ea-p C?l-o?t-h?e-s. It may not be permitted. Similarly, ?Free Shipping.Buy Now? Other restrictions may apply.
“Regarding punctuation. Punctuation should not be used to draw attention. It may not be necessary or repeated more than once in a row. The use of an exclamation point in advertisements may be subject to specific rules. An exclamation mark may not be included in the first line of advertisement text. In some cases, an advertisement can only contain one exclamation mark.
“Regarding grammar, in some cases advertisements must adhere to basic grammar guidelines. Advertisement text can use either logical sentences or phrase forms. Additionally, symbols, numbers or letters may be necessary to convey their true meaning. The advertisement may use them instead of words. You might see the following example:?We have many 4U online! could be considered a violation of the grammar policy. is substituting words.”
“Regarding capitalization. A word may not appear all the capital letters in order to draw attention to it or that phrase. For example,?FREE For example,?FREE? oder?NEW?? may not be allowed. This may not be permitted. Capitalizing the first letter of each word in your ad might be permissible.
“Regarding repetition: Repeated statements should not be used for promotion or in a gimmicky way. The same word cannot be repeated more than three times in a row. An example of this is an ad that says “Deals, Deals and Deals Here?” This may be prohibited. You can replace the advertisement title with text like “Amazing Deals here?” This policy must be followed.”
“Regarding inappropriate terminology: Advertisements, including the display URL may not contain language that could be considered offensive or inappropriate for some users. This could also be applicable to misspellings, selfcensored, and other forms of inappropriate language.
“Regarding unacceptable phrases: Certain ?call-to-action? If they aren’t descriptive of the product or service, phrases might not be used in advertisements. You might see phrases such as?click here? For example, phrases like?click here? and?visit our? These are not allowed. A good example of a?call to action? You might use the phrase?Order your Online Contacts Today? Because it represents the product and site content.
“Regarding superlatives: Superlatives refer to words that emphasise superiority. To ensure that users feel trustworthy and treated fairly, the advertisement text should not include subjective or comparative phrases like?Best?. Your website may need to clearly display this verification. If an advertisement claims it is the “Best of the Web”, the site might be required to show third-party verification. An example of this is a Forbes Magazine seal that indicates the site has won a best-of-the-web award. The ad could be accepted.
“Regarding competition claims: These claims claim that a product/service is superior to a competitor’s product/service. The landing page may need to support competitive claims made in the advertisement text. This builds trust and assures users that they are finding what they want based on the advertisement text. These claims can be supported in many ways, such as a table or chart that compares the product’s features with the competitor’s or a competitive analysis explaining why the product is better. Advertisement text that says “better than product A” could be an example. This could be considered a competitor claim and may need to be supported on the website. This claim could be accepted if the landing page contains a competitive analysis of advertiser A and product B. The advertisement might be approved.
“Regarding Prices: Specific prices may need to be supported within 1-2 Clicks of the landing Page. It is possible that prices in the advertisement text need to be exact. Bulk purchases may also be subject to pricing.
“Regarding discounts offers: Any discount offer that is displayed in the advertisement text might need to be supported within one to two clicks on the landing page. You may support discounts such as?50% discount on all items,?Save $20 when you make your first purchase, and similar phrases. Users may find free offers very attractive and may need to be supported within a few clicks on the landing page. It may be acceptable for the user to infer that the product or service is free even though the word “free” does not appear alongside it.
“These and other requirements of GOOGLE advertisements may be found at ?http://www.GOOGLE.com/ADWORDS/learningcenter/text/index.html?. You should remember that these requirements and guidelines are only a sample of what some web-based search sites like GOOGLE require. Other requirements could exist for different advertising services or media formats.
“Returning back to FIG. “Returning to FIG. If no, factors or levels not meeting the requirements can be adjusted at 228.
“At 232 potential threats to robustness (i.e. potential threats to robustness (i.e. noise factors) may be identified at 232. These potential threats and noise factors could be related to factors that can be controlled. It may be possible to give specific values to controllable factors depending on what you want. If pricing is one of the factors, for example, the price could be set at $100, $200, or any other price you choose. A factor could include an entity, or part of an advertisement that may have an influence on the outcome of the selected performance metrics (such as click-through rate, conversions, etc.).
“In some cases, random events may occur that could directly impact whether people respond to an advertisement. A server could crash, leading to a decrease in sales or traffic. Another example is a competitor’s sudden emergence that could affect the experiment. Uncontrollable events could pose a threat to the experiment’s robustness.
“Unfortunately, some extraordinary events might not be included in the test design. They can have a serious impact on test results, depending on how large they are. A server that is down for more than an hour may not have the same impact as a competitor who appears. Multiple repetitions of the same experiment can be used to overcome this problem. More repetitions of an experiment will increase the likelihood that the results will be reliable, which in turn increases the validity of the results. A test could be conducted for one week, during which there is no competitor for the product being promoted. The results could be different if the test is repeated the next week with two new competitors. To combat this, you can run the test again the next week. To increase the certainty of the results, each repetition can be used. There is a tradeoff between certainty and time. It may be worthwhile to establish how certain the results of the test must be. It may be necessary to determine how certain the test results must be. For example, two repetitions of the test are sufficient to ensure that 85% of the results are valid. Or, does the test require 95% certainty, which would mean five additional weeks of testing. This decision may vary from one case to another and it may not make sense to establish strict guidelines.
“As discussed above with reference to FIG. “As described above with reference to FIG. 2A, an experiment can be designed at 120. The advertisements will be arranged in the correct format. The test could produce misleading or incorrect interpretations if the levels and selected factors are not properly translated into the advertisement format.
“FIG. 10. This flow chart shows an example of how to design the experiment at 120. Based on the array, 1010 can be used to select the best combinations profile. 8. The array at 226 in FIG. may determine the resulting set of unique ads. 2B. 2B. Each array in FIG. Each of the arrays in FIG. 8 may include a corresponding profile for standardized combinations, which is a subset of all possible combinations. These profiles can be found in statistics or mathematics books for each of these arrays. If the L8 array was chosen for testing, the profile of the L8 combinations may be selected, as shown in FIG. 11.”
“On the vertical-axis under?AD number? These numbers correspond to the eight advertisements that can be tested. Horizontally, the row to the right from?Factor? shows seven factors (?A??-?G?). These were chosen for the L8 array. The L4 array would be chosen instead. This combination profile would have four different advertisements along the vertical axis, and three different factors along the horizontal. The numbers?1? and?2? are located on the inside of this table. The numbers?1????? and?2??zunehmen Le dar Get get Da da an bei all des Des darin Dec Und Dar The levels to be tested are indicated by?2? It is possible to look at the table column by column (?A-?G?) in order to check for errors. Each of the??1?s and?2?s should be counted. One approach is to count each column as half a?1 and half a?2 or, in this example, as four??1?s. This indicates that the levels were evenly used. Some approaches might use different levels for all or some of the factors. You should also verify the number of ads being tested. The vertical axis should have eight advertisements. This is done by dividing the number of levels per factor, which in this case is two.
“At 1012 the selected factors or levels can be arranged as shown in FIG. 12 refers to an example advertisement for a software product. These factors and levels could correspond to the levels and factors selected in FIG. 2B. There are seven factors that can be used to test the L8 array. Level 1 and 2 indicate different specifications or characteristics for each factor. Bold text indicates the location of the advertised price and bonus offer levels.
“At 1014 the factors and/or level, for example, as shown in FIG. As per the combinations profile in FIG. 12, 12 can be added to an L8 combinations templates FIG. 11 15. This operation requires care as it can affect one level of the advertisement. One approach is to add the factors to the L8 combinations profile. However, the inside of the table containing the??1?s or??2?s can be temporarily left empty. The blank table that will receive the selected levels can be called the template, while the table with the?1’s and??2?s can be called the profile. FIG. 11.”
“Column A is the combinations profile of FIG. 11 is the first factor with its specifications (levels), in each of eight different advertisements. Column A may be renamed “headline” in the profile. FIG. 13. FIG. 11 The profile in column A shows that advertisements numbers 1-4 have each been given a level 1. FIG. FIG. 12 shows that the headline factor level 1 includes the text “backup software?”. The template may be modified in the column renamed?Headline? Advertisements 1-4, as shown in FIG. 13.”
“One thing you should be aware of is that the factor includes different levels of capitalization. You might consider adding “backup software?” It is possible that the template might not be correctly filled in with lowercase letters. To avoid making mistakes, you can fill in the variables that affect other factors first. Capitalization can be added to the?Headline?” For example, The level of the factor (e.g. Depending on the level (e.g. ?Display URL You may change the capitalization of text. In the profile, factor G corresponds to?Capitalization?. Column G can be renamed in the template to?Capitalization? Fill in the form using lowercase letters for all fields that have a?1? Fill in the profile using lower case letters in all fields that have a?1? in the profile. This may be repeated for columns F and E, which correspond to factors like?Price? The?Bonus Offer? and column E may be repeated. The?Headline?” factor requires that the four first advertisements with a?1? suffice. The template can be filled in.”
“The four remaining advertisements have level 2 as specified in the profile for factor?A? Which was again factor?Headline In the template. The level?2 is an example. This factor can be used to replace a keyword in GOOGLE ADWORDS. This means that if an advertisement is triggered using a specific keyword, it will appear boldly in the GOOGLE advertisement headline. This functionality may make capitalization irrelevant as the keyword will be displayed in the same capitalization that the search query was. Therefore, ?dynamic? The remaining four rows may have?dynamic? This functionality allows you to specify a default headline if an advertisement is triggered using another keyword. Again, capitalization is not important. Capitalization may not be relevant if the default value is set at?1? This factor can be set to?1 as the default value, and capitalization may also be adjusted accordingly. For more information on how to implement this functionality, please refer to the GOOGLE manual located at the URL address. You can rename column B in the template by replacing?Description line 1 with column B. The combinations profile of FIG. may also be used to add the appropriate levels. 11. This is now affected by levels of?Capitalization? This factor is now influenced by the levels,?Capitalization? As shown in FIG. 14. This is the advertisement number 1 in this template. of factor ‘Description Line 1? at level?1? Factor?Price? Level?1? Capitalization factor could be used.”
“Continue with FIG. 10 after adding the factors and levels to the combinations template, as shown in FIG. 15 It is a good idea, as illustrated at 1016, to check frequently whether advertising requirements have been met. The factors and/or levels can be adjusted at 1018 if the answer is not yes. Dynamic elements, such as the?Bonus offer?, may be used. If dynamic elements such as the?Bonus Offer? are used, character counts may be exceeded. Example: advertisement number?2 In the template, ‘Description Line 2’ is used. You may add?Free Trial Reliable Professional? to the template. This is equivalent to 34 characters. If the?Bonus Deal? contains 11 characters, then only 24 characters for the actual factor may remain. If the?Bonus Offer? contains 11 characters, only 24 characters may be left for the actual factor. After all the levels and factors have been added to the combinations template according to the profiles, the template should look something like FIG. 15.”
The combinations template can be used to create advertisements at 1020. To avoid confusion when you combine the different factors and their respective levels, it is recommended that each advertisement be written down. Advertisement number 1 should contain:
“Headline Backup Software”
“Description Line 1 – All formats supported for only $20”
“Description Line 2 – reliable, professional”
“Display URL: www.carsonspage.com”
“Similarly, advertisement number 2 in the web-based advertisement example should contain:
“Headline Backup Software”
“Description Line 1: All Formats Supported for Only $40”
“Description Line 2 – Free Trial, Reliable and Professional”
“Display URL: www.ArchivingExperts.com”
“Further advertisement number 3 should contain:
“Headline Backup Software”
“Description Line 1 – Just $20 to Protect Your Data!”
“Description Line 2 – Free Trial, Easy Backup Process”
“Display URL: www.CarsonsPage.com”
“At 1022, the advertising statistics must be reset or initialized in order to capture the influence of new advertisements that have been created using the multivariate test approach. This will allow for better monitoring of the test. For example, if a GOOGLE advertisement campaign had a single group of advertisements and a single ad, GOOGLE would show statistics on this advertisement from the time it was activated. Multivariate testing is intended to improve the campaign, or at the very least the selected performance metric. It may be necessary to delete or pause the advertisements from before. Sometimes, if advertisements are paused, GOOGLE may include the advertisement in the statistics, which could cause flaws in the test. It is recommended that you delete any previous advertisements or create a new campaign while the campaign is paused. It may also be beneficial to use the same keywords in the new campaign, so that tests can be comparable. The advertisements can be submitted at 1024 and the testing phase can begin. FIGS. 8 and 9 show the eight advertisements used in this example web campaign. 16A-16H, respectively.”
“As explained above with reference to FIG. 2A. After the experiment has been designed, the experiment can be conducted at 130. Here, the advertisements will be submitted and test data gathered. FIG. FIG. 17 is a flowchart that describes an example of how to perform the experiment at 130. The experiment data and statistics may be monitored at 1710.
“At this stage, online ads have been submitted and the results can be measured. This is one advantage of the herein-outlined marketing strategy. Some marketing strategies are not easily quantifiable. A company may experience an increase in revenue, but they may not be able to explain the reason. It is possible to determine the success of new advertisements by comparing their performance to those of prior advertisements. It may be possible to find past results and statistics for web-based campaigns that use GOOGLE. FIG. FIG. 18 illustrates an example of statistics from a web-based advertising campaign using Google. The conversion rate was about 4 percent during this period.
“At 1720 the test statistics can be documented, including the baseline statistics from the previous advertising campaign. You may want to establish a time period in which you can track the results. It may be based on how many clicks are received in a given time frame. It may not be necessary for advertisers to keep track of the results every day if they receive only ten clicks each day. You could record the results weekly, or if there are more than 50 clicks per day, you can keep track of the results each day. Another approach is to record the results every week. This reduces the potential noise factors that could affect test results. People respond differently to ads on weekends. The weekend is one example of noise that can affect the results. Some advertisements may get more clicks on weekends than others during work days. It is also recommended to keep the statistics in order in order to monitor them. It is unlikely that mistakes made at this stage will be corrected. An example of this is if the numbers are added to an incorrect column or used in different intervals during the test. This could lead to some or all of the experiments being flawed.
“FIGS. 19A and 19B illustrate an example spreadsheet to record the statistics. This example shows weekly monitoring. FIGS. FIGS. 19A and 19B might contain more information or performance metrics than what was selected. However, it is possible to track additional information. If you are interested in analyzing other performance metrics, the additional information might be useful.
If weekly monitoring was done, a separate spreadsheet is shown in FIGS. For each week of the test, 19A and 19B can be used. Each advertisement may have a time interval. This could be the dates that correspond to the first or last day of the interval. The?Data 1? column of FIG. FIG. 19 would include the data for advertisement number 1. 19 would contain the data for advertisement number 1 FIGS. FIGS. 20-23 illustrate the example spreadsheet in FIGS. 19A and 19B, with the results from four different one-week time periods.”
“Returning back to FIG. 17 may be monitored and tracked. It may be helpful to review the statistics and look for any irregularities. Sometimes, unexpected fluctuations in data or performance metrics can occur. Potential noise factors could be at play if an online advertisement campaign receives impressions between 330,000 to 370,000 per week and then suddenly sees 120,000 impressions drop in a given time period. These noise factors, and/or uncontrollable incidents can cause bias in the test results. One approach is to extend the testing period or not use statistics from the time period in which noise factors and uncontrollable events could have affected the test.
“As shown in FIG. 2A. The data resulting from the experiment at 130 can be analysed at 140 after it is completed. FIG. FIG. 24 illustrates a flowchart that describes an example approach to analyzing test data. The validity of the statistical data can be checked at 2410. Data interpretation can be used to determine the factors and/or levels that should be combined in order to improve the performance metric(s).
“In one approach, a statistical ratio called the S/N ratio can be used to draw inferences about how to interpret the test results. It generally seeks out to minimize or decrease variation in the outcome. It is important that the S/N ratio be as high and as consistent as possible. This will allow for performance metrics such as conversion rates, which do not significantly fluctuate over time. Calculating the S/N ratios at each factor level is one way to determine which factors are most responsible for variability. These S/N ratios can be compared to show that certain factors have higher ratios than others.
“FIG. “FIG. 25″ shows an example spreadsheet that can be used to calculate S/N for the entire testing period. One approach to calculating the S/N average is to determine the average performance metric for each advertisement. This could be done by taking the average conversions and summating the performance metrics. FIG. 25 shows that the sum of all conversions is 0.027198148. 25 shows that the average conversions add up to 0.027198148. The S/N average may be calculated for the average conversions by:\n10*LOG 10(average conversions/(1?average conversions))\nwhich for the example shown in FIG. 25 may be written as ”
“Next, determine the S/N ratio for each level. As described in FIG. 25, this can be done as follows: 25 where each factor is considered separately. Level 1 of factor 1, 2 of factor 1, 3 of factor 1, and so on. FIGS. FIGS. 26 and 27 provide spreadsheets that can be used to calculate S/N ratios for various performance metrics, such as conversions and clicks. FIGS. 26 and 27 respectively,?L1? 26 and 27,?L1???? and?L2??, respectively. These are the levels 1 and 2. The number of clicks, e.g. The number of clicks (e.g., the number of times that a client selected the advertisement) can be counted across the entire test. There may be other factors involved. Each factor may contain an equal number level 1 and 2 specifications, as shown in factor?B? FIG. 11. FIG. FIG. 28 shows?Description line 1? Levels 1 and 2 correspond to factor?B? The profile. Advertisement numbers 1, 2, 5, and 6 show the factor at level 1. The data below?Sum clicks? is also shown. adapted from FIG. 25 may be added to the table in FIG. 25 may be supplied to the table of FIG. This operation can be repeated for advertisement number 3, 4, 7 and 8, respectively, with 10210, 96989, 2558 and 13671. The spreadsheets shown in FIGS. For almost any type of statistical data that results from the test, 26 and 27 can be created.”
“Preparation of the spreadsheets in FIGS. 26 and 27 describe the number and clicks required for each factor or level. These can be used to calculate the S/N ratios. As discussed above, the average S/N ratio is?Sum conversions? ?Sum Conversions? may be divided by?Sum clicks?. The quotient can then be used in the LOG formula. This operation can be used for all factors and levels, resulting in 14 S/N rates. FIGS. FIGS. 29A, 29B and 29C are examples of a spreadsheet that shows the?Sum clicks? Divided by the ‘Sum Conversions?” The resulting S/N ratios for each level and factor are shown.
“After each of these 14 ratios has been determined, the largest ratio S/N for each factor can be determined. This allows for a selection of most influential levels for each factor. FIG. FIG. 30 shows a spreadsheet that includes the highest S/N ratios for each of these factors. The?RHO?” is next. The?RHO? can be calculated by subtraction of the?MaxS/N for each factor With reference to FIG., subtract the average S/N ratio from the above. 25. The?SUM ROHO? can be derived by adding all the?RHO’s. The?RHO’s for each factor can be subtracted to create the??SUM RHO?. The?Influence? is the sum of all the?RHO?s for each factor. The?influence? is the level of influence that a factor or level has on the test and the performance metrics.
“For instance, the?Price?” FIG. FIG. 30 is the most influential. The price factor may be the most influential on a visitor’s decision to buy a product, as an example. In this case, level 2 was $40.
“Multivariate testing may produce confusing results in some cases. Although it may seem odd that visitors would rather buy the software product for $40 than $20, this scenario is possible. Visitors may have believed that the more expensive product was better than the cheaper one. FIG. 30 for?Description line 1? This may be due to statistical noise. In one approach, either of the levels could be used for the?Description line 1? However, this does not mean that a different value can be used for these levels.
These levels can be combined to create a super advertisement or improved advertisement. The multivariate testing approach to advertising has so far only predicted that the combination the most influential levels would result in an improved advertisement. One approach is to use the above-described data to predict the new performance metric for the advertisement. You can do this by subtracting the “Average S/N” ratio. The?Sum RO? is used to calculate the projected S/N. The projected S/N ratio in the above example would equal?10.355612, or the difference between 5.179238875 and 15.53485053. The average conversions can be calculated by inverting LOG formula. The predicted S/N ratio for the super advertisement predicts that the new super advertisement will have a conversion rate approximately 0.0843648, or 8.5%. Although only one super advertisement was created and tested in this example, it is possible to test multiple super advertisements. You could use, for example, three of the best advertisements.
“Returning To FIG. 24, these factors and/or levels can be combined to create the improved advertisement 2420. The new advertisement could include the following levels, taken from FIG. 30.”
“Factor Headline: Level 2”
“Factor Description Line 1: Level 1”
“Factor Description Line 2: Level 1”
“Factor Display URL: Level 2”
“Factor Price: Level 2”
“Factor Bonus Offer Level 2”
“Factor Capitalization Level 1”
FIG. 31. Sometimes, super advertisements may differ from the ones being tested. Multivariate testing can be used to identify the super advertisement by testing a small portion of all possible advertisements.
Summary for Multivariate optimization method for testing
When creating an advertisement or marketing campaign, marketers and advertisers are often faced with many considerations. The quality or type of responses generated by exposure to an advertisement in a specific environment can be affected by the content and layout of the advertisement. This article describes a method that uses multivariate testing to guide the creation of ads that are optimized for specific responses. A text advertisement on an Internet website is optimized using multivariate testing. Multivariate testing can also be applied to different media formats.
Multivariate testing is a method for improving an advertising campaign or marketing campaign. Multivariate testing can include one or more of these methods: Optimal Design (Discrete Choice), Optimal Design (Taguchi Method), and the Monte Carlo Method.
“A multivariate testing approach to internet or web-based advertising is used in the non-limiting example. It should be noted that these methods can be used to improve advertising and/or marketing campaigns in a variety media formats, such as: Internet page design, email banner advertisements, Internet text links advertisements, text messaging, direct and courier mail, internet marketing, signage, radio and television, personal digital assistants (PDA), cell phones, global positioning system (GPS), in-vehicle displays and telephonic marketing via mobile phones, pagers and telephones or voice over Internet protocol(VOIP), and other media forms, 1A. FIG. FIG. 1A illustrates how a multivariate test approach indicated at 219 may be applied initially to advertising and/or market campaigns indicated at 180 to improve advertising and/or market campaigns indicated at 194.
The advent of online advertising has created a highly competitive market for advertising. Advertisers find internet-based or webpage-based text ads, such as the ones sold under the brand ADWORDS from GOOGLE, both extremely accessible and highly efficient. Businesses that are already established on the Internet market face some disadvantages due to this accessibility and efficiency. New competitors may be able to more easily access the Internet market through web-based advertising, which can reduce the market share for some businesses.
An example of Internet software sales will be shown in order to illustrate the different approaches to improving advertising campaigns. FIG. FIG. 1B illustrates an example computer network system 180. A web server 160 can communicate via the Internet 150 with a number of clients (in this case, 170-178). This is an example of a wide-area network. One client device can be included in each of the network clients 170-178. This could include a mobile phone, a portable digital assistant (PDA), or any other device. These devices can include a graphical display to display web pages to clients. In this example, web server 160 also includes memory 162, which can store computer-readable instructions that web server 160 and clients 170-178 can execute. Web server 160 can also be configured to display advertisements on the different web pages that are displayed by clients.
In this example, a company is advertising a software product that can automatically archive office documents at $20 through an internet campaign. The business began running online advertisements several months ago. There were no competitors, and one single advertisement was enough to draw customers. The cost-per-click (CPC), at that time, was only 5 cents. Each visitor to the website earned the business an average of 80c. This led to a profit margin that was 75 cents per visitor. The business experienced a sharp drop in sales one year later. The business first assumed that the advertisement was not correct. An examination of the online advertisement was therefore conducted. A GOOGLE search query was run as part of the examination. It turned up four advertisements, three from competitors, in the result. While GOOGLE is referred as an example search engine or advertising source throughout, the methods, systems and acts described herein can be applied to other search engines, mail marketing and/or advertising media. In this case, not only was there more competition but also the search rank of the advertisement for the business was lower, from the first to the third position. Each of these new ads was examined by the business and revealed that they were selling substantially the same software product. The business was left with the dilemma of how to fix the situation after concluding that the sales decline was at least partly due to increased competition.
Similar scenarios to this one can occur with almost any product and in any media. Sometimes, it may be difficult for businesses to break even due to the high cost-per-click associated with certain Internet search companies. Companies with large advertising budgets are able to afford to pay more per click to get a higher rank. This makes it harder for smaller companies to compete in this market.
Internet marketers are faced with the challenge of increasing marketing returns at minimal or no cost. An alternative approach is to increase marketing return. Advertising costs can be directly linked to product profit margins. The software company in the above example had a profit margin of 75c per visitor to its website over the past months. The business lost a significant market share to its competitors one year later. The business’s advertisement fell to the third position after it was discovered that competitors were paying higher maximum costs per click for each keyword.
One way to improve their marketing campaign is to increase the advertising budget. This could be done by raising the maximum cost per click up to 50cs. This may allow the business to compete with those of higher rankings. If the company can spend more advertising and has a higher profit margin then they may be able improve their marketing position through an increase in their search rank. The business might not be able to spend enough to attain a higher ranking. The business might decide to raise the cost-per click to 75c, which is the maximum amount that could be afforded while not making a loss. However, this might still not be enough to achieve the highest ranking advertising position. This scenario may lead to the conclusion that a competitor with a higher ranking may be more successful at converting customer traffic into customers. This is how escalating advertising costs and increased competition can threaten a business’s success. While it may work in certain cases to increase advertising spending by increasing profit margins and outbidding competitors, it is not always the best option.
A business might improve its profit margins to be ahead of their competitors by optimizing or optimising the marketing campaign. Businesses would normally allocate a portion of their budget to develop a new, improved marketing campaign that is based on focus groups surveys. Focus group surveys can sometimes provide inaccurate or misleading data, which could lead to a business launching a wrong-directed marketing campaign or advertising campaign.
Advertising campaigns can be more effective if they are based on actual customer responses in real market conditions. Multivariate (MVT), or multivariable, testing can be used to reduce the number and cost of experiments, which in turn will result in a more effective advertisement. This will allow the business to compete for higher search rankings by improving performance metrics like conversion rate. A conversion rate of 4 percent is, as an example, four out of 100 visitors to the website that purchased a product.
Customers can choose from a variety of advertisements to decide if they want to respond to an advertisement. This can give a better indicator of the effectiveness or ineffectiveness of advertising than traditional focus group surveys. This approach can also be applied to web-based advertising media such as landing pages (i.e. The web page to which a user is directed after clicking on an advertisement online or via the internet.
“The methods described in this document can be divided into four operations. FIG. FIG. 2A is an example flowchart that briefly describes these four operations. A more detailed description will be given below, with reference to FIGS. 2B, 10, 17 and 24. FIG. The method could begin with the creation of a hypothesis at 110. The brainstorming session may involve discussing factors such as test requirements, test factors and possible threats to test robustness. The experiment can then be set up at 120. This may include the initialization of test, applying the statistical methodology, transformation of selected variables into advertisements and placement of the advertisement. The experiment can be completed at 130. This may include documenting and monitoring the data and statistics, as well as observing and tracking any noise factors that could impact the experiment. The experiment results can be analysed and interpreted to determine their validity. They may then be implemented at 140 to create an improved “super?” Advertisement, which confirms the predictions made during the experiment.
“As discussed above with reference to FIG. 2A. A hypothesis can be created before the experiment is designed, performed, and analysed. FIG. FIG. 2B illustrates, at 110, a flow diagram that explains in more detail how to develop a hypothesis. One or more performance metrics may be chosen that can be improved or optimized starting at 210. For web-based advertising, performance metrics may include click-through rates (CTR), (e.g. The rate at which clients click on or select the advertisement, or the conversion rate (CR), (e.g. The rate or proportion of clients who purchase the advertised product/engage in the offer presented by an advertisement. An average visitor’s (e.g. a client’s) value, response rate, etc. The click through rate, or CTR, may be used as a performance metric to determine the success of an online advertisement campaign. CTR can be calculated by multiplying the number users who clicked on an advertisement placed on a website by the number times that the advertisements were delivered (i.e. impressions). If the advertisement was sent 100 times (i.e. 100 impressions were delivered), and one person clicked on the advertisement (i.e. clicks recorded), then the CTR would be 1%. Clicks recorded), the resulting CTR is 1%. The conversion rate can also refer to how many visitors purchased products from a webpage, based on the total number of visitors to that webpage. A conversion rate of 10% means that 1/10 visitors to a webpage make a purchase.
While the example above is for Internet advertising, CTR/CR metrics may be similar or different in other media formats. Direct mail marketing metrics may include metrics like response rate, which measures the number of recipients who respond to a specific direct mail advertising campaign.
A business might want to improve their landing page and/or web advertisement conversion rates. It may be beneficial for a business to determine their past and/or present conversion rate to compare with future conversion rates due to adjustments to their advertising campaign. A business might want to improve multiple performance metrics. In this case, they would identify and choose the performance metrics to be improved. This allows you to compare the performance of the previous advertising campaign with the new one.
“At 212 potential test factors may be identified and their levels determined. Before we go into detail about 212, it is helpful to look at some background information about multivariate testing. Multiple terms will be used to refer to the various parameters of the multivariate testing. As used herein, a factor may refer to one entity or a portion of an advertisement. This could potentially impact the output of multivariate testing. A factor may also include media type or format. One factor could be, for example, the amount or magnitude of the price included in an advertisement. A factor could also include a graphic or photograph. A body of text may also be included in a factor.
“Further, there could be more than one level within a factor. The specification of the factor may be included in the levels. An advertisement that uses a price factor like price may have several levels. These levels could include the quantity of the price, such as $100 or $200. Levels may also include information about the image type, copy, content, and text size. The media format is a factor. This means that the different levels can include multiple formats, such as billboard ads, direct mail, television advertisements, direct mail, and television advertisement. The non-limiting examples and approaches discussed herein may not be limited to one media format. They may also include multiple media formats.
“FIGS. The 3rd and 4th versions of the example Internet advertisement show the first and second versions, respectively. They also include a variety of factors and their respective levels. FIG. FIG. 3 shows an Internet advertisement in its first version 310. It includes a title 320 and an abstract with a first and second line 330, 340 and 350. Similarly, FIG. FIG. 4 shows the second version 410, which includes a title 420 and an abstract with a first and second line 430, respectively, as well as a URL address of 450. One example is that the title could be a first factor with a level 1 shown at 320. This would be called?automatic backup?. A second level is shown at 420, which can be referred to as “software makes life easier?”. The URL address, however, may only have one level. For example, it could include?carsonspage.com. 350 to 450 Some factors can include multiple levels while others may only include one level.
“Returning back to FIG. 2B at 212, the potential factors and their corresponding levels may be selected. The selection of testing levels and potential factors may be an important step in the optimization process. These levels and the factors may help to define the test structure. One example is that levels can be chosen to target specific groups of potential customers. Internet advertisements might target consumers more likely to use the Internet. Television advertisements may target viewers who watch television. Direct mail advertisements could target consumers who don’t necessarily have access to the Internet or television.
It may therefore be beneficial to gather a team of competent people for a brainstorming session in order to identify potential factors and levels. People with experience selling the product might be more suited to selecting potential factors and levels that could directly affect the results of the test. The brainstorming session could include sales staff, accounting, customer service, and marketing personnel. In some cases, however, even those with a minimal marketing background may come up with some of the most innovative ideas. It may be beneficial for those who are selecting the possible factors and levels to be open-minded and willing to test new slogans and ideas in order to obtain substantial results. It may be beneficial for at least one person in the brainstorming group to become familiar with specific advertising requirements before they begin.
“The performance metric(s), as defined at 210, may determine the factors and levels that are to be tested. It may be possible to choose factors or levels that could affect click-through rate or response rates, for example, if you are trying to improve a performance metric like click-through rate or response rate. Contradictory levels or factors can be eliminated or reduced in one approach. FIG. 5 shows an example. FIG. 5 illustrates an advertisement 510 that has factors 520 and540 which contradict each other. It may be possible to alter at least one factor or level so that they do not contradict each other.
“It may be possible to use substantially different levels within the same factor in another approach. FIGS. FIGS. 6 and 7 depict two very similar price levels, 620 and 720. A multivariate approach can help you create a better advertisement. It allows you to test substantially different price levels, such as $15 or $50. This is in contrast to the $15 and $16 advertisements. Another approach is to test the most relevant and/or stronger factors. If it is clear that certain factors or levels are not affecting the chosen performance metric(s), it might be possible to reduce/eliminate these factors/or levels from testing. You should remember that not all levels and factors can be tested. It may be possible to test the effect of national gross domestic products (GDPs) on the price level, but it may not be possible to test for its influence on religion, unemployment, gender, age, or sexual preference. Although some of these variables may have an impact on the test’s output, some factors or levels may not be possible to test due to information overload, insufficient data or otherwise unfeasible.
“Now that potential factors or levels have been selected at 212, it is possible to examine the selected factors and/or level in light of the guidelines above. At 214, it can be determined whether any levels or factors are inconsistent. Some levels might not be compatible with certain media formats, for example. If yes, you can select other factors or levels at 212. It can be determined whether the levels selected are significantly different from one another at 216. It may prove more beneficial to test different media formats, such as direct mail and the Internet, than traditional media formats like newspaper advertisements and magazine ads. For some situations, more levels can provide more useful and relevant results. Other levels can be chosen at 212 if the answer is “no”. It can be determined whether the levels and factors are relevant to the performance metrics at 210. If no, then other factors or levels can be chosen at 212. It can be determined whether the levels and/or factors are possible to test at 220. If no, then other factors or levels can be chosen at 212.
“The number of testing levels and factors at 222 can be determined based on the selections of potential factors and/or level. The extent of the test can be determined, for example, by selecting one or more potential levels and factors during brainstorming sessions or another method. The longer and more complex the experiment, the more factors and/or level that must be tested. Multivariate testing aims to reduce the number and complexity of tests to obtain comparable statistical information. However, it is possible to reduce the number and complexity of the factors or levels being tested. This may allow for a compromise between realistic time frames and the amount of knowledge about the impact of the factors on selected performance metrics.
“From 224 it can be determined whether the selected factors or levels are suitable for the testing timeframe and statistical information desired. If no, then the number of factors and/or levels can be increased or decreased. If multiple media formats are being used, one or more may be removed, which will reduce the number of levels associated to that factor.
An orthogonal array can be used to set up tests for different combinations of factors/levels. FIG. 8 shows an example of such an orthogonal array. FIG. 8 is a table that shows some examples of orthogonal arrays. For example, let’s say you select title, price and display URL address. Each factor can be at two levels (e.g. If you choose $100 as the price and $200 for display URL address, then the?L4′ option will be available. array could be chosen. The subscript number is?4. The subscript number?4? is located behind the capital?L? The capital?L? indicates the number of advertisements used (e.g. During multivariate testing, the actual test was performed. The test may include four different web ads if the L4 array was selected. An example of this is where four advertisements could be placed online via an Internet search site like GOOGLE. The first person to visit the website where the ads were placed will see the first of four advertisements. A second advertisement may be displayed to the next visitor. It could have a different combination of one or more factors. It is possible to observe how people react to advertisements by observing the reactions of different groups (e.g. Multivariate testing allows you to infer statistically the impact of the variables and/or limits on selected performance metrics.
“Some experiments can be used to compare the landing page and advertisement. This allows you to select factors and/or levels to determine how performance metrics may be affected by variations like text font, color, pictures or special offers. The L32 array would allow 31 factors to be evaluated for their potential impact on the desired performance metrics. This could result in 32 different landing page and advertisement combinations. Each person who visits the advertisement or landing page could see one of 32 combinations. You can observe the behavior of the person (e.g. When they are presented with different impulses (e.g. different advertisements, different landing pages, etc.) The influential factors and/or levels of influence may be determined.
“Multivariate testing may be used in some cases to determine the best combination of factors or levels that results in the most favorable (e.g. The performance metric(s), which are selected, may be the highest or lowest. Alternately, the multivariate approach could not be applied to tests involving 31 factors with 2 levels each. In such cases, 2,147,483,648 total or 231 advertisements could be tested. This may make it impossible to test all combinations. Multivariate testing allows for testing of far fewer combinations. For example, 32 combinations can be tested using the multivariate approach. Multivariate testing uses a mathematical and statistical approach to create orthogonal arrays. This allows for the testing of fewer combinations, which can reduce any potential loss of information. Other approaches can be used to identify trends at certain levels and factors using fewer tests.
Referring to FIG. 8 are mixed level arrays. The two arrays marked * with an asterisk (*). These arrays can be useful when some factors are tested with a number of levels, and others require testing with a number of levels that is different from the initial number. The L18 array, for example, has one factor with two levels and seven factors with three levels. The table in FIG. FIG. 8.8 shows some of the most commonly used arrays. However, it is important to remember that other arrays can be used depending upon the number of factors or corresponding levels.
“Returning back to FIG. “Returning to FIG. 2B, at 226, at minimum one array could be selected correspondingly to the number or levels of factors that were selected. If seven factors were brainstormed and each factor was assigned two levels, the L8 array might be chosen.
“At 228, selected factors and/or limitations may be compared with the requirements specific for the media format being tested. A web-based marketing campaign that uses a search engine like GOOGLE might have specific requirements for the advertisement. These guidelines may include style, editorial, and character limits. If they are not followed, the advertisement might not be approved. These guidelines prevent testing of levels or factors that are impossible to test.
“FIG. “FIG. 3. It is important to remember that different media formats might have different, more or less requirements than the one shown here. An example of this is the title or headline, which may limit an Internet advertisement to 25 characters. However, there may not be a limit for direct mail media formats like direct mailing. The maximum characters count for the description lines, destination URL, and display URL may be different. To comply with GOOGLE’s advertising policy, the text of the advertisement must be clear and concise, communicate with the audience, be easy to read and compelling without being misleading.
“Other requirements and recommendations may also be related to spelling, punctuation or grammar, grammar, capitalization. repetition, inappropriate language. unacceptable phrases, superlatives, prices, claims, discounts offers.
“Regarding spelling: Proper spelling is crucial to the credibility and clarity of an advertisement. The words should be correctly spelled in the advertisement. This is not true for commonly misspelled words and spelling variations. It may be acceptable if the word is found in an online dictionary. Customers and users should all be able to recognize the spelling error. In some instances, however, you may use any spelling.
“Regarding spacing: Adverts should be spaced between words and after punctuation. For example, C-h-ea-p C?l-o?t-h?e-s. It may not be permitted. Similarly, ?Free Shipping.Buy Now? Other restrictions may apply.
“Regarding punctuation. Punctuation should not be used to draw attention. It may not be necessary or repeated more than once in a row. The use of an exclamation point in advertisements may be subject to specific rules. An exclamation mark may not be included in the first line of advertisement text. In some cases, an advertisement can only contain one exclamation mark.
“Regarding grammar, in some cases advertisements must adhere to basic grammar guidelines. Advertisement text can use either logical sentences or phrase forms. Additionally, symbols, numbers or letters may be necessary to convey their true meaning. The advertisement may use them instead of words. You might see the following example:?We have many 4U online! could be considered a violation of the grammar policy. is substituting words.”
“Regarding capitalization. A word may not appear all the capital letters in order to draw attention to it or that phrase. For example,?FREE For example,?FREE? oder?NEW?? may not be allowed. This may not be permitted. Capitalizing the first letter of each word in your ad might be permissible.
“Regarding repetition: Repeated statements should not be used for promotion or in a gimmicky way. The same word cannot be repeated more than three times in a row. An example of this is an ad that says “Deals, Deals and Deals Here?” This may be prohibited. You can replace the advertisement title with text like “Amazing Deals here?” This policy must be followed.”
“Regarding inappropriate terminology: Advertisements, including the display URL may not contain language that could be considered offensive or inappropriate for some users. This could also be applicable to misspellings, selfcensored, and other forms of inappropriate language.
“Regarding unacceptable phrases: Certain ?call-to-action? If they aren’t descriptive of the product or service, phrases might not be used in advertisements. You might see phrases such as?click here? For example, phrases like?click here? and?visit our? These are not allowed. A good example of a?call to action? You might use the phrase?Order your Online Contacts Today? Because it represents the product and site content.
“Regarding superlatives: Superlatives refer to words that emphasise superiority. To ensure that users feel trustworthy and treated fairly, the advertisement text should not include subjective or comparative phrases like?Best?. Your website may need to clearly display this verification. If an advertisement claims it is the “Best of the Web”, the site might be required to show third-party verification. An example of this is a Forbes Magazine seal that indicates the site has won a best-of-the-web award. The ad could be accepted.
“Regarding competition claims: These claims claim that a product/service is superior to a competitor’s product/service. The landing page may need to support competitive claims made in the advertisement text. This builds trust and assures users that they are finding what they want based on the advertisement text. These claims can be supported in many ways, such as a table or chart that compares the product’s features with the competitor’s or a competitive analysis explaining why the product is better. Advertisement text that says “better than product A” could be an example. This could be considered a competitor claim and may need to be supported on the website. This claim could be accepted if the landing page contains a competitive analysis of advertiser A and product B. The advertisement might be approved.
“Regarding Prices: Specific prices may need to be supported within 1-2 Clicks of the landing Page. It is possible that prices in the advertisement text need to be exact. Bulk purchases may also be subject to pricing.
“Regarding discounts offers: Any discount offer that is displayed in the advertisement text might need to be supported within one to two clicks on the landing page. You may support discounts such as?50% discount on all items,?Save $20 when you make your first purchase, and similar phrases. Users may find free offers very attractive and may need to be supported within a few clicks on the landing page. It may be acceptable for the user to infer that the product or service is free even though the word “free” does not appear alongside it.
“These and other requirements of GOOGLE advertisements may be found at ?http://www.GOOGLE.com/ADWORDS/learningcenter/text/index.html?. You should remember that these requirements and guidelines are only a sample of what some web-based search sites like GOOGLE require. Other requirements could exist for different advertising services or media formats.
“Returning back to FIG. “Returning to FIG. If no, factors or levels not meeting the requirements can be adjusted at 228.
“At 232 potential threats to robustness (i.e. potential threats to robustness (i.e. noise factors) may be identified at 232. These potential threats and noise factors could be related to factors that can be controlled. It may be possible to give specific values to controllable factors depending on what you want. If pricing is one of the factors, for example, the price could be set at $100, $200, or any other price you choose. A factor could include an entity, or part of an advertisement that may have an influence on the outcome of the selected performance metrics (such as click-through rate, conversions, etc.).
“In some cases, random events may occur that could directly impact whether people respond to an advertisement. A server could crash, leading to a decrease in sales or traffic. Another example is a competitor’s sudden emergence that could affect the experiment. Uncontrollable events could pose a threat to the experiment’s robustness.
“Unfortunately, some extraordinary events might not be included in the test design. They can have a serious impact on test results, depending on how large they are. A server that is down for more than an hour may not have the same impact as a competitor who appears. Multiple repetitions of the same experiment can be used to overcome this problem. More repetitions of an experiment will increase the likelihood that the results will be reliable, which in turn increases the validity of the results. A test could be conducted for one week, during which there is no competitor for the product being promoted. The results could be different if the test is repeated the next week with two new competitors. To combat this, you can run the test again the next week. To increase the certainty of the results, each repetition can be used. There is a tradeoff between certainty and time. It may be worthwhile to establish how certain the results of the test must be. It may be necessary to determine how certain the test results must be. For example, two repetitions of the test are sufficient to ensure that 85% of the results are valid. Or, does the test require 95% certainty, which would mean five additional weeks of testing. This decision may vary from one case to another and it may not make sense to establish strict guidelines.
“As discussed above with reference to FIG. “As described above with reference to FIG. 2A, an experiment can be designed at 120. The advertisements will be arranged in the correct format. The test could produce misleading or incorrect interpretations if the levels and selected factors are not properly translated into the advertisement format.
“FIG. 10. This flow chart shows an example of how to design the experiment at 120. Based on the array, 1010 can be used to select the best combinations profile. 8. The array at 226 in FIG. may determine the resulting set of unique ads. 2B. 2B. Each array in FIG. Each of the arrays in FIG. 8 may include a corresponding profile for standardized combinations, which is a subset of all possible combinations. These profiles can be found in statistics or mathematics books for each of these arrays. If the L8 array was chosen for testing, the profile of the L8 combinations may be selected, as shown in FIG. 11.”
“On the vertical-axis under?AD number? These numbers correspond to the eight advertisements that can be tested. Horizontally, the row to the right from?Factor? shows seven factors (?A??-?G?). These were chosen for the L8 array. The L4 array would be chosen instead. This combination profile would have four different advertisements along the vertical axis, and three different factors along the horizontal. The numbers?1? and?2? are located on the inside of this table. The numbers?1????? and?2??zunehmen Le dar Get get Da da an bei all des Des darin Dec Und Dar The levels to be tested are indicated by?2? It is possible to look at the table column by column (?A-?G?) in order to check for errors. Each of the??1?s and?2?s should be counted. One approach is to count each column as half a?1 and half a?2 or, in this example, as four??1?s. This indicates that the levels were evenly used. Some approaches might use different levels for all or some of the factors. You should also verify the number of ads being tested. The vertical axis should have eight advertisements. This is done by dividing the number of levels per factor, which in this case is two.
“At 1012 the selected factors or levels can be arranged as shown in FIG. 12 refers to an example advertisement for a software product. These factors and levels could correspond to the levels and factors selected in FIG. 2B. There are seven factors that can be used to test the L8 array. Level 1 and 2 indicate different specifications or characteristics for each factor. Bold text indicates the location of the advertised price and bonus offer levels.
“At 1014 the factors and/or level, for example, as shown in FIG. As per the combinations profile in FIG. 12, 12 can be added to an L8 combinations templates FIG. 11 15. This operation requires care as it can affect one level of the advertisement. One approach is to add the factors to the L8 combinations profile. However, the inside of the table containing the??1?s or??2?s can be temporarily left empty. The blank table that will receive the selected levels can be called the template, while the table with the?1’s and??2?s can be called the profile. FIG. 11.”
“Column A is the combinations profile of FIG. 11 is the first factor with its specifications (levels), in each of eight different advertisements. Column A may be renamed “headline” in the profile. FIG. 13. FIG. 11 The profile in column A shows that advertisements numbers 1-4 have each been given a level 1. FIG. FIG. 12 shows that the headline factor level 1 includes the text “backup software?”. The template may be modified in the column renamed?Headline? Advertisements 1-4, as shown in FIG. 13.”
“One thing you should be aware of is that the factor includes different levels of capitalization. You might consider adding “backup software?” It is possible that the template might not be correctly filled in with lowercase letters. To avoid making mistakes, you can fill in the variables that affect other factors first. Capitalization can be added to the?Headline?” For example, The level of the factor (e.g. Depending on the level (e.g. ?Display URL You may change the capitalization of text. In the profile, factor G corresponds to?Capitalization?. Column G can be renamed in the template to?Capitalization? Fill in the form using lowercase letters for all fields that have a?1? Fill in the profile using lower case letters in all fields that have a?1? in the profile. This may be repeated for columns F and E, which correspond to factors like?Price? The?Bonus Offer? and column E may be repeated. The?Headline?” factor requires that the four first advertisements with a?1? suffice. The template can be filled in.”
“The four remaining advertisements have level 2 as specified in the profile for factor?A? Which was again factor?Headline In the template. The level?2 is an example. This factor can be used to replace a keyword in GOOGLE ADWORDS. This means that if an advertisement is triggered using a specific keyword, it will appear boldly in the GOOGLE advertisement headline. This functionality may make capitalization irrelevant as the keyword will be displayed in the same capitalization that the search query was. Therefore, ?dynamic? The remaining four rows may have?dynamic? This functionality allows you to specify a default headline if an advertisement is triggered using another keyword. Again, capitalization is not important. Capitalization may not be relevant if the default value is set at?1? This factor can be set to?1 as the default value, and capitalization may also be adjusted accordingly. For more information on how to implement this functionality, please refer to the GOOGLE manual located at the URL address. You can rename column B in the template by replacing?Description line 1 with column B. The combinations profile of FIG. may also be used to add the appropriate levels. 11. This is now affected by levels of?Capitalization? This factor is now influenced by the levels,?Capitalization? As shown in FIG. 14. This is the advertisement number 1 in this template. of factor ‘Description Line 1? at level?1? Factor?Price? Level?1? Capitalization factor could be used.”
“Continue with FIG. 10 after adding the factors and levels to the combinations template, as shown in FIG. 15 It is a good idea, as illustrated at 1016, to check frequently whether advertising requirements have been met. The factors and/or levels can be adjusted at 1018 if the answer is not yes. Dynamic elements, such as the?Bonus offer?, may be used. If dynamic elements such as the?Bonus Offer? are used, character counts may be exceeded. Example: advertisement number?2 In the template, ‘Description Line 2’ is used. You may add?Free Trial Reliable Professional? to the template. This is equivalent to 34 characters. If the?Bonus Deal? contains 11 characters, then only 24 characters for the actual factor may remain. If the?Bonus Offer? contains 11 characters, only 24 characters may be left for the actual factor. After all the levels and factors have been added to the combinations template according to the profiles, the template should look something like FIG. 15.”
The combinations template can be used to create advertisements at 1020. To avoid confusion when you combine the different factors and their respective levels, it is recommended that each advertisement be written down. Advertisement number 1 should contain:
“Headline Backup Software”
“Description Line 1 – All formats supported for only $20”
“Description Line 2 – reliable, professional”
“Display URL: www.carsonspage.com”
“Similarly, advertisement number 2 in the web-based advertisement example should contain:
“Headline Backup Software”
“Description Line 1: All Formats Supported for Only $40”
“Description Line 2 – Free Trial, Reliable and Professional”
“Display URL: www.ArchivingExperts.com”
“Further advertisement number 3 should contain:
“Headline Backup Software”
“Description Line 1 – Just $20 to Protect Your Data!”
“Description Line 2 – Free Trial, Easy Backup Process”
“Display URL: www.CarsonsPage.com”
“At 1022, the advertising statistics must be reset or initialized in order to capture the influence of new advertisements that have been created using the multivariate test approach. This will allow for better monitoring of the test. For example, if a GOOGLE advertisement campaign had a single group of advertisements and a single ad, GOOGLE would show statistics on this advertisement from the time it was activated. Multivariate testing is intended to improve the campaign, or at the very least the selected performance metric. It may be necessary to delete or pause the advertisements from before. Sometimes, if advertisements are paused, GOOGLE may include the advertisement in the statistics, which could cause flaws in the test. It is recommended that you delete any previous advertisements or create a new campaign while the campaign is paused. It may also be beneficial to use the same keywords in the new campaign, so that tests can be comparable. The advertisements can be submitted at 1024 and the testing phase can begin. FIGS. 8 and 9 show the eight advertisements used in this example web campaign. 16A-16H, respectively.”
“As explained above with reference to FIG. 2A. After the experiment has been designed, the experiment can be conducted at 130. Here, the advertisements will be submitted and test data gathered. FIG. FIG. 17 is a flowchart that describes an example of how to perform the experiment at 130. The experiment data and statistics may be monitored at 1710.
“At this stage, online ads have been submitted and the results can be measured. This is one advantage of the herein-outlined marketing strategy. Some marketing strategies are not easily quantifiable. A company may experience an increase in revenue, but they may not be able to explain the reason. It is possible to determine the success of new advertisements by comparing their performance to those of prior advertisements. It may be possible to find past results and statistics for web-based campaigns that use GOOGLE. FIG. FIG. 18 illustrates an example of statistics from a web-based advertising campaign using Google. The conversion rate was about 4 percent during this period.
“At 1720 the test statistics can be documented, including the baseline statistics from the previous advertising campaign. You may want to establish a time period in which you can track the results. It may be based on how many clicks are received in a given time frame. It may not be necessary for advertisers to keep track of the results every day if they receive only ten clicks each day. You could record the results weekly, or if there are more than 50 clicks per day, you can keep track of the results each day. Another approach is to record the results every week. This reduces the potential noise factors that could affect test results. People respond differently to ads on weekends. The weekend is one example of noise that can affect the results. Some advertisements may get more clicks on weekends than others during work days. It is also recommended to keep the statistics in order in order to monitor them. It is unlikely that mistakes made at this stage will be corrected. An example of this is if the numbers are added to an incorrect column or used in different intervals during the test. This could lead to some or all of the experiments being flawed.
“FIGS. 19A and 19B illustrate an example spreadsheet to record the statistics. This example shows weekly monitoring. FIGS. FIGS. 19A and 19B might contain more information or performance metrics than what was selected. However, it is possible to track additional information. If you are interested in analyzing other performance metrics, the additional information might be useful.
If weekly monitoring was done, a separate spreadsheet is shown in FIGS. For each week of the test, 19A and 19B can be used. Each advertisement may have a time interval. This could be the dates that correspond to the first or last day of the interval. The?Data 1? column of FIG. FIG. 19 would include the data for advertisement number 1. 19 would contain the data for advertisement number 1 FIGS. FIGS. 20-23 illustrate the example spreadsheet in FIGS. 19A and 19B, with the results from four different one-week time periods.”
“Returning back to FIG. 17 may be monitored and tracked. It may be helpful to review the statistics and look for any irregularities. Sometimes, unexpected fluctuations in data or performance metrics can occur. Potential noise factors could be at play if an online advertisement campaign receives impressions between 330,000 to 370,000 per week and then suddenly sees 120,000 impressions drop in a given time period. These noise factors, and/or uncontrollable incidents can cause bias in the test results. One approach is to extend the testing period or not use statistics from the time period in which noise factors and uncontrollable events could have affected the test.
“As shown in FIG. 2A. The data resulting from the experiment at 130 can be analysed at 140 after it is completed. FIG. FIG. 24 illustrates a flowchart that describes an example approach to analyzing test data. The validity of the statistical data can be checked at 2410. Data interpretation can be used to determine the factors and/or levels that should be combined in order to improve the performance metric(s).
“In one approach, a statistical ratio called the S/N ratio can be used to draw inferences about how to interpret the test results. It generally seeks out to minimize or decrease variation in the outcome. It is important that the S/N ratio be as high and as consistent as possible. This will allow for performance metrics such as conversion rates, which do not significantly fluctuate over time. Calculating the S/N ratios at each factor level is one way to determine which factors are most responsible for variability. These S/N ratios can be compared to show that certain factors have higher ratios than others.
“FIG. “FIG. 25″ shows an example spreadsheet that can be used to calculate S/N for the entire testing period. One approach to calculating the S/N average is to determine the average performance metric for each advertisement. This could be done by taking the average conversions and summating the performance metrics. FIG. 25 shows that the sum of all conversions is 0.027198148. 25 shows that the average conversions add up to 0.027198148. The S/N average may be calculated for the average conversions by:\n10*LOG 10(average conversions/(1?average conversions))\nwhich for the example shown in FIG. 25 may be written as ”
“Next, determine the S/N ratio for each level. As described in FIG. 25, this can be done as follows: 25 where each factor is considered separately. Level 1 of factor 1, 2 of factor 1, 3 of factor 1, and so on. FIGS. FIGS. 26 and 27 provide spreadsheets that can be used to calculate S/N ratios for various performance metrics, such as conversions and clicks. FIGS. 26 and 27 respectively,?L1? 26 and 27,?L1???? and?L2??, respectively. These are the levels 1 and 2. The number of clicks, e.g. The number of clicks (e.g., the number of times that a client selected the advertisement) can be counted across the entire test. There may be other factors involved. Each factor may contain an equal number level 1 and 2 specifications, as shown in factor?B? FIG. 11. FIG. FIG. 28 shows?Description line 1? Levels 1 and 2 correspond to factor?B? The profile. Advertisement numbers 1, 2, 5, and 6 show the factor at level 1. The data below?Sum clicks? is also shown. adapted from FIG. 25 may be added to the table in FIG. 25 may be supplied to the table of FIG. This operation can be repeated for advertisement number 3, 4, 7 and 8, respectively, with 10210, 96989, 2558 and 13671. The spreadsheets shown in FIGS. For almost any type of statistical data that results from the test, 26 and 27 can be created.”
“Preparation of the spreadsheets in FIGS. 26 and 27 describe the number and clicks required for each factor or level. These can be used to calculate the S/N ratios. As discussed above, the average S/N ratio is?Sum conversions? ?Sum Conversions? may be divided by?Sum clicks?. The quotient can then be used in the LOG formula. This operation can be used for all factors and levels, resulting in 14 S/N rates. FIGS. FIGS. 29A, 29B and 29C are examples of a spreadsheet that shows the?Sum clicks? Divided by the ‘Sum Conversions?” The resulting S/N ratios for each level and factor are shown.
“After each of these 14 ratios has been determined, the largest ratio S/N for each factor can be determined. This allows for a selection of most influential levels for each factor. FIG. FIG. 30 shows a spreadsheet that includes the highest S/N ratios for each of these factors. The?RHO?” is next. The?RHO? can be calculated by subtraction of the?MaxS/N for each factor With reference to FIG., subtract the average S/N ratio from the above. 25. The?SUM ROHO? can be derived by adding all the?RHO’s. The?RHO’s for each factor can be subtracted to create the??SUM RHO?. The?Influence? is the sum of all the?RHO?s for each factor. The?influence? is the level of influence that a factor or level has on the test and the performance metrics.
“For instance, the?Price?” FIG. FIG. 30 is the most influential. The price factor may be the most influential on a visitor’s decision to buy a product, as an example. In this case, level 2 was $40.
“Multivariate testing may produce confusing results in some cases. Although it may seem odd that visitors would rather buy the software product for $40 than $20, this scenario is possible. Visitors may have believed that the more expensive product was better than the cheaper one. FIG. 30 for?Description line 1? This may be due to statistical noise. In one approach, either of the levels could be used for the?Description line 1? However, this does not mean that a different value can be used for these levels.
These levels can be combined to create a super advertisement or improved advertisement. The multivariate testing approach to advertising has so far only predicted that the combination the most influential levels would result in an improved advertisement. One approach is to use the above-described data to predict the new performance metric for the advertisement. You can do this by subtracting the “Average S/N” ratio. The?Sum RO? is used to calculate the projected S/N. The projected S/N ratio in the above example would equal?10.355612, or the difference between 5.179238875 and 15.53485053. The average conversions can be calculated by inverting LOG formula. The predicted S/N ratio for the super advertisement predicts that the new super advertisement will have a conversion rate approximately 0.0843648, or 8.5%. Although only one super advertisement was created and tested in this example, it is possible to test multiple super advertisements. You could use, for example, three of the best advertisements.
“Returning To FIG. 24, these factors and/or levels can be combined to create the improved advertisement 2420. The new advertisement could include the following levels, taken from FIG. 30.”
“Factor Headline: Level 2”
“Factor Description Line 1: Level 1”
“Factor Description Line 2: Level 1”
“Factor Display URL: Level 2”
“Factor Price: Level 2”
“Factor Bonus Offer Level 2”
“Factor Capitalization Level 1”
FIG. 31. Sometimes, super advertisements may differ from the ones being tested. Multivariate testing can be used to identify the super advertisement by testing a small portion of all possible advertisements.
Click here to view the patent on Google Patents.