The Google Analytics team recently launched a feature called “content experiments” . I believe this feature is an amazing boon to a Product Manager struggling with the challenge of what design or layout to build for a particular improvement to a website page and prove that it works for the customer. This feature is extremely helpful for an e-commerce website and I almost feel that the Google Analytics team smartly identified this need and came up with a modified solution tailor made for e-commerce retailers (although it can literally work for anyone else).
Google Analytics is probably the most preferred start-up analytics tool as it is free and comes loaded with a lot of features that helps any business measure its performance effectively. Having used Omniture, WebTrekk and Google Analytics at different times in my career, I have come to respect Google Analytics as a very user-friendly tool. While top-notch analytics and highly insightful reports can be generated with a great degree of accuracy using Omniture or WebTrekk, those tools are highly expensive to purchase. They are also very complicated to use. Google on the other hand gives a whole bunch of standard reports that give the complete picture on the performance of a business. Any analytics tool is usually confusing to use and a lot of insights that are generated from metrics such as pageviews, bounce rate, exit rate, conversion rate and so on should only be interpreted to the extent that it is useful to make good business decisions. A lot of noise is generated in analytics and a Product Manager should not make brash decisions merely based on a certain metric they have analyzed.
Coming back to this new feature called “content experiments”, Google defines it as a A/B/n test that one can conduct on say a product page of an e-commerce website. The flexibility comes from the fact that one can test multiple options of the same page and at the same time, also test various combination of components displayed in those pages. This, in my mind, is a combination of both a A/B test and a multi-variate test. The blog world is still confused with what content experiments can really allow with many accusing Google that they no longer will be able to conduct multivariate testing!. I believe that content experiments may not be exactly similar to a multivariate test, but the option to conduct an experiment with five variations of a single page allows a smart tester to come up with the right amount of changes that can be effectively tested with the customers. Google is doing away with Website Optimizer and slowly integrating content experiments into Google Analytics as the future of testing for its users.
While recently working on some new variations of a product page, the design process with the UI team led to the realization that subjectively speaking, there was more than one ideal variation of the product page that people liked in the company. This is a very common situation that a product manager faces in any organization. My team of Product Managers smartly came up with the idea of having specific event-based tracking across various CTA, buttons, content and links on the product page. This, we hoped, will allow us to use Google Analytics, look up under Events and track how each of the various components in the product page performed. This could then help us determine what components (or images or content or features) was widely used or accepted by a customer. This is a powerful tool for Product Managers to shut highly opinionated HIPPOs and other noisy characters in an organization from talking out of turn. Because, we now have data (however accurately representative it may be of the absolute truth) to silence the critics.
However, we were still left with one particular challenge. We had glaringly different design approaches that we couldn’t nail down for the product page. So, an A/B test was finalized with two different versions of the product page. If we had content experiments available, we could have actually used the various combinations we came up with and tested more than two combinations of the product page in one go. Given that we can randomly display these variations of the product page to different segments of visitors, we would have easily determined which version of the product is the winner. In fact, testing different designs and layouts of banners on a website (in home page, category page etc.) can also now be achieved in a very effective manner.
Now, content experiments in itself cannot be called as a game changer as it is not introducing anything new in the analytics market that doesn’t exist today. In fact, Google’s website optimizer can help one achieve almost similar results. But, Content Experiments is going to make Google Analytics a one-stop shop for all needs that an internal analytics or Product team in a company has by making it easy to create experiments helpful in making data-driven decisions on website changes.