Technology and Society Book Reviews

Link to index page Link to privacy reviews Link to commerce, security, and the law reviews Link to culture reviews. Link to politics, security, and the law reviews Link to ethics, rhetoric, and metaphysics reviews. Link to science fiction reviews.

Title: The Power of Experiments

Authors: Michael Luca and Max H. Bazerman

Publisher: MIT Press

Copyright: 2020

ISBN13: 978-0-262-04837-8

Length: 211

Price: $25.95

Reviewer note: I received a promotional copy of this book.

Many of us spend a significant portion of our lives online. Marketers, web designers, and product developers take full advantage of that exposure by constantly testing which pitches, nudges, and products drive consumer actions (and revenue) most effectively.

In The Power of Experiments, Harvard Business School professors Michael Luca and Max H. Bazerman describe how organizations use the data available from sources such as web traffic, transactions, and surveys to evaluate their campaigns.  They start with a well-known example from behavioral science: a 2010 effort by Her Majesty’s Revenue and Customs (HMRC) in the United Kingdom to improve citizen compliance in filing tax returns.

Encouraging Beneficial Behavior

The goal of the experiment, initiated by Behavioural Insights Team leader David Halpern and actualized by Michael Hallsworth, was to determine which of several versions of a reminder letter resulted in the highest compliance rate. After the trial produced a clear winner, the team kept experimenting by varying the initial address (first name versus Mr. or Mrs.) and appeals such as “Most people with a debt like yours have paid it by now”.

The success of the HMRC effort points to the value of similar experiments, but it took time for researchers in some fields, particularly economics, to accept that approach. Early work on prospect theory by Daniel Kahneman and Amos Tversky in the late 1970s and early 1980s illustrated that individuals don’t use strict utility (also called expected utility or expected value) to come to purely rational decisions.

These discoveries gave birth to the field of behavioral economics, which addresses how changes in stimuli such as problem framing, color of a button, or other inputs affect human behavior. Much of the rest of The Power of Experiments describes campaigns implementing techniques from behavioral economics in the policy realm, technology sector, and as a means for promoting socially beneficial behavior.

Case Studies

Several examples, such as Facebook’s manipulation of user timelines to induce emotional responses or Uber and Lyft changing their algorithms to change how easy it was for customers to get a ride are probably familiar to readers who are even a little familiar with online marketing and experimentation. One example I wasn’t familiar with was the experiment StubHub, an online ticket reselling platform owned by eBay, conducted regarding how it presented its fees.

StubHub makes its money by charging buyers for shipping and handling as well as a service fee. The question was whether StubHub should present the total price, fees included, at the start of the transaction, or whether it should hide the fees until the end of the transaction. StubHub tested each presentation over ten days, which the authors note is a typical length for this type of experiment, and assessed their results.

It turned out that buyers were more willing to purchase tickets if the fees were hidden until the end of the transaction. Rather than seeing the total price at the start and deciding to abandon the purchase early on, it’s reasonable to argue the buyer had made it all the way to the last step of the process and was invested in completing the sale. Valuing time that can’t be reclaimed is an example of the sunk cost fallacy, but it is a common behavior.

Conclusions

The Power of Experiments is a terrific overview of how organizations, especially online companies but also government and other groups, can use A/B testing to optimize their results. Luca and Bazerman also discuss issues of ethics and consent that affect these efforts, perhaps limiting what some parties can or should do.

I believe The Power of Experiments is an excellent book for managers and executives who oversee online marketing or other behavioral elements of an operation. While most professional marketers will be familiar with the concepts and examples presented in the book, Luca and Bazerman’s easy style and approachable writing provide an excellent introduction or refresher for non-specialists at all levels. Highly recommended.

 

Curtis Frye is the editor of Technology and Society Book Reviews. He is the author of more than 30 books, including Improspectives, his look at applying the principles of improv comedy to business and life. His list includes more than 20 books for Microsoft Press and O'Reilly Media; he has also created more than 50 online training courses for lynda.com. He received his undergraduate degree in political science from Syracuse University and his MBA from the University of Illinois. In addition to his writing, Curt is a keynote speaker and entertainer. You can find more information about him at www.curtisfrye.com and follow him as @curtisfrye on Twitter.

 

Link to new reviews.Link to list of reviews by publisherLink to page with contact information.