Insights, The Marketing Technology Office

Black(out) Friday: How To Prevent A Best Buy Blunder

holiday_ecommerceFor retail giants across the U.S., the stakes don’t get much higher than Thanksgiving weekend. According to reports from ComScore, Thanksgiving day and Black Friday broke online sales records this year with $1 billion and $1.5 billion, respectively. Cyber Monday remained the largest online shopping day of the year, and surpassed all expectations this year to become the biggest online shopping day in recorded history, with over $2 billion in online sales.

Basically, Thanksgiving weekend is the Super Bowl. You run drills for it; you test and retest rich content elements for efficacy across multiple screens; you increase your server space. But inevitably, some retailers still fumble the ball. This year, it was Best Buy’s turn.

Constantly doing battle with both online giants like Amazon and big-box giants like Wal-Mart, Best Buy got tired of getting shellacked by the competition on retail’s holiest of holy days. So in 2013, they did some things about it. Opening its doors on Thanksgiving day for the first time, the store drew massive lines by offering Turkey Day-only deals that dropped TV prices below even the bargain-basement of Amazon. They even offered a price-match guarantee, which included online retailers. The strategy paid off, and the company easily made up the hit from the price drops, besting competitors in the process.

Clearly, a repeat of this performance was necessary for Best Buy to prove it wasn’t a fluke. Then this happened:



This message appeared to Best Buy consumers twice on Black Friday; once in the morning around 9am, lasting for about 2 hours, and then again around 5:30pm. According to the angry shouts on social media and the real-time blogging call-outs, the slip-up did not go unnoticed.

How could this happen?

According to  a statement released to Re/code, a Best Buy rep said  “A concentrated spike in mobile traffic triggered issues that led us to shut down in order to take proactive measures to restore full performance.”

The truth is there are many reasons why a site could fail during the busy shopping season. While some retailers completely underestimate the traffic, big box stores like Best Buy, Wal-Mart and Target prepare for Black Friday all year round.

More likely, the outages were a result of some other unforeseen circumstance. If Best Buy introduced a new feature, for example, and were not expecting massive numbers of visitors to utilize the feature at the same time, outages could occur. Considering the complexities of today’s marketing ecosystems and the inherent real-time system inter-dependencies, the potential for performance and availability issues can be significant.  While there is no way to fully  guarantee availability, retailers should consider the following, during the holidays and any other time marketing campaigns can drive significant traffic:

1. Make Sure Marketing & IT Are On The Same Page

Web traffic spikes during the holidays can come from a variety of sources. From mass emailing a coupon to giving away a TV on Twitter, your marketing department rolls out all their best campaign ideas around the holidays. But at many large companies, Marketing and IT are not always aligned, and may not inform each other of their activities and related requirements and expectations. One of the fundamental ways to prepare for the holiday season’s high site traffic is to tightly coordinate and communicate efforts between marketing and IT. This can be easier said than done at large organizations or where marketing teams work with a mix of off-prem (ex. Software-as-a-Service) and on-prem systems.  Campaign briefs and plans should include technical aspects outlining supporting systems, testing scenarios and roles for key IT resources. Bonus points if you can utilize technology resources to shape and improve marketing campaigns, giving IT a seat at the table.

2. Always Have A Back-Up Plan

When your holiday strategy relies around a few key shopping days, there is only so much you can do to prevent your infrastructure from overloading. In addition to typical regression testing of critical systems, IT and Marketing teams should align on what happens when systems fail. Whether it is a backup network, moving to the cloud or coordinating messaging on social media, having a strategy that involves both marketing and IT can create a much more unified message to the end consumer. Long lead times can help with this. While marketing teams may be anxious to get to market, the more runway IT teams have to test critical systems, the more detailed their testing strategy can become.

And remember, testing does not stop at your office door. Third-party tools like content delivery networks (CDNs), content and experience management, commerce, tag management, ad management, video management, recommendation engines, ratings/social/comments engines, etc. all need to be tested, tuned and supported with an alternative plan. Starting with a technology roadmap and smart technology selection, marketing and IT should begin meeting with vendors early in Q3 to ensure that all vendor integrations are equipped to handle larger-than-usual traffic spikes. Meeting with vendors ahead of time, letting them know about promotions and big shopping days you expect this season can help everyone be prepared for what is ahead.

3. Sweat The Small Stuff

While adding servers and CDNs can prevent slow sites, it won’t always save you when getting massive spikes in traffic. During the 2013 Super Bowl, Coke created a microsite that crashed when it received 78,000 hits per second, despite an infrastructure of 30 servers. This is where availability and performance tools come in. Availability and performance tools are designed to reduce stress on servers, wringing every drop of speed out of your stack by maximizing efficiency at common bottleneck points.

One tool modern marketers use is load balancers. Like a dealer in a poker game, a load balancer distributes demand to servers one at a time. There are simple load balancers that simply deal in order, and then there are more complicated load balancers distributing based on algorithms and common patterns. Some data management strategies work with these load balancers better than others. For example, some applications are coded to have what are called “Sticky Sessions”–where traffic that originated from one server must always be pinned to that server. This prevents a load balancer from doing its job.

Optimizing session data on busy shopping days can help reduce server stress as well. Are you using distributed sessions, server clustering or saving all data locally? No matter how you handle session data, caching critical files can drastically improve page load speed, especially for online retailers. Data for many popular online retail features like personalization, pricing and promotions can be tied to a user’s session data and cached in order to reduce calls to the server. But be careful how caching impacts frequent changes, like prices. If Best Buy wants to offer a different price based on user activity and session data, pricing should not be cached.

Truthfully, we don’t know what caused Best Buy to go down on Black Friday. It could be a variety of factors. It could be human error. What we do know is that running a site that is well tested, well coded and can scale has a variety of positive side effects–from improved search engine rankings to improved user experience to increased sales. No strategy is bulletproof, but by having a plan and working through the technology implications, your online presence will be more successful no matter how consumers respond to that “irresistible” offer.

Back To Posts