A Project Management Spotlight on Performance Testing



By Bill Sundermann

Texas, USA

During the software and operational challenges Southwest Airlines faced in late December, 2022, project management professionals likely asked themselves “Why wasn’t the system tested to handle these use cases of heavy customer travel combined with the likelihood of bad weather impacting flying conditions?” Other airlines were able to recover quickly from the disruption caused by the frigid cold in Chicago, Denver, and other northern airports in the path of the storm.

The Dallas Morning News reported Southwest Airlines canceled more than 4,800 flights over two days as its operational meltdown continued into a second week with no indication of when it would end. Dallas-based Southwest canceled 2,509 flights nationwide on a Wednesday, about 62% of its schedule, according to Flightaware.com. The problems escalated, when the airline decided it needed to cut about two-thirds of its schedule over the coming days to try to reset the operation. Between December 22 and 28, the carrier canceled 13,353 flights, affecting as many as 2.3 million people. (1)

“The tools we use to recover from disruption serve us well 99% of the time, but clearly, we need to double down on our already existing plans to upgrade systems for these extreme circumstances so that we never again face what’s happening right now,” CEO Bob Jordan said in a video apology to customers during the disruption. “It almost became a running joke around the company that we aren’t able to make certain changes because it would involve technology,” said Lyn Montgomery, president of the union representing the 16,000 flight attendants at Southwest, TWU Local 556.

Southwest uses internally built and maintained systems called SkySolver and Crew Web Access for pilots and flight attendants. They can sign on to those systems to pick flights and then make changes when flights are canceled or delayed or when there is an illness. SkySolver and Crew Web Access are both available as mobile apps, but those systems often break down during even mild weather events, and employees end up making phone calls to Southwest’s crew scheduling help desk to find better routes. During periods of heavy operational trouble, the system gets bogged down with too much demand.  In a Wall Street Journal article, Southwest said the disruption will reduce its pretax earnings by $725 million to $825 million. (2)

In another high-profile system failure in November 2022, the Ticketmaster sales platform was overwhelmed from high demand for Taylor Swift concert tickets.  CNN Business reported that as sales for the singer’s new Eras Tour began, heavy demand snarled the ticketing site, infuriating fans who couldn’t buy tickets (3). Customers complained about Ticketmaster not loading, saying the platform didn’t allow them to access tickets, even if they had a pre-sale code for verified fans. On a Thursday, Ticketmaster announced that the sale to the general public, which was scheduled to begin the following Friday, had been cancelled due to “extraordinarily high demands on ticketing systems and insufficient remaining ticket inventory to meet that demand.” The issues for Ticketmaster started earlier that week on Tuesday, when the site’s sale kicked off for “verified fans” — a mechanism aimed at eliminating bots that gives presale codes to individuals. The “verified fan” platform was created in 2017 to help Ticketmaster handle situations of enormous demand (more than 3.5 million people pre-registered to be a Swift “verified fan”). That unprecedented demand, combined with a “staggering number of bot attacks as well as fans who didn’t have invite codes” drove “unprecedented traffic” to its site, Ticketmaster said, and, essentially, broke it.


To read entire paper, click here

Editor’s note: Second Editions are previously published papers that have continued relevance in today’s project management world, or which were originally published in conference proceedings or in a language other than English.  Original publication acknowledged; authors retain copyright.  This paper was originally presented at the 15th UT Dallas PM Symposium in May 2023.  It is republished here with the permission of the author and conference organizers.

How to cite this paper: Sundermann, B. (2023). A Project Management Spotlight on Performance Testing; presented at the 15th University of Texas at Dallas Project Management Symposium in Richardson, TX, USA in May 2023; republished in the PM World Journal, Vol. XII, Issue XI, November. Available online at https://pmworldlibrary.net/wp-content/uploads/2023/11/pmwj135-Nov2023-Sundermann-project-management-spotlight-on-performance-testing.pdf

About the Author

Bill Sundermann

Texas, USA


Bill Sundermann is a certified PMP with 27 years of experience in the financial services software industry. As a senior project manager, he is passionate about training and sharing knowledge in the areas of agile coaching, team building, and effective messaging. He can be contacted at bsundermann@verizon.net