Skytrax Airline Ratings Vindicated (Sort of)

Airline-rating site Skytrax exaggerated some claims about its methodology, but there’s no “smoking gun” evidence that it manipulated the results. That’s the final conclusion of an investigation by the Advertising Standards Association (ASA), the British watchdog on consumer advertising. The investigation was triggered by a complaint from KwikChex, a British price-comparison website. And although the ASA supported KwikChex, the results apparently will not have any significant impact on present or future Skytrax airline and airport ratings—arguably the world’s most widely cited. Nor do they invalidate the concept of user-generated travel reviews.

KwikChex challenged five specific claims, all upheld by the ASA:

• Skytrax claims its reviews are “trusted” and from “real travelers with real opinions,” but ASA found that although Skytrax describes a rigorous verification system, it cannot adequately prove authenticity.

• Skytrax claims that its scores are based on results from “more than 5 million reviews,” but ASA found that visitors to the Skytrax website could access only about 400,000 of those submissions. ASA did not disprove the 5 million figure, but says that Skytrax can’t prove it, either.

• Skytrax labels its results as “Official Star Quality Ratings,” but ASA found insufficient evidence of industry-wide agreement and, therefore, the term “Official” is misleading.

• Skytrax claims its ratings are “real time,” but it continues to display ratings for airlines that have folded.

• Skytrax claims that the terms “5-Star Airlines” and other similar terms are trademarked, but ASA found insufficient substantiation of that claim.

Overall, although KwikChex and the ASA can claim “victory,” my take is that the findings are largely in the realm of nit-picking and do not go to the heart of the question of ratings integrity.

I certainly have some questions of my own about the Skytrax ratings. As I noted earlier, placing Spirit in the top 10 domestic North American airlines and rating complaint-prone American Eagle at number six are absurd conclusions that defy realism. As does listing those two lines among the regular “domestic” lines but classing larger Alaska and JetBlue among the “regional” lines.

ASA, apparently, has some clout. Although its Skytrax findings were not issued until November 7, Skytrax has already dropped the “official” label on its awards and otherwise modified its online posting. Would that our FTC could get such swift action on its many consumer-abuse complaints.

The ASA investigation failed to shed any additional light on the question that has surrounded travel ratings for years: How valid are user-based reviews? Certainly, ASA found no inherent bias in Skytrax or any deliberate manipulation of customer submissions—the two most common criticisms of user-based reviews. The results, themselves, illustrate the other major criticism of such reviews: Too many typical consumers are not in a position to evaluate any given supplier by any sort of standards that are consistent with those of others. I’ve certainly found such biases in many user-based airline and hotel reviews:

Travel Smarter! Sign up for our free newsletter.

By proceeding, you agree to our Privacy Policy and Terms of Use.

Sign Up

• As far as I can tell, many airline reviews—especially those in the slick upscale travel magazines—seem to be heavily biased toward business class.

• Domestic airlines often suffer because so many comments come from short-haul trips, where overall service factors are almost always lower than on long-haul international flights.

No matter what you think about user-generated results, however, they’re here to stay, especially with hotels. Most big OTA sites include user-derived ratings, either their own or from TripAdvisor or other third-party sources. Yes, individual hotel operators can still submit—or bribe guests to submit—rave reviews on their own properties or bad marks on competitors. But there’s safety in numbers, and the big sites almost always base their ratings on big numbers, not individual responses. Can you rely on them for total accuracy? No. But are they useful guides? Most of us would say “yes.”And the Skytrax investigation doesn’t really change that.  

Ed Perkins on Travel is copyright (c) 2012 Tribune Media Services, Inc.

You Might Also Like:

 

• Some Progress on Hotel-Fee Scams, but No Results Yet
• Airline Consumer Committee: No Surprises
• Which Destination Has the Best-Reviewed Hotels?