There's been a lot of heated discussion in the outsourcing industry of late regarding the issue of lists, rankings, awards and proclamations of vendors, advisors, tought-leaders etc. As per expected, emotions are strained when some come out looking good, and some not-so-good.
However, with such a ramshackle swirl of information out there... from consultants, media, bloggers, analysts, associations, researchers, there has never been a more critical need for the ultimate target - the buyer - to have balanced, unbiased and accurate information and advice. The real issue here is that rankings or "best of" lists are only as valuable as the rigor and independence of the evaluation methodology.
Deborah Kops, who has guested here with some intriguing views on outsourcing issues, has offered to share her thoughts with us on this whole issue. Deborah is widely recognized as one of the outsourcing industry's most experienced voices, having spent the majority of her career on the advisory and practitioner-side of global services, leading global transformation efforts at Deutsch Bank and Bank of America, before helping to establish PwC's outsourcing division. Today, Deborah is Chief Marketing Officer for WNS Global Services, a leading offshore BPO and KPO provider, and has greater exposure than most people in the industry to these "lists", now she is leading marketing for a services provider with revenues of half-a-billion dollars. Over to you Deborah:
In May 2006, I published an article in Global Services magazine called “The Book of Lists.” A series of events drove me to take up a pen; I had recently served as a judge on several rankings, and had been amused that I was personally ranked a “superstar” on a list in a category that I in no way, shape or form could be termed an expert, even by my father who thought me a wunderkind.
The article seemed to have been overlooked by the chattering classes. I got no feedback and relegated the topic to my personal annals, thinking, “there I go again—seeing disconnects where no one else does.” And subsequently moving to the provider bench, I experienced the full force of “list mania;” the requests for lists and rankings were incessant, but I grumbled privately as I pushed my team to find the most arcane of data for every inquiry, concerned that missing the submission of even one application was the ultimate of marketing faux pas.
Now, two years later, the industry is all abuzz on the topic of lists. So I went back to that article to see if it still was relevant. I think it is.
Two years ago, I wrote about those determined to grab onto the global services delivery trend by pronouncing the definitive judgment on the industry’s best and brightest providers and individuals, identifying the latest trend or dimensioning the marketplace.
And I found, after reviewing what was then a relatively small group of list makers caught in the act, that, one list’s number four was another’s number 26. If in the variations within any crop of “global,” “best,” “emerging” and “top” service providers or advisors is any indication, beauty is definitely in the eye of the beholder. And what is magic about being in the top 100 or five or 10 in a market as big and diverse as our global services industry?
We generally acknowledge the usefulness of lists. Lists give us a simple framework to evaluate relativity and test importance in an increasingly complex world. They can serve as an elegant communication tool by allowing us to compare, contrast, and prioritize a range of factors and opinions. But faced with all these lists, what is the business line sponsor or the corporate sourcing staff to think? Are these lists bogus exercises or helpful tools to evaluate the supply side of the services-delivery equation?
What do the lists tell us?
First and foremost, independent provider and advisor lists serve as a good public barometer of brand recognition and a market reflection of image in a relatively immature industry without one industry-wide voice, unlike the Royal Institute of Chartered Architects, or the American Medical Association. They are used as stand-in as the services industry’s only current broad indicator of brand value. As the industry matures, the use of scientifically-constructed and tested brand indices should become more common; however, at this juncture in services evolution, it’s about all we have to rely upon.
Provider claims of excellence or advisor claims of differentiated methodology can sometimes be supported by the independent opinion of third party lists, especially if the industry’s best and brightest are sitting judgment. While ranked lists are by no means scientific or purportedly meant as an endorsement, they are often used in collateral to provide seeming accreditation in an industry that has no common, recognized standards of excellence of its own at this point in its evolution.
Lists can uncover market trends. At their most useful, list makers may be comprised of industry watchers — journalists, consultants, advisors, buyers and academics — who look closely and thoughtfully at the applicants, trying to isolate new trends in service provision or employee management, allowing them to determine whether certain offerings are finally ready to meet with market acceptance.
Lists are also a gauge of marketing prowess. Honest list evaluators privately admit that the completeness of the applications and the quality of presentation, not to mention grammar, weigh heavily in the evaluation process. The ability to succinctly communicate a value proposition to industry insiders may be a good test of market positioning and messaging. If the insider gets the proposition, chances are that the marketplace might just understand the message, too. And for those lists that are styled as vox populi? Well, those are a reflection of marketing prowess, too. Some of us on the sell-side have the time and the wherewithal to focus on getting our clients and friends to answer questionnaires.
Lists both influence and reflect current industry values. Size and scale seem to matter. It appears that there is a direct correlation between service provider or adviser size and relative ranking on most of the lists, which include a best category, or are ordinal in evaluation. Over time, as provider and advisor track records become more transparent, and the industry really centers on what is a value proposition or whether the advisor’s deal was really well structured, rankings may imply other values.
Emerging offerings may first reach the buyer’s attention through the list process. As a relatively easy marketing and inexpensive marketing exercise, emerging or stage one outsourcers or advisors can avail themselves of an opportunity to broadcast the news of their process innovation or new approaches to delivery, judges permitting.
Over time, lists can serve as a good indication of the maturity of the marketplace. As new entrants and new offerings enter the marketplace, lists serve as a good archive of marketplace movement and development year-on-year. Over time, call it the ancestry.com database for the industry.
What Don’t the Lists Tell Us?
Read the methodological fine print if it is available. Although the data points evaluated may be substantial, not all lists are formulated based on the same or consistent level of rigor. Consider factors of sample size, statistical application, and criteria weighting. And realize that, human nature being what it is, each evaluator will bring his own personal biases for certain delivery strategies, provider profiles and approaches to the exercise. At the end of any judged exercise, for example, individual judges’ lists are assimilated into one publishable list, further adding a degree of whim, taste and fancy to the process. These lists are not the last word, but rather should be the first in selection.
A key component of editorial calendars and other sponsorships in the outsourcing industry, rankings and lists can provide a service to communities by identifying players and trends. But let’s not delude ourselves; they are also a business imperative for publishers, associations and pundits to build membership and/or circulation and sell adverts, publications and reprints, playing on the sell side’s need for recognition. They generally make someone money! This is not necessarily a negative, but is rather the way the world goes around. What’s critical is that that which is editorial and that which is financial should be kept completely independent at all costs. Pay for play in any form must not be the modus operandi.
And another problem inherent in list making is that the lists do not necessarily always evaluate the entire universe of sell-side providers or advisors. They often evaluate those who “opt in,” having deemed the application exercise a good use of marketing time and dollars to develop an estimation of market perception. Some providers and advisors determine that, for a range of business reasons, that inclusion is not beneficial at a moment in time because of business performance or the priority of other corporate events. Careful consumers of lists understand that the tallies are by no means exhaustive or inclusive of all offerings that may meet a corporate need, but merely serve as a boost to research.
The Effective Use of Lists...
In the right context, sell-side rankings can be exceptionally useful tools. Between all the lists, most providers and advisors — established and emerging — are covered, providing some of the initial legwork to identify the universe of players. This enables the quick assembly of a short list for further desktop investigation. The rankings can also softly suggest relative movement of the industry over time. As provider/advisor positioning changes, factors such as market size, buyer values and performance may be implied.
Sometimes corporate managers cross-reference their decisions through third-party opinion. Using a list to effectively support selection ex-post facto could be a good way to help outsourcing sponsors become comfortable with a value proposition or an advisor selection, especially when the consultant’s or provider’s brand is not yet a household name.
...And the ineffective use of lists
Now comes the rub. When I was younger and absolutely intransigent on most issues, the family retort always was ‘who died and made you queen?’ By extrapolation, who is to say that one list is the absolute authority on the industry best and brightest? Are the publishers, the industry pundits, the academics, the consultants the last word? Each of us can at best see only the smallest piece of the puzzle, and the “truth” we trumpet, whether backed by the numbers of ballots, or the experiences on which we base our opinions, is in the end only perception at a point in time.
Watching the debate in a range of media--blogs, articles, private conversations, and analyst calls-- is certainly a distraction from the day-to-day hardships of this economy and this industry, but it takes me back to my school days, watching Tommy Brennan beat the dickens out of Karl Kurth on the playground of Cumberland School. At the end of the day, it’s a completely useless fight because of the inherent and unavoidable biases in lists--- no matter how every list maker beats his methodological chest.
The only list which ultimately matters is the client’s own ranking of advisors and service providers. Factors such as process scope, geographic reach, value proposition, qualifications of the team customer approach and performance record...these are just a few of the criteria that drive evaluation. Best advice as when reviewing the growing crop of lists — each to his own.