Vinnie Mirchandani has his latest take on the constant controversy of third-party researchers, consultants and associations compiling rankings of service providers. This time the IAOP's Global Services 100 is being questioned.
We've also had some banter about the Black Book of Outsourcing on this site, which made such a noise with its constant rankings of service providers, that Datamonitor decided to buy them to hop on this bandwagon. And we've never even got to discussing the Global Services 100, or several other rankings that come out periodically. Moreover, some "analyst" firms make a living ranking service providers, while barely bothering to talk to their customers, and selling white papers to the winners so they can flout their success (you all know who you are).
Personally, as an analyst and advisor, I find these lists useful – I sometimes find out about some provider I didn't know a lot about, and they draw attention to who's doing well at the
moment. But that's all I care about. If these entities produced a directory of service providers, it would be a valuable resource to the industry at large. But that isn't really the case; why produce free information of you can't sell it as marketing collateral?
What worries me is the following:
These entities make the majority (or all) of their revenues from service providers;
However accurate these rankings may be, they always make errors which can cause buyers to make poor purchasing decisions, or service providers to be unfairly discounted from a down-select process
So what can entities to do make these rankings more credible?
If I am advising a client in an area where I need some additional input or validation, I want to talk to the resident expert in that field. At the end of the day, it's the recognized experts who have withstood the test of time that can give you the real deal when it comes to rankings. A true expert in a niche area, for example application development services, would be in constant contact with all the service providers, their clients' successes and traumas, and have a unique view on who really is delivering the goods, versus who is struggling. And we don't really want some fluffy top-ten score, we want to know exactly where each service provider has scale, expertise, language, technology and process acumen. That's the real info customers need. The only benefit of rankings is for service providers to send out press releases boasting about their "performance" and add more marketing vapourware to PowerPoint decks. Oh, and they need to pony up significant cash to whomever compiled the list. But it's cheap point-scoring, so what do they care?
Hence, in my world, a service provider ranking is only really credible when the expert's personal name and brand is attached. Someone is being held accountable, as opposed to some faceless corporate brand, which masks any real accountability. At least an analyst firm is attaching an analyst's name to that ranking list, so that analyst has some skin in the game. A good analyst will normally produce correct performance indicators. A poor analyst will make mistakes, or simply not have good industry connections or guidance, and likely not last long in this economy.
Strong stuff I know, but it has to be said… your views are more than welcome.