Humour

Making horse sense of benchmarking - I

A two-part guide to understand the baffling business of benchmarking, by T A Balasubramanian

Another busy day in the life of Doodh Byramji. Better known as Doodh, or Doodhi, Byramji is the fully baffled design engineer of Baffle Technologies, otherwise called Baff-Tech.

Today, Byramji plans to probe deep into the phenomenon of benchmarking. He has already been confronted on this topic by his CEO, Baidyanath Baffle, the founder and owner of Baff-Tech, who wanted to know why he is spending so much time comparing benchmarks.

“Doodhi, please tell me. What does all this benchmarking mean? You say you spend a lot of time with benchmarks before you come up with your baffling product ideas. Are you involved in some new furniture purchases? Or has somebody been doodling on the benches in Baff-Tech without my knowledge?”

Byramji is at a loss for an explanation. “Benchmarking? No, I think it does not have anything to do with any benches-wenches, Mr Baffle. I use these in my reports just like some people use bar codes, but I have no idea what they represent. Give me a day or two, and I will have a report ready.”

So here, once more, is the action captured in the intrepid diary of Byramji, carefully entered, with every detail.

9:20 am: First, it is time to visit my old friend Nawab Ghallstone, the famous IT systems tester. The IT media fondly refer to him as the “Prince of Benchmark.” One writer went so far as to claim that Ghallstone could test anything you gave him, including your patience.

9:40 am: I have arrived at Ghallstone’s workplace, which is given the exciting name “Ghallstone Benchmark Labs.” I wait for a while, sitting on a simple reception bench. I notice that the entire place is full of benches—in all shapes and colours.

There are many machines inside the place, with many people walking around slowly in white and silver spacesuits that make them look like astronauts. Some of them are carrying instruments that look like long swords with speedometers attached.

After a few minutes, I am greeted by a short man in a silvery suit with a beard and a big welcoming smile, who walks in casually, carrying one of the rods. I give him my card.

“Come in Byramji. Or can I call you Doodh? It is always a pleasure to meet people who want something tested. We test everything here: hardware, software, middleware and swimwear. The last is, of course, just a new business angle we have added. We have to diversify to keep up with the times, you know,” he says winking.

He escorts me inside the bustling hall, and I am bewildered by all the hyper activity going on around.

“Actually, I don’t have anything to be tested. I have heard that you are one of the best benchmarking consultants, so I came to learn more about what you do.”

“Yes, we are the best,” says Ghallstone beaming. “As we say in our mission statement, if your system does not pass our tests, it is a complete flop. If it passes, it is a megaflop.”

“Are you a real nawab?” I ask, awed by the prospect of talking to a decadent Indian scion from the princely era of the past.

“I was a nawab, yes,” says Ghallstone sadly. “But, as you can see, we princes too have to earn a living, what with inflation eating into our old money. So I started this benchmarking business.”

“I want to know more about benchmarking,” I say meekly.

“First, let me explain what a benchmark is,” says Nawab Ghallstone, stroking his beard. “It is a test we use to compare the performance of different systems. I have some experience, you know, with horses.”

“Horses?”

“Yes, horses. My father was the Nawab of Ghoda. Nawab Ghoda Ghallstone used to breed racehorses of all kinds. He designed many kinds of tests for comparing the performance of stallions. Some of the world’s best racehorses have been benchmarked at Ghoda. But that was all in the past. These days, we don’t test horses; we test systems of all kinds.”

“So how do you conduct these benchmark tests?”

“What we do is, given a benchmark, we try to do as well as we can on it, and make sure that your system has the fastest benchmark—I mean, it is the fastest system—in the world.”

“Which one is it?”

“Which one is what?”

“You make sure that my system has the fastest benchmark in the world or do you make sure that my system is the fastest in the world?”

Ghallstone studies me intensely, again making me feel like a mouse before an owl. “You know the old story about the tax consultant?” he asks slowly.

“No, please tell me,” I say puzzled.

“Well, a businessman goes to a tax consultant and tells him that he has a problem because he has made a lot of money and has to pay too much tax. So he asks the consultant to fix his tax returns. The consultant asks him how much tax he has to pay. The man says, a million. Then the consultant asks: what do you want it to be?”

“Now I am beginning to understand,” I nod.

“Well, Doodh, you seem to be smart. All we ask our clients is the same question as the consultant in the story: what do you want your benchmark to be? If it has to be the best in the world, we find a benchmark that will make your system the best. Keep the customer delighted, you know.”

“Can you give me an example?”

“Now take my own Ghallstone test. If you say your system should be better than IBM, then I will find something about your system that has to be better—may be the speed with which your software can load into the disc—and then create a special Ghallstone test to highlight that feature. You can then claim that you have a solution that is tested and proven to be better than IBM. Simple, eh?”

“Ingenious, Nawabji. I had no idea benchmarking was such a clever business.”

“Well, my father told me how he used to rate horses this way. He would always find a horse that ran slower than your favourite and benchmark your chosen one against that. Result? Your horse would be rated the best every time!”

“This is a new perspective on horse sense, Nawabji,” I say impressed.

Nawab Ghallstone thumps my back appreciatively.

(to be continued next week)