Humour

Making horse sense of benchmarking-II

T A Balasubramanian concludes his two-part guide on understanding the baffling business of benchmarking

11:40 am: I am still talking to the esteemed Nawab Ghallstone, writing notes and going deeper into the mysterious ways in which Ghallstone Benchmark Labs does its business. Sitting next to me on a large blue bench, sipping soda, is the Nawab, who is twirling his long curly moustache, evidently hugely pleased to have an audience.

“How did this benchmarking business start?” I ask, looking at the buzzing speedometer-rod instrument he is holding.

“It’s like the cosmetics industry, where they can turn ordinary homely-looking girls into ravishing stunners with clever brush strokes and a few coats of paint. Makers of computer systems and software discovered that customers feel safe if they are given scientific-sounding names and codes with decimal points that seem to make their systems unbeatable toppers in a shoot-out against the world’s best.”

“Where do you get these names, Nawabji?”

“Branding techniques, Doodh. For instance, the suffix “stone” has become a standard extension for naming computer benchmarks.”

“You mean like the cosmetics people use the prefix “fair” on some cosmetic products to make customers think they can become the fairest of them all?”

“Ah, good point, Doodh. However, I would draw the line with the cosmetics analogy and not carry it to the dark side,” he growls. “The benchmark branding ball was set rolling with the Whetstone, written in Algol, used for evaluating the power of computers. That came out of the Whetstone Laboratory in Britain,” huffs Ghallstone.

“Ah, that explains how you named your Lab, Nawabji,” I say brightly.

“Well, there was no Nawab Whet in Britain, Doodh. But yes, my name is a benchmark too, if you notice. Created by my dad, the Nawab of Ghoda. Ghoda Ghallstone Senior was the first, but his junior is the finest name in the benchmarking business,” he says modestly. “Anyway, Whets-tone was only the beginning. Now, if you think about it, the logical successor for a Whetstone is…”

“A Dhrystone?” I hazard brightly.

“You’re brilliant, Doodh! The Dhrystone it is. Dhrystone was named as a pun on the Whetstone. Dhrystones per second is a measure of the number of iterations of the main code loop you get when the Dhrystone score is divided by 1,857. Why 1,857, you may ask? Well, that particular figure is the number of Dhrystones per second on the VAX 11/785, the original 1 MIPS machine. At least it used to be in those days when my father tested horses, and the VAX was considered a computer stallion. Are you following me?” he asks.

“Yes, I understand, Nawabji. Stones for the old, stones for the new, something murky, something blue, eh?”

“Well, Doodh, don’t we all want to hide our warts and show our best profile? So what’s wrong if most system designers talk only about those benchmarks that show their products in a nice angelic halo. You could unbend a little and be sympathetic if they yield to this uncontrollable urge to over-represent the significance of benchmarks to customers, again to show off their best profiles. You can say this is bench marketing, or if you want to be dour, bench

racketeering. You see these packets of salt here?”

“I was about to ask you about that, Nawabji” I say.

“Well, we always recommended that our clients should deliver their benchmarks along with a packet of salt. A pinch won’t do,” says Ghallstone looking at me with a mellow smile.

“What are your current benchmark projects?” I ask changing the subject quickly.

“We are working on one with the code name Basic Instinct 3, or BI3. This is a test that compares skin surface areas on any component. A big Hollywood producer was one of the first customers. He wanted to benchmark the quantity of skin shown by the leading lady in his new animated movie with industry favourites like Chicago and Moulin Rouge. So we created a saturation skin show test, and gave it a special name.”

“Which is?” I ask curiously.

“Ah, we’re back to branding, Doodh. We call it the Sharonstone Test. Movies can be benchmarked too. We measure the speed of motion pictures in moving illusions per second, or MIPS. If you want to test specifically for skin saturation, you need a fixed point of reference, and that happens to be one Sharon moving illusion per second, or SMIPS. That is our reference benchmark, what you saw in the original Basic Instinct. We benchmarked the movie Chicago at 6 SMIPS. Moulin Rouge zoomed up to 8 SMIPS. And as for Bollywood, the best we have so far is Jism, with 2 SMIPS.”

“That’s entertainment,” I say encouragingly.

“Then we have a literary benchmark that tests thriller novels to find out if they match the original sizzling scenes in the novels written by—you guessed it—Irving Stone. So an Ihrvingstone index of 3 would mean that you’re reading a racy story thrice as exciting as anything old Stone himself has produced. And a rating of minus 5 would mean a damp squib of the novel. Literary critics love this benchmark. They’re kicked that we have now given them another arrow to shoot at those terrible authors they enjoy dragging over the stones, anyway.”

“That’s like getting stoned, literally,” I add, warming to the theme.

“But getting back to IT, another project involves a test for comparing the freshness of media such as tape or plastic. Sometimes, when you store magnetic tape or vinyl disks for too long in stuffy places, they get coated by fungus.”

“Ah, I see. Rolling stones gather no moss?” I chip in brightly.

“Exactly. For these cases, we have borrowed from the maxim and created the Rhollingstone Test. If your Rhollingstone Index is 1, your media is clean. If it is 9, you can assume that your media has got so much muck on it that it has become useless.”

“And for the future?” I ask, “Leave no stone untested?”

“Oh, yes. We have a whole bag of other benchmarks coming up soon, Doodh. You can expect to see Rhosettastone, a natural language translation benchmark. There is Khidneystone, our medical expert systems benchmark and Lhivingstone, for anthropology in Africa. And the big one called Schissorspaperstone, which is a series of benchmarks that can be used to prove any machine architecture superior to any other.”

4:40 pm: I leave Ghallstone Labs, considerably heavier in my head. As I head home, I reflect that Nawab Ghallstone certainly lived up to his royal name, and it would now be my turn to translate all this gyan and present it to my CEO, Baffleji.

As for the benchmarks that I plan to use in my work, all I can say is that Baff-Tech needs a lot of fast horses to produce some stable systems. Or should I say, “Some stables, fast, to produce horse systems?”