One of our clients (who shall remain anonymous) recently paid their platform vendor (who shall also remain anonymous) for a data warehouse performance assessment. This is the third time we’ve witnessed this particular client go through this process in the last 15 years. This go me thinking about the whole process…
For a performance analysis exercise to be of real benefit I think three broad objectives need to be achieved:
1) Tools – whoever is carrying out the analysis had better have better tools in their kitbag than those already at the disposal of the in-house technical team.
Why would I bother paying a third party to come up with the same performance metrics that I’ve been collecting for decades?
Telling me that I run at an average of x% CPU busy, that my machine is using y% of total capacity and that department z are the biggest users of machine resource in hardly likely to be new news.
2) Recommendations – whatever the findings uncover the recommendations that follow had better be 100% ‘doable’.
This means they need to have an acceptable level of risk, not take too long to implement and be able to be achieved within the available constraints such as skills, space, CPU, batch processing windows etc.
There is no point at all in describing theoretically achievable improvements that have no chance of being implemented.
3) Impact – this is where most investigations fall short.
If you can’t tell me with any certainty how much a recommendation will improve my world why on earth would I make a change to my production system?
“Queries will run faster”. Really? By how much? How will that benefit me?
“Less CPU will be required to run your workloads”. Excellent! Exactly how much are we talking? Oh, you’re not sure…
Most of the performance reports we have reviewed end up as shelf-ware – full of well intended recommendations of which few, if any, are actually implemented.
Those recommendations that are implemented usually add up to a collective incremental gain that fails to justify the implementation cost, let alone the cost of the report as well.
There is rarely, if ever, a silver bullet uncovered in such investigations. For their to be a silver bullet would surely suggest a lot of bright folks had been asleep at the wheel for a long time.
That would never happen, surely!?!?!
