Category Archives: General BI

Database Operation Complexity Reference

I mentioned this previously, but I’ve been reading “Principles of Distributed Database Systems.” I’m enjoying it, and it’s helping me solidify many of the concepts that I apply daily in my capacity as a Principal BI Solutions consultant. Database theory, and specifically as it relates to tuning is part of any professionals work with Oracle. We’ve all deciphered the performance implications and clues for improvements from EXPLAIN PLAN. I’ve always been told you want to be able to give Oracle the clues/configurations to enable filter of results (selection) before joining. It always made sense but I had never understood fully the concepts behind these recommendations. Until now…

I don’t espouse to understand all of the complexities behind these issues but I ran across a great reference chart. I wanted to post it here along with some numbers for demonstration as a quick reference for any professionals with understanding of quadratic, logarithmic, and linear scales to match those up with the operations we use on a day to day basis.

This is from the book mentioned above, however I’m sure it’s rather common knowledge.

OPERATION COMPLEXITY
SELECT O(n)
PROJECT (without dedup)
PROJECT (with dedup) O(n*log n)
GROUP
JOIN
SEMIJOIN
DIVISION
SET OPERATORS
CARTESIAN PRODUCT O(n²)

A quick look at some numbers in the orders mentioned yield the following “costs” in n operation times.

n O(n) O(n*log n) O(n²)
5 5 3.49 25
10 10 10 100
100 100 200 10000
1000 1000 3000 1000000
1000000 1000000 6000000 1E+12

Hope this helps provide a useful way to match up the database operations we use on a daily basis with the theoretical cost of such operations. Cheers!

Web Analytics meets SimCity

I have a customer with a few web properties. I am helping them sort out some of their web reporting needs and integrating them into a BI Environment. A few weeks back I mentioned to them, mostly jokingly, they should take a look at VisitorVille. They took the advice (althought it twasn’t really advice, just water cooler talk) and gave it a go. They pulled it straight away for technical reasons but I think they are still planning on hooking it up at some point.

It’s a rather unique product/service, and I’m not sure it’s applicable for many businesses, but I must say it is impressive looking. I think it just goes to show, there are many different and appropriate ways to view data enriched by a Business Intelligence system. I think there will always be wonderful and impressive ways to view data based on the particular data set, metaphor, and audience. I’m betting that the in between (data consumption, extraction, transformation, derivation, application of real time business logic, etc) will remain much the same, but presentation is where the creative types can add immense value and impact.

Too bad the business users can’t use VisitorVille like SimCity. Build another skyscraper (web page), a bus terminal from out of town (referral from an outside site) as easy as the real SimCity game allows. 🙂 Website owners would LOVE that… Manage your website from a game terminal.