Category Archives: Professional

At first, it’s A then B testing not A/B testing

Had a great conversation today with Aydin Ghajar; a guy who really knows his stuff when it comes to building viral loops for consumer services.  The issue of rates/conversions yield big differences in ultimate outcomes for viral systems.  One of the things I’ve been grappling with thinking about A/B testing in MVPs is the complexity this adds early on.  After all, Lean is about doing things efficiently to learn what you need to match a customer/market need.

What happens when the basics to install the ability to learn how well A or B (or C or D) is non trivial?  A/B testing that isn’t trivial website button color, text takes some effort (days, but still).  My friend made an astute observation (paraphrased):

At first, your A/B testing while you’re still validating early assumptions is NOT parallel A/B testing (50 visitors this week into TWO 25 visitor buckets) but rather sequential: A is done one week, evaluated.  B is done week after, evaluated.

Why?  First off, A/B testing is complicated and best deferred until a bit later in MVP.  Why else? At first, when you’re testing your early hypothesis your volumes are small enough that you’re decreasing the accuracy of and time to determine if A or B is true.

In our example, if we get 50 users per week and we split them we’ll get 25 users in A and 25 users in B per week.  Week one, we get 25 of each, and get an action or two in each.  The statistic relevance or confidence of 25 user sample is LOW; it’s probably not enough to validate or invalidate the effectiveness or value of A or B (whatever you’re testing).  So, next week you get another 25 in each.  At 50 users, you’re confidence increases and now you’re ready to make a real determination on if A or B worked.

What’s the problem?  When you’re early on, getting things figured FAST is important.  While not right for every thing, I’d much rather have tried A in one week (50 sample size), find the results, and move on to B.  Same learning (faster in some cases) and more efficient (aka cheaper) initially.

So, I’m in perfect agreement with my friend.  A/B testing happens sequentially at first, then in parallel once practical (implementation/sample size).

Lean Startup Method improves my showers!

I’ve been doing a little bit of reading, and opted for a quick weekend in LA to give myself a crash course in the Lean Startup method.  Why?  I wanted to develop some of my own abilities and instincts to move faster to a product/market fit.  In my review of the last 3 years starting an open source DB company, I realized I was WILDLY under informed on what customers really wanted, needed.  I am determined that, whatever I do next, I will spend MUCH more time working on customer development and product market fit than feature and solution development.  The Lean Startup Method allows you work through, in a rough and tumble way, to QUICKLY disqualify your current business model/solution as viable.

That’s a pretty stock answer/elevator pitch for the Lean Startup method.  What in the world does it have to do with my showers?

There are these people; inventors, entrepreneurs, creative types that get ideas ALL the time.  They get an idea, they think about it’s place in the world.  They think how’d they build this product.  They think about what it would look like.  How awesome it would be, etc.  Many, myself included, typically see problems/solutions all the time.  At the moment, I have no fewer than 15 ideas for new business that vary from Food Preparation, to Easy Re-ording of Groceries, to population/search analytics, to improving pace of play on golf courses, utility trading, etc.

These ideas percolate; I think about them in the shower.  I think about them in the car.  They pick at me; I think about how great they could be.  I figure out a piece of the puzzle, put that into a mental cabinet and then go about my day.  Days later, it pops back into my head, and I start working on another aspect of the idea.  I call this my own Entrepreneurial Porn.  It’s fluffy, stimulating but is a lazy time consuming alternative to actually building a business.  I only stop thinking about an idea after a year or so has passed since I thought about it, or I find some critical problem with the SOLUTION that means it won’t work.  This comes after a significant amount of mental power/effort is expended.

Here’s the real curse:  I invent new ideas (into the queue) faster than they head out (solution won’t work).  I turn over and spend many of my waking hours (probably sleeping ones too) thinking about, and creatively looking upon ideas that will go nowhere.  Emotionally I feel like I’ve wasted this time and feel guilty about it, and it increases frustration internally knowing I’m not building a great business because I’m spending too much time on Entrepreneurial porn.

My hope now is that Lean Startup Method will provide  me the ability to banish these ideas from my shower time, driving time, and other moments that I could be thinking about  viable businesses, my loved ones and be a more present during the day.  I now have the ability to spend a few hours (or day or two) to figure out why the idea won’t work; then I don’t have to spend the thoughtless and wasted hours of time mulling over the solution.

Point in case on the curse; while I write this blog from 38,000 ft I just got a new idea inspired by observing a woman using her Windows Tablet.  One more idea to try and exorcise!  Entrepreneurial Devils be Damned!

Good Riddens Ecto

I was using Ecto, a Mac OS X blog client for the past year or so.  Overall I really DIDN’T like it, except that it had the ability to allow me to “Paste” images into the blog using the clipboard.  You’d be surprised at how many blog clients still force you save a file on to your computer, then browse to it, and then upload it, then include it in your post.  A real drag when you’re trying to do a few simple software screen shots.

I upgraded to a new Mac recently and the Ecto license key I had didn’t come over from the old Mac.  According to the Ecto website, just email them and they’ll send it along.  I emailed on 8/11/2009 and no response.  Not good customer service at all. 

But I really wanted to blog about the encrypted variables; I downloaded and gave ScribeFire a try and I’m switching.  It can paste images same as Ecto and it’s built right into Firefox.  Excellent!

So… Ecto customer service really goofed on this one.  You’ve got a customer who isn’t happy, gave me an opportunity to find something I like better, and a chance to blog about the whole thing!

NYC 4/9 and 4/10

I’ll be roaming in Manhattan this Thursday and Friday. Have some plans, but actually have some bits of time.

Anyone want to shoot the breeze about Pentaho / BI / Open Source? Email me: ngoodman@ignorethispart.com bayontechnologies.com

Happy New Year 2009!

I resisted the urge to post a “2008 recap” and “2009 predictions” since that seemed to be well covered in lots of different circles/blogs.

Ahhh… Who am I kidding? I’m just lazy! 2008 was a crappy year (personally, but not professionally) and 2009 is off to a great start (personally, but not professionally)!

Already I’m very much enjoying 2009 even though the consulting work is shaping up pretty light these first few weeks.

<shamelessplug>
Need any help with Mondrian/Kettle/Pentaho? I’m available for smaller 3-20 day engagements remotely and onsite in North America.
</shamelessplug>

The best part about the start of the year, was I was able to get some time testing, updating, and deploying to my demo server the two projects that Bayon has been sponsoring over the past few months.

JDBCKettle – Allows for Kettle transformations to be used in an EII fashion. This allows you to use a (set of) kettle transformations and access via SQL.

PentahoFlashCharts – Updated to OFC 2.0 and Pentaho 2.0.stable it also includes new XML Template for building charts. Right now it’s diverged from the Pentaho chart standard but I hope to get back to the standard pentaho chart definition before this goes to an initial Beta release.

I’ll be blogging more about these projects in the coming days.

Happy New Year!

Oracle ACE: In Absentia

So… A few years back I spent a LOT of time with Oracle ETL and BI products. I learned them inside and out, gave some user conference presentations, wrote a bunch of blogs, even Alpha tested a version of Oracle Warehouse Builder. Then I found “Open Source BI” and I’ve been heading breakneck into the world of MySQL, Pentaho, … A choice I do NOT regret – my consultancy is busier than ever and I love the Open Source BI play.

However – I miss seeing some of the old Oracle peeps at Open World. This year, I even registered for my free ACE pass to OOW but didn’t make it because I started two new projects this week. What I realized this year, was that I’m WAY out of touch with what’s going on in the land of Big Red O. The words and products for BI whiz past me – they don’t even look anything like they did just a couple of years back.

I hope everyone had a good time at OOW this year! I don’t see a path back to the land of Oracle anytime soon for me. :(

Readers: Thank you for the Latte

About 10 days ago I decided to experiment with AdWords. I was mostly interested in what ads would be placed in my content. For the most part there’s not been any big surprises and most the indexing and ads presented are spot on. Oracle pages for Oracle consultancies. Pentaho Pages show ads for Talend (these guys buy themselves to the top of pretty much every somewhat related page/term). The ads to begin with were a bit bizarre, but once Google had indexed all seemed normal.

Well… The best news about my experiment is that YOU, my READERs have contributed USD 5.50 to my Latte fund! The next big quad shot espresso beverage drink will be that much sweeter. Seriously, thanks for reading, and thanks for the latte!

PS – The ads will probably go away when I get a few minutes to take it out.

bayon is back

For readers who have been perusing since the early days of this blog (bayon blog) you’ll know what I’m talking about. If you’re a reader that has joined in the past year and half you’re probably wondering “What is bayon?”

bayon is a boutique consulting firm specializing in Business Intelligence implementations; it’s my company that I’ve operated since 2002. I put it on the back burner when I put on a Pentaho jersey and played a few games on the Pentaho team. I’m leaving (actually, left) Pentaho. My time at Pentaho was great. The Pentaho tribe is a great group of kind, honest, smart people. Rare to find the intersection of good people and good technologists.

I’ve felt the siren call of helping customers in a more entrenched way. Consulting does that I think. So, not like it’s a big announcement, but it is belated as my last day at Pentaho was nearly two months ago:

I’m now working at bayon full time building a dedicated practice around Open Source BI technologies in the enterprise. Bayon has joined the Pentaho partner program as a Certified Systems Integrator.

So there you have it. Shingle is out.

If you are interested in Pentaho, Open Source ETL, Open Source BI, etc don’t hesitate to be in touch.

PS – It’s also worth noting that my leaving has no reflection on the progress of the business. Quite the opposite really; some would consider me foolish for leaving when the company is doing as well as it is!

Well, I noticed

My workday goes much smoother because I listen to a variety of Online Radio stations. Today they all went silent or played public awareness campaigns from SaveNetRadio.org. People have been wondering if anyone will notice. Well, like the title says: I noticed!

I don’t know all the mechanics, but it comes to this. A lot of these small, boutique-ish online radio stations will shut down because the cost structure of the compensation will be, in their opinion, unsuitable.

I, for one, being a proponent of open content, software, and standards think there must be some underlying disconnect between the “Copyright Royalty Board” and broadcasters.

These small, hobbyist online broadcasters are part of larger shift in broadcasting/economies. Web 2.0-ish user generated content and participatory systems of consumer and producer.

The CRB probably needs to take another look at what it’s doing to see if it’s just trying to hang on to old ideologies in a new world.

I support Net Radio. :) You should too.
SaveNetRadio.org

Pet Peeve: EST != EDT

I work with people all over the country and the world. What that means is that we often schedule meetings, calls, webex meetings, remote consulting sessions, etc. Lacking some great shared calendar in the cloud that we can use to do this adhoc (I’m sure there’s some web 2.0 startup who does this so please comment if you know of something GOOD) this means that people email and put suggested and adjusted times in emails.

For instance, just yesterday, I received the following email:

The regular 10am EST XYZ meeting tomorrow is cancelled until further notice.

What’s the issue with this email? Well, we don’t have a 10am EST meeting. We have a meeting scheduled at 10am Eastern (ie, when the clock in the eastern time zone hits 10am during the summer months).

EDT and EST have VERY SPECIFIC timezone offsets.

EDT = UTC – 4
EST = UTC – 5

I use generally, and think many others also use “Eastern” to refer ONLY to local time. Ie, what the clock on the wall says in New York regardless of EDT/EST.

Let’s take the above example:

  • 10:00 EST on June 22 (someone sends an email requesting a meeting)
  • 10:00 EST = 13:00 UTC (given the definition of EST, with an offset of -5 hours)
  • 13:00 UTC = 11:00 EDT (ie, makes sense right, 10:00 EST = 11:00 EDT)
  • 11:00 Eastern = 10:00 EST (on June 22 when New York is in EDT the actual meeting time)

Obviously you assume that someone requesting a meeting for 10am EST on day that falls on EDT was ACTUALLY requesting a meeting at 10am EDT. However, why bother doing that?

My suggestion to people that can’t keep it all straight:

Use Eastern/Pacific instead of EDT/PSTs. Eastern/Pacific is clear that it’s local time but you haven’t confused it by requesting an incorrect time.