{"id":330,"date":"2009-01-30T14:31:30","date_gmt":"2009-01-30T21:31:30","guid":{"rendered":"http:\/\/www.nicholasgoodman.com\/bt\/blog\/?p=330"},"modified":"2009-01-30T14:31:30","modified_gmt":"2009-01-30T21:31:30","slug":"the-death-of-prevrow-rowclone","status":"publish","type":"post","link":"http:\/\/www.nicholasgoodman.com\/bt\/blog\/2009\/01\/30\/the-death-of-prevrow-rowclone\/","title":{"rendered":"The death of prevRow = row.clone()"},"content":{"rendered":"<p><strong>UPDATE: This step is available in <\/strong><strong><a href=\"http:\/\/sourceforge.net\/project\/showfiles.php?group_id=140317&amp;package_id=186321&amp;release_id=657167\">Kettle 3.2 M1.<\/a><\/strong><\/p>\n<p>For those that have done more involved Kettle projects you&#8217;ll know how valuable the Javascript step is.  It&#8217;s the Swiss Army knife of Kettle development.  The calculator step is a nice thought, but the limited set of functions and the constriction of having to enter it in pulldowns can make more complex calculations more difficult.<\/p>\n<p>Those that have done &#8220;observed metric&#8221; type calculations in Kettle will know this bit of Javascript well:<\/p>\n<blockquote><p>var prevRow;<br \/>\nvar PREV_ORDER_DATE;<\/p>\n<p>if ( prevRow != null &#38;&#38; prevRow.getInteger(&#8220;customernumber&#8221;, -1) == customernumber.getInteger() )<br \/>\nPREV_ORDER_DATE = prevRow.getDate(&#8220;orderdate&#8221;, null);<br \/>\nelse<br \/>\nPREV_ORDER_DATE = null;<\/p>\n<p><strong>prevRow = row.Clone();<\/strong><\/p><\/blockquote>\n<p>This little bit of Javascript allowed you to &#8220;look forward&#8221; (or back depending on  your sorting) and calculate the difference between items:<\/p>\n<ul>\n<li>\n\tWatching a set of &#8220;balances&#8221; fly by and calculate the transactions (this balance &#8211; prev balance) = transaction amount<br \/>\n\tWeb Page duration (next click time &#8211; this click time) = time spent viewing this web page<br \/>\n\tOrder Status time (next order status time &#8211; this order status time) = Amount of time spent in this order status (warehouse waiting)<\/li>\n<\/ul>\n<p>In other words, lining data up and peaking ahead and backwards is a common analytic calculation.  In <a href=\"http:\/\/www.orafaq.com\/node\/55\">Oracle\/ANSI SQL<\/a>, there&#8217;s a whole set of functions  that perform these type of functions.<\/p>\n<p>This week I committed to the Kettle 3.2x source code a step to perform the LEAD\/LAG functions that I&#8217;ve had to hand write several times in Javascript.  It&#8217;s been long overdue as I told Matt I designed the step in my head two years ago and he&#8217;s been patiently waiting for me to get off my *ss and do something about it.<\/p>\n<p>You can find more information about the step on its <a href=\"http:\/\/wiki.pentaho.com\/display\/EAI\/Analytic+Query\">Wiki page<\/a>, along with a few examples in the samples\/transformations\/ directory.<\/p>\n<p>The step allows you peek N rows forward, and N rows backward over a group and grab the value and include it in the current row.  The step allows you to set the group (at which to reset the LEAD\/LAG), and setup each function (Name, Subject, Type, N rows)<br \/>\n<a href=\"http:\/\/www.nicholasgoodman.com\/bt\/blog\/wp-content\/uploads\/2009\/01\/200901301239.jpg\" onclick=\"window.open('http:\/\/www.nicholasgoodman.com\/bt\/blog\/wp-content\/uploads\/2009\/01\/200901301239.jpg','popup','width=595,height=290,scrollbars=no,resizable=yes,toolbar=no,directories=no,location=no,menubar=no,status=yes,left=0,top=0');return false\"><img decoding=\"async\" loading=\"lazy\" src=\"http:\/\/www.nicholasgoodman.com\/bt\/blog\/wp-content\/uploads\/2009\/01\/200901301239-tm.jpg\" border=\"1\" alt=\"200901301239\" hspace=\"4\" vspace=\"4\" width=\"205\" height=\"100\" \/><\/a><br \/>\nUsing a group field (groupseq) and LEADing\/LAGing ONE row (N = 1) we can get the following dataset:<br \/>\n<a href=\"http:\/\/www.nicholasgoodman.com\/bt\/blog\/wp-content\/uploads\/2009\/01\/200901301238.jpg\" onclick=\"window.open('http:\/\/www.nicholasgoodman.com\/bt\/blog\/wp-content\/uploads\/2009\/01\/200901301238.jpg','popup','width=506,height=153,scrollbars=no,resizable=yes,toolbar=no,directories=no,location=no,menubar=no,status=yes,left=0,top=0');return false\"><img decoding=\"async\" loading=\"lazy\" src=\"http:\/\/www.nicholasgoodman.com\/bt\/blog\/wp-content\/uploads\/2009\/01\/200901301238-tm.jpg\" border=\"1\" alt=\"200901301238\" hspace=\"4\" vspace=\"4\" width=\"330\" height=\"100\" \/><\/a><br \/>\nAny additional calculations (such as the difference, etc) can be calculated like any other fields.<\/p>\n<p>This was my first commit to the Kettle project, and a very cool thing happened.  I checked in the base step and in true open source fashion, Samatar (another dev) noticed, and created an icon for my step which was great since I had no idea what to make as the icon.  Additionally, hours after my first commit he had included a French translation for the step.  He and I didn&#8217;t discuss it ahead of time, or even know each other.  That&#8217;s the way open source works&#8230; well.  \ud83d\ude42<\/p>\n<p><strong>RIP prevRow = row.clone()<\/strong>.  You are dead to me now.  Long live the Analytic Query step<\/p>\n","protected":false},"excerpt":{"rendered":"<p>UPDATE: This step is available in Kettle 3.2 M1. For those that have done more involved Kettle projects you&#8217;ll know how valuable the Javascript step is. It&#8217;s the Swiss Army knife of Kettle development. The calculator step is a nice thought, but the limited set of functions and the constriction of having to enter it [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[15,9,11],"tags":[],"_links":{"self":[{"href":"http:\/\/www.nicholasgoodman.com\/bt\/blog\/wp-json\/wp\/v2\/posts\/330"}],"collection":[{"href":"http:\/\/www.nicholasgoodman.com\/bt\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/www.nicholasgoodman.com\/bt\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/www.nicholasgoodman.com\/bt\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"http:\/\/www.nicholasgoodman.com\/bt\/blog\/wp-json\/wp\/v2\/comments?post=330"}],"version-history":[{"count":0,"href":"http:\/\/www.nicholasgoodman.com\/bt\/blog\/wp-json\/wp\/v2\/posts\/330\/revisions"}],"wp:attachment":[{"href":"http:\/\/www.nicholasgoodman.com\/bt\/blog\/wp-json\/wp\/v2\/media?parent=330"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/www.nicholasgoodman.com\/bt\/blog\/wp-json\/wp\/v2\/categories?post=330"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/www.nicholasgoodman.com\/bt\/blog\/wp-json\/wp\/v2\/tags?post=330"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}