\n However, we\u2019ve had some known issues. In fact, until PDI 4.2 GA and LucidDB 0.9.4 GA it\u2019s pretty problematic unless you run through the process of patching LucidDB outlined on this page: Known Issues<\/a>.<\/p>\n In some ways, we have to admit, that we released this piece of software too soon. Early and often comes with some risk, and many have felt the pain of some of the issues that have been discovered with the streaming loader.<\/p>\n In some ways, we\u2019ve built an unnatural approach to loading for PDI: PDI wants to PUSH data into a database<\/b>. LucidDB wants to PULL data from remote sources<\/b>, with it\u2019s integrated ELT and DML based approach (with connectors to databases, salesforce, etc). Our streaming loader \u201cfakes\u201d a pull data source, and allows PDI to \u201cpush\u201d into it.<\/p>\n There\u2019s mutliple threads involved, when exceptions happen users have received cruddy error messages such as \u201cBroken Pipe\u201d that are unhelpful at best, frustrating at worse. Most all of these contortions will have sorted themselves out and by the time 4.2 GA PDI and 0.9.4 GA of LucidDB are released the streaming loader should be working A-OK. Some users would just assume avoid the patch instructions above and have posed the question: In a general sense, if not the streaming loader how would I load data into LucidDB?<\/i><\/p>\n Again, LucidDB likes to \u201cpull\u201d data from remote sources. One of those is CSV files. Here\u2019s a nice, easy, quick (30k r\/s on my MacBook) method to load a million rows using PDI and LucidDB:<\/p>\n![\"201106292256.jpg\"](\"http:\/\/www.nicholasgoodman.com\/bt\/blog\/wp-content\/uploads\/2011\/06\/201106292256.jpg\")
<\/a><\/p>\n