At the next Orange County Java Users Group meeting I will be giving an intro talk on Talend Open Studio. I chose to speak on Talend because I found it to be a highly productive tool for creating data integration (ETL/ELT), schema migration, and batch processing jobs. The full abstract is below.
If you can attend, please RSVP on our new site hosted by Meetup.
Abstract: Talend Open Studio for Data Integration is an Eclipsed-based Java code generator that allows you to create ETL (extract, transform, load) jobs. It does this primarily through a visual programming editor containing over 600 components, including 450+ connectors for databases, files, and applications. However, it can be extended by injecting Java or Groovy code where needed, or by developing custom components in Java. In essence, it’s a Swiss Army knife for your data. There are specialized versions as well, targeting big data (Hadoop), BPM, data quality, and more.
In this introductory talk I will cover what TOS DI is capable of, best practices, and give a live demo showing you how to:
- connect to databases and work with files
- map input and output schemas
- filter and join data
- insert Java code for more advanced data manipulation
As time permits, I will delve further into the feature set.
“Let me show you a miracle.” said the beautiful Ukrainian woman. “Give me your hand.” Without a second thought I placed my hand in hers. Then, despite my immediate protestations, she pulls out a fancy 2-sided emery board and begins buffing and polishing my right index fingernail to a brilliant shine. “Really, that’s OK. I don’t need-” She interrupts, “You have girlfriend? Wife?” “Why, no but-” Oh right, she’s trying to sell me 2 of these. Thus I was introduced to Kyiv. I will say, the product worked and that fingernail was shiny for 2 weeks just like she said, but I never saw any Ukrainian men walking around with ultra-shiny fingernails. Just gullible Americans. Continue reading Kyiv: Thoughts and Observations
Tokyo is a city of wonders and isolation. I wanted to wait until I left to write this up, because I didn’t want to besmirch the city. Keep in mind, I had the highest hopes for Tokyo, and expectations are everything.
Tokyo is clean and modern, perhaps more than any other city I’ve been in. What was interesting was how something had to go wrong for me to realize that everything was going right. While walking near the prime minister’s residence, I smelled something bad. Was that… the sewers? Yes, it seems so. In Bangkok, you are constantly walking past a sewer that had become ripe, but that was the only time it happened in Tokyo. Someone beeped, and I realized that unlike Cebu, people aren’t doing that constantly. The weather predictions were accurate. In Bangkok and Cebu, not once was the current weather report correct with regards to precipitation, much less what would happen in a few hours. A train is a minute late. Huh, that never happens. Unfortunately, the biggest timing problem in Tokyo was the cherry blossoms arriving 2 weeks early, followed by rain and powerful winds, ruining my chance of seeing them on all but one tree. Thus were dashed by dreams of walking around Yoyogi Park while listening to Cherry Blossom Girl. Continue reading Tokyo: Parting Thoughts
This is making the rounds and is pretty interesting:
My framework of choice, Grails, doesn’t look spectacularly fast, although I already knew that. However, it is nice to see that it is notably faster than RoR (which appears to be a complete dog) and Django, and will probably improve quite a bit with v2.2 on Groovy 2.0 and Java 7.
However, it seems a big part of the performance lag is because it’s built on Spring, and to my surprise Spring isn’t a great performer, either. It is nice to see that straight up Servlets and Vert.x are beasts, so another score for our dear JVM.
Another thing that caught my attention is that Grails’s ranking improves quite a bit once you move it from EC2 to dedicated hardware. IaaS and PaaS save you some admin work, but also costs you some dev time, but at some point it looks like you’ll get a nice gain from dedicated hardware. It could make something like Rackspace managed hosting on dedicated hardware look reasonable.
With the attention it’s getting, I also expect a shakeup as framework providers help the benchmarkers improve performance. It will be interesting to see how this plays out.
I’ve recently arrived in Cebu City, Philippines. My impressions of the city are colored, of course, by living in the US, but also by spending the previous 2 months in Bangkok. Here’s what struck me. Continue reading Cebu City: First Impressions
I’ve been in Bangkok for a week and it’s an interesting city. Even though I heavily researched it (including spending time on expat forums and talking to former residents) there were some surprises. Here are some of them. Continue reading Bangkok: First Impressions, Observations, and Misconceptions
I am beginning a grand adventure. Well, for me, anyway. At the end of this year, I will be flying to Bangkok, Thailand to start a 6 month (or longer) trek abroad. You know, like Caine in Kung Fu. Well, not exactly.
This isn’t a vacation. It’s an experiment in nomadic living, inspired by the likes of Expat Software and The 4-Hour Work Week. Even after the success of Tim Ferriss’s book, this is a pretty foreign concept to most people. Upon hearing it, they think I’m going on a really long vacation, and by extension, I’m some secret millionaire. I’m not (yet). Or that I’ll be backpacking and living in hostels and villages. I’m not doing that either.
I’ll simply be living in other countries, much like I would here. The countries I’ve chosen are more affordable than Southern California (I know, not saying much), although that does get offset by renting fully furnished apartments in major cities, as well as airfare. My goal is to spend the same or less per month than I would here, while making things a lot more interesting – you know, meeting people and getting in adventures. But for the most part, I’ll be coding and learning new things.
Why now? What prompted this? There are several factors: Continue reading Basically, I’m just gonna walk the earth.
I recently had to restore a MySQL backup. It had been a while since I used MySQL, having switched to Postgresql sometime back. First thing I noticed was the lack of a GUI admin tool that PG spoils you with. Making matters worse, the command line tool isn’t even added to your path; you can’t just run “mysql” like you can in a Unix shell, much less use the redirect shortcut. And I was getting errors.
Here’s what you need to do to restore a MySQL backup on Windows:
- Run the command line tool from the start menu
- Open your backup file in a text editor. Does it start with a command to create or “use” the database? If not
- Create it, if necessary
- Type “use database” filling in your DB name
- Type “source path-to-SQL-file“. BUT, you must follow these rules:
- Use the full source command, not the \. shortcut
- Have no spaces in your path. I copied mine to a root of a drive. Note that spaces in the file name is OK, just not the path.
- Do not quote the file name, even if it has spaces. This gave error 22.
- Use forward slashes in the path, e.g., C:/path/to/filename.sql. Otherwise you’ll get error 2.
- Do not end with a semicolon.
Follow all those rules and it should work fine.
I’m pretty excited about Grails, but am often asked for social proof. Well, by every measure, Groovy is the #1 JVM language after Java, and Grails is the #1 full stack web framework. But for some, that’s not enough. They want to know who is making it #1? Major companies. Below I’ve listed the household names. I could not get details for some companies, but a Google search will verify the claim (e.g., the Disney developer posting a Grails question to a mailing list).
Last edit: November 6th, 2013 Continue reading Companies Using Grails
I needed to generate an XML file from database tables and the plan was to use Talend Open Studio. Talend is an ETL tool that generates data integration jobs in Java. The community edition is free and I’d been using it for several other data tasks for an ecommerce client. Overall, I think it’s quicker than hand coding in Java, but you can still dip into Java code if you need to and embed the jobs in other programs.
Unfortunately, it’s not so good when it comes to generating moderately complex XML files. By moderately complex, I mean lists of lists like this:
Continue reading Merging XML Files With Groovy