Hadoop, trying it.

Full Project: https://github.com/tsolakp/hadoop-sample

There has been a lot of talk about Hadoop lately but with very little explanation of what it actually is. Unfortunately some of tutorial have not been simple enough to give me a big picture of what this technology is all about.
After some reading http://developer.yahoo.com/hadoop/tutorial/ comprehensive tutorial on Yahoo and trying I want to share my observations.

What it is and what it is not.

First of all Hadoop is simply a batch application. It is designed to process large amount of data (for example in forms of large data files) in parallel over distributed system where each machine is called a node.

It is not a database like the related Apache projects called Hive or HBase.

It consist of two major parts. One is distributed batch app that manages job on each node and second is the HDFS (file system) which allows Hadoop to transparently distribute large files across nodes so that each job on the node could access and process the file or its parts.

One probably obvious but crucial aspect of Hadoop is that it makes sure the processing code (i.e Mapper’s or Reducer’s Java classes) are run closer to the data which means that unlike Spring Batch, for example, it consists of multiple processes running on each node and controller by master “namenode”.
This makes it difficult to debug, but fortunately you can run Hadoop jobs just like plain Java “main” application for testing before deploying to “pseudo” (all nodes are run on single machine) or “real” (nodes are actually separate machines) environments.

Continue reading Hadoop, trying it.

EasyMock gets more power with PowerMock.

There are a lot of mocking libraries for Java, out of which EasyMock is my favorite. It has been serving well for me until I had to unit test legacy code which did not completely adhere to IOC (inversion of control) principles and had a lot of static method calls and direct creation of service/helper objects instead of using dependency injection. Unfortunately EasyMock could not handle mocking of static methods and objects which were created directly in the class or method being tested along with some other limitations such handling final classes.
This forced me to use other mocking libraries until I encountered PowerMock. It provides all the ammunition (and more) in dealing with these issues and has nice integration with EasyMock.
Here I’ll show you how to use EasyMock with PowerMock in order to handle common mocking test cases which were not possible with EasyMock before.

Continue reading EasyMock gets more power with PowerMock.

Ubuntu and Dual Monitor with ATI x300.

I am kind of new to linux, otherwise I probably would not be writing this article or spend so much time figuring out the solution I am about to describe.
Anyways, I have this old IBM T43 that I use for my work and ever since I installed Ubuntu on it I just could not get laptop’s monitor to behave as second monitor
and have big Dell one to be as the primary one.
I installed “Monitors” app for Gnome and every time I would select Dell as primary the Gnome would freak out and show total garbage on both screens.
After some Googling all I could conclude is that the main problem was my ATI X300 mobile graphics card that was not supported any more by AMD/ATI and
the only supported open source drivers (that I was using) did not work very well in dual monitor setup.
Also the only suggested fixes required complicated xrandr commands or xorg.conf changes that did not work in my case.
What actually worked is this pretty easy fix that I got when paying more attention to what Gnome’s “Monitors” app was doing when applying the changes.

Continue reading Ubuntu and Dual Monitor with ATI x300.

JSP myths.

Since JPS’s came out as part of J2EE there have been lots of talks of how they were so ASP like and were badly designed.
But since there was not any alternative that time the Java community had to embrace it and used it. In the end JSP made into most J2EE based web applications.

Then in the last few years JSP started to get bad press again with advance of AJAX, component based frameworks like Wicket , JSF, Tapestry and with template based frameworks like Freemarker.

Most of bad press was around the difficulty to work with JSP as HTML developer, JSP containing too much Java code, JPS being difficult to debug, JSP not being suitable to develop component based web applications, JSP becoming obsolete since what was done in JSP can now be done with AJAX on client browser.

There are some valid points and I am personally not a fan of JSP either but want to get things straight and show that those issues have nothing to do with JSP, that JSP is not dead yet and can very well play nicely with other web frameworks and AJAX.

Continue reading JSP myths.

Maven2 eclipse plugin and custom or pom packages.

Update: This approach will also work with “ear” projects and only requires adding plugin definition without changes to eclipse maven plugin.

Recently I had to create maven project which was producing war file and then packaging it into tar before deploying to maven repository without deploying the war itself. This forced me to declare my pom package as “pom”. Everything was fine until I tried to run “mvn eclipse:eclipse” which gave me error that it does not support “pom” packages.
After googling I stumble upon this ticket

http://jira.codehaus.org/browse/MECLIPSE-310

which suggested to specify “war” (or “jar”) packaging in eclipse plugin configuration like so

<build>

        <plugins>

            <plugin>
                <groupid>org.apache.maven.plugins</groupid>
                <artifactid>maven-eclipse-plugin</artifactid>
                <configuration>
                    <packaging>war</packaging>
                </configuration>
            </plugin>

    ….

</plugins></build>

Continue reading Maven2 eclipse plugin and custom or pom packages.

HOA and Software Development. Whats common?

I was so shocked when I first encountered HOAs (Home Owner Association) that I started researching online about it to just to see how someone in his right mind will allow bunch of people to tell him how he needs to live in his house and at the same time pay for HOA.
I was not alone and there are even forums devoted to this where sometimes you can see comments from these so called HOA proponents who always bring up this common “Purple House” example.

Purple House.

What if your neighbor paints his house purple? According to HOA folks there is no way you can prevent it from happening without HOA rules and the value of your house will suddenly go to zero.
So, everyone in the neighborhood gets scared and agrees to set up an HOA for the community. And since HOA is simply a corporation it requires board of directors, insurance, monthly fee and at least a monthly meeting.
This of course, will have all hard working people to avoid it since they won’t have time for these things, leaving only those who have nothing to do in their lives but seek a power to control others. Usually these are old people or retired lawyers :)
Once the board gets elected, they start coming up with rules to supposedly avoid Purple House case, such as requiring everyone to paint their house with only few colors and bunch of other restriction that board (remember old people) finds reasonable.
Eventually these rules start to apply to the height of your lawn, how your car is parked, how your kids play outside or how you do your barbecue. This certainly will anger a lot of people but they won’t have any legal case against HOA and will end up paying HOA to be told what to do just to avoid the Purple House which would probably have never happened and could have been dealt with more relaxed City restrictions.

So how does it apply to Development?
Continue reading HOA and Software Development. Whats common?

Issues with depepency injection in latest frameworks.

Here I am going to show some of important limitations to Java and Guice approached to dependency injections, particularly where wiring logic should go.

Java Dependency Injection

public interface LogService {
public void log(String result);
}

@Named(“FileLogService”)
public class FileLogService implements LogService{
public void log(String result){}
}

@Named(“ConsoleLogService”)
public class ConsoleLogService implements LogService{
public void log(String result){}
}

public class LogClient{

@Inject
@Named(“FileLogService”)
private LogConsole logConsole = null;

public void doSomething(){
logConsole.log(“doSomething”);
}
}

This is one way of injecting dependency into client code using Java annotation. As you can see we just hard coded LogClient to FileLogService and there is no way to have LogClient use ConsoleLogService.

Continue reading Issues with depepency injection in latest frameworks.

Spring WS client.

In my previous post I described how to create sample Spring Web Service.
In this article I am going to show how to write generic Web Services client for it (or for any other Web Services).

The idea here is to define java interface that will mirror WS request/response messages and have a Spring proxy to take care of WS calls and XML marshalling.
In previous article we defined SampleEndpoint that was Java back bone of our WS.
What we whould like to do is to have Java interface that will mimic SampleEndpoint public methods and have Spring create WS proxy for it to be injected into our WS clients.

Continue reading Spring WS client.

Spring WS+JAXB

Download projects

Here is sample Spring WS application that shows how to expose your bean method as WS service and handle XML marshaling using JAXB.

There are 4 major parts involved here.

Continue reading Spring WS+JAXB

Spring MVC+Custom Multy Action support.

Download projects

Test (default tomcat 6.0.14 install)http://localhost:8080/eqs-pim-webapp-1.0.0/login.htm
Enter nothing on login.

This article tries to show how to put together relatively complex web application using Spring MVC and is an attempt to show some of the best practices of web application development and packaging.
To address some of the Java web development problems and to achieve simplicity of Ruby on Rails additional abstract classes and tag libraries have been introduce as part of this sample application.
You can use included source code as template for your future projects.
Continue reading Spring MVC+Custom Multy Action support.