Java/Tomcat Web Services

I've decided I want to have my Java algorithms library available as a web services API. This post will describe how I set about to accomplish that.

First of all, I must give credit where it is due. Most of the information on how to do this for Tomcat (my container of choice right now) came from this document by Peter Yeung. Mostly the steps laid out in that document were sufficient for my project, although there were a few gotchas that took me some time to work out. I think he was using an older version of the Java Web Services Developer Pack. It looks like there have been a few changes, including the introduction of bugs!

So, to start, I downloaded the above mentioned JWSDP as noted above. This definitely makes the job of creating the web service infrastructure easier. It has utilities to generate the WSDL and a bunch of SOAP utility classes, stuff for serialization and such. Who would want to deal with all that? Not me! (Indeed, one could create highly functional web services without ever knowing WSDL using this an other similar utilities out there.)

The actual programming steps are then quite simple. First, one needs to create an interface to define the functionality of web service. For my Warshall algorithm, it looks like this:

package algorithms.server;
import java.rmi.Remote;
import java.rmi.RemoteException;

public interface Warshall extends Remote{
 public PathMatrix getPathMatrix(byte[][] adjacencyMatrix) 
  throws RemoteException;
}

Then I created a WarshallImpl class that implements that interface (I won't bother posting it here since it's just a POJO, basically the Warshall.java I posted earlier, but I will make the whole project available to download.) Also I created the PathMatrix class rather than just returning the matrix as an array since I want to be able to send a status back to the client, like if it made an unreasonable request.

Now, you need two XML files. One is a JaxRPC web service description file that the JWSDP deploy utility will use to get its necessary information. It should be named jaxrpc-ri.xml. It looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<!-- configuration file for JWSDP wsdeploy tool -->
<webServices xmlns="http://java.sun.com/xml/ns/jax-rpc/ri/dd" version="1.0"
targetNamespaceBase="http://com.test/wsdl"  
       typeNamespaceBase="http://com.test/types"  
>
<!-- the endpoint name becomes the service name in the WSDL -->
<endpoint name="warshallService"
interface="algorithms.Warshall"
implementation="algorithms.WarshallImpl"/>
<endpointMapping endpointName="warshallService" urlPattern="/warshallService"/>
</webServices>

Then you need a deployment descriptor. This was the first gotcha from Peter Yeung's instructions. He said to make an empty web.xml, i.e. just web-app tags with no content. This doesn't work. The utility will generate some kind of null pointer exception. After some Googling, I came up with this solution:

<?xml version="1.0" encoding="UTF-8"?>
<web-app>
<session-config>
<session-timeout>60</session-timeout>
</session-config>
</web-app>

Put both the web.xml and raxrpc-ri.xml in a config directory under your project root, and put the java source in a src directory. Also make a build directory, a dist directory and a web directory. One might choose to simplify this by keeping the configuration files in the web/WEB-INF directory, and building straight to to web/WEB-INF/classes. Or, just make a nice ant script to automate it all. I'm in the process of doing the latter, which I'll include with my project when I upload it. This will make it all very easy. In the meantime, after building the project I have to copy the conf and build contents into the web/WEB-INF and web/WEB-INF/classes directories.

Now we get to the magic parts. First, made a war file from the web directory:

jar -cvf build/myWEB-INF.war -C web/ .

Then I ran
%JWSDP_HOME%/jaxrpc/bin/wsdeploy.bat -verbose -o dist/algorithms.war build/myWEB-INF.war assuming that %JWSDP_HOME% is the install directory of JWSDP. This makes a new war file that is the web service app itself, that can then be deployed to Tomcat. Just dump this war file in the Tomcat webapps folder and the web service is deployed. That's it, done. Oh, do make sure you put the needed .jar files from the JWSDP distribution into the Tomcat lib. This is %JWSDP_HOME%\jaxrpc\lib, %JWSDP_HOME%\saaj\lib, %JWSDP_HOME%\jwsdp-shared\lib and %JWSDP_HOME%\jaxp\lib\endorsed.

OK, so now we have a functional web service. We can check that it works like so: http://localhost:8082/algorithms/warshallService?WSDL. That shows us our WSDL, which we didn't have to bother to write.

Now, how about a client? JWSDP also contains an automatic client maker. Nice! It generates all the client stub classes for us. All that's needed is another simple XML configuration file to give it instructions. For my client, it looks like this:

<?xml version="1.0"?>
<configuration xmlns="http://java.sun.com/xml/ns/jax-rpc/ri/config">
<!-- WSDL URL and generated package name -->
<wsdl location="http://localhost:8082/algorithms/warshallService?WSDL" packageName="clientStub"></wsdl>
</configuration>

I named it /config/wscompile_config.xml under the project root.

Then I ran:

%JWSDP_HOME%/jaxrpc/bin/wscompile -d build -gen:client -keep config/wscompile_config.xml

That generates all the client stub classes. Finally, make a jar of it:

jar -cvf lib/wsstub.jar -C build clientStub/

OK, now we're good to go... mostly. This was the other big gotcha. To actually use this we will need the necessary jars from the JWSDP distribution. The one's we need are from: jwsdp-shared\lib, \fastinfoset\lib and \jaxrpc\lib. Now I come to the biggest gotcha of this project You also need the saaj jars, but the ones included with the JWSDP dist DIDN'T WORK. It seems like there is some kind of incompatibility with some of the other jars, or maybe something. But I kept getting a class cast exception, specifically:

java.lang.ClassCastException: com.sun.xml.internal.messaging.saaj.soap.ver1_1.Message1_1Impl cannot be cast to com.sun.xml.messaging.saaj.soap.MessageImpl

This one drove me crazy. Finally I solved it by downloading the lib files directly from the SAAJ (SOAP with Attachments API for Java) project, https://saaj.dev.java.net/. That indeed solved the problem. And I was done with my client. My source, the part I had to write myself, looks like this:

package algorithms.client;
import java.rmi.RemoteException;
import java.util.Random;

import javax.xml.rpc.ServiceException;

import clientStub.*;

public class WarshallClient {
 public static void main (String[] args) {
  WarshallService warshallService = new WarshallService_Impl();
  Warshall warshall = null; 
  try {
   warshall = warshallService.getWarshallPort();
  } catch (ServiceException e) {
   e.printStackTrace();
  }
  try {
   PathMatrix pm = warshall.getPathMatrix(new byte[][]{{0,0,0,1},{1,0,1,1},{1,0,0,1},{1,0,1,0}});
   System.out.println();
   System.out.printf("Status=%s\n",pm.getStatus());
   System.out.print("Path matrix:");
   for(int i=0;i<pm.getPathMatrix().length;i++) {
    System.out.println();
    for(int j=0;j<pm.getPathMatrix().length;j++) {
     System.out.print(pm.getPathMatrix()[i][j] + ",");
    }
   }
  } catch (RemoteException e) {
   e.printStackTrace();
  }
 }
}

WarshallService, WarshallService_Impl and Warshall were all generated automatically by the wscompile utiltiy. Warshall.java is just a stub for the actual web service. The rest is just to test that it's actually working. And it does! Now, making web services is about as easy as making the POJOs. Nice! Now I can add my algorithms API to the cloud! I'm sure thousands of programmers are waiting anxiously. :)

Anyway, I'll post the whole source as soon as I have a chance to clean it up a little. In the meantime, I hope my experience proves useful to others.

Java Interview Questions

Here are a few of the more notable Java programming questions I've had at interviews lately.

Q: How do sets and lists differ in handling multiple nulls?

A: In a Set, there will have be single null, while in a List there will be as many nulls as were inserted. This is because sets consolidate equal inputs into a single entry (no duplicates) while lists allow multiples of the same value. BTW, this comes from the mathematical definition of sets and lists.

Q: Given the following: Set s = new HashSet<String>(); what is the behavior of this set? For instance what happens if I add, say, an Integer?

A: The Set will contain the Integer. No compiler or runtime errors will occur. Whether or not a collection is a generic depends on the declaration type, not the assignment type.

Q: In an HttpSession, if one adds a HashMap as a session attribute in a request, then in another request retrieves the HashMap and adds another entry: is it necessary to set the attribute again for yet another request to retrieve the new HashMap entry?

A: Yes. If you don't then the HashMap will appear to the subsequent request as if you haven't updated the HashMap. In other words, if one modifies the object in a session attribute, the attribute needs to be re-set with the object.

Q: Are there memory leaks in Java?

A: Not in the traditional sense as they are understood in C programming, for instance. By that I mean that one doesn't need to worry about explicitly freeing memory after allocating it. The garbage collector takes care of that. There are programming habits that lead to effective memory leakage, however. For example one might have a private instance variable that is initialized in, say, a constructor. That variable might not be referenced anywhere else in the class but as long as an application retains a reference to the class object itself the variable can't be garbage collected. Now this is probably really an example of the bad programming technique of using instance variables when a local variable will do but, still, it illustrates how allocated memory can stay allocated even though it is not reachable. In a more complex example it might not be as obvious as in this simplified version.

Warshall's Algorithm in Java

For fun (I have a nerdy definition of fun, I admit) I decided the other day, hanging out at Peet's Coffee, to bang out Warshall's algorithm in Java. Ah, the life of an unemployed programmer.

Before I present it, I suppose I should explain a little bit about what Warshall's algorithm does, in case there some readers who have never heard of it (like me less that a year ago). I'm not going to go deep, it being a well established and easily Googleable subject. For instance, this page does a better job than I'm likely to do in any case.

Here it is in a nutshell. Given a directed graph find out which nodes have paths between them. This is also known as the transitive closure of the graph for those who prefer the more impressive sounding mathematical terminology. For Warshall's algorithm one starts with the adjacency matrix of the graph, which is a matrix in which all nodes with direct connections (edges) between them are identified with a 1, and all other pairs of nodes with a 0. Warshall's algorithm generates from this data a new matrix with a 1 for all nodes that are connected by a path, 0 for all others. By "path" I mean that by traveling from node to adjacent node (those with an edge between them) one can arrive at the other node in a pair of nodes. Since this is a directed graph one must follow the direction of the edge in question. Think of it as a game of catch where each player only throws to certain other players. The adjacency matrix would identify which players throw to specific other players; a 1 row 1 column 2 if player 1 throws to player 2, a 0 otherwise. It's quite possible, based on this scenario, that some players never get the ball thrown to them. A fair PE teacher could implement Warshall's algorithm to identify that situation. If only more PE teachers understood algorithms the world would be a more equitable place. (One interesting side note, row 1 column 1 will always be 0 in the adjacency matrix since a player never throws the ball to himself, but it could be 1 in the path matrix since hopefully the player can get the ball back... or maybe not!)

The way Warshall's algorithm works is like this. Enumerating through each node numbered by k, starting with k=0 (i.e. node #0), decide which node pairs i,j have paths through node k. This can be determined by evaluating wither or not a path from i to k and from k to j already exists on the path matrix, which was initialized as the adjacency matrix. If it does, then the path from i to j can be added to the path matrix. Our paths will thus grow organically as we iterate through k. When we done we will have every path identified in our matrix since every path with every intermediate node will have been identified.

Anyway, now that I've probably confused everybody (Google "directed graph" to become un-befuddled) here's the Java code. I've also included a test case class which I think helps to understand it. So, without further ado:

package com.blogspot.ezetter.algorithms;

public class Warshall {
 public byte[][] getPathMatrix(byte [][] adjacencyMatrix) { 
  int numberOfNodes=adjacencyMatrix[0].length;
  byte [][] pathMatrix = new byte[numberOfNodes][numberOfNodes];
  System.arraycopy(adjacencyMatrix,0,pathMatrix,0,numberOfNodes);
  for(int k=0;k<numberOfNodes; k++) {
   for(int i=0;i<numberOfNodes; i++) {
    if (pathMatrix[i][k]==0){
     continue;
    }
    for(int j=0;j<numberOfNodes; j++) {
     if(pathMatrix[i][k]==1 &&
      pathMatrix[k][j]==1) {
       pathMatrix[i][j]=1;
     }
    }
   }
  }
  return pathMatrix;
 }
}

And the test case:

package com.blogspot.ezetter.algorithms;
import junit.framework.TestCase;

public class WarshallTests extends TestCase {
 private byte[][] adjacencyMatrix = 
  {{0,0,0,1},{1,0,1,1},{1,0,0,1},{1,0,1,0}};
 private byte[][] expectedPathMatrix = 
  {{1,0,1,1},{1,0,1,1},{1,0,1,1},{1,0,1,1}};
 private byte[][] adjacencyMatrix2 = 
  {{0,1,0},{0,0,1},{0,0,0}};
 private byte[][] expectedPathMatrix2 = 
  {{0,1,1},{0,0,1},{0,0,0}};
 
 public void testGetPathMatrix() {
  byte[][] pathMatrix = new Warshall().getPathMatrix(adjacencyMatrix);
  for(int i=0;i<adjacencyMatrix.length;i++) {
   for(int j=0;j<adjacencyMatrix.length;j++) {
    assertEquals("Path matrix at i=" + i + ", j=" + j + ":",
     expectedPathMatrix[i][j],pathMatrix[i][j]);
   }
  }   
  pathMatrix = new Warshall().getPathMatrix(adjacencyMatrix2);
  for(int i=0;i<adjacencyMatrix2.length;i++) {
   for(int j=0;j<adjacencyMatrix2.length;j++) {
    assertEquals("Path matrix at i=" + i + ", j=" + j + ":",
     expectedPathMatrix2[i][j],pathMatrix[i][j]);
   }
  }   
 }

Spring MVC vs Struts Part 2

This Part 2 of my experience switching from Struts to Spring MVC. Part 1 is here.

Now, before I get attacked by a Spring expert I am not claiming that this as a "best practices" post. I'm just describing my experience with Spring MVC and the solution I came up with. I would truly welcome a Spring veteran to come along and explain the right way to do it.

What I wanted was to retrieve certain information from the model and inject it into the view whether one was coming to the controller from the form submission or from any other source, like a link. At first I tried using the formBackingObject since it is always called but since one doesn't have access to the ModelAndView there that doesn't work well. I tried using handleRequest which is also always called but since I was therefore overriding the method from a superclass, which calls onSubmit, I was masking that functionality and causing my onSubmit to be uncalled. I then tried using showForm, which is called to display the form. This ended up being my final, relatively satisfying, solution. It did leave one problem. I wanted to return to the same form after submission but that proved tricky to do from onSubmit. My solution was to call showForm from onSubmit and then return the ModelAndView returned by showForm.

This has been working pretty well, although not entirely aesthetically satisfying. I wish it was simpler but as with many things, once you figure it out, it's not too bad. If you doubt it's complex check out the workflow in the Javadoc. One has to consider both the workflow in the class and all it's superclasses. On the plus side this give one a great deal of control over how a request is handled.

But when my headache reaches a certain threshold, I long for Struts.

Note: I will post the actual source soon, in a more detailed description of this project.

Spring MVC vs Struts

I mentioned a few posts ago that I recently switched from Struts 2 to Spring MVC as my web development platform for a current project. I also mentioned that so far I don't regret my decision although I have some reservations. What I like about Spring MVC is that 1. it's more closely integrated with Spring, and Spring is most assuredly a worthy invention and 2. it does have in certain respects more flexibility than Struts. I'll probably have more to say on those strengths later but for now the point is that that's also Spring MVCs weakness. With flexibility often comes complexity and the Spring MVC controller model is definitely complex relative to Struts'.

Struts has a very tight and simple coupling between the view and the controller. In Webwork (the predecessor to Struts 2) it was extremely simple. You registered your controllers in xwork.xml and referenced them in the actions of your forms and your controller's execute would be automatically called by Webwork on each form submit. A get request directly from a URL is handled in exactly the same way. In Struts 2 it's similar, although a little more flexible. In the struts.xml file one can configure specific methods of the controller to be called, so you aren't just stuck with execute. That was a nice addition.

In Spring MVC it's more complex. First of all Spring MVC mirrors the HTTPServlet doGet/doPost methods more closely as opposed to Struts higher level of abstraction. But Spring adds more methods to handle different cases like onSubmit for form sumission, showForm to display the view and formBackingObject to link the model to the view. These are all chained together depending on the way the controller is entered, say a URL get vs. a form post. Contrast this with the simple description for Struts I gave above. Just one method called, configurable in the struts.xml, no matter how the form is called.

Where I ran into trouble was to be able to have identical things happen whether a user came to the controller via a link or a form submission. I want to check if the user is logged in and I want to retrieve certain information from the model whether this is the initial display of the view or a form submission. The problem is, which methods to use? There are so many to choose from and some don't get called in certain circumstances. Do I use doPost, handleRequest, formBackingObject, showForm or some combination thereof?

Since this post is getting a little too long I think my solution is going to have to be in a "to be continued".

So, stay tuned.

The ORM Paradigm

I recently gave a presentation for a job interview on the value of Object Relational Mapping tools and, specifically, JPA. For the already enlightened it does contain technical information on using JPA which might still be of interest, I've decided JPA is the best ORM implementation yet.

Say to your programming environment: "be the database".

The presentation can be found at this link.

Static Analysis Tools

I'm evaluating static analysis tools for my Master's program at Cal Lutheran University. I'm finding the subject intriguing. Static analysis programs test code for potential flaws and are contrasted with dynamic testing tools like JUnit. They check code for deviations from established best practices, good design standards and possible serious issues. In my experience so far these tools generally do a good job. At the minimum they are useful for learning. One might not even always agree with the results but they are often interesting from an academic standpoint. It's especially fun to run them on popular open source projects. It's therapeutic for the ego to realize how pervasive certain bad practices are. By that token, running them on one's own projects might help one avoid such embarrassments.

The best of breed in the Java space are: FindBugs, PMD and Checkstyle. Each of the tools have different focuses, Checkstyle being oriented towards formatting conventions for instance (although also evaluating for more serious issues). One interesting thing is that they all seem to find different problems so they work well in conjunction. They each have IDE integration, specifically Eclipse plugins. Also they each have build tool integration with ANT. I think a development environment can benefit greatly by using both these features: use them in the IDE for real time preemption and then integrate into the automatic builds to ensure clean code across the team (I dub the concept Bug Brother). Few could argue the benefits of clean code for bug avoidance and maintenance. Best practices are in many cases well established across the industry but even good programmers can find it difficult to keep them all straight and to maintain the discipline to apply them. Tools like these can keep us honest. Let's use them.

Here's a sample Ant build.xml snippet from my project (the one I hope to share more on soon):

 <taskdef name="findbugs" 
          classpath="${findbugs.home}/lib/findbugs.jar" 
          classname="edu.umd.cs.findbugs.anttask.FindBugsTask" />

    <target name="findbugs" depends="build">
       <findbugs home="${findbugs.home}" output="xml"
                     outputFile="bcel-fb.xml">
            <sourcePath path="${src.dir}" />
            <class location="${build.dir}" />
            <auxClasspath path="${web.dir}/WEB-INF/lib/spring.jar" />
            <auxClasspath path="${web.dir}/WEB-INF/lib/spring-webmvc.jar" />
            <auxClasspath path="${web.dir}/WEB-INF/lib/persistence.jar" />
            <auxClasspath path="${web.dir}/WEB-INF/lib/commons-logging.jar" />
            <auxClasspath path="${web.dir}/WEB-INF/lib/servlet-api.jar" />
       </findbugs>
    </target>
 
<taskdef resource="checkstyletask.properties"
         classpath="${checkstyle.home}/checkstyle-all-5.0-beta2.jar"/>
   
<target name="checkstyle"
        description="Generates a report of code convention violations.">

  <checkstyle config="sun_checks.xml"
              failureProperty="checkstyle.failure"
              failOnViolation="false">
    <formatter type="xml" tofile="checkstyle_report.xml"/>
    <fileset dir="${src.dir}" includes="**/*.java"/>
  </checkstyle>
</target>
 
<path id="pmd.classpath">
    <pathelement location="${build}"/>
    <fileset dir="${pmd.home}/lib/">
        <include name="*.jar"/>
    </fileset>
</path>

<taskdef name="pmd" classname="net.sourceforge.pmd.ant.PMDTask" 
classpathref="pmd.classpath"/>

<target name="pmd">
      <pmd>
  <ruleset>basic,imports,unusedcode,strings</ruleset>
          <formatter type="net.sourceforge.pmd.renderers.HTMLRenderer"
 toFile="pmdout.html"/>
          <fileset dir="src">
              <include name="**/*.java"/>
          </fileset>
      </pmd>
</target>
   
<target name="analysis" depends="build,findbugs,checkstyle,pmd"/>

My April Fools Program

I meant to post this yesterday but, oldest excuse in the book, got busy. Anyway, not wanting to waste the opportunity and not willing to wait a year I'm posting it now. It is an example using a new feature of Java reflection. Here goes:

package april.fools;

public class Thp {
 
 Thp(Object input) {
  while(Class.doesHalt(this.getClass(),input));
 }
 
 public static void main(String [] args) {
  try {
   new Thp(Class.forName("april.fools.Thp"));
  } catch (ClassNotFoundException e) {
   e.printStackTrace();
  }
 }
}

I hope you appreciate it.

A Project Log (plog)

In the course of these postings I intend to update readers on the progress of a project I am undertaking. The gist of my project is to create a web application to facilitate user creatable online courses. I intend it to become a synthesis of an online university, a blog and a social networking site. As such it will contain functionality for creating course materials in the form of blog-type postings, designing quizzes both multiple choice and free form and common social network features similar to those found on linkedin, twitter, etc. I think it's an ambitious project and I don't know where it will end up but it will at least be a good learning experience. I plan to release it under GPL and since I will be sharing my step by step design experiences it should be easily implementable by anybody who wishes to steal my idea (giving me deserved credit, no doubt).

Anyway I have already started on the project but I will in the course of posting about it back up and start from the beginning thereby giving the full design cycle. It will be a Java web application using Spring MVC as the framework and JPA for data access. I started the project on Struts 2, with which I'm already familiar, but decided to switch to Spring since I wanted to learn it. So far I haven't regretted it although there are things I like about Struts that I miss in Spring MVC (more later) and it has been a steep learning curve. But, hey, learning is what it's all about.

So, in subsequent posts I will lay out my basic design and then move through implementation details as I encounter (encountered) them. Hopefully it will be an interesting experience for the reader, as it has been for me.