Archive for the 'Technology under the hood' Category

Hudson - too easy

Saturday, August 2nd, 2008

Continuous integration is a lovely thing. Automated testing is also lovely as you can see where things work or not almost instantly in different environments and scenarios.

Over the years I have used a few CI servers and tools - most notably, CruiseControl (Java and .NET) and Continuum (from the Maven crew). These systems, whilst useful, cam e with a few annoyances, most notably their slight fiddliness in getting up and running. Whislt they worked, it seemed that there were always tweaks required here and there to get things running smoothly and when things went wrong they we not that easy to correct.

After chatting with a few friends, I thought I would give Hudson a try as it seems like a new contender for CI server of the month. Spurned on by glowing feedback from said friends I though I would see how easy it would be to try and aggregate our disparate CI servers (For Java and .NET) into a single hudson installation - something advertised to be easy on the tin.

Here are the steps I followed:
- download WAR, runit
- config master/slave
- config maven app
- config .NET app
- force some builds
- too easy

A simple distributed build system that works cross platform sounded too good to be true, however it seems that hudson is well on the way to being just that. I couldn’t believe how easy it was to get a distributed build/CI system up and running and building/testing our Java and .NET apps with all reports and config managed centrally. I especially like the simple reports on build trends and the weather paradigm used for displaying build stability.

After this simple proof of concept, we moved the setup to a cluster of VMWare instances (Linux and windows) so we can now build multi-platform apps and manage things centrally for all projects. This is now our main CI platform and seems to be working well.

Hurray for progress!

No Fluff Just Stuff, April 2008

Tuesday, June 17th, 2008

With forecasts of April snow, our 2Paths contingent of two headed off at the crack of dawn one Friday for Bellevue, WA to attend the No Fluff Just Stuff - Pacific Northwest Software Symposium (NFJS). Having only spent 10 minutes at the border (whew!), we arrived early enough in the Seattle area to fit in some Fluff (aka shopping and dining) before getting into the Stuff.

After a quick introduction by Evangelist Scott Davis, and with hopes of winning an IPhone by the end of the weekend, I headed right into his double-bill of sessions about Groovy. The first one entitled “Groovy, the Blue Pill: Writing Next Generation Java Code in Groovy” was a good gentle introduction to neophytes like myself on Groovy basics. Now that my interest was piqued, I was eager to continue on with Scott’s next session “Groovy, The Red Pill: Metaprogramming, the Groovy Way to Blow a Buttoned-Down Java Developer’s Mind”. Never mind the Blue Pill - the Red Pill is where it’s at!

Scott’s enthusiasm for Groovy was definitely contagious, and my mind started churning, trying to think of ways to start introducing Groovy into current projects or future projects at 2Paths. An allure of Groovy is that it’s a dynamic language and can be run on the JVM. Groovy is implemented in Java, so the two languages offer seamless interoperability. Groovy compiles down to Java bytecode, so Groovy code can call Java code, and vice-versa. This would make integration into our existing infrastructure much simpler.

The Blue Pill was starting to take effect, as Scott showed us the terseness of the language. No need to labouriously write getters and setters, amongst other many nifty language shortcuts. Who needs semicolons or parentheses anymore? Because Groovy really is Java under the hood, programmers unwilling to let go of that extra baggage can continue to use it and nothing will break. We were then shown demos on how much more quickly we could bang out Unit Tests in Groovy.

In the next session, the Red Pill stuff was pretty mind-blowing. Introducing method pointers, closures, the ExpandoMetaClass. Method pointers make our code more readable and understandable by allowing us to alias methods with semantics tailored to our own business logic. Closures are an exciting and powerful inclusion in Groovy that aren’t available in Java, allowing us to pass around executable snippets of code. And last but certainly not least, there’s the ExpandoMetaClass that lets us either do a lot of really cool stuff, or get ourselves into a heap of trouble. With the ExpandoMetaClass we can intercept methods and inject methods into any Class. Scott showed us some fun examples that got us all fired up wanting to try more!

With my introduction to Groovy setting the stage for the weekend, I went on to choose more sessions on Groovy (Design Patterns, Testing, Groovy with Spring), but also attended other sessions such as Spring Configuration, Regular Expressions, and GIS for Web Developers. I found most of the other sessions mostly affirming knowledge I already had, but augmenting it with good snippets here and there that were new to me. Venkat Subramaniam was the other main Groovy Prophet, and I came home with his book “Programming Groovy” in hand, eager to start integrating Groovy at least into our testing infrastructure.

The atmosphere at NFJS was inspiring, as the speakers all had great enthusiasm for the technologies they were touting, and the audience was generally equally as keen. It felt great to be in a large conference hall amongst so many other like-minded programmers, all sharing, and getting fired up about what we do, and our seemingly unlimited potential. I fully recommend NFJS (regardless of me not winning an IPhone), and would love the opportunity to go again!

Seriously, No BullFluff

Friday, October 5th, 2007

Last week’s No Fluff Just Stuff symposium was undoubtedly the best Java event I’ve attended thus far, and the icing on the cake was Brian Sletten’s talk on “Beyond Cute-sy Mashups”. It was worth showing up for that alone. It’s not very often that it happens, but I’m stoked.

Below is my newly updated ‘Bleeding Edge Learning Curve Road Map’:

No Fluff Just Stuff

Friday, September 28th, 2007

The day started curiously as I was waiting for the 57, where I came across a quaint wannabe bus driver from Somalia. Naturally, I asked him why he had opted for life over here in Canada. “Here the supermarkets don’t sell bullets side by side with the vegetables”, was his honest reply. I never did get his name, but he did manage to leave an impression.

Anyhow, arriving at the square dusky-coloured Delta hotel, Barbie presented me with my name tag for this year’s No Fluff Just Stuff Western Canada Software Symposium. I inquired if it wasn’t necessary to register, at which she bluntly retaliated, “Are you lying about having paid? Well, you look like a honest guy, but nice of you to ask!”. Charming.

Scott Davis kicked off the show, and before we knew it, he had bewildered us Java techies with some Groovy magic. As his finale, he had Grails spooning off a fully-CRUD enabled web app in a split-second jiffy. Admittedly Grails does hit that sweet spot, which no doubt will have me succumbing to its allure, yet one growing pet peeve I have with it is that it does come with shackles of its own. For one, it imposes its own build tool, namely gant. But since, for better or for worse, most of us use maven as our build tool of choice, i would cast an early warning to the coders behind Grails - choice is key.

Thus, and surprisingly so, the much talked about shiny new toy didn’t earn my first merit, instead the honour went to a talk on Agile testing strategies. Jared, a likable confectionery-throwing speaker, not only talked a good talk, he also walked the walk. Test Driven Development (TDD) is a notion that has been much paraded about on the Agile front, yet until now I have remained skeptical. Until now, I didn’t fully take on board what its potential benefits could be. Enlightened, I now realize that by using TDD this has the knock on effect of it driving the design, hence driving the developer in keeping the code testable, which in turn keeps it simple, modular, and loosely-coupled. Genius.

And with that, I will finish off Day 1 with the quote of the day:
“I can’t fix stupid” ~Jared Richardson’s take on dumb developers.

Sweet Online Reference Tool

Thursday, June 7th, 2007

Got API

Unit Test Your Web Security

Thursday, May 31st, 2007

Web-based business applications require stringent security measures. Within a secure website, a user must be authenticated by the system and for each different user role a predetermined set of authorization rights must be imposed. This isn’t anything new, and luckily, there exists an open source Java security framework, known as Acegi, which makes the job of keeping those hackers at bay a relatively straightforward one.

Acegi is a powerful, flexible security solution for the Java enterprise application. Working in tandem with its best friend, Spring, it provides applications with comprehensive authentication and authorization capabilities.

Integrating Acegi into your Spring enabled web application is generally an easy, although perhaps sometimes tedious, process of adding a bunch of XML wiring into your Spring context. And after you add that little extra touch of custom configuration for your specific web application, you are good to go.

Or are you? How do you know the thing actually works? And, more importantly, how can you be sure that it will continue to work for every project build? Here’s a JUnit test that shows you how.


package com.twopaths.test.core.security;

import java.io.IOException;

import javax.servlet.Filter;
import javax.servlet.FilterChain;
import javax.servlet.ServletContextEvent;
import javax.servlet.ServletContextListener;
import javax.servlet.ServletException;

import org.acegisecurity.Authentication;
import org.acegisecurity.context.SecurityContextHolder;
import org.acegisecurity.ui.webapp.AuthenticationProcessingFilter;
import org.springframework.mock.web.MockFilterChain;
import org.springframework.mock.web.MockFilterConfig;
import org.springframework.mock.web.MockHttpServletRequest;
import org.springframework.mock.web.MockHttpServletResponse;
import org.springframework.mock.web.MockServletContext;
import
  org.springframework.test.AbstractTransactionalDataSourceSpringContextTests;
import org.springframework.web.context.ContextLoader;
import org.springframework.web.context.ContextLoaderListener;

public class AcegiSecurityTest
  extends AbstractTransactionalDataSourceSpringContextTests {

  private static final String SECURITY_CONTEXT_CLASSPATH =
    "classpath:/com/twopaths/test/core/security-context.xml";

  private static final String TEST_USERNAME = "test";
  private static final String TEST_PASSWORD = "test";

  private MockServletContext servletContext;
  private MockFilterConfig mockConfig;
  private FilterChain mockFilterChain;
  private Filter authenticationFilter;

  @Override
  protected String[] getConfigLocations() {
    return new String[] {SECURITY_CONTEXT_CLASSPATH};
  }

  @Override
  protected void onSetUpInTransaction() throws Exception {
    // Insert user dummy data
    jdbcTemplate.execute("insert into end_user values(1, 'ROLE_USR', '" +
      TEST_USERNAME + "', '" +
      TEST_PASSWORD + "', 'true')");
  }

  @Override
  protected void onSetUpBeforeTransaction() throws Exception {
    // Init servlet context mock
    servletContext = new MockServletContext("");
    servletContext.addInitParameter(
      ContextLoader.CONFIG_LOCATION_PARAM,
      SECURITY_CONTEXT_CLASSPATH);

    // Init servlet context listener
    ServletContextListener contextListener = new ContextLoaderListener();
    ServletContextEvent event = new ServletContextEvent(servletContext);
    contextListener.contextInitialized(event);

    // Init filter config and filter chain mocks
    mockConfig = new MockFilterConfig(servletContext);
    mockFilterChain = new MockFilterChain();

    // Init authentication filter
    authenticationFilter = (Filter)
      this.getApplicationContext().getBean("authenticationProcessingFilter");
    authenticationFilter.init(mockConfig);
  }

  public void testThatTestUserLogsInOk() throws Exception {
    // Build mock request
    MockHttpServletRequest request =
      new MockHttpServletRequest("POST", "/j_acegi_security_check");
    request.setParameter(
      AuthenticationProcessingFilter.ACEGI_SECURITY_FORM_USERNAME_KEY,
      TEST_USERNAME);
    request.setParameter(
      AuthenticationProcessingFilter.ACEGI_SECURITY_FORM_PASSWORD_KEY,
      TEST_PASSWORD);

    // Build mock response
    MockHttpServletResponse response = new MockHttpServletResponse();

    // Run authentication filter
    authenticationFilter.doFilter(request, response, mockFilterChain);

    // Get authentication instance
    Authentication authentication =
      SecurityContextHolder.getContext().getAuthentication();

    // Verify authentication instance
    assertNotNull(authentication);
    assertEquals(TEST_USERNAME, authentication.getName());

    // Verify response redirected URL
    assertEquals("/home", response.getRedirectedUrl());
  }

}

Automating Asynchronous Jobs with Java 5 and Spring

Friday, May 25th, 2007

<< The Spiel >>

Java 5 brings a new arsenal of weaponry when it comes to creating highly scalable concurrent applications. The JVM has been improved to allow classes to take advantage of hardware-level concurrency support, and a rich set of new concurrency classes has been provided to make it easier to develop thread-safe, well-tested, high-performance concurrent applications.

The following code example makes full use of these new features and leverages the power of Spring to fully automate an asynchronous job to run periodically for a given duration (or forever if required). There is also a couple of extra goodies in the bag. Not only do jobs get kicked off, we also get a result back when they finish. And if the unimaginable happens and the job hangs, no problem, a timeout will occur, thus ensuring that no threads are left hanging around to hinder our application’s robustness.

<< The Brains >>

package com.twopaths.core.asynchronous;

import static java.util.concurrent.TimeUnit.*;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.FutureTask;
import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.ScheduledFuture;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;

import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.springframework.beans.BeansException;
import org.springframework.context.ApplicationContext;
import org.springframework.context.ApplicationContextAware;

public class AsynchronousProcessor<T>
    implements Runnable, ApplicationContextAware {

  protected final Log logger = LogFactory.getLog(this.getClass());

  protected static boolean started = false;

  private ScheduledFuture handle;

  // Specifies which bean in spring to instantiate as a new job
  // Default is job
  private String jobBean = "job";

  // Specified initial delay before kicking off the first job
  // Default is 1 second
  private int initialDelay = 1;

  // Specifies the delay between finishing and starting a new job
  // Default is 10 seconds
  private int delay = 10;

  // Specifies how long to wait for a job to complete before timing out
  // Default is 60 seconds
  private int timeout = 60;

  // Specifies how long job will keep running
  // Default is 1 hour. Set to zero to run forever.
  private int duration = 60 * 60;

  protected final ScheduledExecutorService scheduler =
      Executors.newScheduledThreadPool(1);
  protected final ExecutorService THREADPOOL =
      Executors.newCachedThreadPool();

  private ApplicationContext applicationContext;

  public void setApplicationContext(ApplicationContext applicationContext)
      throws BeansException {
    this.applicationContext = applicationContext;
  }

  public AsynchronousProcessor() {
    this.start();
  }

  protected synchronized void start() {
    if (!this.started) { // Ensures that it only starts up once
      this.started = true;
      final ScheduledFuture handle =
        scheduler.scheduleWithFixedDelay(this, this.getInitialDelay(),
        this.getDelay(), SECONDS);
      if (this.getDuration() > 0) {
        scheduler.schedule(new Runnable() {
          public void run() {handle.cancel(true);}
        }, this.getDuration(), SECONDS);
      }
      this.setHandle(handle);
    }
  }

  public void run() {
    try {
      Callable<T> job =
        (Callable<T>) applicationContext.getBean(this.getJobBean());
      T result = this.call(job, this.getTimeout());
      this.logger.info(result);
    } catch (Exception e) {
      this.logger.error("Exception occured when running job", e);
    }
  }

  protected <T> T call(Callable<T> c, long timeout)
      throws InterruptedException, ExecutionException, TimeoutException {
    FutureTask<T> t = new FutureTask<T>(c);
    THREADPOOL.execute(t);
    T result = t.get(timeout, SECONDS);
    return result;
  }

  public void destroy() {
    if ((this.getHandle() != null) && (!this.getHandle().isCancelled())) {
      this.getHandle().cancel(true);
    }
  }

  public String getJobBean() {
    return jobBean;
  }
  public void setJobBean(String jobBean) {
    this.jobBean = jobBean;
  }

  public ScheduledFuture getHandle() {
    return handle;
  }
  protected void setHandle(ScheduledFuture handle) {
    this.handle = handle;
  }

  public int getDelay() {
    return delay;
  }
  public void setDelay(int delay) {
    this.delay = delay;
  }

  public int getInitialDelay() {
    return initialDelay;
  }
  public void setInitialDelay(int initialDelay) {
    this.initialDelay = initialDelay;
  }

  public int getTimeout() {
    return timeout;
  }
  public void setTimeout(int timeout) {
    this.timeout = timeout;
  }

  public int getDuration() {
    return duration;
  }
  public void setDuration(int duration) {
    this.duration = duration;
  }

}

<< The Slave >>

package com.twopaths.core.asynchronous;

import java.util.concurrent.Callable;

public class HelloWorldJob<T> implements Callable<T> {

  private String message = "Hello World"; // default

  public String getMessage() {
    return message;
  }
  public void setMessage(String message) {
    this.message = message;
  }

  public T call() {
    return (T) message;
  }

}

<< The Glue >>

<?xml version=”1.0″ encoding=”UTF-8″?>
<beans xmlns=”http://www.springframework.org/schema/beans”>

  <bean id=”job”
  class=”com.twopaths.core.asynchronous.HelloWorldJob”
  scope=”prototype”/>

  <bean id=”asynchronousProcessor”
  class=”com.twopaths.core.asynchronous.AsynchronousProcessor”
  destroy-method=”destroy”/>

</beans>

Hibernate Tip of the Day - Is it a feature or a bug?

Thursday, May 10th, 2007

Annotations have undoubtedly been a breath of fresh air since coming onto the Java scene, but here’s one big slap in the face which you can’t help but feel is totally unwarranted.

I’m talking about mixing your field and method annotations in Hibernate. This really bares its ugly head when annotating relationships.

Let’s have a look-see at the following code snippet:

@Entity
public class Book {

@Id
@GeneratedValue
private Long id;
private Publisher publisher;

public Long getId() {
return id;
}
public void setId(Long id) {
this.id = id;
}

@ManyToOne
public Publisher getPublisher() {
return publisher;
}
public void setPublisher(Publisher publisher) {
this.publisher = publisher;
}
}

So what does Hibernate think of my code? Like a friend who you come to depend on just as they give you the slip and leave you in the lurch, it spits out this exiguous exception:

org.hibernate.MappingException: Could not determine type for: Publisher,
for columns: [org.hibernate.mapping.Column(publisher)]

The real conundrum though, is it a feature or a bug?

Unbelievably, Emmanuel Bernard, the project lead for Hibernate Annotations, states that this is a feature.

Well, Emmanuel, this is one feature (a.k.a. a mighty kick in the cojones) which we could all certainly do without.

Daylight Savings Time changes

Friday, March 9th, 2007

What is DST?

Daylight Savings Time (DST) is an adjustment made to the local standard time, allowing people to make the best use of daylight during the different seasons. The mnemonic device “Spring ahead, Fall back”, refers to how one should adjust a clock to correctly navigate the move to and from DST.

New start and end dates

As an energy-saving move, the United States extended their DST by four weeks, requiring less artificially-generated hours during business hours. Most of Canada and Bermuda followed suit to keep in synch with their neighbour. Starting in 2007, DST will begin two weeks earlier, on the second Sunday in March, and will end two weeks later, on the first Sunday in November.

Impact on software

In order correctly represent the time, and to make date/time calculations, your software will need to have the DST rules internally coded. This is a well-understood issue, and your current system should seamlessly handle the adjustments required to accommodate DST. However, unless you have a very recently created system, some aspects may not refer to the new start and end dates. If this is the case, in the four new weeks of DST, the correct current time may not be represented by your system, and time span calculations for dates that cross on or more of the DST transition dates may be incorrect.

Risk assessment

Before making a change to your system, determine whether the system will be affected by the new daylight savings time dates. If it does not rely on calculating, storing and displaying date/time values, you might not need to make any change to your system, even if it does not been recently updated.

If you determine that your system will be affected, make an assessment to ensure the impact is sufficient to warrant making the change. What is the cost to your business to have incorrect time and/or date/time calculations for the period, versus the cost in bringing your system down to apply the required changes? Do you have any system upgrades already scheduled, during which you could apply updates to revise the DST rules, gaining two birds with one system outage?

Making the change

Once you’ve decided you need to make the upgrade, you will need to understand your system, to determine which portions will need to be updated. At 2Paths, we focus on J2EE web applications using Postgres databases. For such systems, the operating system, Java Runtime Environment (JRE) and database are all candidates for upgrades. The latest versions of each have the new DST rules are in place, and once these have been added, your system should handle the new Daylight Savings Time dates correctly.

Simple, lightweight Java development

Wednesday, February 28th, 2007

Through the short arc in the history of the software development industry, developers have taken on ever larger and more complex problems. Programmed routines became formalized as components; applications were developed to communicate with each other, creating building blocks that allowed software architects to erect ambitious towers of code resembling the Biblical Babel. The result was a crisis in software development in the 1990s, where it became evident that the complex systems under development were becoming impossible to control.

There have been several responses to this crisis, and one of the most promising is the move toward lightweight development. This movement embraces the principle of simplicity, and by reducing the complexity of the code, other aspects of a project — performance, testing, maintenance, management — are likewise made easier to address.

At 2Paths, we focus on Java enterprise development, which allows us to address enterprise-scale problems that are out of reach from more limited technologies such as Microsoft VisualBasic and PHP. We maintain simplicity through our commitment to using lightweight, open source technologies, including Spring and Hibernate — which are well-accepted in the movement — as well as our own Tequila framework. To make the most effective use of these tools, we also incorporate lightweight coding practices whenever possible, such as adhering to an architecture based on REST and a POJO-driven domain.

In addition, we follow lightweight development processes. Our agile methodology is based on XP and Scrum, with elements of each chosen when most applicable to our own business. We practice test-driven development to allow us the freedom to nimbly change direction as our clients’ needs require. The resulting test framework allows us to effectively refactor our code, extending our lightweight development long into the lifecycle of our projects.

The focus on lightweight development contributes to our simplicity, leaving us more free to concentrate on productivity and refinement, while leveraging accepted best practices in our industry.