May 2011 Archives

Injecting context in Java serialization

| | TrackBacks (0)

Serialization is a useful feature built into Java. It converts objects into a bytestream that can be saved to a file or transmitted over a network and then reconstructed later into the same object (perhaps in a different Java VM). The basics are detailed here, with some more advanced features here and the complete API is here. Despite all this documentation, I recently came a across a problem not described. What if an object being deserialized requires a reference to some non-serializable object in the
application. That is how can an object be reconstructed when it contains part of the application's context that can't be serialized. For instance, what if the object has a network connection as a member variable. The network connection can not be meaningfully serialized, so when the object is deserialized it has somehow reconnect itself to the network, which requires some reference to the network context. This may sound like a contrived example, but it occurs in the Red Dwarf Server (previously known as Sun's Darkstar project) where network connections are transparently moved around a server cluster.

The best way to solve this problem is to ensure that the serializable objects are self-contained and do not need any outside context when deserialized. Let's assume that is not possible and move on. Next best is to hook the object to its required context immediately after the deserialization. However, this may not be practicable because the deserialized object is not easily accessible (for example, if it is deep in an object graph - which is the case in Red Dwarf and my code).

Red Dwarf solves the problem by defining a static accessor on the context class and then on the serializable object adding a serialization override method like the below:
private void readObject(ObjectInputStream in) throws Exception {
  in.defaultReadObject();
  this.contextObject = Context.getContextObject();
}
This solution works, but I don't like it. Creating a static context makes it hard to write tests and add a second context if required (I have been caught out by this before). It could improved by making the static context use ThreadLocal storage, but there are other ways.


Checking out the Java serialization source code, I thought there may be some way of extending the deserialization process. The main extension points seem to be the readObject(ObjectInputStream in), writeObject(ObjectOutputStream in) and readResolve() methods added to the serializable object. These are found by reflection on their signature in a private method and can not be changed. There is a method readObjectOverride() on ObjectInputStream(), which allows a bespoke deserialization process to be defined. However, so many of the methods on ObjectInputStream are private that this would be like writing a new process from scratch.

There are two other options. The ObjectInputStream can be subclassed and contain a reference to the context object. This new class can detected within readObject(ObjectInputStream in) with instanceof and the context dereferenced. An alternative is to keep the subclass of ObjectInputStream and set enableResolveObject(true) in the constructor (this requires the program to have security permission). This means that after deserialization the resolveObject(Object obj) method on the stream subclass will be called and the return value passed as the final result of the deserialization. Thus this method can perform extra initialisation or even replace the given object with a different one. The code below shows both these techniques - you probably only want to use one at a time.
// imports skipped for brevity
public class SerializationTest {

  public static class A implements Serializable {
    public int a;

    public A(int x) {a = x;}

    private void readObject(ObjectInputStream in) throws Exception {
      in.defaultReadObject();
      if (in instanceof MyObjectInputStream) {
        System.out.println("Found context=" + ((MyObjectInputStream) in).context);
      }
    }
  }
	
  public static class MyObjectInputStream extends ObjectInputStream {
    private final int context;

    public MyObjectInputStream(InputStream in, int context) throws Exception {
      super(in);
      enableResolveObject(true); // requires permissions
      this.context = context;
    }

    protected Object resolveObject(Object obj) throws IOException {
      return new A(3);
    }
  }

  public static void main(String[] args) throws Exception {
    A test = new A(2);
    ByteArrayOutputStream bos = new ByteArrayOutputStream();
    ObjectOutputStream out = new ObjectOutputStream(bos);
    out.writeObject(test);
    out.close();

    byte[] buf = bos.toByteArray();
    MyObjectInputStream in = new MyObjectInputStream(new ByteArrayInputStream(buf), 9);
    A test2 = (A) in.readObject();
    System.out.println("Serialized object=" + test2.a);
  }
}

Honours

| | TrackBacks (0)

It was always my plan at university to do honours - it was a step on the way to a PhD. The Computer Science degree at The University of Western Australia was nominally a three year degree, but a small number of students with good grades were invited to do a fourth year in order to graduate "with honours". I got the required grades and thus got an invitation to do the extra year. Fourteen other students did the same out of a third-year class of just over 100. A few good students who must have had invitations refused - I knew a couple of them, including one who got very good grades. Although on the whole, the honours class was representative of the better students in our year. That year I worked harder than I ever had before (but have a couple of times more since). I needed to achieve 1st class (the highest grade available) to stand a good chance of getting into the PhD program. This is the story of that year.

Myth at the heart of the Roman Empire

| | TrackBacks (0)

Available on iTunes and OU Podcasts

Like the previously reviewed Myth in the Greek and Roman Worlds, these podcasts are part of The Open University course A330: Myth in the Greek and Roman Worlds. These podcasts examine how Rome represents its foundation myths through a number of historical objects, ranging in origin from the classical period to the 17th century. The series is composed of 10 episodes with: 5 video podcasts on specific artifacts; 2 audio and a video podcast introducing or giving an overview of the topic; and, two audio podcasts shared with "Myth in the Greek and Roman Worlds" discussing the role of myth in the ancient world. The video podcasts are in documentary format, normally showing a talking head or an object being described. The video podcasts are all around 5 minutes long (50MB) and recorded at 640×360 resolution, or a smaller iPod optimised format. Transcripts are available in PDF format.

The well-known story of Rome's Foundation is that Romulus and Remus (descendants of Aeneas, a prince of Troy) were abandoned as infants, suckled by a she-wolf and founded the city with Romulus killing Remus in an argument over the exact location of the new city. There is apparently some evidence of other slightly different foundation myths, where other families take a more prominent role. The myth in its current form became dominant during the reign of Augustus who claimed ancestry from Romulus. Augustus was adept at manipulating Rome's foundation myths to help consolidate power in himself and his family. Suetonius writes that as an alternative to Augustus, Octavian was offered the name Romulus as the second founder of Rome.

The House of Augustus is the first artifact examined. Presented by Augustus as a small domestic building, its location was further reinforced his connection with the gods and foundation myths. It is on the Palantine amid rows of temples and close to the Lupercal, where Romulus and Remus were supposedly found by the she-wolf. In 13BC the Roman Senate commissioned the Ara Pacis, an altar to honour Augustus for bringing peace to Rome. It is the subject of one of the podcasts and again links Augustus to the foundation myths. The altar displays scenes of nature, Rome's foundation and a religious procession including the figure of Augustus himself. Augustus is also connected to the last classical object investigated - the Lapis Niger. This artifact is a number of black basalt paving stones covering an important shrine in the Comitium area of the Roman Forum. Although it is not known what exactly the stones cover, it is know that the site is of archaic importance in the city's foundation.

It was originally though that the Capitoline Wolf (the famous status of Romulus and Remus suckling the she-wolf) was classical or Etruscan. However, it is now known that the wolf statue probably dates from the medieval period, with the twins added during the renaissance. Still the statue's image is plastered all over Rome as a symbol of the city as the traditional seat of power, even in modern Italy. Many people have used the foundation myths. In the 17th century the Popes had the d'Arpino frescoes painted in the salon of the Palazzo dei Conservatori by Giuseppe Cesari. These paintings are detailed in the last of the artifact podcasts. They depict scenes in the foundation of Rome together with a large statue of Pope Innocent X.

The artifact podcasts are interesting, but most of the others can be skipped if already seen as part of a previous Open University course on myth.

Election Bullshit

| | TrackBacks (0)

On Thursday the UK held a referendum on whether it should use the Alternative Vote (AV) electoral system. I didn't hold much hope for an AV victory. While I believe AV is a preferable over the current First-Past-The-Post (FPTP) electoral voting system, FPTP is ok and democracy generally runs acceptably in Britain. It can be hard to get people out to vote on what can seem like a minor issue and hard to get people to want change when there doesn't feel like a need for change. So with the political parties split and voting voluntary, I expected AV to lose, and it did - by a large margin. No problem. I'm not going to leave the UK over the electoral system, at least not when there are so many other good reasons to leave.

So what has made me care enough to write this, after the battle has been lost? The day after the election I received a circular promoting the "No to AV" campaign (good timing guys). There is an image of the circular on Flickr at this address. The third bullet point states that Australia wants to get rid of it's AV-like electoral system. I left Australia 8 years ago, and I still read the Australia papers nearly every day. I have seen no mention at all of this - not from the press (even the Murdoch press) or friends. I can honestly say I have seen infinitely more support for WA succession (a fairly fringe position). After some investigation it seems that this view is based on a single survey - generally considered flawed. I can't imagine there will be a referendum on voting system in Australia for many years. The rest of the flyer seems similarly dishonest. Many of the points based on the assumption that the person who gets the most votes is the "winner" and "best" candidate, which is clearly not true. Such a person has a plurality by definition, but "best" and "winner" are defined by the voting system used - which is the question at hand.

The reason I prefer AV is because it give a result more representative of the whole constituency, rather than just the plurality. Ignoring the possibility of Proportional Representation (which, to my mind, is a different argument entirely), if we assume constituency based democracy, where one person represents an geographic area, then it is better to have the person who most closely matches the views of the majority of constituents (or at least those that vote). If you disagree, well you are in the majority (in an election with only two options AV and FPTP are effectively the same). I just wish that the campaign to build that majority didn't print bullshit.