In another of my recent series of generic utility Java classes, I present a multi-channel asynchronous throttler. My project connected to an external webservice which imposed draconian restrictions if usage went above a certain level – defined as whether there were more than X service calls in Y seconds. I was determined to stay under this level. Furthermore, some types of service calls were more expensive than others and had their own limits in addition to the overall limit. That is for certain types of service call I had stay under two limits – a call specific limit and the overall limit that applied to all calls. I refer to this as multi-channel throttling. Also, I wanted the throttler to be asynchronous, that is I did not want to stop procesing while waiting for the webservice to respond to my call. Looking on the web I found a number of throttlers, but none that matched my multi-channel, asynchronous requirements.
The class I coded is below. The interface allows for both synchronous (with the submitSync methods) and asynchronous (with the submit methods, returning a Future object) Runnable code to be executed, and throttled if necessary. Using Runnable allows the throttler to be generic – there is no application specific code in it. The throttles are defined using the Rate inner class. At least one Rate, the overall throttle, must be defined. Extra channel throttles can be passed into the constructor as a Map of Rates keyed by any object. Then calls to submit(Object channelKey, Runnable task) will throttle based on both the overall throttle and the channel throttle with the specified key if it exists.
The Rate is specified in terms of X calls in Y time period. The time period being defined using a Quantity pattern, a scandalously underused pattern in business code – use it! The throttle uses a rolling time window rather than quantising time into discrete blocks. So in continuous time, the throttle is never exceeded (a mistake I saw in a few implementations). The ordering of service calls may not be preserved when multiple calls are scheduled for the start of the next period – the delayed calls will execute in a random order. Apart from this (and across channels) the class will attempt to maintain the order of the service calls. Although it may not always use the optimal ordering of throttled calls. While the time period can be specified down to nanoseconds, this throttle is only millisecond accurate as that is all I needed. I created the TimeProvider interface for the testing reasons detailed below.
As I’m trying to use TDD more, I also have a test class below (using JUnit3). Writing the throttler for tests required a few changes. Firstly, the scheduler used to delay throttled calls can be passed in so I use the DeterministicScheduler from the JMock project (a discussion of that class is here). Furthermore, as it is not practically possible to mock the Java System class and thus control internal time, I have had to create a TimeProvider interface. I can use this to properly mock time and provide a default implementation that just redirects to System.currentTimeMillis().