the complexity threshold

I was visiting my friend Joe when I was reminded of my theory as to why BEEP (RFC 3080) hasn’t really taken off like it should have. The catalyst for this discussion was my remembering a fairly recent discussion on the (mostly inert) beepwg mailing list. In this discussion, I was reminded that it is seemingly fairly common for implementers (or potential implementers) to really not understand certain aspects of how BEEP works. In this case, it was the use of SEQ frame. This was a particular point of concern when I was writing my own BEEP implementation, Net::BEEP::Lite.

But, let’s step back for a minute. Imagine if you are working on a new project that needs an application-level transport, and one isn’t immediately obvious. You now have two choices: 1) write your own protocol, or 2) reuse an existing protocol. Option 2 is very attractive, because option 1 seems hard.

So, you look for a transport protocol to reuse. BEEP is very attractive for this, as this is its purpose.

Now, lets say you choose your transport, but, after a little investigation there are no suitable implementations to reuse. Now, as a developer, you have to ask yourself a question: can I both write my application AND write enough transport code to make it work? This is what I call a complexity threshold. In general, if you can write the transport code in about 3 days or less, I would think that the answer to the question would be yes. Of course, this threshold is somewhat subjective, but, in general, BEEP fails this threshold. It would probably take three days just to understand what needed to be written. In other words, BEEP is complex enough that it always looks like it needs to be its own project.

Now, of course, this threshold is moot if you can find an implementation to reuse. However, this leads to the next issue: how likely is it that you will be able to find an implementation that you can reuse? In BEEP’s case, this is largely an issue of either BEEP being easy enough to understand and implement by “average” developers, or there being a driving application that attracts enough talent to create good implementations. (There are other issues when trying to reuse other application transports, but I’ll most likely cover them in a different post.)

If the second condition is true, then it almost doesn’t matter how hard BEEP is to implement (although, one could certainly argue that a hard to implement protocol wouldn’t likely have a driving application in the first place). However, in BEEP’s case, this doesn’t appear to be true. So it comes down to how difficult BEEP is for someone to write casually. This is the second complexity threshold.

Now, after surveying a number of open-source BEEP implementations, I found them to either be incomplete (in some critical way), broken, and/or quite difficult to use. This, and my own experience, leads me to believe that BEEP is above this complexity threshold as well. Exacerbating this somewhat is the fact that, on the surface, BEEP really isn’t that complex or difficult to understand. But when you delve a bit further, I think it turns out to be more difficult that it first looked, and I think some developers overreact to this, making their implementation more complicated still. Of course, an excellent example, or more documentation, can effect how complex BEEP is to implement. And, of course, how difficult something is to develop is in the eye of the beholder.