In his book Patterns of Network Architecture, John Day came to the conclusion that all layers in a network are really the same: repeated sets of the same functionality, the same mechanisms, but with different policies and different syntax. This is a powerful idea: that each genuine layer will in fact be a reflection of the same concepts - a distributed interprocess communication facility. Each would need to consider managing discovery and directory data, routing, flow control, error handling, management information, who is allowed to participate in the conversation and other basic factors, but with a different scope and with different reach.
“In one sense, nothing has changed. In the traditional range of operation, the layers are still pretty much what they were/ In another sense, a lot has changed. We no longer have an architecture of individually crafted protocols, but an architecture of common repeating units. This takes network architecture from craft to rational discipline.”
- John Day1
Messaging is no more, and no less, than one or more additional layers of distributed IPC. As we build our systems we route between processes based on topics, queues, keys or other distribution algorithms, we handle failures and errors, and we manage the rate and flow of data. We are bound to require the same functionality, and by identifying our needs precisely, we can build not only systems which get the job done now, but will predictably do so in the future, and cooperate with others. This is the final nail in the occasionally heard argument that you can replace messaging with astute network programming - of course you can, but you are doing the same job via different means, implementing again what already exists.
The corollary of this is that when we build distributed systems we are always messaging. As soon as a system moves out of our control, or even moves to another core of a machine, certain guarantees and expectations fall by the way side. If we address and embrace that by intelligently by separating our application specific functionality from the infrastructure to route and manage that interaction, we can build applications which do scale, which can handle requirements, and which properly account for the realities of distributed systems.
Looking back at the evolution from basics of communication with remote systems, to the concept of packets and messages, the power of topic based addressing, and the cutting edge of data distribution and distributed messaging, it is clear that there is always room for powerful new layers to be added that span existing and future networks. I hope this essay has been informative, and I would be more than interested in hearing your own responses, experiences and opinions on the topics discussed, on what messaging means, how it is being used, and how it can be used in the future.
-
John Day, Patterns in Network Architecture: A Return to Fundamentals, Prentice-Hall 2007 ↩