Replacing DIS/HLA with AMQP/ProcotolBuffers (Part 1)

Update (2017-03-06):   It's been a while since I wrote this.  If you're doing distributed simulation for a defense related reasons, you're probably still using HLA or DIS.  It's never going away.  There are only a few products that do HLA and you probably know what they are.  There's also a newer standard called DDS created OMG.  OMG is the same organization behind CORBA.

-------

High Level Architecture (HLA) is way of passing simulation data around. HLA was meant to replace DIS. Both systems are crap.

DIS is still used because it's simple, standardized and well understood. Anyone can implement it and inter-operate with other implementations. The biggest problems is that since it's standardized, implementing capabilities outside of the spec means that the new feature/packet won't be supported on others systems.

HLA was supposed to solve the above problems. However, it's cause more problems than it's really solved. It's the reason why the Air Force and others outside of the Navy are sticking with DIS.

One problem with HLA is that there are multiple standards. There's a DOD standard that everyone inside the Navy modeling and simulation world uses and the IEEE 1516 standard. I've not seen anyone actually using the 1516 standard.

Another issue is what the standard actually specifies. HLA while a standard is not an product that you would use. Run-Time Infrastructure (RTI) is the middle-ware that is used. The standard specifies ways in data is distributed and programming interfaces, but it does not specify a wire protocol. As a result, implementations of an RTI are usually not compatible on the wire. So, competing implementations can't talk to one another. This leads to vendor lock-in in that everyone that wants to play in a sandbox must agree to use a common version.

There are various RTI implementations. Raytheon makes one called RTI-NG Pro. It's the "standard" that the Navy has chosen. From what I understand it based on the original DMSO source code. The code has been closed up and you can now buy the "standard" RTI from them.

Other implementations are around. Portico is an open source effort to implement the DoD standard and 1516. RTI-S, while not compliant with the HLA standards, it implements the programming interface and can be easily swapped out with other implementations. It has features that make it desirable to use. RTI-S has funded by a few government agencies. It's government open source in that you can get it if you need it and you're working on a government contract. There was some talk of open sourcing, but I'll believe it when I see it.

The nice feature of the HLA is that the data to be shared between the federates is defined in the Federation Object Model (FOM). As long as all of the players agree on the FOM (and an RTI), it's easy to play together. The FOM can be shared and everyone can implement those pieces that interests them.

The FOM side steps the problem of having to re-convene a DIS committee to update the standard for the new features that are need. It's more agile and easier to change when the players decide that the original understanding was faulty. (What? We were wrong? Doesn't happen here.)

So, we come back to our original problems: DIS => Simple easy to use. HLA => More flexible but must sell soul for licenses.

I think that there's hope for the above problems and TENA isn't it.

What's needed is to combine the simplicity of DIS with the flexibility of the FOM while making it opensource. I believe that Google's ProtocolBuffers (PB) may be a potential solution to part of the problem.

Read the link to get more detail, but a PB is a way of specifying data to be sent on the network or stored in a file in a binary format. The PB is actually specified in a simple text file. It's not even xml. The file with PB definition is processed by a code generator. The resulting code is used to marshal/unmarshal the data. There are code generators for C++, Java, Python and Ruby.

While code generation and simple format specification are great features, the best one is the ability to be backwards compatible with previous specifications. This is accomplished with the required and optional tags for fields in the specification. (Click the PB link above and you'll see an example.)

The PB could be used with UDP, TCP, or Multi-cast to push data onto the network. Just a quick glance at the PB group at Google, produced this example of using PB on the network.

Since DIS is simple and easy to understand, it probably won't take much effort to create a PB version of DIS. NPS has created an XML version of the specification. While it will take some work, parsing the xml and generating the necessary PB files should reasonably straight forward.

While it's nice to have DIS in the PB format, it's just DIS in a new format. As mentioned above, PB provides an easy way to extend an existing packet with new data while maintaining backwards capability with applications that might not have the new data fields. This would allow experimentation and extension without worrying about convening a new committee meeting.

This same capability could be extended to the FOM for HLA. The FOM format is well defined but it has it's own issues and could be more flexible. (Why did it have to have ID numbers specified?) It would not take much to parse it and generate the appropriate PB files.

Since there is no wire standard for HLA, it's not a problem to just broadcast the packets need for HLA. However, HLA specifies certain capabilities that is needed compliance. Data Distribution Management (DDM) is one item that would need to be addressed. DDM allows players to subscribe to data that meets special requirements. PB by itself can't do this. There is potential fix that is open source way and has some standards support.

Advanced Message Queue Protocol (AMQP) may be the standard that is needed to fix the problem. QPID is an opensource implementation that is used by RedHat and others. I'll cover it in Part 2.

Update:
Thrift is an Apache Incubator project that could serve as an alternative to PB.
Benchmark of PB vs other systems.

Update:
Fixed problem with link to DIS XML spec developed by MOVES.