Route Table Guide¶
RIC Message Router – RMR¶
Messages sent via the RIC Message Router (RMR) are routed to an endpoint (another application) based on a combination of the message type (MT) and subscription ID (SID) supplied in the message. RMR determines the endpoint by matching the MT and SID combination to an entry in a route table which has been supplied dynamically by a Route Manager service, or as a static table loaded during RMR initialisation. It is also possible to route messages directly to an endpoint which is the managed entity “owner,” using the managed entity ID (MEID).
For most xAPP developers the format of the RMR route table is not important beyond understanding how to create a static table for local testing. For developers of a Route Manager service, the need is certainly a requirement. This document describes the overall syntax of a route table and the interface between the Route Manager service and RMR.
Contents of a Route Table¶
The table consists of a start record, one or more entry records, and an end record. Each entry record defines one message type, with an optional sender application, and the endpoint(s) which accept the indicated message type. All table records contain fields separated with vertical bars (|), and allow for trailing comments with the standard shell comment symbol (hash, #) provided that the start of the comment is separated from the last token on the record by one or more spaces. Leading and trailing white space in each field is ignored. Figure 1 illustrates a very basic route table with two message types, 1000 and 2000, and two subscription IDs for message type 1000.
newrt | start | rt-0928 rte | 2000 | logger:30311 mse | 1000 | 10 | forwarder:43086 mse | 1000 | 21 | app0:43086,app1:43086 newrt | end | 3
Figure 1: A basic route table.
Entry record syntax¶
Two types of table entries are supported for compatibility with the original RMR implementation, but only the mse entry type is needed and that should be the entry used when creating new tables. The following shows the syntax for both entry types:
rte | <msg-type>[,<sender-endpoint>] | <endpoint-group>[;<endpoint-group>;...] mse | <msg-type>[,<sender-endpoint>] | <sub-id> | <endpoint-group>[;<endpoint-group>;...]
is the table entry type
is the integer message type
is the endpoint description of the message sender; only that sender will read the entry from the table, so a single table may be used for all senders when a common message type is delivered to varying endpoints based on senders. If the sender endpoint is omitted from the entry, then the entry will be used by all applications.
is the subscription id (integer) for subscription-based messages, or -1 if the message type is not subscription-based. An mse entry with a sub-id of -1 is the same as an rte entry with the same message type.
is one or more, comma separated, endpoint descriptions.
When an application sends a message with the indicated type, the message will be sent to one endpoint in the group in a round-robin ordering. If multiple endpoint groups are given, then the message is sent to a member selected from each group; 3 groups, then three messages will be sent. The first group is required.
Table entries must end with a record termination sequence which may be one of the following three sequences:
a single newline (\n)
a DOS style CRLF pair (\r\n)
a single carriage return (\r)
Care must be taken when manually editing a static table; some editors do not add a final record termination sequence to the last line of a file. RMR expects the final record to have a termination sequence to ensure that the record was not truncated (especially important when receiving dynamic tables).
The route table parser within RMR assumes that route table entries are sent via RMR messages as a stream. To ensure synchronisation and prevent malformed tables because of broken sessions or lost packets, each table must begin and end with an newrt record. Each newrt record has one of two possible syntax layouts as described below.
newrt | begin [| table-id-string] newrt | end [| record-count]
Figure 2: Illustration of the newrt records in the table.
The table-id-string is an optional string which is used by RMR when sending an acknowledgement back to the Route Manager service (see the Route Manager Interface section for more details). If a record-count is given as the final field on the end record, RMR will verify that the number of mse and rte entries in the table matches the count; if there is a mismatch in values the table is not used.
The endpoint description is either the hostname or IP address followed by a port number; the two are separated by a single colon. The illustration below assumes that host names (e.g. forwarder and app1) are defined; they also make the tables easier to read. The port number given is the port number that the user application provides to RMR when the RMR initialisation function is invoked (and thus is the port that RMR is listening on).
Creating a table from the two entry types is fairly simple, however there are some subtleties which should be pointed out to avoid unexpected behaviour. For this discussion the following complete table will be used.
newrt | start | rt-0928 rte | 2000 | logger:30311 mse | 1000 | 10 | forwarder:43086 mse | 1000,forwarder:43086 | 10 | app2:43086 mse | 1000 | -1 | app0:43086,app1:43086; logger:20311 newrt | end | 4
(1) (2) (3) (4) (5) (6)
Figure 3: A complete RMR routing table (line numbers to the right for reference).
Table Entry Ordering¶
Whether a table is read from a file on disk, or is received from a Route Manager service, RMR parses the records to build an internal route table keeping only the relevant information. Entries are read in the order they appear (from the file or in messages received), and RMR will use only one entry for each MT/SID pair.
For most tables, the ordering of entries is not important, but when there are entries which duplicate the MT/SID pair ordering becomes significant. RMR will use the last valid entry for a MT/SID pair that it encounters. An entry is considered valid if there is no sender identified with the message type (line 3), and when the sender (host and port) match the the applications’ location and the port provided to RMR for listening.
Using the table in figure 3 as an example, there are two entries which match the MT/SID pair of 1000/10. When this table is parsed on any host, RMR will recognise and add the first entry (line 3) to the internal representation; this entry is valid for all applications. The second 1000/10 entry (line 4) is valid when the table is parsed on the forwarder host, and only by the application which is listening on port 43086. For this application the entry will override the more generic entry for the MT/SID combination.
As a rule, the ordering of entries for a given MT/SID pair should be from most generic to most specific.
Route Manager Communications¶
During initialisation RMR will use the value of the
RMR_RTG_SVC environment variable to connect to the Route
Manager service in order to request a route table. The
connection between RMR and the Route Manager is also an RMR
session and thus RMR messages will be used to exchange
requests and responses.
During initialisation, RMR establishes a wormhole connection to the Route Manager and sends a message type of 21 to request a new table. RMR will continue to send table requests until a table is received and accepted; in other words it is fine for the Route Manager to ignore the requests if it is not ready to respond.
Sending Tables To RMR¶
Table entry data is expected to arrive via RMR message with a message type of 20. The message may contain one or more entries provided that the entries are newline separated. Current versions of RMR support very large messages, however to ensure compatibility with an xAPP built using an older version of RMR (pre 3.8), messages should be limited to 4 KiB.
Table Acceptance and Acknowledgement¶
When RMR receives the table end entry (newrt|end), it will send a state message back to the Route Manager to indicate the state of the received table. The message type is 22 and the payload will contain UTF-8 tokens which indicate the state. The second token will be the table ID supplied on the start record, or the string “<id-missing>.” When the state is an error state, RMR might add a final set of tokens which contain the reason for the failure.
Upon receipt of a status message which indicates an “OK” response, the Route Manager can assume that the table has been installed and is in use. Any other response indicates that RMR did not use the table and has dropped it; the previous table is still in use.
Using A Static Route Table¶
A static route table can be provided to assist with testing,
or to provide a bootstrap set of route information until a
dynamic table is received from a routing manager. The
RMR_SEED_RT is checked during RMR
initialisation and if set is expected to reference a file
containing a route table. This table will be loaded and used
until overlaid by a table sent by the Route Manager.
To simulate dynamic reloads during testing, and for some
specialised use cases, the static table will be reloaded
periodically if the
RMR_RTG_SVC environment variable is
set to -1. When set to -1 RMR will not listen for Route
Manager connections, nor will it attempt to request a
If the file given by the
RMR_SEED_RT variable does not
exist, and the
RMR_RTG_SVC variable is set to -1, RMR
will block until the table is created. This simulates a
delayed dynamic load during testing, and can be used when the
xAPP is reading the route table saved by another local
process rather than one sent directly by the Route Manager.
To assist with debugging, and to allow an application to
share the route table received from Route Manager, RMR will
stash the route table updates it received. Updates are
stashed in a file named by the
variable, and if that variable is not present, the
RR_SEED_RT variable will be used with an added
The primary use of route table stashing is to assist with
debugging of applications, and because there are risks for an
application to share its table, table sharing is NOT
recommended. Table sharing can be enabled by setting the
RMR_STASH_RT variable for the application that will be
the source of the route table updates, and using the file
named for tha application when defining the
RMR_SEED_RT variable for applications which are to read
the table information. Obviously, all applications must be
running in the same container, on the same host, or have a
common disk volum between their environments. Known risks to
using table sharing include
An update to the table (not a complete table) may be received prior to one or more readers accessing the file, and thus the reader may not receive a valid or complete table.
Any entry which has a sender:port associated with the message type will likely be ignored by all readers.
Routing Using MEID¶
Starting with version 1.13.0, RMR provides the ability to
select the endpoint for a message based on the MEID (managed
entity ID) in the message, rather than selecting the endpoint
from the round-robin list for the matching route table entry.
When the MEID is used, the message is sent to the endpoint
which owns, or is responsible for the managed entity.
Should the owner change messages will be routed to the new
owner when the route table is updated. To make use of MEID
routing, there must be one or more route table entries which
list the special endpoint name
%meid instead of providing
a round robin list. As an example, consider the following
route table entry:
mse| 1000,forwarder:43086 | 10 | %meid
Figure 4: Sample route entry with the meid flag.
The final field of the entry doesn’t specify a round-robin group which means that when an application attempts to send a message with type 1000, and the subscription ID of 10, the MEID in the message will be used to select the endpoint.
MEID endpoint selection¶
To select an endpoint for the message based on the MEID in a message, RMR must know which endpoint owns the MEID. This information, known as an MEID map, is provided by the Route Manager over the same communication path as the route table is supplied. The following is the syntax for an MEID map.
meid_map | start | <table-id> mme_ar | <owner-endpoint> | <meid> [<meid>...] mme_del | <meid> [<meid>...] meid_map | end | <count> [| <md5sum> ]
Figure 5: Meid map table.
The mme_ar records are add/update records and allow for the list of MEIDs to be associated with (owned by) the indicated endpoint. The <owner-endpoint> is the hostname:port, or IP address and port, of the application which owns the MEID and thus should receive any messages which are routed based on a route table entry with %meid as the round-robin group. The mme_del records allow for MEIDs to be deleted from RMR’s view. Finally, the <count> is the number of add/replace and delete records which were sent; if RMR does not match the <count> value to the number of records, then it will not add the data to the table. Updates only need to list the ownership changes that are necessary; in other words, the Route Manager does not need to supply all of the MEID relationships with each update.
The optional <md5sum> field on the end record should be the MD5 hash of all of the records between the start and end records. This allows for a precise verification that the transmitted data was correctly received.
If a static seed file is being used for the route table, a second section can be given which supplies the MEID map. The following is a small example of a seed file:
newrt|start | id-64306 mse|0|-1| %meid mse|1|-1|172.19.0.2:4560 mse|2|-1|172.19.0.2:4560 mse|3|-1|172.19.0.2:4560 mse|4|-1|172.19.0.2:4560 mse|5|-1|172.19.0.2:4560 newrt|end meid_map | start | id-028919 mme_ar| 172.19.0.2:4560 | meid000 meid001 meid002 meid003 meid004 meid005 mme_ar| 172.19.0.42:4560 | meid100 meid101 meid102 meid103 mme_del | meid1000 meid_map | end | 1
Figure 6: Illustration of both a route table and meid map in the same file.
The tables above will route all messages with a message type of 0 based on the MEID. There are 10 meids which are owned by two different endpoints. The table also deletes the MEID meid1000 from RMR’s view.
Reserved Message Types¶
RMR is currently reserving message types in the range of 0 through 99 (inclusive) for its own use. Please do not use these types in any production or test environment as the results may be undesired.
Appendix A – Glossary¶
Many terms in networking can be interpreted with multiple meanings, and several terms used in various RMR documentation are RMR specific. The following definitions are the meanings of terms used within RMR documentation and should help the reader to understand the intent of meaning.
A programme which uses RMR to send and/or receive messages to/from another RMR based application.
An error that RMR has encountered which will prevent further successful processing by RMR. Critical errors usually indicate that the application should abort.
An RMR based application that is defined as being capable of receiving one or more types of messages (as defined by a routing key.)
A key/value pair which is set externally to the application, but which is available to the application (and referenced libraries) through the
getenvsystem call. Environment variables are the main method of communicating information such as port numbers to RMR.
An abnormal condition that RMR has encountered, but will not affect the overall processing by RMR, but may impact certain aspects such as the ability to communicate with a specific endpoint. Errors generally indicate that something, usually external to RMR, must be addressed.
The name of the host as returned by the
gethostbynamesystem call. In a containerised environment this might be the container or service name depending on how the container is started. From RMR’s point of view, a host name can be used to resolve an endpoint definition in a route table.)
Internet protocol. A low level transmission protocol which governs the transmission of datagrams across network boundaries.
A TCP socket used to await incoming connection requests. Listen sockets are defined by an interface and port number combination where the port number is unique for the interface.
A series of bytes transmitted from the application to another RMR based application. A message is comprised of RMR specific data (a header), and application data (a payload).
A data structure used to describe a message which is to be sent or has been received. The message buffer includes the payload length, message type, message source, and other information.
A signed integer (0-32000) which identifies the type of message being transmitted, and is one of the two components of a routing key. See Subscription ID.
The portion of a message which holds the user data to be transmitted to the remote endpoint. The payload contents are completely application defined.
A set of information which defines the current state of the underlying transport connections that RMR is managing. The application will be give a context reference (pointer) that is supplied to most RMR functions as the first parameter.
The method of selecting an endpoint from a list such that all endpoints are selected before starting at the head of the list.
A series of “rules” which define the possible endpoints for each routing key.
Route table manager
An application responsible for building a route table and then distributing it to all applicable RMR based applications.
The process of selecting an endpoint which will be the recipient of a message.
A combination of message type and subscription ID which RMR uses to select the destination endpoint when sending a message.
The sender of a message.
A signed integer value (0-32000) which identifies the subscription characteristic of a message. It is used in conjunction with the message type to determine the routing key.
The endpoint selected to receive a message.
Transmission Control Protocol. A connection based internet protocol which provides for lossless packet transportation, usually over IP.
Also called a process thread, or pthread. This is a lightweight process which executes in concurrently with the application and shares the same address space. RMR uses threads to manage asynchronous functions such as route table updates.
An optional portion of the message buffer that the application may populate with data that allows for tracing the progress of the transaction or application activity across components. RMR makes no use of this data.
A fixed number of bytes in the message buffer) which the application may populate with information related to the transaction. RMR makes use of the transaction ID for matching response messages with the &c function is used to send a message.
An error state that is believed to be short lived and that the operation, if retried by the application, might be successful. C programmers will recognise this as
A warning occurs when RMR has encountered something that it believes isn’t correct, but has a defined work round.
A direct connection managed by RMR between the user application and a remote, RMR based, application.