Developer-Guide

Introduction

This document provides information required to work on O-DU High code-base.

Coding Style

O-DU High uses C languages. The coding guidelines followed are:

  1. A new file should have License Header and Footer with exception of auto-generated files like files generated by ASN tool. Refer to the diagram below for License header.

  2. Every block must be indented by 3 spaces.

  3. Any header file must be included only in .c file, not in other header files.

  4. The line width should not exceed more than 120 characters.

Figure 17 License Header and Footer

Figure 17 : License Header and Footer

O-DU High code

Refer to O-DU High code-base at: https://gerrit.o-ran-sc.org/r/gitweb?p=o-du/l2.git;a=tree

Technical Details

Below section references coding specifics of O-DU High components.

Thread Management

Creation of Thread:

In O-DU High, multiple threads are created using below macro

ODU_CREATE_TASK (priority, stskId)

  1. Creates a thread by declaring a thread id

  2. Inputs

    • priority - Priority of the task

    • stskId - Thread Id

Setting a core affinity:

ODU_SET_THREAD_AFFINITY (tskId, mode, coreId, tskAssociatedTskId)

  1. Sets the processor/core affinity for a thread based on the mode supplied by the caller.

  2. Inputs

    • tskId - thread Id

    • mode - mode according to which the affinity is set

    • coreId - coreId to which the affinity has to be set

    • tskAssociatedTskId - thread Id of the associated layer

  3. Returns ROK on success and RFAILED on failure

Registering Entities:

All logical entities in O-DU High must be registered into the database.

ODU_REG_TTSK (ent, inst, ttype, prior, initTsk, actvTsk)

  1. Inputs

    • ent - Id of the entity to activate. Example: ENTDUAPP, ENTSCTP, ENTEGTP etc

    • Inst - Instance of the entity to activate. It distinguishes between multiple instances of the same entity on a given processor. Example: RLC_UL_INST (Instance id 0) and RLC_DL_INST (Instance id 1) belong to the same entity id, ENTRLC.

    • ttype - Type of entity

    • prior - Priority, ranges from 0(Highest) to 3(Lowest).

    • initTsk - Initialization function(xxActvInit) of the entity being registered gets invoked. Example: duActvInit initializes DU APP

    • actvTsk - This function(xxActvTsk) is responsible to receive any incoming message to that entity. Example: duActvTsk is triggerred when a message comes to DU APP

Attaching Entity to Thread:

Every entity must be attached to a thread to schedule its activation based on priority and incoming events. Any number of entities can be attached to a system task.

ODU_ATTACH_TTSK (ent, inst, stskId)

  1. Inputs

    • ent - Entity Id of the task

    • inst - Instance Id of the task

    • stskId - Thread Id to use

Memory Management

Configuration

Memory is divided into multiple regions(identified by region id) and each region is divided into multiple pools(identified by pool id). The configurations are present in mt_ss.h and mt_ss.c at <rsys_directory>/l2/src/mt. Currently, the number of regions configured are 6 and each region has 5 pools.

Region and pool used by each layer is identified by following macros:

  • MAC - MAC_MEM_REGION and MAC_POOL

  • SCH - SCH_MEM_REGION and SCH_POOL

  • RLC UL - RLC_MEM_REGION_UL and RLC_POOL

  • RLC_DL - RLC_MEM_REGION_DL and RLC_POOL

  • DU APP - DU_APP_MEM_REGION and DU_POOL

Static Memory

Macros are defined at each layer for static memory allocation/deallocation from that layer’s region and pool.

XX_ALLOC(bufPtr, size)

  1. Allocates static buffer

  2. Inputs:

    • bufPtr - pointer to store address of the memory allocated

    • size - size of memory to be allocated

  3. Result:

    • If allocation is sucessful, butPtr stores memory address

    • If allocation fails, bufPtr is NULL.

XX_FREE(bufPtr, size)

  1. Frees static buffer

  2. Inputs:

    • bufPtr - pointer to memory to be freed

    • size - size of memory to be freed

Here, XX stands for various ODU-High entity i.e.

  • MAC - MAC_ALLOC & MAC_FREE

  • SCH - SCH_ALLOC & SCH_FREE

  • RLC - RLC_ALLOC & RLC_FREE

  • DU APP - DU_ALLOC & DU_FREE

Sharable Memory

One of the methods of communication between layers is through sharabale memory. The sender will allocate sharable buffer from its own region and pool. This memory will be freed by receiving layer and returned back to sender’s region and pool.

XX_ALLOC_SHRABL_BUF(bufPtr, size)

  1. Allocates sharable buffer

  2. Inputs:

    • bufPtr - pointer to store address of the memory allocated

    • size - size of memory to be allocated

  3. Result:

    • If allocation is sucessful, butPtr stores memory address

    • If allocation fails, bufPtr is NULL.

XX_FREE_SHRABL_BUF(region, pool, bufPtr, size)

  1. Frees sharabale buffer

  2. Inputs:

    • region - region where this buffer is allocated from

    • pool - pool where this buffer is allocated from

    • bufPtr - pointer to memory to be freed

    • size - size of memory to be freed

Here, XX stands for various ODU-High entities i.e.

  • MAC - MAC_ALLOC_SHRABL_BUF & MAC_FREE_SHRABL_BUF

  • SCH - Since scheduler communicates only with MAC and is tightly coupled, sharable buffers are not needed.

  • RLC - RLC_ALLOC_SHRABL_BUF & RLC_FREE_SHRABL_BUF

  • DU APP - DU_ALLOC_SHRABL_BUF & DU_FREE_SHRABL_BUF

Message Buffer

A message is an ordered sequence of bytes. It stores both the control information and the data being communicated. Message buffers are allocated from dynamic memory.

ODU_GET_MSG_BUF(region, pool, mBuf)

  1. Allocates memory for message buffer

  2. Inputs:

    • region - region of sending layer

    • pool - pool of sending layer

    • mBuf - pointer to message buffer

ODU_PUT_MSG_BUF(mBuf)

  1. Frees memory for message

  2. Inputs:

    • mBuf - message pointer

WLS Memory

WLS memory is allocated for message exchanges between O-DU High and O-DU Low.

LWR_MAC_ALLOC(ptr, size)

  1. Allocates WLS memory block

  2. Inputs:

    • ptr - pointer to store address of the memory allocated

    • size - size of memory to be allocated

  3. Result:

    • If allocation is sucessful, ptr stores memory address

    • If allocation fails, ptr is NULL.

LWR_MAC_FREE(ptr, size)

  1. Frees WLS block

  2. Inputs:

    • bufPtr - pointer to memory to be freed

    • size - size of memory to be freed

Intra O-DU High Communication

O-DU high entities communicate with each other through one of the following:

Types of Communication

Direct API Call

Interface APIs invoked from one entity translate into direct function calls into the destination entity. Control returns to the calling entity after the called entity has completed processing the called function.

Macro to select this communication mode : ODU_SELECTOR_TC

Serialization

Interface API invoked from one entity is packed into a message and then sent to destination entity through system services. Control returns to the caller immediately after the message is posted, before the destination has seen or processed it. There are two serialization methods supported:

  1. Pack/Unpack data

    • The interface data is packed into the message. Receiver will unpack this, parameter by parameter.

    • Macro to select this communication mode : ODU_SELECTOR_LC

  2. Pack/Unpack pointer

    • The pointer to data is packed and sent. Receiver will unpack the pointer and directly access data at this address.

    • Macro to select this communication mode : ODU_SELECTOR_LWLC

Below figure depicts the mode of communication between various entities registered in O-DU High. Here,

  • TC stands for Direct API call

  • LC stands for Serialization by packing/unpacking of data

  • LWLC stands for Serialization by packing/unpacking of pointers

Figure 18 Mode of communication between O-DU High entities

Figure 18: Mode of communication between O-DU High entities

Steps of Communication

  1. Fill Post Structure

    Information needed by system services to route API to the destination layer is stored in post structure.

    typedef struct pst
    {
    ProcId dstProcId; /* destination processor ID */
    ProcId srcProcId; /* source processor ID */
    Ent dstEnt; /* destination entity */
    Inst dstInst; /* destination instance */
    Ent srcEnt; /* source entity */
    Inst srcInst; /* source instance */
    Prior prior; /* priority */
    Route route; /* route */
    Event event; /* event */
    Region region; /* region */
    Pool pool; /* pool */
    Selector selector; /* selector */
    uint16_t spare1; /* spare for alignment */
    } Pst;
  2. Pack API into message

    At sender, API is packed i.e. the data is stored into a message in ordered sequence of bytes. At receiver, the data is unpacked from the message and its corresponding handler is invoked.

    1. If pst->selector is LC, each parameter is packed/unpacked one by one using one of the below.

      • oduPackUInt8(val, mBuf) - Packs 8-bits value(val) into message(mBuf)

      • oduUnpakcUInt8(val, mBuf) - Unpacks 8-bits from message(mBuf) and stores in val

      • oduPackUInt16(val, mBuf) - Packs 16-bits value(val) into message(mBuf)

      • oduUnpakcUInt16(val, mBuf) - Unpacks 16-bits from message(mBuf) and stores in val

      • oduPackUInt32(val, mBuf) - Packs 32-bits value(val) into message(mBuf)

      • oduUnpakcUInt32(val, mBuf) - Unpacks 16-bits from message(mBuf) and stores in val

      The sequence in which the parameters are unpacked must be reverse of the packing sequence.

    2. If pst->selector is LWLC, pointer to the interface structure is packed/unpacked.

      • oduPackPointer(ptr, mBuf) - Packs pointer value(ptr) into message(mBuf)

      • oduUnpackPointer(ptr, mBuf) - Unpacks pointer value from message(mBuf) and stores in ptr

  3. Post the message

    Once the post information is filled and API is packed into a message, it is posted to destination using:

    ODU_POST_TASK(pst, mBuf)

    1. Inputs

      • pst - post structure mentioned above

      • mBuf - message

Below figure summarized the above steps of intra O-DU High communication

Figure 19 Communication between entities

Figure 19: Steps of Communication between O-DU High entities

Communication with Intel O-DU Low

Intel O-DU Low communicates with O-DU High over WLS interface. Hence, Intel’s “wls_lib.h” library is required for using the following APIs for communication.

  1. WLS_Open

    void* WLS_Open(const char *ifacename, unsigned int mode, uint64_t *nWlsMacMemorySize, uint64_t *nWlsPhyMemorySize)

    1. Description

      • Opens the WLS interface and registers as instance in the kernel space driver.

      • Control section of shared memory is mapped to application memory.

    2. Inputs:

      • ifacename - pointer to string with device driver name (/dev/wls)

      • mode - mode of operation (Master or Slave). Here, O-DU High acts as MASTER.

      • nWlsMacMemorySize - returns the value of WLS MAC memory Size as O-DU High acts as MASTER

      • nWlsPhyMemorySize - returns the value of WLS PHY memory Size as O-DU High acts as MASTER

    3. Returns pointer handle to WLS interface for future use by WLS functions

  2. WLS_Ready

    int WLS_Ready(void *h)

    1. Description

      • Checks the state of remote peer of WLS interface

    2. Inputs - handle of WLS interface

    3. Returns 0 if peer is available i.e. one to one connection is established

  3. WLS_Close

    int WLS_Close(void *h)

    1. Description

      • Closes the WLS interface and de-registers as instance in the kernel space driver

      • Control section of shared memory is unmapped form user space application

    2. Input - handle of WLS interface to be closed

    3. Returns 0 if operation is successful

  4. WLS_Alloc

    void* WLS_Alloc(void* h, unsigned int size)

    1. Description

      • Allocates memory block for data exchange shared memory. Memory block is backed by huge pages.

      • Memory is allocated only once for L2, and divided into various regions.

    2. Input

      • h - handle of WLS interface

      • size - size of memory block to allocate

    3. Returns

      • Pointer to allocated memory block

      • NULL on memory allocation failure

  5. WLS_Free

    int WLS_Free(void* h, void* pMsg)

    1. Description

      • Frees memory block for data exchanged on shared memory.

    2. Input

      • h - handle of WLS interface

      • pMsg - pointer to WLS memory

    3. Returns 0 if operation is sucessful

  6. WLS_Put

    int WLS_Put(void* h, unsigned long long pMsg, unsigned int MsgSize, unsigned short MsgTypeID, unsigned short Flags)

    1. Description

      • Puts memory block (or group of blocks) allocated from WLS memory into the interface to transfer to remote peer

    2. Input

      • h - handle of WLS interface

      • pMsg - pointer to memory block (physical address) with data to be transfered to remote peer

      • MsgSize - size of memory block to send (should be less than 2 MB)

      • MsgTypeID - application specific identifier of message type

      • Flags - Scatter/Gather flag if memory block has multiple chunks

    3. Returns 0 if operation is successful

  7. WLS_Check

    int WLS_Check(void* h)

    1. Description

      • Checks if there are memory blocks with data from remote peer

    2. Input - handle of WLS interface

    3. Returns number of blocks available for “get” operation

  8. WLS_Wait

    int WLS_Wait(void* h)

    1. Description

      • Waits for new memory block from remote peer

      • Blocking call

    2. Input - the handle of WLS interface

    3. Returns number of blocks available for “get” operation

  9. WLS_Get

    unsigned long long WLS_Get(void* h, unsigned int *MsgSize, unsigned short *MsgTypeID, unsigned short *Flags)

    1. Description

      • Gets memory block from interface received from remote peer.

      • Non-blocking operation

    2. Input

      • h - handle of WLS interface

      • MsgSize - pointer to set size of memory block

      • MsgTypeID - pointer to application specific identifier of message type

      • Flags - pointer to Scatter/Gather flag if memory block has multiple chunks

    3. Returns

      • Pointer to memory block (physical address) with data received from remote peer

      • NULL if error or no blocks available

  10. WLS_WGet

    unsigned long long WLS_WGet(void* h, unsigned int *MsgSize, unsigned short *MsgTypeID, unsigned short *Flags)

    1. Description

      • Gets memory block from interface received from remote peer

      • It is a blocking operation and waits for next memory block from remote peer

    2. Input

      • h - handle of WLS interface

      • MsgSize - pointer to set size of memory block

      • MsgTypeID - pointer to application specific identifier of message type

      • Flags - pointer to Scatter/Gather flag if memory block has multiple chunks

    3. Returns

      • Pointer to memory block (physical address) with data received from remote peer

      • NULL if error or no blocks available

  11. WLS_WakeUp

    int WLS_WakeUp(void* h)

    1. Description

      • Performs “wakeup” notification to remote peer to unblock “wait” operations pending

    2. Input - handle of WLS interface

    3. Returns 0 if operation is successful

  12. WLS_VA2PA

    unsigned long long WLS_VA2PA(void* h, void* pMsg)

    1. Description

      • Converts virtual address (VA) to physical address (PA)

    2. Input

      • h - handle of WLS interface

      • pMsg - virtual address of WLS memory block

    3. Returns

      • Physical address of WLS memory block

      • NULL, if error

  13. WLS_PA2VA

    void* WLS_PA2VA(void* h, unsigned long long pMsg)

    1. Description

      • Converts physical address (PA) to virtual address (VA)

    2. Input

      • h - handle of WLS interface

      • pMsg - physical address of WLS memory block

    3. Returns

      • Virtual address of WLS memory block

      • NULL, if error

  14. WLS_EnqueueBlock

    int WLS_EnqueueBlock(void* h, unsigned long long pMsg)

    1. Description

      • Used by the Master to provide memory blocks to slave for next slave-to-master data transfer

    2. Input

      • h - handle of WLS interface

      • pMsg - physical address of WLS memory block

    3. Returns 0 if opertaion is successful

  15. WLS_DequeueBlock

    unsigned long long WLS_DequeueBlock(void* h)

    1. Description

      • Used by the Master and Slave to get block from master-to-slave queue of available memory blocks

    2. Input - handle of WLS interface

    3. Returns

      • Physical address of WLS memory block

      • NULL, if error

  16. WLS_NumBlocks

    int WLS_NumBlocks(void* h)

    1. Description

      • Returns number of current available block provided by the Master for new transfer of data from slave

    2. Input - handle of WLS interface

    3. Returns number of available blocks in slave to master queue

Scheduler Framework with plug-in support

5G NR SCH module is encapsulated within 5G NR MAC of ODU-High. Any communication to/from SCH will happen only through MAC. The scheduler framework in ODU-High provides support to plug-in multiple scheduling algorithms easily.

Design

5G NR Scheduler Framework Design

Figure 20: 5G NR Scheduler Framework Design

  • The code for scheduler has been divided into 2 parts i.e. the common APIs and scheduler-specific APIs.

  • Any code (structure/API) which is specific to a scheduling algorithm must be within scheduler-specific files such as sch_rr.c and sch_rr.h for round-roubin scheduler.

  • Function pointers are used to identify and call APIs belonging to the scheduling algorithm in use at any given point in time.

  • All supported scheduling algorithm are listed in SchType enum in sch.h file.

  • All function pointers are declared in SchAllApis structure in sch.h

  • For each scheduling algorithm, function pointers must be initialised to scheduler-specific APIs during scheduler initialisation.

Call Flow

  • In any call flow, a common API calls the scheduler-specific API using function pointer and its output is returned back to the common API, which will be further processed and communicated to MAC.

Call flow example of Multi-Scheduling Algorithm framework

Figure 21: Example of a call flow in multi-scheduling algorithm framework

Additional Utility Functions

  1. ODU_START_TASK(startTime, taskId)

    1. Gives current time through input parameter

    2. Input

      • startTime - stores current time to be returned

      • taskId - task id of calling entity

  2. ODU_STOP_TASK(startTime, taskId)

    1. Calculates difference of start time and current time.

    2. Input

      • startTime - start time of this task

      • taskId - taskId of calling entity

  3. ODU_SET_PROC_ID(procId)

    1. Processors are identified by processor identifiers (ProcId) that are globally unique. It sets the procId for the local processor. In O-DU High, procId is 0 (DU_PROC)

    2. Inputs

      • procId - process id to be set

  4. ODU_GET_PROCID()

    1. Finds and returns the local processor id on which the calling task is running

    2. Inputs

      • void

  5. ODU_CAT_MSG(mbuf1, mbuf2, order)

    1. Concatenates the given two message.

    2. Inputs

      • mbuf1 - pointer to message buffer 1

      • mbuf2 - pointer to message buffer 2

      • order - order in which the messages are concatenated

  6. ODU_GET_MSG_LEN(mBuf, lngPtr)

    1. Determines length of the data contents of a message

    2. Inputs

      • mBuf - pointer to the message buffer

      • lngPtr - pointer to store length value

  7. ODU_EXIT_TASK()

    1. Gracefully exits the process

    2. Inputs

      • void

  8. ODU_PRINT_MSG(mBuf, src, dst)

    1. Prints information about message buffer.

    2. Inputs

      • mBuf - pointer to the message buffer

      • src - source Id

      • dest - destination Id

  9. ODU_REM_PRE_MSG(dataPtr, mBuf)

    1. Removes one byte of data from the beginning of a message

    2. Inputs

      • dataPtr - pointer to the location where one byte of data is placed

      • mBuf - pointer to the message buffer

  10. ODU_REM_PRE_MSG_MULT(dst, cnt, mBuf)

    1. Removes the specified number of bytes of data from the beginning of a message

    2. Inputs

      • dst - pointer to the location where the data bytes are placed.

      • cnt - number of bytes to be removed from the message.

      • mBuf- pointer to the message.

  11. ODU_REG_TMR_MT(ent, inst, period, func)

    1. Registers timer function of an entity with system services

    2. Inputs

      • ent - entity ID of task registering the timer.

      • inst - instance of task registering the timer.

      • period - period in system ticks between system service sccessive scheduling of the timer function in the entity

      • func - timer function.

  12. ODU_SEGMENT_MSG(mBuf1, idx, mBuf2)

    1. Segments a message into two messages at the specified index.

    2. Inputs

      • mBuf1 - Message 1, original message to be segmented

      • idx - index in message 1 from which message 2 is created.

      • mBuf2 - pointer to message buffer 2 (new message).

  13. ODU_ADD_PRE_MSG_MULT(src, cnt, dst)

    1. Copies consecutive bytes of data to the beginning of a message

    2. Inputs

      • src - source buffer

      • cnt - number of bytes

      • dst - destination message

  14. ODU_ADD_PRE_MSG_MULT_IN_ORDER(src, cnt, dst)

    1. Copies consecutive bytes of data to the beginning of a message and keeps the bytes order preserved

    2. Inputs

      • src - source buffer

      • cnt - number of bytes

      • dst - destination message

  15. ODU_ADD_POST_MSG_MULT(src, cnt, dst)

    1. Copies consecutive bytes of data to the end of a message

    2. Inputs

      • src - source buffer

      • cnt - number of bytes

      • dst - destination message

  16. ODU_COPY_MSG_TO_FIX_BUF(src, srcIdx, cnt, dst, ccnt)

    1. Copies data from a message buffer into a fixed buffer

    2. Inputs

      • src - source message

      • srcIdx - start index of source buffer to be copied

      • cnt - number of bytes to be copied

      • dst - destination buffer

      • ccnt - number of bytes copied

  17. ODU_COPY_FIX_BUF_TO_MSG(src, dst, dstIdx, cnt, ccnt)

    1. Copies data from a fixed buffer to a message buffer

    2. Inputs

      • src - source buffer

      • dst - destination message

      • dstIdx - index in destination message to starting copying bytes from

      • cnt - number of bytes to be copied

      • ccnt - number of bytes copied

O1 Module

Coding Style

O1 uses GNU C++ language.

ODU - O1 Communication

O1 module runs as a thread in O-DU High.

Alarm communication between the threads happen on a Unix socket.

O-DU High sends alarm messages in the following structure using Alarm Interface APIs.

Alarm Structure
typedef struct
{
MsgHeader msgHeader; /* Alarm action raise/clear */
EventType eventType; /* Alarm event type */
char objectClassObjectInstance[OBJ_INST_SIZE]; /* Name of object that raise/clear an alarm */
char alarmId[ALRM_ID_SIZE]; /* Alarm Id */
char alarmRaiseTime[DATE_TIME_SIZE]; /* Time when alarm is raised */
char alarmChangeTime[DATE_TIME_SIZE]; /* Time when alarm is updated */
char alarmClearTime[DATE_TIME_SIZE]; /* Time when alarm is cleared */
char probableCause[TEXT_SIZE]; /* Probable cause of alarm */
SeverityLevel perceivedSeverity; /* Severity level of alarm */
char rootCauseIndicator[TEXT_SIZE]; /* Root cause of alarm */
char additionalText[TEXT_SIZE]; /* Additional text describing alarm */
char additionalInfo[TEXT_SIZE]; /* Any additional information */
char specificProblem[TEXT_SIZE]; /* Any specific problem related to alarm */
}AlarmRecord;

O1 - Netconf Communication

O1 communicates with the Netconf server using sysrepo and libyang APIs