Non-RT RIC DMaaP Adapter

DMaaP Adapter

Introduction

This is a generic information producer using the Information Coordination Service (ICS) Data Producer API. It can get information from DMaaP (ONAP) or directly from Kafka topics. The information can be filtered, transformed, aggregated and then delivered to data consumers using REST calls (POST) or via Kafka.

The DMaaP Adapter registers itself as an information producer along with its information types in Information Coordination Service (ICS). The information types are defined in a configuration file.

A data consumer can create an information job (data subscription) using the ICS consumer API (for R-Apps) or the A1-EI (Enrichment Information) API (for NearRT-RICs) based on the registered information types. This service will get data from DMaaP MR or Kafka topics and deliver it to the data consumers based on their created subscription jobs.

So, a data consumer may be decoupled from DMaaP and/or Kafka this way.

The service is implemented in Java Spring Boot (DMaaP Adapter Service).

_images/Architecture.png

This product is a part of NONRTRIC.

Data Delivery

When a data consumer creates an Information Job, either a URL for REST callbacks, or a Kafka Topic can be given as output for the job. After filtering, aggregation and data transformation the data will be delivered to the output. Several data consumers can receive data from one Kafka Topic.

_images/DataDelivery.png

The output will be the same regardless if the information is received from DMaaP of from Kafka. If the data is not buffered/aggregated, and the output is a Kafka Stream, both the keys and the values are forwarded (after filtering/transformation). If the output is HTTP, only the the values are forwarded (in the HTTP body).

Data Aggregation

When an Information Job is created, a bufferTimeout can be defined for aggregation of information. If this feature is used, the subscribed data will be buffered and will be delivered in chunks.

The data will then be wrapped in a JSON array in a manner similar to DMaaP. The type configuration can define if the received data is Json. If not, each object is quoted (the output will then be an array of strings). If the data type is Json, the output will be an array of Json objects.

Configuration File

The configuration file defines which DMaaP and Kafka topics that should be listened to and registered as subscribeable information types. There is an example configuration file in config/application_configuration.json

Each entry will be registered as a subscribe information type in ICS. The following attributes can be used in each entry:

  • id, the information type identifier.

  • dmaapTopicUrl, a URL to use to retrieve information from DMaaP. Defaults to not listen to any topic.

  • kafkaInputTopic, a Kafka topic to listen to. Defaults to not listen to any topic.

  • useHttpProxy, indicates if a HTTP proxy shall be used for data delivery (if configured). Defaults to false. This parameter is only relevant if a HTTPproxy is configured in the application.yaml file.

  • dataType, this can be set to “pmData” which gives a possibility to perform a special filtering of PM data.

  • isJson, this indicates that the received is Json objects (from Kafka a stream of objects and from DMaaP an array of quoted json objects). Default value is false. If the received data is Json objects, the data sent to the consumer does not need to be quoted. When buffering is used the output will be an array of json objects ‘[{},{}]’ as opposed to an array of strings ‘[“string1”, “string2”]’. When buffering is not used, the output content-type will be ‘application/json’ as opposed to ‘text/plain’. When buffering is used, the output content-type will ‘application/json’ regardless of this parameter.

These parameters will be used to choose which parameter schemas that defines which parameters that can be used when creating an information job/data subscription.

Below follows an example of a configuration file.

{
   "types": [
      {
         "id": "DmaapInformationType",
         "dmaapTopicUrl": "/dmaap-topic-1",
         "useHttpProxy": true,
         "isJson" : true
      },
      {
         "id": "KafkaInformationType",
         "kafkaInputTopic": "TutorialTopic",
      },
      {
         "id": "PmInformationType",
         "dmaapTopicUrl": "/dmaap-topic-2",
         "dataType": "PmData",
         "isJson" : true
      }
   ]
}

Information Job Parameters

When an information consumer creates an information job,it can provide type specific parameters. The allowed parameters are defined by a Json Schema. The following schemas can be used by the component (are located in dmaapadapter/src/main/resources):

typeSchema.json

This schema will by default be registerred for the type. The following properties are defined:

  • kafkaOutputTopic, optional parameter which enables the information job to output data to a Kafka topic instead of a direct call to a data consumer. The output of a job can be directed to HTTP or to Kafka regardless if the input is retrieved from DMaaP or from Kafka.

  • filterType, selects the type of filtering that will be done. This can be one of: “regexp”, “json-path”, “jslt”.

    • regexp is for standard regexp matching of text. Objects that contains a match of the expression will be pushed to the consumer.

    • json-path can be used for extracting relevant data from json.

    • jslt, which is an open source language for JSON processing. It can be used both for selecting matching json objects and for extracting or even transforming of json data. This is very powerful.

  • filter, the value of the filter expression.

  • bufferTimeout can be used to buffer several json objects received from Kafka when kafkaInputTopic is defined into one json array. If bufferTimeout is used, the delivered data will be a Json array of the objects received. If not, each received object will be delivered in a separate call. This contains:

    • maxSize, the maximum number of objects to collect before delivery to the consumer

    • maxTimeMiliseconds, the maximum time to delay delivery (to buffer).

  • maxConcurrency, defines max how many paralell REST calls the consumer wishes to receive. 1, which is default, means sequential. A higher value may increase throughput.

Below follows examples of some filters.

{
  "filterType":"regexp",
  "filter": ".*"
}

{
  "filterType":"jslt",
  "filter": "if(.event.commonEventHeader.sourceName == \"O-DU-1122\") .event.perf3gppFields.measDataCollection.measInfoList[0].measValuesList[0].measResults[0].sValue"
}

{
  "filterType":"json-path",
  "filter": "$.event.perf3gppFields.measDataCollection.measInfoList[0].measTypes.sMeasTypesList[0]"
}

Below follows an example of using bufferTimeout and maxConcurrency.

{
   "bufferTimeout":{
      "maxSize":123,
      "maxTimeMiliseconds":456
   },
   "maxConcurrency":1
}

typeSchemaPmData.json

This schema will be registered when the configured dataType is “pmData”. This will extend the filtering capabilities so that a special filter for PM data can be used. Here it is possible to define which meas-types (counters) to get from which resources.

The filterType parameter is extended to allow value “pmdata” which can be used for PM data filtering.

  • sourceNames an array of source names for wanted PM reports.

  • measObjInstIds an array of meas object instances for wanted PM reports. If a given filter value is contained in the filter definition, it will match (partial matching). For instance a value like “NRCellCU” will match “ManagedElement=seliitdus00487,GNBCUCPFunction=1,NRCellCU=32”.

  • measTypes selects the meas types to get

  • measuredEntityDns partial match of meas entity DNs.

  • measObjClass matching of the class of the measObjInstId. The measObjInstId must follow the 3GPP naming conventions for Managed Objects (3GPP TS 32.106-8). Example, for a distinguished name “ManagedElement=RNC-Gbg-1,ENodeBFunction=1”, the MO class will be “ENodeBFunction”.

All PM filter properties are optional and a non given will result in “match all”. The result of the filtering is still following the structure of a 3GPP PM report.

Below follows an example of a PM filter.

{
  "filterType":"pmdata",
  "filter": {
    "sourceNames":[
       "O-DU-1122"
    ],
    "measObjInstIds":[
       "UtranCell=dGbg-997"
    ],
    "measTypes":[
       "succImmediateAssignProcs"
    ],
    "measuredEntityDns":[
       "ManagedElement=RNC-Gbg-1"
    ]
  }
}

Developer Guide

This document provides a quickstart for developers of the Non-RT RIC DMaaP Adapter.

Additional developer guides are available on the O-RAN SC NONRTRIC Developer wiki.

DMaaP Adapter Service

This Java implementation is run in the same way as the Information Coordinator Service.

The following properties in the application.yaml file have to be modified: * server.ssl.key-store=./config/keystore.jks * app.webclient.trust-store=./config/truststore.jks * app.configuration-filepath=./src/test/resources/test_application_configuration.json

Kubernetes deployment

Non-RT RIC can be also deployed in a Kubernetes cluster, it/dep repository. hosts deployment and integration artifacts. Instructions and helm charts to deploy the Non-RT-RIC functions in the OSC NONRTRIC integrated test environment can be found in the ./nonrtric directory.

For more information on installation of NonRT-RIC in Kubernetes, see Deploy NONRTRIC in Kubernetes.

For more information see Integration and Testing documentation on the O-RAN-SC wiki.

API-Docs

Here we describe the APIs to access the Non-RT RIC DMaaP Adapter.

DMaaP Adapter

The DMaaP Adapter provides support for push delivery of any data received from DMaaP or Kafka.

See DMaaP Adapter API for full details of the API.

The API is also described in Swagger-JSON and YAML:

API name

swagger-icon

yaml-icon

DMaaP Adapter API

link

link

Release-Notes

This document provides the release notes for the release of the DMaaP Adapter.

Version history DMaaP Adater

Date

Ver.

Author

Comment

2021-12-13

1.0.0

Henrik Andersson

E Release Initial version

2022-02-07

1.0.1

Henrik Andersson

E Maintenance Release Non root user in Docker Bugfixes

2022-06-28

1.1.0

Henrik Andersson

F Release

2022-08-25

1.1.1

Patrik Buhr

F Maintenance Release

Release Data

E Release

Project

Non-RT RIC

Repo/commit-ID

nonrtric/b472c167413a55a42fc7bfa08d2138f967a204fb

Release designation

E

Release date

2021-12-13

Purpose of the delivery

Initial release.

E Maintenance Release

Project

Non-RT RIC

Repo/commit-ID

nonrtric/4df1f9ca4cd1ebc21e0c5ea57bcb0b7ef096d067

Release designation

E

Release date

2022-02-09

Purpose of the delivery

Improvements and bug fixes

F Release

Project

Non-RT RIC DMaaP Adapter

Repo/commit-ID

nonrtric/plt/dmaapadapter 4bfffa671f94f2f3591f78918de408ec363d6616

Release designation

F

Release date

2022-06-28

Purpose of the delivery

Moved to separate repo. Minor improvements.

F Maintenance Release

Project

Non-RT RIC DMaaP Adapter

Repo/commit-ID

nonrtric/plt/dmaapadapter 8e52ee5eef1679b2cf642fa27eb7e9d4ad2fbe9d

Release designation

F

Release date

2022-08-25

Purpose of the delivery

Major improvements.