User-Guide

Overview

Kserve Adapter works with the AIML Framework and is used to deploy, delete and update AI/ML models on Kserve.

Steps to build and run kserve adapter

Prerequisites

  1. Install go

  2. Install make

Steps

git clone "https://gerrit.o-ran-sc.org/r/aiml-fw/aihp/ips/kserve-adapter"
cd kserve-adapter
Update ENV variables in Makefile under run section.
Update pkg/helm/data/sample_config.json with model url. This can be obtained from AIMFW dashboard(Training Jobs-> Training Job status -> Select Info for a training job -> Model URL)
Execute below commands
make build
make run

Steps to run kserve adapter using AIMLFW deployment scripts

Follow the steps in this link: AIMLFW installation guide

Demo steps

Prerequisites

  1. Install chart museum

  2. Build ricdms binary

Steps for the demo

  1. Run ric dms

    export RIC_DMS_CONFIG_FILE=$(pwd)/config/config-test.yaml
./ricdms

  2. Run kserve adapter

    Create namespace called ricips

    kubectl create ns ricips
Update ENV variables in Makefile under run section.
Update :file: pkg/helm/data/sample_config.json with model url. This can be obtained from AIMFW dashboard(Training Jobs-> Training Job status -> Select Info for a training job -> Model URL)
Execute below commands
make build
make run

  1. Generating and upload helm package

    curl --request POST --url 'http://127.0.0.1:10000/v1/ips/preparation?configfile=pkg/helm/data/sample_config.json&schemafile=pkg/helm/data/sample_schema.json'

  2. Check uploaded charts

    curl http://127.0.0.1:8080/api/charts

  3. Deploying the model

    curl --request POST --url 'http://127.0.0.1:10000/v1/ips?name=inference-service&version=1.0.0'

  4. Check deployed Inference service

    kubectl get InferenceService -n ricips

  5. Perform predictions

    Use below command to obtain Ingress port for Kserve.

    kubectl get svc istio-ingressgateway -n istio-system

    Obtain nodeport corresponding to port 80. In the below example, the port is 31206.

    NAME                   TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)                                                                      AGE
istio-ingressgateway   LoadBalancer   10.105.222.242   <pending>     15021:31423/TCP,80:31206/TCP,443:32145/TCP,31400:32338/TCP,15443:31846/TCP   4h15m

Create file predict_inference.sh with below contents:

model_name=sample-xapp
curl -v -H "Host: $model_name.ricips.example.com" http://<VM IP>:<Ingress port for Kserve>/v1/models/$model_name:predict -d @./input_qoe.json

Update the VM IP and the Ingress port for Kserve above.

Create file input_qoe.json with below contents:

{"signature_name": "serving_default", "instances": [[[2.56, 2.56],
        [2.56, 2.56],
        [2.56, 2.56],
        [2.56, 2.56],
        [2.56, 2.56],
        [2.56, 2.56],
        [2.56, 2.56],
        [2.56, 2.56],
        [2.56, 2.56],
        [2.56, 2.56]]]}

Use command below to trigger predictions

source predict_inference.sh