Installation Guide¶

Abstract
Preface
Hardware Requirements
Software Installation and Deployment
References

Abstract ¶

This document describes how to install O-RAN INF image, example configuration for better real time performance, and example deployment of Kubernetes cluster and plugins.

The audience of this document is assumed to have basic knowledge in Yocto/Open-Embedded Linux and container technology.

Version history

Date	Ver.	Author	Comment
2019-11-02	1.0.0	Jackie Huang	Initail version

Preface ¶

Before starting the installation and deployment of O-RAN INF, you need to download the ISO image or build from source as described in developer-guide.

Hardware Requirements ¶

Following minimum hardware requirements must be met for installation of O-RAN INF image:

HW Aspect	Requirement
# of servers	1
CPU	2
RAM	4G
Disk	20G
NICs	1

Software Installation and Deployment ¶

1. Installation from the O-RAN INF ISO image ¶

Please see the README.md file for how to build the image.
The Image is a live ISO image with CLI installer: oran-image-inf-host-intel-x86-64.iso

1.1 Burn the image to the USB device ¶

Assume the the usb device is /dev/sdX here

$ sudo dd if=/path/to/oran-image-inf-host-intel-x86-64.iso of=/dev/sdX bs=1M

1.2 Insert the USB device in the target to be booted.¶

1.3 Reboot the target from the USB device.¶

1.4 Select “Graphics console install” or “Serial console install” and press ENTER ¶

1.5 Select the hard disk and press ENTER ¶

Notes: In this installer, you can only select which hard disk to install, the whole disk will be used and partitioned automatically.

e.g. insert “sda” and press ENTER

1.6 Remove the USB device and press ENTER to reboot ¶

2. Configuration for better real time performance ¶

Notes: Some of the tuning options are machine specific or depend on use cases, like the hugepages, isolcpus, rcu_nocbs, kthread_cpus, irqaffinity, nohz_full and so on, please do not just copy and past.

Edit the grub.cfg with the following example tuning options

# Notes: the grub.cfg file path is different for legacy and UEFI mode
#   For legacy mode: /boot/grub/grub.cfg
#   For UEFI mode: /boot/EFI/BOOT/grub.cfg

grub_cfg="/boot/grub/grub.cfg"
#grub_cfg="/boot/EFI/BOOT/grub.cfg"

# In this example, 1-16 cores are isolated for real time processes
root@intel-x86-64:~# rt_tuning="crashkernel=auto biosdevname=0 iommu=pt usbcore.autosuspend=-1 nmi_watchdog=0 softlockup_panic=0 intel_iommu=on cgroup_enable=memory skew_tick=1 hugepagesz=1G hugepages=4 default_hugepagesz=1G isolcpus=1-16 rcu_nocbs=1-16 kthread_cpus=0 irqaffinity=0 nohz=on nohz_full=1-16 intel_idle.max_cstate=0 processor.max_cstate=1 intel_pstate=disable nosoftlockup idle=poll mce=ignore_ce"

# optional to add the console setting
root@intel-x86-64:~# console="console=ttyS0,115200"

root@intel-x86-64:~# sed -i "/linux / s/$/ $console $rt_tuning/" $grub_cfg

Reboot the target

root@intel-x86-64:~# reboot

3. Kubernetes cluster and plugins deployment instructions (All-in-one)¶

This instruction will show you how to deploy kubernetes cluster and plugins in an all-in-one example scenario after the above installation.

3.1 Change the hostname (Optional)¶

# Assuming the hostname is oran-aio, ip address is <aio_host_ip>
# please DO NOT copy and paste, use your actaul hostname and ip address
root@intel-x86-64:~# echo oran-aio > /etc/hostname
root@intel-x86-64:~# export AIO_HOST_IP="<aio_host_ip>"
root@intel-x86-64:~# echo "$AIO_HOST_IP oran-aio" >> /etc/hosts

3.2 Disable swap for Kubernetes ¶

root@intel-x86-64:~# sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab
root@intel-x86-64:~# systemctl mask dev-sda4.swap

3.3 Set the proxy for docker (Optional)¶

If you are under a firewall, you may need to set the proxy for docker to pull images

root@intel-x86-64:~# HTTP_PROXY="http://<your_proxy_server_ip>:<port>"
root@intel-x86-64:~# mkdir /etc/systemd/system/docker.service.d/
root@intel-x86-64:~# cat << EOF > /etc/systemd/system/docker.service.d/http-proxy.conf
[Service]
Environment="HTTP_PROXY=$HTTP_PROXY" "NO_PROXY=localhost,127.0.0.1,localaddress,.localdomain.com,$AIO_HOST_IP,10.244.0.0/16"
EOF

3.4 Reboot the target ¶

root@intel-x86-64:~# reboot

3.5 Initialize kubernetes cluster master ¶

root@oran-aio:~# kubeadm init --kubernetes-version v1.16.2 --pod-network-cidr=10.244.0.0/16
root@oran-aio:~# mkdir -p $HOME/.kube
root@oran-aio:~# cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
root@oran-aio:~# chown $(id -u):$(id -g) $HOME/.kube/config

3.6 Make the master also works as a worker node ¶

root@oran-aio:~# kubectl taint nodes oran-aio node-role.kubernetes.io/master-

3.7 Deploy flannel ¶

root@oran-aio:~# kubectl apply -f /etc/kubernetes/plugins/flannel/kube-flannel.yml

Check that the aio node is ready after flannel is successfully deployed and running

root@oran-aio:~# kubectl get pods --all-namespaces |grep flannel
kube-system   kube-flannel-ds-amd64-bwt52        1/1     Running   0          3m24s

root@oran-aio:~# kubectl get nodes
NAME       STATUS   ROLES    AGE     VERSION
oran-aio   Ready    master   3m17s   v1.15.2-dirty

3.8 Deploy kubernetes dashboard ¶

Deploy kubernetes dashboard

root@oran-aio:~# kubectl apply -f /etc/kubernetes/plugins/kubernetes-dashboard/kubernetes-dashboard-admin.rbac.yaml
root@oran-aio:~# kubectl apply -f /etc/kubernetes/plugins/kubernetes-dashboard/kubernetes-dashboard.yaml

Verify that the dashboard is up and running

# Check the pod for dashboard
root@oran-aio:~# kubectl get pods --all-namespaces |grep dashboard
kube-system   kubernetes-dashboard-5b67bf4d5f-ghg4f   1/1     Running   0          64s

Access the dashboard UI in a web browser with the https url, port number is 30443.

For detail usage, please refer to Doc for dashboard

3.9 Deploy Multus-CNI ¶

root@oran-aio:~# kubectl apply -f /etc/kubernetes/plugins/multus-cni/multus-daemonset.yml

Verify that the multus-cni is up and running

root@oran-aio:~# kubectl get pods --all-namespaces | grep -i multus
kube-system   kube-multus-ds-amd64-hjpk4              1/1     Running   0          7m34s

For further validating, please refer to the Multus-CNI quick start

3.10 Deploy NFD (node-feature-discovery)¶

root@oran-aio:~# kubectl apply -f /etc/kubernetes/plugins/node-feature-discovery/nfd-master.yaml
root@oran-aio:~# kubectl apply -f /etc/kubernetes/plugins/node-feature-discovery/nfd-worker-daemonset.yaml

Verify that nfd-master and nfd-worker are up and running

root@oran-aio:~# kubectl get pods --all-namespaces |grep nfd
default       nfd-master-7v75k                        1/1     Running   0          91s
default       nfd-worker-xn797                        1/1     Running   0          24s

Verify that the node is labeled by nfd:

root@oran-aio:~# kubectl describe nodes|grep feature.node.kubernetes
                   feature.node.kubernetes.io/cpu-cpuid.AESNI=true
                   feature.node.kubernetes.io/cpu-cpuid.AVX=true
                   feature.node.kubernetes.io/cpu-cpuid.AVX2=true
                   (...snip...)

3.11 Deploy SRIOV CNI ¶

Provision VF drivers and devices

Enumerate PF Devices

root@oran-aio:~/dpdk-18.08/usertools# lspci -D |grep 82599
0000:04:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
0000:04:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

Correlate the PF device to eth interfaces and bring them up

root@oran-aio:~# ethtool -i eth4  |grep bus-info
bus-info: 0000:04:00.0
root@oran-aio:~# ethtool -i eth5  |grep bus-info
bus-info: 0000:04:00.1
root@oran-aio:~# ifconfig eth4 up
root@oran-aio:~# ifconfig eth5 up

Load VF Driver modules

root@oran-aio:~# modprobe ixgbevf
root@oran-aio:~# modprobe uio
root@oran-aio:~# modprobe igb-uio
root@oran-aio:~# modprobe vfio
root@oran-aio:~# modprobe vfio-pci
root@oran-aio:~# lsmod |grep ixgbevf
ixgbevf                61440  0
root@oran-aio:~# lsmod |grep vfio
vfio_pci               40960  0
vfio_virqfd            16384  1 vfio_pci
vfio_iommu_type1       24576  0
vfio                   24576  2 vfio_iommu_type1,vfio_pci
irqbypass              16384  2 vfio_pci,kvm

Bind VF drivers to VF devices

root@oran-aio:~# cat /sys/bus/pci/devices/0000\:04\:00.0/sriov_totalvfs
root@oran-aio:~# cat /sys/bus/pci/devices/0000\:04\:00.1/sriov_totalvfs
root@oran-aio:~# cat /sys/bus/pci/devices/0000\:04\:00.0/sriov_numvfs
root@oran-aio:~# cat /sys/bus/pci/devices/0000\:04\:00.1/sriov_numvfs
root@oran-aio:~# echo 8 > /sys/bus/pci/devices/0000\:04\:00.0/sriov_numvfs
root@oran-aio:~# echo 8 > /sys/bus/pci/devices/0000\:04\:00.1/sriov_numvfs

root@oran-aio:~# lspci -D |grep 82599
04:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
04:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
04:10.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
04:10.1 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
04:10.2 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
04:10.3 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
04:10.4 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
04:10.5 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
04:10.6 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
04:10.7 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
04:11.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
04:11.1 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
04:11.2 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
04:11.3 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
04:11.4 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
04:11.5 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
04:11.6 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
04:11.7 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)

root@oran-aio:~# dpdk-devbind -b vfio-pci 0000:04:11.0 0000:04:11.1 0000:04:11.2 0000:04:11.3 0000:04:11.4 0000:04:11.5 0000:04:11.6 0000:04:11.7

root@oran-aio:~# dpdk-devbind --status-dev net

Network devices using DPDK-compatible driver
============================================
04:11.0 '82599 Ethernet Controller Virtual Function 10ed' drv=vfio-pci unused=ixgbevf,igb_uio
04:11.1 '82599 Ethernet Controller Virtual Function 10ed' drv=vfio-pci unused=ixgbevf,igb_uio
04:11.2 '82599 Ethernet Controller Virtual Function 10ed' drv=vfio-pci unused=ixgbevf,igb_uio
04:11.3 '82599 Ethernet Controller Virtual Function 10ed' drv=vfio-pci unused=ixgbevf,igb_uio
04:11.4 '82599 Ethernet Controller Virtual Function 10ed' drv=vfio-pci unused=ixgbevf,igb_uio
04:11.5 '82599 Ethernet Controller Virtual Function 10ed' drv=vfio-pci unused=ixgbevf,igb_uio
04:11.6 '82599 Ethernet Controller Virtual Function 10ed' drv=vfio-pci unused=ixgbevf,igb_uio
04:11.7 '82599 Ethernet Controller Virtual Function 10ed' drv=vfio-pci unused=ixgbevf,igb_uio

Network devices using kernel driver
===================================
04:00.0 '82599ES 10-Gigabit SFI/SFP+ Network Connection 10fb' if=eth4 drv=ixgbe unused=igb_uio,vfio-pci
04:00.1 '82599ES 10-Gigabit SFI/SFP+ Network Connection 10fb' if=eth5 drv=ixgbe unused=igb_uio,vfio-pci
04:10.0 '82599 Ethernet Controller Virtual Function 10ed' if=eth6 drv=ixgbevf unused=igb_uio,vfio-pci
04:10.1 '82599 Ethernet Controller Virtual Function 10ed' if=eth14 drv=ixgbevf unused=igb_uio,vfio-pci
04:10.2 '82599 Ethernet Controller Virtual Function 10ed' if=eth7 drv=ixgbevf unused=igb_uio,vfio-pci
04:10.3 '82599 Ethernet Controller Virtual Function 10ed' if=eth15 drv=ixgbevf unused=igb_uio,vfio-pci
04:10.4 '82599 Ethernet Controller Virtual Function 10ed' if=eth8 drv=ixgbevf unused=igb_uio,vfio-pci
04:10.5 '82599 Ethernet Controller Virtual Function 10ed' if=eth16 drv=ixgbevf unused=igb_uio,vfio-pci
04:10.6 '82599 Ethernet Controller Virtual Function 10ed' if= drv=ixgbevf unused=igb_uio,vfio-pci
04:10.7 '82599 Ethernet Controller Virtual Function 10ed' if=eth17 drv=ixgbevf unused=igb_uio,vfio-pci

Build SRIOV CNI

root@oran-aio:~# HTTP_PROXY="http://<your_proxy_server_ip>:<port>"

root@oran-aio:~# wget https://dl.google.com/go/go1.14.1.linux-amd64.tar.gz
root@oran-aio:~# tar -zxvf go1.14.1.linux-amd64.tar.gz
root@oran-aio:~# PATH=$PATH:/root/go/bin/
root@oran-aio:~# git clone https://github.com/intel/sriov-cni
root@oran-aio:~# cd sriov-cni
root@oran-aio:~/sriov-cni# make
root@oran-aio:~/sriov-cni# cp build/sriov /opt/cni/bin

root@oran-aio:~# cd ~/
root@oran-aio:~# git clone https://github.com/intel/sriov-network-device-plugin
root@oran-aio:~# cd sriov-network-device-plugin
root@oran-aio:~/sriov-network-device-plugin# git fetch origin pull/196/head:fpgadp
root@oran-aio:~/sriov-network-device-plugin# git checkout fpgadp
root@oran-aio:~/sriov-network-device-plugin# make image
root@oran-aio:~/sriov-network-device-plugin# docker images |grep sriov-device-plugin
nfvpe/sriov-device-plugin                             latest              f4e6bbefad67        5 minutes ago       25.5MB

Deploy SRIOV CNI

root@oran-aio:~/sriov-network-device-plugin# cat <<EOF> deployments/sriovdp_configMap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: sriovdp-config
  namespace: kube-system
data:
  config.json: |
    {
        "resourceList": [{
                "resourceName": "intel_sriov_netdevice",
                "selectors": {
                    "vendors": ["8086"],
                    "devices": ["154c", "10ed"],
                    "drivers": ["i40evf", "ixgbevf"]
                }
            },
            {
                "resourceName": "intel_sriov_dpdk",
                "selectors": {
                    "vendors": ["8086"],
                    "devices": ["154c", "10ed"],
                    "drivers": ["vfio-pci"]
                }
            },
            {
                "resourceName": "mlnx_sriov_rdma",
                "isRdma": true,
                "selectors": {
                    "vendors": ["15b3"],
                    "devices": ["1018"],
                    "drivers": ["mlx5_ib"]
                }
            }
        ]
    }
EOF

root@oran-aio:~/sriov-network-device-plugin# kubectl create -f deployments/sriovdp_configMap.yaml
root@oran-aio:~/sriov-network-device-plugin# kubectl create -f deployments/k8s-v1.16/sriovdp-daemonset.yaml

root@oran-aio:~/sriov-network-device-plugin# kubectl get pods --all-namespaces |grep kube-sriov-device-plugin
kube-system   kube-sriov-device-plugin-amd64-6lm8n   1/1     Running   0          12m

root@oran-aio:~/sriov-network-device-plugin# kubectl -n kube-system logs kube-sriov-device-plugin-amd64-6lm8n
I0327 02:14:46.488409   14488 manager.go:115] Creating new ResourcePool: intel_sriov_netdevice
I0327 02:14:46.488427   14488 factory.go:144] device added: [pciAddr: 0000:04:10.0, vendor: 8086, device: 10ed, driver: ixgbevf]
I0327 02:14:46.488439   14488 factory.go:144] device added: [pciAddr: 0000:04:10.1, vendor: 8086, device: 10ed, driver: ixgbevf]
I0327 02:14:46.488446   14488 factory.go:144] device added: [pciAddr: 0000:04:10.2, vendor: 8086, device: 10ed, driver: ixgbevf]
I0327 02:14:46.488459   14488 factory.go:144] device added: [pciAddr: 0000:04:10.3, vendor: 8086, device: 10ed, driver: ixgbevf]
I0327 02:14:46.488467   14488 factory.go:144] device added: [pciAddr: 0000:04:10.4, vendor: 8086, device: 10ed, driver: ixgbevf]
I0327 02:14:46.488473   14488 factory.go:144] device added: [pciAddr: 0000:04:10.5, vendor: 8086, device: 10ed, driver: ixgbevf]
I0327 02:14:46.488479   14488 factory.go:144] device added: [pciAddr: 0000:04:10.6, vendor: 8086, device: 10ed, driver: ixgbevf]
I0327 02:14:46.488485   14488 factory.go:144] device added: [pciAddr: 0000:04:10.7, vendor: 8086, device: 10ed, driver: ixgbevf]
I0327 02:14:46.488502   14488 manager.go:128] New resource server is created for intel_sriov_netdevice ResourcePool
I0327 02:14:46.488511   14488 manager.go:114]
I0327 02:14:46.488516   14488 manager.go:115] Creating new ResourcePool: intel_sriov_dpdk
I0327 02:14:46.488529   14488 factory.go:144] device added: [pciAddr: 0000:04:11.0, vendor: 8086, device: 10ed, driver: vfio-pci]
I0327 02:14:46.488538   14488 factory.go:144] device added: [pciAddr: 0000:04:11.1, vendor: 8086, device: 10ed, driver: vfio-pci]
I0327 02:14:46.488545   14488 factory.go:144] device added: [pciAddr: 0000:04:11.2, vendor: 8086, device: 10ed, driver: vfio-pci]
I0327 02:14:46.488551   14488 factory.go:144] device added: [pciAddr: 0000:04:11.3, vendor: 8086, device: 10ed, driver: vfio-pci]
I0327 02:14:46.488562   14488 factory.go:144] device added: [pciAddr: 0000:04:11.4, vendor: 8086, device: 10ed, driver: vfio-pci]
I0327 02:14:46.488569   14488 factory.go:144] device added: [pciAddr: 0000:04:11.5, vendor: 8086, device: 10ed, driver: vfio-pci]
I0327 02:14:46.488575   14488 factory.go:144] device added: [pciAddr: 0000:04:11.6, vendor: 8086, device: 10ed, driver: vfio-pci]
I0327 02:14:46.488581   14488 factory.go:144] device added: [pciAddr: 0000:04:11.7, vendor: 8086, device: 10ed, driver: vfio-pci]
I0327 02:14:46.488591   14488 manager.go:128] New resource server is created for intel_sriov_dpdk ResourcePool

Test intel_sriov_netdeivce

root@oran-aio:~/sriov-network-device-plugin# cat <<EOF> deployments/sriov-crd.yaml
apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
  name: sriov-net1
  annotations:
    k8s.v1.cni.cncf.io/resourceName: intel.com/intel_sriov_netdevice
spec:
  config: '{
  "type": "sriov",
  "cniVersion": "0.3.1",
  "name": "sriov-network",
  "vlan": 100,
  "ipam": {
    "type": "host-local",
    "subnet": "10.56.217.0/24",
    "routes": [{
      "dst": "0.0.0.0/0"
    }],
    "gateway": "10.56.217.1"
  }
}'
EOF

root@oran-aio:~/sriov-network-device-plugin# kubectl create -f deployments/sriov-crd.yaml
root@oran-aio:~/sriov-network-device-plugin# kubectl create -f deployments/pod-tc1.yaml
root@oran-aio:~/sriov-network-device-plugin# kubectl get pods  |grep testpod1
root@oran-aio:~/sriov-network-device-plugin# ip link |grep 'vlan 100'
  vf 3 MAC a6:01:0a:34:39:e1, vlan 100, spoof checking on, link-state auto, trust off, query_rss off

root@oran-aio:~/sriov-network-device-plugin# kubectl exec -it testpod1 -- ip addr show |grep a6:01:0a:34:39:e1 -C 2
  valid_lft forever preferred_lft forever
21: net1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
  link/ether a6:01:0a:34:39:e1 brd ff:ff:ff:ff:ff:ff
  inet 10.56.217.3/24 brd 10.56.217.255 scope global net1
     valid_lft forever preferred_lft forever

Test intel_sriov_dpdk

root@oran-aio:~/sriov-network-device-plugin# cat <<EOF> deployments/sriovdpdk-crd.yaml
apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
  name: sriov1-vfio
  annotations:
    k8s.v1.cni.cncf.io/resourceName: intel.com/intel_sriov_dpdk
spec:
  config: '{
  "type": "sriov",
  "cniVersion": "0.3.1",
  "vlan": 101,
  "name": "sriov1-vfio"
}'
EOF

root@oran-aio:~/sriov-network-device-plugin# cat <<EOF> deployments/dpdk-1g.yaml
apiVersion: v1
kind: Pod
metadata:
  name: dpdk-1g
  annotations:
    k8s.v1.cni.cncf.io/networks: '[
       {"name": "sriov1-vfio"},
       {"name": "sriov1-vfio"}
    ]'
spec:
  restartPolicy: Never
  containers:
  - name: dpdk-1g
    image: centos/tools
    imagePullPolicy: IfNotPresent
    volumeMounts:
    - mountPath: /mnt/huge-2048
      name: hugepage
    - name: lib-modules
      mountPath: /lib/modules
    - name: src
      mountPath: /usr/src
    command: ["/bin/bash", "-ec", "sleep infinity"]
    securityContext:
      privileged: true
      capabilities:
        add:
        - ALL
    resources:
      requests:
        memory: 4Gi
        hugepages-1Gi: 4Gi
        intel.com/intel_sriov_dpdk: '2'
      limits:
        memory: 4Gi
        hugepages-1Gi: 4Gi
        intel.com/intel_sriov_dpdk: '2'
  imagePullSecrets:
  - name: admin-registry-secret
  volumes:
  - name: hugepage
    emptyDir:
      medium: HugePages
  - name: lib-modules
    hostPath:
      path: /lib/modules
  - name: src
    hostPath:
      path: /usr/src
  imagePullSecrets:
  - name: admin-registry-secret
EOF

root@oran-aio:~/sriov-network-device-plugin# kubectl create -f deployments/sriovdpdk-crd.yaml
root@oran-aio:~/sriov-network-device-plugin# kubectl create -f deployments/dpdk-1g.yaml

root@oran-aio:~/sriov-network-device-plugin# root@oran-aio:~/sriov-network-device-plugin# kubectl get pods | grep dpdk
dpdk-1g    1/1     Running   0          13s

root@oran-aio:~/sriov-network-device-plugin# ip link |grep 101
  vf 7 MAC 00:00:00:00:00:00, vlan 101, spoof checking on, link-state auto, trust off, query_rss off
  vf 6 MAC 00:00:00:00:00:00, vlan 101, spoof checking on, link-state auto, trust off, query_rss off

Now test with dpdk

### build following package and copy to target server: bitbake bison;bitbake kernel-devsrc
root@oran-aio:~/sriov-network-device-plugin# rpm -ivh ~/bison-3.0.4-r0.corei7_64.rpm
root@oran-aio:~/sriov-network-device-plugin# rpm -ivh ~/kernel-devsrc-1.0-r0.intel_x86_64.rpm

root@oran-aio:~/sriov-network-device-plugin# kubectl exec -it $(kubectl get pods -o wide | grep dpdk | awk '{ print $1 }') -- /bin/bash
[root@dpdk-1g /]# export |grep INTEL
  declare -x PCIDEVICE_INTEL_COM_INTEL_SRIOV_DPDK="0000:04:11.6,0000:04:11.5"

[root@dpdk-1g /]# yum -y install wget ncurses-devel unzip libpcap-devel ncurses-devel libedit-devel pciutils lua-devel

[root@dpdk-1g /]# cd /opt
[root@dpdk-1g /]# wget https://fast.dpdk.org/rel/dpdk-18.08.tar.xz
[root@dpdk-1g /]# tar xf dpdk-18.08.tar.xz
[root@dpdk-1g /]# cd dpdk-18.08/
[root@dpdk-1g /]# sed -i 's/CONFIG_RTE_EAL_IGB_UIO=y/CONFIG_RTE_EAL_IGB_UIO=n/g' config/common_linuxapp
[root@dpdk-1g /]# sed -i 's/CONFIG_RTE_LIBRTE_KNI=y/CONFIG_RTE_LIBRTE_KNI=n/g' config/common_linuxapp
[root@dpdk-1g /]# sed -i 's/CONFIG_RTE_KNI_KMOD=y/CONFIG_RTE_KNI_KMOD=n/g' config/common_linuxapp
[root@dpdk-1g /]# export RTE_SDK=/opt/dpdk-18.08
[root@dpdk-1g /]# export RTE_TARGET=x86_64-native-linuxapp-gcc
[root@dpdk-1g /]# export RTE_BIND=$RTE_SDK/usertools/dpdk-devbind.py
[root@dpdk-1g /]# make install T=$RTE_TARGET
[root@dpdk-1g /]# cd examples/helloworld
[root@dpdk-1g /]# make
[root@dpdk-1g /]# NR_hugepages=2
[root@dpdk-1g /]# ./build/helloworld -l 1-4 -n 4 -m $NR_hugepages
...
    hello from core 2
    hello from core 3
    hello from core 4
    hello from core 1