【OAI】【debug】分立机器基站接入核心网并调试可用解决之路

问题描述

在两台机器上,分别为服务器A与主机B,其中服务器A上部署OAI核心网,主机B上部署容器版ueransim。两机器用一根网线连接。两台机器的连接如此。但是ueransim接入核心网后无法通过创建的uesimtun传输数据到核心网。

解决思路1——机器网卡设置到192.168.70.0网段

将机器物理网卡网段设置到与核心网所在网段一致,192.168.70.x,ueransim直接接入。

测试过程

两机器分别设置如下:

核心网A:

ens38:
                  addresses: [192.168.70.12/24]
                  dhcp4: no
                  gateway4: 192.168.70.1
                  nameservers:
                          addresses: [8.8.8.8,144.144.144.144,255.255.255.255]

主机B:

ens39:
                  addresses: [192.168.70.13/24]
                  dhcp4: no
                  gateway4: 192.168.70.1
                  nameservers:
                          addresses: [8.8.8.8,144.144.144.144,255.255.255.255]

两者可以互相ping通,但开启核心网后却无法互相ping通

进行如下修改,将对方网口作为自己的网关后,在开通核心网后可以ping通,但依旧无法ping通核心网

核心网A:

ens38:
                  addresses: [192.168.70.12/24]
                  dhcp4: no
                  gateway4: 192.168.70.12
                  nameservers:
                          addresses: [8.8.8.8,144.144.144.144,255.255.255.255]

主机B:

ens39:
                  addresses: [192.168.70.13/24]
                  dhcp4: no
                  gateway4: 192.168.70.13
                  nameservers:
                          addresses: [8.8.8.8,144.144.144.144,255.255.255.255]

经过在服务器A抓包,发现服务器A的192.168.70.12上有抓到来自192.168.70.13寻找192.168.70.1的ARP,但是却没有成功找到的返回
有来自192.168.70.13的寻找192.168.70.1的icmp包,但只有request, no response found

然后通过如下方法解决,让主机B可以ping到192.168.70.1:
在主机B上添加到192.168.70.0/24网段的路由:

sudo route add -net 192.168.70.0/24 gw 192.168.70.12

在服务器A添加到主机B的路由:

sudo route add -host 192.168.70.13 gw 192.168.70.12

在这里插入图片描述
然后启动ueransim测试:
在这里插入图片描述

结论

无法启动,ueransim需要接入demo-oai等网桥,貌似无法接入另一台机器上的网桥。

但这样的网络配置可以作为非容器化ueransim的一种配置,中间不用换网段到192.168.12.x

解决思路2 docker-swarm

Docker Compose和Docker Swarm旨在实现完全集成,这意味着您可以将Compose应用程序指向Swarm集群,并使它们都像使用单个Docker主机一样正常工作。引用
manager是我们管理集群的入口,我们的docker命令都是在manager上执行,node节点上是不能执行dockr命令的

测试过程

这里我将服务器A作为管理节点,主机B作为工作节点

在服务器A上,通过ssh接入想作为管理节点的服务器,然后执行swarm init命令,指定swarm管理机的IP地址:

#安装docker-ce
sudo apt-get update
sudo apt-get install docker-ce
#指定为管理节点,该节点ip为192.168.12.3
lab@lab-virtual-machine:~$ docker swarm init --advertise-addr 192.168.12.3

出现以下提示

Swarm initialized: current node (uowf6o6oftuh47y2qjgm7jqeh) is now a manager.

To add a worker to this swarm, run the following command:

    docker swarm join --token SWMTKN-1-2f3q3ajwud8mg8xw62zbs7k0hpw9ypss7qgqcyjdzw5dns4qxf-a9mnk42uf9tjh58yj8s00t4ta 192.168.12.3:2377

To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.

其中这里表示如何加入工作节点:

docker swarm join --token SWMTKN-1-2f3q3ajwud8mg8xw62zbs7k0hpw9ypss7qgqcyjdzw5dns4qxf-a9mnk42uf9tjh58yj8s00t4ta 192.168.12.3:2377

在主机B上将主机B设置为工作节点:

docker swarm join --token SWMTKN-1-2f3q3ajwud8mg8xw62zbs7k0hpw9ypss7qgqcyjdzw5dns4qxf-a9mnk42uf9tjh58yj8s00t4ta 192.168.12.3:2377

出现以下提示:

This node joined a swarm as a worker.

到此搭建了一个简单的swarm集群。

查看接入的节点:

lab@lab-virtual-machine:~$ docker node ls

在这里插入图片描述
查看创建的网络:

docker network ls

在这里插入图片描述
创建一个新网络:
网络模式需要选择overlay

docker network create -d overlay testnet

在这里插入图片描述
可是OAI核心网的网络模式是bridge,是否可以将其变成overlay?

将所有docker-compose-basic-vpp-nrf.yaml中的网络由bridge改为overlay:

networks:
    public_net:
        driver: overlay
        name: demo-oai-public-net
        ipam:
            config:
                - subnet: 192.168.70.0/24
        driver_opts:
            com.docker.network.bridge.name: "demo-oai"
    public_net_access:
        driver: overlay
        name: oai-public-access
        ipam:
            config:
                - subnet: 192.168.72.0/24
        driver_opts:
            com.docker.network.bridge.name: "cn5g-access"
    public_net_core:
        driver: overlay
        name: oai-public-core
        ipam:
            config:
                - subnet: 192.168.73.0/24
        driver_opts:
            com.docker.network.bridge.name: "cn5g-core"

然后使用docker stack启动

之前 swarm 集群中docker service create一次只能部署一个微服务,我们可以使用 docker stack + compose 一次启动多个服务
stack相关操作:

deploy      Deploy a new stack or update an existing stack
ls          List stacks
ps          List the tasks in the stack
rm          Remove one or more stacks
services    List the services in the stack
lab@lab-virtual-machine:~/oai-cn5g-fed/docker-compose$ docker stack deploy --compose-file docker-compose-basic-vpp-nrf.yaml oai

在这里插入图片描述
查看运行的服务:

docker service ls

在这里插入图片描述

查看容器运行状态:

docker service ps 容器名字

在这里插入图片描述

启动ueransim;

lab@lab-virtual-machine:~/oai-cn5g-fed/docker-compose$ sudo docker stack deploy --compose-file docker-compose-ueransim-vpp.yaml ueransim
[sudo] password for lab: 
WARN[0000] network public_net_access: network.external.name is deprecated in favor of network.name 
WARN[0000] network public_net: network.external.name is deprecated in favor of network.name 
Ignoring unsupported options: privileged

Ignoring deprecated options:

container_name: Setting the container name is not supported.

Error response from daemon: 2 matches found based on name: network demo-oai-public-net is ambiguous

其中前两个警告WARN【0000】是由于yaml文件网络external的格式不对,更改为如下后会消失:

networks:
    public_net:
        external: true
        name: demo-oai-public-net
    public_net_access:
        external: true
        name: oai-public-access

后面的警告Error response from daemon: 2 matches found based on name: network demo-oai-public-net is ambiguous是由于机器上docker network里的demo-oai-public-net有local与swarm两种
在这里插入图片描述
将其中local的部分删掉:

docker network rm f6dd0838cf68
docker network rm 34b108d99783
docker network rm f9b3e7f7b442

再次启动ueransim:

lab@lab-virtual-machine:~/oai-cn5g-fed/docker-compose$ sudo docker stack deploy --compose-file docker-compose-ueransim-vpp.yaml ueransim
Ignoring unsupported options: privileged

Ignoring deprecated options:

container_name: Setting the container name is not supported.

Creating service ueransim_ueransim

在这里插入图片描述

但是查看ueransim的日志,发现未能启动成功:
甚至连gnb都没有成功接入OAI核心网

lab@lab-virtual-machine:~/oai-cn5g-fed/docker-compose$ docker service logs 5iarfozcjdvi
WARNING: Error loading config file: /home/lab/.docker/config.json: open /home/lab/.docker/config.json: permission denied
ueransim_ueransim.1.4wxgx35hss0j@lab-virtual-machine    | Now setting these variables '@GTP_IP@ @IGNORE_STREAM_IDS@ @LINK_IP@ @MCC@ @MNC@ @NCI@ @NGAP_IP@ @NGAP_PEER_IP@ @SD_0@ @SD_1@ @SD_2@ @SST_0@ @SST_1@ @SST_2@ @TAC@'
ueransim_ueransim.1.4wxgx35hss0j@lab-virtual-machine    | Now setting these variables '@AMF_VALUE@ @APN@ @GNB_IP_ADDRESS@ @IMEI@ @IMEI_SV@ @IMSI@ @KEY@ @MCC@ @MNC@ @OP@ @OP_TYPE@ @PDU_TYPE@ @SD_C@ @SD_D@ @SD_R@ @SST_C@ @SST_D@ @SST_R@'
ueransim_ueransim.1.4wxgx35hss0j@lab-virtual-machine    | Done setting the configuration
ueransim_ueransim.1.4wxgx35hss0j@lab-virtual-machine    | ### Running ueransim ###
ueransim_ueransim.1.4wxgx35hss0j@lab-virtual-machine    | Running gnb
ueransim_ueransim.1.4wxgx35hss0j@lab-virtual-machine    | UERANSIM v3.2.5
ueransim_ueransim.1.4wxgx35hss0j@lab-virtual-machine    | [2023-04-17 07:30:40.606] [rls-udp] [error] RLS failure [Socket bind failed: Cannot assign requested address]
ueransim_ueransim.1.4wxgx35hss0j@lab-virtual-machine    | [2023-04-17 07:30:40.607] [sctp] [info] Trying to establish SCTP connection... (192.168.70.132:38412)
ueransim_ueransim.1.4wxgx35hss0j@lab-virtual-machine    | [2023-04-17 07:30:40.607] [gtp] [error] GTP/UDP task could not be created. Socket bind failed: Cannot assign requested address
ueransim_ueransim.1.jygw0gvxxm32@lab-virtual-machine    | Now setting these variables '@GTP_IP@ @IGNORE_STREAM_IDS@ @LINK_IP@ @MCC@ @MNC@ @NCI@ @NGAP_IP@ @NGAP_PEER_IP@ @SD_0@ @SD_1@ @SD_2@ @SST_0@ @SST_1@ @SST_2@ @TAC@'
ueransim_ueransim.1.4wxgx35hss0j@lab-virtual-machine    | [2023-04-17 07:30:40.613] [sctp] [error] Binding to 192.168.70.141:0 failed. SCTP bind failed: Cannot assign requested address
ueransim_ueransim.1.jygw0gvxxm32@lab-virtual-machine    | Now setting these variables '@AMF_VALUE@ @APN@ @GNB_IP_ADDRESS@ @IMEI@ @IMEI_SV@ @IMSI@ @KEY@ @MCC@ @MNC@ @OP@ @OP_TYPE@ @PDU_TYPE@ @SD_C@ @SD_D@ @SD_R@ @SST_C@ @SST_D@ @SST_R@'
ueransim_ueransim.1.4wxgx35hss0j@lab-virtual-machine    | Running ue
ueransim_ueransim.1.jygw0gvxxm32@lab-virtual-machine    | Done setting the configuration
ueransim_ueransim.1.4wxgx35hss0j@lab-virtual-machine    | UERANSIM v3.2.5
ueransim_ueransim.1.4wxgx35hss0j@lab-virtual-machine    | [2023-04-17 07:30:41.589] [nas] [info] UE switches to state [MM-DEREGISTERED/PLMN-SEARCH]
ueransim_ueransim.1.jygw0gvxxm32@lab-virtual-machine    | ### Running ueransim ###
ueransim_ueransim.1.4wxgx35hss0j@lab-virtual-machine    | [2023-04-17 07:30:43.791] [nas] [error] PLMN selection failure, no cells in coverage
ueransim_ueransim.1.4wxgx35hss0j@lab-virtual-machine    | [2023-04-17 07:30:45.993] [nas] [error] PLMN selection failure, no cells in coverage
ueransim_ueransim.1.jygw0gvxxm32@lab-virtual-machine    | Running gnb
ueransim_ueransim.1.4wxgx35hss0j@lab-virtual-machine    | [2023-04-17 07:30:46.592] [rrc] [warning] Acceptable cell selection failed, no cell is in coverage
ueransim_ueransim.1.jygw0gvxxm32@lab-virtual-machine    | UERANSIM v3.2.5
ueransim_ueransim.1.4wxgx35hss0j@lab-virtual-machine    | [2023-04-17 07:30:46.592] [rrc] [error] Cell selection failure, no suitable or acceptable cell found
ueransim_ueransim.1.4wxgx35hss0j@lab-virtual-machine    | [2023-04-17 07:30:47.094] [nas] [info] UE switches to state [MM-DEREGISTERED/NO-CELL-AVAILABLE]
ueransim_ueransim.1.jygw0gvxxm32@lab-virtual-machine    | [2023-04-17 07:31:53.735] [rls-udp] [error] RLS failure [Socket bind failed: Cannot assign requested address]
ueransim_ueransim.1.4wxgx35hss0j@lab-virtual-machine    | [2023-04-17 07:31:16.604] [rrc] [warning] Acceptable cell selection failed, no cell is in coverage
ueransim_ueransim.1.4wxgx35hss0j@lab-virtual-machine    | [2023-04-17 07:31:16.605] [rrc] [error] Cell selection failure, no suitable or acceptable cell found
ueransim_ueransim.1.jygw0gvxxm32@lab-virtual-machine    | [2023-04-17 07:31:53.737] [sctp] [info] Trying to establish SCTP connection... (192.168.70.132:38412)
ueransim_ueransim.1.4wxgx35hss0j@lab-virtual-machine    | [2023-04-17 07:31:16.823] [nas] [error] PLMN selection failure, no cells in coverage
ueransim_ueransim.1.jygw0gvxxm32@lab-virtual-machine    | [2023-04-17 07:31:53.737] [gtp] [error] GTP/UDP task could not be created. Socket bind failed: Cannot assign requested address
ueransim_ueransim.1.jygw0gvxxm32@lab-virtual-machine    | [2023-04-17 07:31:53.743] [sctp] [error] Binding to 192.168.70.141:0 failed. SCTP bind failed: Cannot assign requested address
ueransim_ueransim.1.jygw0gvxxm32@lab-virtual-machine    | Running ue
ueransim_ueransim.1.jygw0gvxxm32@lab-virtual-machine    | UERANSIM v3.2.5
ueransim_ueransim.1.jygw0gvxxm32@lab-virtual-machine    | [2023-04-17 07:31:54.719] [nas] [info] UE switches to state [MM-DEREGISTERED/PLMN-SEARCH]
ueransim_ueransim.1.jygw0gvxxm32@lab-virtual-machine    | [2023-04-17 07:31:56.921] [nas] [error] PLMN selection failure, no cells in coverage
ueransim_ueransim.1.jygw0gvxxm32@lab-virtual-machine    | [2023-04-17 07:31:59.124] [nas] [error] PLMN selection failure, no cells in coverage
ueransim_ueransim.1.jygw0gvxxm32@lab-virtual-machine    | [2023-04-17 07:31:59.722] [rrc] [warning] Acceptable cell selection failed, no cell is in coverage
ueransim_ueransim.1.jygw0gvxxm32@lab-virtual-machine    | [2023-04-17 07:31:59.722] [rrc] [error] Cell selection failure, no suitable or acceptable cell found
ueransim_ueransim.1.jygw0gvxxm32@lab-virtual-machine    | [2023-04-17 07:32:00.225] [nas] [info] UE switches to state [MM-DEREGISTERED/NO-CELL-AVAILABLE]
ueransim_ueransim.1.jygw0gvxxm32@lab-virtual-machine    | [2023-04-17 07:32:29.734] [rrc] [warning] Acceptable cell selection failed, no cell is in coverage

删除oai与ueransim服务:

docker stack rm oai
docker stack rm ueransim

将ueransim与oai合并到一起启动:
创建文件1.yaml

version: '3.8'
services:
    mysql:
        container_name: "mysql"
        image: mysql:8.0
        volumes:
            - ./database/oai_db2.sql:/docker-entrypoint-initdb.d/oai_db.sql
            - ./healthscripts/mysql-healthcheck2.sh:/tmp/mysql-healthcheck.sh
        environment:
            - TZ=Europe/Paris
            - MYSQL_DATABASE=oai_db
            - MYSQL_USER=test
            - MYSQL_PASSWORD=test
            - MYSQL_ROOT_PASSWORD=linux
        healthcheck:
            test: /bin/bash -c "/tmp/mysql-healthcheck.sh"
            interval: 10s
            timeout: 5s
            retries: 30
        networks:
            public_net:
                ipv4_address: 192.168.70.131
    oai-udr:
        container_name: "oai-udr"
        image: oaisoftwarealliance/oai-udr:v1.5.0
        environment:
            - TZ=Europe/Paris
            - UDR_NAME=OAI_UDR
            - UDR_INTERFACE_NAME_FOR_NUDR=eth0
            - MYSQL_IPV4_ADDRESS=192.168.70.131
            - MYSQL_USER=test
            - MYSQL_PASS=test
            - MYSQL_DB=oai_db
            - WAIT_MYSQL=120
            - USE_FQDN_DNS=yes
            - REGISTER_NRF=yes
            - NRF_IPV4_ADDRESS=192.168.70.130
            - NRF_FQDN=oai-nrf
        depends_on:
            - mysql
            - oai-nrf
        networks:
            public_net:
                ipv4_address: 192.168.70.136
    oai-udm:
        container_name: "oai-udm"
        image: oaisoftwarealliance/oai-udm:v1.5.0
        environment:
            - TZ=Europe/Paris
            - UDM_NAME=OAI_UDM
            - SBI_IF_NAME=eth0
            - REGISTER_NRF=yes
            - USE_FQDN_DNS=yes
            - UDR_IP_ADDRESS=192.168.70.136
            - UDR_FQDN=oai-udr
            - NRF_IPV4_ADDRESS=192.168.70.130
            - NRF_FQDN=oai-nrf
        depends_on:
            - oai-udr
        networks:
            public_net:
                ipv4_address: 192.168.70.137
    oai-ausf:
        container_name: "oai-ausf"
        image: oaisoftwarealliance/oai-ausf:v1.5.0
        environment:
            - TZ=Europe/Paris
            - AUSF_NAME=OAI_AUSF
            - SBI_IF_NAME=eth0
            - USE_FQDN_DNS=yes
            - REGISTER_NRF=yes
            - UDM_IP_ADDRESS=192.168.70.137
            - UDM_FQDN=oai-udm
            - NRF_IPV4_ADDRESS=192.168.70.130
            - NRF_FQDN=oai-nrf
        depends_on:
            - oai-udm
        networks:
            public_net:
                ipv4_address: 192.168.70.138
    oai-nrf:
        container_name: "oai-nrf"
        image: oaisoftwarealliance/oai-nrf:v1.5.0
        environment:
            - TZ=Europe/Paris
            - NRF_INTERFACE_NAME_FOR_SBI=eth0
        networks:
            public_net:
                ipv4_address: 192.168.70.130
    oai-amf:
        container_name: "oai-amf"
        image: oaisoftwarealliance/oai-amf:v1.5.0
        environment:
            - TZ=Europe/paris
            - MCC=208
            - MNC=95
            - REGION_ID=128
            - AMF_SET_ID=1
            - SERVED_GUAMI_MCC_0=208
            - SERVED_GUAMI_MNC_0=95
            - SERVED_GUAMI_REGION_ID_0=128
            - SERVED_GUAMI_AMF_SET_ID_0=1
            - SERVED_GUAMI_MCC_1=460
            - SERVED_GUAMI_MNC_1=11
            - SERVED_GUAMI_REGION_ID_1=10
            - SERVED_GUAMI_AMF_SET_ID_1=1
            - PLMN_SUPPORT_MCC=208
            - PLMN_SUPPORT_MNC=95
            - PLMN_SUPPORT_TAC=0xa000
            # Slice 0 (222, 123)
            - SST_0=222
            - SD_0=123
            # Slice 0 (128, 12)
            - SST_1=128
            - SD_1=12
            - AMF_INTERFACE_NAME_FOR_NGAP=eth0
            - AMF_INTERFACE_NAME_FOR_N11=eth0
            # One single SMF instance
            - SMF_INSTANCE_ID_0=1
            - SMF_FQDN_0=oai-smf
            - SMF_IPV4_ADDR_0=192.168.70.133
            - SELECTED_0=true
            - NF_REGISTRATION=yes
            - SMF_SELECTION=yes
            - USE_FQDN_DNS=yes
            - EXTERNAL_AUSF=yes
            - EXTERNAL_UDM=no
            - EXTERNAL_NSSF=no
            - NRF_IPV4_ADDRESS=192.168.70.130
            - NRF_FQDN=oai-nrf
            - AUSF_IPV4_ADDRESS=192.168.70.138
            - AUSF_FQDN=oai-ausf
            - UDM_IPV4_ADDRESS=192.168.70.137
            - UDM_FQDN=oai-udm
            - INT_ALGO_LIST=[ "NIA1" , "NIA2"]
            - CIPH_ALGO_LIST=[ "NEA1" , "NEA2"]
        depends_on:
            - mysql
            - vpp-upf
            - oai-ext-dn
            - oai-ausf
        networks:
            public_net:
                ipv4_address: 192.168.70.132
    oai-smf:
        container_name: "oai-smf"
        image: oaisoftwarealliance/oai-smf:v1.5.0
        environment:
            - TZ=Europe/Paris
            - SMF_INTERFACE_NAME_FOR_N4=eth0
            - SMF_INTERFACE_NAME_FOR_SBI=eth0
            - DEFAULT_DNS_IPV4_ADDRESS=172.21.3.100
            - DEFAULT_DNS_SEC_IPV4_ADDRESS=8.8.8.8
            - AMF_IPV4_ADDRESS=192.168.70.132
            - AMF_FQDN=oai-amf
            - UDM_IPV4_ADDRESS=192.168.70.137
            - UDM_FQDN=oai-udm
            - UPF_IPV4_ADDRESS=192.168.70.201
            - UPF_FQDN_0=vpp-upf.node.5gcn.mnc95.mcc208.3gppnetwork.org
            - NRF_IPV4_ADDRESS=192.168.70.130
            - NRF_FQDN=oai-nrf
            - DEFAULT_CSCF_IPV4_ADDRESS=127.0.0.1  # only needed when ims is being used
            - USE_LOCAL_SUBSCRIPTION_INFO=yes  #Set to yes if SMF uses local subscription information instead of from an UDM
            - REGISTER_NRF=yes
            - DISCOVER_UPF=yes
            - USE_FQDN_DNS=yes
            - USE_NETWORK_INSTANCE=yes
            - ENABLE_USAGE_REPORTING=yes
            # Slice 0 (1, 0xFFFFFF)
            - DNN_NI0=oai
            - TYPE0=IPv4
            - DNN_RANGE0=12.1.1.151 - 12.1.1.253
            - NSSAI_SST0=1
            - SESSION_AMBR_UL0=200Mbps
            - SESSION_AMBR_DL0=400Mbps
            # Slice 1 (1, 1)
            - DNN_NI1=oai.ipv4
            - TYPE1=IPv4
            - DNN_RANGE1=12.1.1.51 - 12.1.1.150
            - NSSAI_SST1=1
            - NSSAI_SD1=1
            - SESSION_AMBR_UL1=100Mbps
            - SESSION_AMBR_DL1=200Mbps
            # Slice 2 (222, 123)
            - DNN_NI2=default
            - TYPE2=IPv4
            - DNN_RANGE2=12.1.1.2 - 12.1.1.50
            - NSSAI_SST2=222
            - NSSAI_SD2=123
            - SESSION_AMBR_UL2=50Mbps
            - SESSION_AMBR_DL2=100Mbps
            # Slice 3 for ims
            - DNN_NI3=ims
            - TYPE3=IPv4v6
            - DNN_RANGE3=14.1.1.2 - 14.1.1.253
        extra_hosts:
            - "vpp-upf.node.5gcn.mnc95.mcc208.3gppnetwork.org:192.168.70.201"
        depends_on:
            - oai-amf
        networks:
            public_net:
                ipv4_address: 192.168.70.133
    vpp-upf:
        privileged: true
        container_name: "vpp-upf"
        image: oaisoftwarealliance/oai-upf-vpp:v1.5.0
        environment:
            - IF_1_IP=192.168.70.201
            - IF_1_TYPE=N4
            - IF_2_IP=192.168.72.201
            - IF_2_TYPE=N3
            - IF_2_NWI=access.oai.org
            - IF_3_IP=192.168.73.201
            - IF_3_TYPE=N6
            - IF_3_IP_REMOTE=192.168.73.135 # EXT-DN IP Address
            - IF_3_NWI=internet.oai.org
            - NAME=VPP-UPF
            - MNC=95
            - MCC=208
            - REALM=3gppnetwork.org
            - VPP_MAIN_CORE=0
            - VPP_CORE_WORKER=1
#           - VPP_PLUGIN_PATH=/usr/lib64/vpp_plugins/                # RHEL7
            - VPP_PLUGIN_PATH=/usr/lib/x86_64-linux-gnu/vpp_plugins/ # Ubntu18.04
            - SNSSAI_SD=123
            - SNSSAI_SST=222
            - DNN=default
            - REGISTER_NRF=yes
            - NRF_IP_ADDR=192.168.70.130
            - NRF_PORT=80
            - HTTP_VERSION=1
        depends_on:
            - oai-nrf
        healthcheck:
            test: /bin/bash -c "pgrep vpp"
            interval: 10s
            timeout: 5s
            retries: 5
        networks:
            public_net:
                ipv4_address: 192.168.70.134
            public_net_access:
                ipv4_address: 192.168.72.134
            public_net_core:
                ipv4_address: 192.168.73.134
    oai-ext-dn:
        privileged: true
        init: true
        container_name: "oai-ext-dn"
        image: oaisoftwarealliance/trf-gen-cn5g:latest
        entrypoint: /bin/bash -c \
              "iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE;"\
              "ip route add 12.1.1.0/24 via 192.168.73.201 dev eth0; ip route; sleep infinity"
        command: ["/bin/bash", "-c", "trap : SIGTERM SIGINT; sleep infinity & wait"]
        healthcheck:
            test: /bin/bash -c "iptables -L -t nat | grep MASQUERADE"
            interval: 10s
            timeout: 5s
            retries: 5
        networks:
            public_net_core:
                ipv4_address: 192.168.73.135
    ueransim:
        container_name: ueransim
        image: ueransim:latest
        privileged: true
        environment:
            # GNB Congig Parameters
            - MCC=208
            - MNC=95
            - NCI=0x000000010
            - TAC=0xa000
            - LINK_IP=192.168.70.141
            - NGAP_IP=192.168.70.141
            - GTP_IP=192.168.72.141
            - NGAP_PEER_IP=192.168.70.132
            - SST=222
            - SD=123
            - IGNORE_STREAM_IDS=true
            # UE Config Parameters
            - NUMBER_OF_UE=1
            - IMSI=208950000000041
            - KEY=0C0A34601D4F07677303652C0462535B
            - OP=63bfa50ee6523365ff14c1f45f88737d
            - OP_TYPE=OPC
            - AMF_VALUE=8000
            - IMEI=356938035643803
            - IMEI_SV=0035609204079514
            - GNB_IP_ADDRESS=192.168.70.141
            - PDU_TYPE=IPv4
            - APN=default
            - SST_0=222
            - SD_0=123
            - SST_C=222
            - SD_C=123
            - SST_D=222
            - SD_D=123
        healthcheck:
            test: /bin/bash -c "ifconfig uesimtun0"
            interval: 10s
            timeout: 5s
            retries: 5
        networks:
            public_net:
                ipv4_address: 192.168.70.141
            public_net_access:
                ipv4_address: 192.168.72.141
networks:
    public_net:
        driver: overlay
        name: demo-oai-public-net
        ipam:
            config:
                - subnet: 192.168.70.0/24
        driver_opts:
            com.docker.network.bridge.name: "demo-oai"
    public_net_access:
        driver: overlay
        name: oai-public-access
        ipam:
            config:
                - subnet: 192.168.72.0/24
        driver_opts:
            com.docker.network.bridge.name: "cn5g-access"
    public_net_core:
        driver: overlay
        name: oai-public-core
        ipam:
            config:
                - subnet: 192.168.73.0/24
        driver_opts:
            com.docker.network.bridge.name: "cn5g-core"

启动1.yaml:

lab@lab-virtual-machine:~/oai-cn5g-fed/docker-compose$ docker stack deploy --compose-file 1.yaml oaiue
WARNING: Error loading config file: /home/lab/.docker/config.json: open /home/lab/.docker/config.json: permission denied
Ignoring unsupported options: privileged

Ignoring deprecated options:

container_name: Setting the container name is not supported.

Creating network oai-public-access
Creating network oai-public-core
Creating network demo-oai-public-net
Creating service oaiue_ueransim
Creating service oaiue_vpp-upf
Creating service oaiue_oai-amf
Creating service oaiue_oai-udr
Creating service oaiue_oai-udm
Creating service oaiue_oai-ausf
Creating service oaiue_oai-nrf
Creating service oaiue_mysql
Creating service oaiue_oai-ext-dn
Creating service oaiue_oai-smf

这样运行的容器乱七八糟的分布在多个机器上,需要通过如下方式指定不同容器运行的节点:

利用docker-compose.yml文件中的deploy.placement.constraints属性

#指定manage或worker
deploy:
  replicas: 1
     placement:
        constraints:
          ## 常用方式指定 manager或者work节点
          - node.role == worker 
#指定主机名
deploy:
  replicas: 1
     placement:
        constraints:
          - node.hostname == docker1 

指定标签的方法:
node label 可以给一个或多个机器打上一个标签,然后再compose文件中指定节点标签,就可以部署在打上对应标签的机器上。具体步骤如下:

现在主节点机器上对所有节点机器进行打标签操作,命令如下:

docker node update --label-add role=标签名称  主机名
 
例如我要对一个主机名为docker1的机器打上一个标签为db的标签
docker node update --label-add role=db  docker1

deploy:
  replicas: 1
     placement:
        constraints:
          - node.labels.role == db

除此以外还有指定节点ID的方式

这里我采用指定manager与worker的方式:
将核心网部署在manager节点上,将ueransim部署在worker节点上

version: '3.8'
services:
    mysql:
        container_name: "mysql"
        image: mysql:8.0
        volumes:
            - ./database/oai_db2.sql:/docker-entrypoint-initdb.d/oai_db.sql
            - ./healthscripts/mysql-healthcheck2.sh:/tmp/mysql-healthcheck.sh
        environment:
            - TZ=Europe/Paris
            - MYSQL_DATABASE=oai_db
            - MYSQL_USER=test
            - MYSQL_PASSWORD=test
            - MYSQL_ROOT_PASSWORD=linux
        healthcheck:
            test: /bin/bash -c "/tmp/mysql-healthcheck.sh"
            interval: 10s
            timeout: 5s
            retries: 30
        networks:
            public_net:
                ipv4_address: 192.168.70.131
        deploy:
          mode: global
          placement: 
             constraints: 
               - node.role == manager


    oai-udr:
        container_name: "oai-udr"
        image: oaisoftwarealliance/oai-udr:v1.5.0
        environment:
            - TZ=Europe/Paris
            - UDR_NAME=OAI_UDR
            - UDR_INTERFACE_NAME_FOR_NUDR=eth0
            - MYSQL_IPV4_ADDRESS=192.168.70.131
            - MYSQL_USER=test
            - MYSQL_PASS=test
            - MYSQL_DB=oai_db
            - WAIT_MYSQL=120
            - USE_FQDN_DNS=yes
            - REGISTER_NRF=yes
            - NRF_IPV4_ADDRESS=192.168.70.130
            - NRF_FQDN=oai-nrf
        depends_on:
            - mysql
            - oai-nrf
        networks:
            public_net:
                ipv4_address: 192.168.70.136
        deploy:
          mode: global
          placement: 
             constraints: 
               - node.role == manager

    oai-udm:
        container_name: "oai-udm"
        image: oaisoftwarealliance/oai-udm:v1.5.0
        environment:
            - TZ=Europe/Paris
            - UDM_NAME=OAI_UDM
            - SBI_IF_NAME=eth0
            - REGISTER_NRF=yes
            - USE_FQDN_DNS=yes
            - UDR_IP_ADDRESS=192.168.70.136
            - UDR_FQDN=oai-udr
            - NRF_IPV4_ADDRESS=192.168.70.130
            - NRF_FQDN=oai-nrf
        depends_on:
            - oai-udr
        networks:
            public_net:
                ipv4_address: 192.168.70.137
        deploy:
          mode: global
          placement: 
             constraints: 
               - node.role == manager

    oai-ausf:
        container_name: "oai-ausf"
        image: oaisoftwarealliance/oai-ausf:v1.5.0
        environment:
            - TZ=Europe/Paris
            - AUSF_NAME=OAI_AUSF
            - SBI_IF_NAME=eth0
            - USE_FQDN_DNS=yes
            - REGISTER_NRF=yes
            - UDM_IP_ADDRESS=192.168.70.137
            - UDM_FQDN=oai-udm
            - NRF_IPV4_ADDRESS=192.168.70.130
            - NRF_FQDN=oai-nrf
        depends_on:
            - oai-udm
        networks:
            public_net:
                ipv4_address: 192.168.70.138
        deploy:
          mode: global
          placement: 
             constraints: 
               - node.role == manager

    oai-nrf:
        container_name: "oai-nrf"
        image: oaisoftwarealliance/oai-nrf:v1.5.0
        environment:
            - TZ=Europe/Paris
            - NRF_INTERFACE_NAME_FOR_SBI=eth0
        networks:
            public_net:
                ipv4_address: 192.168.70.130
        deploy:
          mode: global
          placement: 
             constraints: 
               - node.role == manager

    oai-amf:
        container_name: "oai-amf"
        image: oaisoftwarealliance/oai-amf:v1.5.0
        environment:
            - TZ=Europe/paris
            - MCC=208
            - MNC=95
            - REGION_ID=128
            - AMF_SET_ID=1
            - SERVED_GUAMI_MCC_0=208
            - SERVED_GUAMI_MNC_0=95
            - SERVED_GUAMI_REGION_ID_0=128
            - SERVED_GUAMI_AMF_SET_ID_0=1
            - SERVED_GUAMI_MCC_1=460
            - SERVED_GUAMI_MNC_1=11
            - SERVED_GUAMI_REGION_ID_1=10
            - SERVED_GUAMI_AMF_SET_ID_1=1
            - PLMN_SUPPORT_MCC=208
            - PLMN_SUPPORT_MNC=95
            - PLMN_SUPPORT_TAC=0xa000
            # Slice 0 (222, 123)
            - SST_0=222
            - SD_0=123
            # Slice 0 (128, 12)
            - SST_1=128
            - SD_1=12
            - AMF_INTERFACE_NAME_FOR_NGAP=eth0
            - AMF_INTERFACE_NAME_FOR_N11=eth0
            # One single SMF instance
            - SMF_INSTANCE_ID_0=1
            - SMF_FQDN_0=oai-smf
            - SMF_IPV4_ADDR_0=192.168.70.133
            - SELECTED_0=true
            - NF_REGISTRATION=yes
            - SMF_SELECTION=yes
            - USE_FQDN_DNS=yes
            - EXTERNAL_AUSF=yes
            - EXTERNAL_UDM=no
            - EXTERNAL_NSSF=no
            - NRF_IPV4_ADDRESS=192.168.70.130
            - NRF_FQDN=oai-nrf
            - AUSF_IPV4_ADDRESS=192.168.70.138
            - AUSF_FQDN=oai-ausf
            - UDM_IPV4_ADDRESS=192.168.70.137
            - UDM_FQDN=oai-udm
            - INT_ALGO_LIST=[ "NIA1" , "NIA2"]
            - CIPH_ALGO_LIST=[ "NEA1" , "NEA2"]
        depends_on:
            - mysql
            - vpp-upf
            - oai-ext-dn
            - oai-ausf
        networks:
            public_net:
                ipv4_address: 192.168.70.132
        deploy:
          mode: global
          placement: 
             constraints: 
               - node.role == manager

    oai-smf:
        container_name: "oai-smf"
        image: oaisoftwarealliance/oai-smf:v1.5.0
        environment:
            - TZ=Europe/Paris
            - SMF_INTERFACE_NAME_FOR_N4=eth0
            - SMF_INTERFACE_NAME_FOR_SBI=eth0
            - DEFAULT_DNS_IPV4_ADDRESS=172.21.3.100
            - DEFAULT_DNS_SEC_IPV4_ADDRESS=8.8.8.8
            - AMF_IPV4_ADDRESS=192.168.70.132
            - AMF_FQDN=oai-amf
            - UDM_IPV4_ADDRESS=192.168.70.137
            - UDM_FQDN=oai-udm
            - UPF_IPV4_ADDRESS=192.168.70.201
            - UPF_FQDN_0=vpp-upf.node.5gcn.mnc95.mcc208.3gppnetwork.org
            - NRF_IPV4_ADDRESS=192.168.70.130
            - NRF_FQDN=oai-nrf
            - DEFAULT_CSCF_IPV4_ADDRESS=127.0.0.1  # only needed when ims is being used
            - USE_LOCAL_SUBSCRIPTION_INFO=yes  #Set to yes if SMF uses local subscription information instead of from an UDM
            - REGISTER_NRF=yes
            - DISCOVER_UPF=yes
            - USE_FQDN_DNS=yes
            - USE_NETWORK_INSTANCE=yes
            - ENABLE_USAGE_REPORTING=yes
            # Slice 0 (1, 0xFFFFFF)
            - DNN_NI0=oai
            - TYPE0=IPv4
            - DNN_RANGE0=12.1.1.151 - 12.1.1.253
            - NSSAI_SST0=1
            - SESSION_AMBR_UL0=200Mbps
            - SESSION_AMBR_DL0=400Mbps
            # Slice 1 (1, 1)
            - DNN_NI1=oai.ipv4
            - TYPE1=IPv4
            - DNN_RANGE1=12.1.1.51 - 12.1.1.150
            - NSSAI_SST1=1
            - NSSAI_SD1=1
            - SESSION_AMBR_UL1=100Mbps
            - SESSION_AMBR_DL1=200Mbps
            # Slice 2 (222, 123)
            - DNN_NI2=default
            - TYPE2=IPv4
            - DNN_RANGE2=12.1.1.2 - 12.1.1.50
            - NSSAI_SST2=222
            - NSSAI_SD2=123
            - SESSION_AMBR_UL2=50Mbps
            - SESSION_AMBR_DL2=100Mbps
            # Slice 3 for ims
            - DNN_NI3=ims
            - TYPE3=IPv4v6
            - DNN_RANGE3=14.1.1.2 - 14.1.1.253
        extra_hosts:
            - "vpp-upf.node.5gcn.mnc95.mcc208.3gppnetwork.org:192.168.70.201"
        depends_on:
            - oai-amf
        networks:
            public_net:
                ipv4_address: 192.168.70.133
        deploy:
          mode: global
          placement: 
             constraints: 
               - node.role == manager

    vpp-upf:
        privileged: true
        container_name: "vpp-upf"
        image: oaisoftwarealliance/oai-upf-vpp:v1.5.0
        environment:
            - IF_1_IP=192.168.70.201
            - IF_1_TYPE=N4
            - IF_2_IP=192.168.72.201
            - IF_2_TYPE=N3
            - IF_2_NWI=access.oai.org
            - IF_3_IP=192.168.73.201
            - IF_3_TYPE=N6
            - IF_3_IP_REMOTE=192.168.73.135 # EXT-DN IP Address
            - IF_3_NWI=internet.oai.org
            - NAME=VPP-UPF
            - MNC=95
            - MCC=208
            - REALM=3gppnetwork.org
            - VPP_MAIN_CORE=0
            - VPP_CORE_WORKER=1
#           - VPP_PLUGIN_PATH=/usr/lib64/vpp_plugins/                # RHEL7
            - VPP_PLUGIN_PATH=/usr/lib/x86_64-linux-gnu/vpp_plugins/ # Ubntu18.04
            - SNSSAI_SD=123
            - SNSSAI_SST=222
            - DNN=default
            - REGISTER_NRF=yes
            - NRF_IP_ADDR=192.168.70.130
            - NRF_PORT=80
            - HTTP_VERSION=1
        depends_on:
            - oai-nrf
        healthcheck:
            test: /bin/bash -c "pgrep vpp"
            interval: 10s
            timeout: 5s
            retries: 5
        networks:
            public_net:
                ipv4_address: 192.168.70.134
            public_net_access:
                ipv4_address: 192.168.72.134
            public_net_core:
                ipv4_address: 192.168.73.134
        deploy:
          mode: global
          placement: 
             constraints: 
               - node.role == manager

    oai-ext-dn:
        privileged: true
        init: true
        container_name: "oai-ext-dn"
        image: oaisoftwarealliance/trf-gen-cn5g:latest
        entrypoint: /bin/bash -c \
              "iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE;"\
              "ip route add 12.1.1.0/24 via 192.168.73.201 dev eth0; ip route; sleep infinity"
        command: ["/bin/bash", "-c", "trap : SIGTERM SIGINT; sleep infinity & wait"]
        healthcheck:
            test: /bin/bash -c "iptables -L -t nat | grep MASQUERADE"
            interval: 10s
            timeout: 5s
            retries: 5
        networks:
            public_net_core:
                ipv4_address: 192.168.73.135
        deploy:
          mode: global
          placement: 
             constraints: 
               - node.role == manager

    ueransim:
        container_name: ueransim
        image: ueransim:latest
        privileged: true
        environment:
            # GNB Congig Parameters
            - MCC=208
            - MNC=95
            - NCI=0x000000010
            - TAC=0xa000
            - LINK_IP=192.168.70.141
            - NGAP_IP=192.168.70.141
            - GTP_IP=192.168.72.141
            - NGAP_PEER_IP=192.168.70.132
            - SST=222
            - SD=123
            - IGNORE_STREAM_IDS=true
            # UE Config Parameters
            - NUMBER_OF_UE=1
            - IMSI=208950000000041
            - KEY=0C0A34601D4F07677303652C0462535B
            - OP=63bfa50ee6523365ff14c1f45f88737d
            - OP_TYPE=OPC
            - AMF_VALUE=8000
            - IMEI=356938035643803
            - IMEI_SV=0035609204079514
            - GNB_IP_ADDRESS=192.168.70.141
            - PDU_TYPE=IPv4
            - APN=default
            - SST_0=222
            - SD_0=123
            - SST_C=222
            - SD_C=123
            - SST_D=222
            - SD_D=123
        healthcheck:
            test: /bin/bash -c "ifconfig uesimtun0"
            interval: 10s
            timeout: 5s
            retries: 5
        networks:
            public_net:
                ipv4_address: 192.168.70.141
            public_net_access:
                ipv4_address: 192.168.72.141
        deploy:
          mode: global
          placement: 
             constraints: 
               - node.role == worker

networks:
    public_net:
        driver: overlay
        name: demo-oai-public-net
        ipam:
            config:
                - subnet: 192.168.70.0/24
        driver_opts:
            com.docker.network.bridge.name: "demo-oai"
    public_net_access:
        driver: overlay
        name: oai-public-access
        ipam:
            config:
                - subnet: 192.168.72.0/24
        driver_opts:
            com.docker.network.bridge.name: "cn5g-access"
    public_net_core:
        driver: overlay
        name: oai-public-core
        ipam:
            config:
                - subnet: 192.168.73.0/24
        driver_opts:
            com.docker.network.bridge.name: "cn5g-core"

经过较长时间的等待后,启动结果如下:
核心网网元正常启动,ueransim未成功
在这里插入图片描述
对ueransim进行重启:

docker service ls  
docker service update --force 07ms7mdsk907

重启后依旧无法接入

将ueransim与核心网都部署到服务器A上,依旧同样的报错

再尝试将所有容器全部部署到主机B上
在这里插入图片描述
依旧不行

通过设置容器依赖关系,让ueransim依赖与其他核心网网元,这样ueransim会在其他网元全部启动后才启动:

        depends_on:
            - oai-nrf
            - mysql
            - vpp-upf
            - oai-ext-dn
            - oai-ausf
            - oai-amf
            - oai-smf
            - oai-udm
            - oai-udr

结点服务的可视化界面管理visualizer:

docker pull dockersamples/visualizer:latest
docker run -itd --name visualizer -p 8099:8080 -e HOST=192.168.12.3 -e PORT=8080 -v /var/run/docker.sock:/var/run/docker.sock dockersamples/visualizer:latest

然后在浏览器输入 192.168.12.3:8090访问
reference1
reference2
reference3:docker-compose+swarm

结论

使用docker swarm可以实现容器的分立部署,让一个系统的容器自定义的分配到不同的主机上运行,我创建的项目中成功启动了OAI核心网与ueransim容器,核心网可以正常运行,但无论是将ueransim和OAI核心网分配到多台主机上还是一台主机上,ueransim始终无法正常接入,且其内部的网络关系被隐藏了难以进行调试,使用docker swarm方法失败。

讨论: 原始方案路由关系详解

分立部署核心网和ueransim
原始方案实现了OAI在服务器A上的部署,ueransim在主机B上的部署,并且ueransim接入了核心网,创建了PDU session,生成了用户应用接口uesimtun0:12.1.1.x,并且在ueransim容器内部可以ping到核心网demo-oai:192.168.70.1及其他网元以及进行TCP客户端&服务器的通信,但在通过uesimtun0进行通信却不行。
在此我们先详细分析此方案的路由关系,网络连接关系,在探讨其中可能存在的问题及解决方法。

https://www.cnblogs.com/davis12/p/14392125.html#%E4%BA%8C%E5%AE%B9%E5%99%A8%E7%9A%84ip%E5%8F%AF%E4%BB%A5%E8%A2%AB%E5%AE%BF%E4%B8%BB%E6%9C%BA%E4%BB%A5%E5%8F%8A%E5%85%B6%E5%AE%83%E4%B8%BB%E6%9C%BA%E7%9B%B4%E6%8E%A5%E8%AE%BF%E9%97%AE%E5%B1%80%E5%9F%9F%E7%BD%91

https://www.cnblogs.com/xingyys/p/11517656.html

https://www.zhihu.com/question/587187783

https://blog.csdn.net/hanxiaotongtong/article/details/125253806

0422调试问题发现

1.ueransim 非容器分立部署可以接入建立PDU session与uesimtun0但无法通过nr-binder绑定12.1.1.x传输数据:

基站侧的路由添加既需要添加到192.168.70.0/24的也需要1添加192.168.72.0/24的,
分析:在mini的核心网中只需要添加到一个路由可能是该路由既承载了控制面也承载了用户面,而在basic的核心网中,70网段承载了控制面,只添加此路由也可以建立PDUsession,但无法传输用户面数据,再添加到72网段的路由就可以了

2.sudo sysctl net.ipv4.conf.all forwarding=1的意义
在核心网添加这个命令,开启的是路由转发,使容器网桥上的各个网元可以被访问到
具体体现为,开启了这个在基站可以ping到网元如amf:192.168.70.132, 而不开启就ping不到

0422 讨论:解决问题路径发现

经过测试,在分立部署情况下,使用非容器的ueransim可以成功接入核心网并传输数据,没有复杂的网桥间路由,只有在基站侧添加到核心网两个网桥的路由,可以成功接通。
在容器ueransim情况下,只添加两个这样的网桥基站也可以接入,但无法使用12.1.1.x传输数据,然后在核心网侧添加到60网桥的路由后也无法传输

于是想到:是否可以把容器当作虚拟机来使用,让容器可以直接直连网口ens40,这样也许就可以在容器内部添加两个路由,并在容器内部运行ueransim程序,然后如同在虚拟机中一样接入核心网并传输数据。

关键:物理网口直连容器(查询结果如下),在容器内部直接访问物理网口(无太多相关结果)

https://blog.csdn.net/fgf00/article/details/52575500

https://blog.csdn.net/smallfish1983/article/details/38820441
此办法是将容器默认网桥或自建网桥与物理网卡绑定在一起,已经尝试过应该不行

https://blog.csdn.net/ithaibiantingsong/article/details/81386307
该文章写了两种方法,其一是端口映射,在创建容器时指定,使容器的中应用进程的端口暴露到宿主机上,外部网络对容器内部程序进行访问时只需要通过:宿主机IP+暴露端口号的方式进行访问 。
**Think:**如果是我用过的TCP客户端与服务器,我可以比较轻松的设置IP与端口号以及进行通信。但是ueransim这个程序如何的端口号如何获取?获取后对它进行访问可以进行怎样的操作? 归根结底端口号是传输层的事情,而要想使ueransim不做更改的接入OAI核心网需要进行数据链路层的连接。
与之相似的是指定容器的网络模式为host,这样创建的容器直接使用主机的命名空间,访问容器中运行的应用程序就像访问在主机中运行的应用程序一样。
**这种可以试一下:**将容器ueransim设置为host模式,看是否能直接接入服务器A上的核心网。进一步,由于官方的ueransim启动程序的问题,可能不支持直接设置为host自动启动,可能需要构建自己的ueransim容器,然后将其设置为host。

0424,尝试docker网络模式host解决

方法简述

在分立部署情况下,使用非容器的ueransim可以成功接入核心网并传输数据,没有复杂的网桥间路由,只有在基站侧添加到核心网两个网桥的路由,可以成功接通。
在容器ueransim情况下,只添加两个这样的网桥基站也可以接入,但无法使用12.1.1.x传输数据,然后在核心网侧添加到60网桥的路由后也无法传输

于是想到:是否可以把容器当作虚拟机来使用,让容器可以直接直连网口ens40,这样也许就可以在容器内部添加两个路由,并在容器内部运行ueransim程序,然后如同在虚拟机中一样接入核心网并传输数据。

经过查询使用docker的host网络模式,让docker容器与主机公用一个网络命名空间,使ueransim容器内的网络与主机网络完全互通,就可以访问主机物理接口

测试过程

docker host
docker-compose host
服务器A上核心网部分启动:

sudo sysctl net.ipv4.conf.all.forwarding=1
sudo iptables -P FORWARD ACCEPT
docker-compose -f docker-compose-basic-vpp-nrf.yaml up -d

主机B上ueransim更改:

#添加主机B到服务器A核心网的路由
gNB-host$: sudo ip route add  192.168.70.0/24 via 192.168.12.3 dev ens40
gNB-host$: sudo ip route add  192.168.72.0/24 via 192.168.12.3 dev ens40

将ueransim容器设置为host模式

version: '3.8'
services:
    ueransim:
        container_name: ueransim
        image: ueransim:latest
        privileged: true
        environment:
            # GNB Congig Parameters
            - MCC=208
            - MNC=95
            - NCI=0x000000010
            - TAC=0xa000
            - LINK_IP=192.168.70.141
            - NGAP_IP=192.168.70.141
            - GTP_IP=192.168.72.141
            - NGAP_PEER_IP=192.168.70.132
            - SST=222
            - SD=123
            - IGNORE_STREAM_IDS=true
            # UE Config Parameters
            - NUMBER_OF_UE=1
            - IMSI=208950000000041
            - KEY=0C0A34601D4F07677303652C0462535B
            - OP=63bfa50ee6523365ff14c1f45f88737d
            - OP_TYPE=OPC
            - AMF_VALUE=8000
            - IMEI=356938035643803
            - IMEI_SV=0035609204079514
            - GNB_IP_ADDRESS=192.168.70.141
            - PDU_TYPE=IPv4
            - APN=default
            - SST_0=222
            - SD_0=123
            - SST_C=222
            - SD_C=123
            - SST_D=222
            - SD_D=123
        network_mode: "host"
           
        healthcheck:
            test: /bin/bash -c "ifconfig uesimtun0"
            interval: 10s
            timeout: 5s
            retries: 5
#networks:
#    hostnet:
#     external: true
#    name: host片

直接启动无法接入,gnb都无法接入,日志如下:

lab@lab-virtual-machine:~/oai-cn5g-fed/docker-compose/ueransim yz$ docker-compose -f docker-compose-ueransim-vpp.yaml up
Creating ueransim ... done
Attaching to ueransim
ueransim    | Now setting these variables '@GTP_IP@ @IGNORE_STREAM_IDS@ @LINK_IP@ @MCC@ @MNC@ @NCI@ @NGAP_IP@ @NGAP_PEER_IP@ @SD_0@ @SD_1@ @SD_2@ @SST_0@ @SST_1@ @SST_2@ @TAC@'
ueransim    | Now setting these variables '@AMF_VALUE@ @APN@ @GNB_IP_ADDRESS@ @IMEI@ @IMEI_SV@ @IMSI@ @KEY@ @MCC@ @MNC@ @OP@ @OP_TYPE@ @PDU_TYPE@ @SD_C@ @SD_D@ @SD_R@ @SST_C@ @SST_D@ @SST_R@'
ueransim    | Done setting the configuration
ueransim    | ### Running ueransim ###
ueransim    | Running gnb
ueransim    | UERANSIM v3.2.5
ueransim    | [2023-04-24 04:26:34.612] [rls-udp] [error] RLS failure [Socket bind failed: Cannot assign requested address]
ueransim    | [2023-04-24 04:26:34.613] [sctp] [info] Trying to establish SCTP connection... (192.168.70.132:38412)
ueransim    | [2023-04-24 04:26:34.613] [gtp] [error] GTP/UDP task could not be created. Socket bind failed: Cannot assign requested address
ueransim    | [2023-04-24 04:26:34.614] [sctp] [error] Binding to 192.168.70.141:0 failed. SCTP bind failed: Cannot assign requested address
ueransim    | Running ue
ueransim    | UERANSIM v3.2.5
ueransim    | [2023-04-24 04:26:35.615] [nas] [info] UE switches to state [MM-DEREGISTERED/PLMN-SEARCH]
ueransim    | [2023-04-24 04:26:37.816] [nas] [error] PLMN selection failure, no cells in coverage
ueransim    | [2023-04-24 04:26:40.018] [nas] [error] PLMN selection failure, no cells in coverage
ueransim    | [2023-04-24 04:26:40.617] [rrc] [warning] Acceptable cell selection failed, no cell is in coverage
ueransim    | [2023-04-24 04:26:40.617] [rrc] [error] Cell selection failure, no suitable or acceptable cell found
ueransim    | [2023-04-24 04:26:41.119] [nas] [info] UE switches to state [MM-DEREGISTERED/NO-CELL-AVAILABLE]
ueransim    | [2023-04-24 04:27:10.629] [rrc] [warning] Acceptable cell selection failed, no cell is in coverage
ueransim    | [2023-04-24 04:27:10.629] [rrc] [error] Cell selection failure, no suitable or acceptable cell found
ueransim    | [2023-04-24 04:27:10.857] [nas] [error] PLMN selection failure, no cells in coverage
ueransim    | [2023-04-24 04:27:40.641] [rrc] [warning] Acceptable cell selection failed, no cell is in coverage
ueransim    | [2023-04-24 04:27:40.641] [rrc] [error] Cell selection failure, no suitable or acceptable cell found
ueransim    | [2023-04-24 04:27:41.685] [nas] [error] PLMN selection failure, no cells in coverage
ueransim    | [2023-04-24 04:28:10.653] [rrc] [warning] Acceptable cell selection failed, no cell is in coverage
ueransim    | [2023-04-24 04:28:10.653] [rrc] [error] Cell selection failure, no suitable or acceptable cell found
ueransim    | [2023-04-24 04:28:12.513] [nas] [error] PLMN selection failure, no cells in coverage

进入容器内部:

lab@lab-virtual-machine:~/oai-cn5g-fed/docker-compose/ueransim yz$ docker exec -it ueransim /bin/bash

查看容器内的网络发现与宿主机的一模一样:

root@lab-virtual-machine:/ueransim/bin# ifconfig
docker0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 172.17.0.1  netmask 255.255.0.0  broadcast 172.17.255.255
        ether 02:42:27:d3:5c:f4  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

ens33: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.2.106  netmask 255.255.255.0  broadcast 192.168.2.255
        inet6 fe80::f5f:7af1:a9b:2e90  prefixlen 64  scopeid 0x20<link>
        ether 00:0c:29:a6:b4:4b  txqueuelen 1000  (Ethernet)
        RX packets 17259  bytes 4920218 (4.9 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 6106  bytes 637253 (637.2 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

ens39: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.14.33  netmask 255.255.255.0  broadcast 192.168.14.255
        inet6 fe80::20c:29ff:fea6:b47d  prefixlen 64  scopeid 0x20<link>
        ether 00:0c:29:a6:b4:7d  txqueuelen 1000  (Ethernet)
        RX packets 3589  bytes 343412 (343.4 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 966  bytes 64547 (64.5 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

ens40: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.12.33  netmask 255.255.255.0  broadcast 192.168.12.255
        inet6 fe80::20c:29ff:fea6:b469  prefixlen 64  scopeid 0x20<link>
        ether 00:0c:29:a6:b4:69  txqueuelen 1000  (Ethernet)
        RX packets 68731  bytes 4201164 (4.2 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 415  bytes 35145 (35.1 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

ens41: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.13.33  netmask 255.255.255.0  broadcast 192.168.13.255
        inet6 fe80::20c:29ff:fea6:b473  prefixlen 64  scopeid 0x20<link>
        ether 00:0c:29:a6:b4:73  txqueuelen 1000  (Ethernet)
        RX packets 755  bytes 174401 (174.4 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 971  bytes 65182 (65.1 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 1917  bytes 204293 (204.2 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 1917  bytes 204293 (204.2 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

路由也与宿主机的一模一样:

root@lab-virtual-machine:/ueransim/bin# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.2.1     0.0.0.0         UG    100    0        0 ens33
0.0.0.0         192.168.14.1    0.0.0.0         UG    20101  0        0 ens39
0.0.0.0         192.168.12.1    0.0.0.0         UG    20102  0        0 ens40
0.0.0.0         192.168.13.1    0.0.0.0         UG    20103  0        0 ens41
169.254.0.0     0.0.0.0         255.255.0.0     U     1000   0        0 ens39
172.17.0.0      0.0.0.0         255.255.0.0     U     0      0        0 docker0
192.168.2.0     0.0.0.0         255.255.255.0   U     100    0        0 ens33
192.168.12.0    0.0.0.0         255.255.255.0   U     102    0        0 ens40
192.168.13.0    0.0.0.0         255.255.255.0   U     103    0        0 ens41
192.168.14.0    0.0.0.0         255.255.255.0   U     101    0        0 ens39
192.168.70.0    192.168.12.3    255.255.255.0   UG    0      0        0 ens40
192.168.72.0    192.168.12.3    255.255.255.0   UG    0      0        0 ens40

ping到其他IP也都如宿主机可以ping通:

root@lab-virtual-machine:/ueransim/bin# ping 192.168.70.1
PING 192.168.70.1 (192.168.70.1) 56(84) bytes of data.
64 bytes from 192.168.70.1: icmp_seq=1 ttl=64 time=0.930 ms
^C
--- 192.168.70.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.930/0.930/0.930/0.000 ms
root@lab-virtual-machine:/ueransim/bin# ping 192.168.70.132
PING 192.168.70.132 (192.168.70.132) 56(84) bytes of data.
64 bytes from 192.168.70.132: icmp_seq=1 ttl=63 time=0.910 ms
^C
--- 192.168.70.132 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.910/0.910/0.910/0.000 ms
root@lab-virtual-machine:/ueransim/bin# ping 192.168.12.3  
PING 192.168.12.3 (192.168.12.3) 56(84) bytes of data.
64 bytes from 192.168.12.3: icmp_seq=1 ttl=64 time=0.905 ms
^C
--- 192.168.12.3 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.905/0.905/0.905/0.000 ms

可以确定,此容器内部网络环境与在容器外一模一样,那么接下来先需要做的就是解决基站为何不能接入的问题:

#安装vim 用于编辑配置文件
apt-get update
apt-get install vim

基站与ue启动用代码:

/ueransim/bin/nr-gnb -c /ueransim/etc/custom-gnb.yaml 
/ueransim/bin/nr-ue -c /ueransim/etc/custom-ue.yaml -n $NUMBER_OF_UE

基站启动报错日志:

root@lab-virtual-machine:/ueransim/bin# /ueransim/bin/nr-gnb -c /ueransim/etc/custom-gnb.yaml 
UERANSIM v3.2.5
[2023-04-24 04:39:15.209] [rls-udp] [error] RLS failure [Socket bind failed: Cannot assign requested address]
[2023-04-24 04:39:15.210] [sctp] [info] Trying to establish SCTP connection... (192.168.70.132:38412)
[2023-04-24 04:39:15.210] [gtp] [error] GTP/UDP task could not be created. Socket bind failed: Cannot assign requested address
[2023-04-24 04:39:15.212] [sctp] [error] Binding to 192.168.70.141:0 failed. SCTP bind failed: Cannot assign requested address

打开启动基站配置文件:

vim /ueransim/etc/custom-gnb.yaml 

配置文件如下:


mcc: '208'          # Mobile Country Code value
mnc: '95'           # Mobile Network Code value (2 or 3 digits)

nci: '0x000000010'  # NR Cell Identity (36-bit)
idLength: 32        # NR gNB ID length in bits [22...32]
tac: 0xa000              # Tracking Area Code

linkIp: 192.168.70.141   # gNB's local IP address for Radio Link Simulation (Usually same with local IP)
ngapIp: 192.168.70.141   # gNB's local IP address for N2 Interface (Usually same with local IP)
gtpIp:  192.168.72.141    # gNB's local IP address for N3 Interface (Usually same with local IP)

# List of AMF address information
amfConfigs:
  - address: 192.168.70.132
    port: 38412

# List of supported S-NSSAIs by this gNB
slices:
  - sst: 222
    sd:  123
  - sst: 1
    sd:  0
  - sst: 129
    sd:  129

# Indicates whether or not SCTP stream number errors should be ignored.
ignoreStreamIds: true
                     

将其修改为如下配置后,gnb可以成功接入:

mcc: '208'          # Mobile Country Code value
mnc: '95'           # Mobile Network Code value (2 or 3 digits)

nci: '0x000000010'  # NR Cell Identity (36-bit)
idLength: 32        # NR gNB ID length in bits [22...32]
tac: 40960              # Tracking Area Code

linkIp: 127.0.0.1   # gNB's local IP address for Radio Link Simulation (Usually same with local IP)
ngapIp: 192.168.12.33   # gNB's local IP address for N2 Interface (Usually same with local IP)
gtpIp:  192.168.12.33   # gNB's local IP address for N3 Interface (Usually same with local IP)

# List of AMF address information
amfConfigs:
  - address: 192.168.70.132
    port: 38412

# List of supported S-NSSAIs by this gNB
slices:
  - sst: 222
    sd:  123
  - sst: 222
    sd:  123
  - sst: 222
    sd:  123

# Indicates whether or not SCTP stream number errors should be ignored.
ignoreStreamIds: true

接入日志如下:

在这里插入图片描述

然后进行ue的接入:
在这里插入图片描述
查看ue的配置文件如下:

vim /ueransim/etc/custom-ue.yaml
# IMSI number of the UE. IMSI = [MCC|MNC|MSISDN] (In total 15 digits)
supi: 'imsi-208950000000041'
# Mobile Country Code value of HPLMN
mcc: '208'
# Mobile Network Code value of HPLMN (2 or 3 digits)
mnc: '95'

# Permanent subscription key
key: '0C0A34601D4F07677303652C0462535B'
# Operator code (OP or OPC) of the UE
op: '63bfa50ee6523365ff14c1f45f88737d'
# This value specifies the OP type and it can be either 'OP' or 'OPC'
opType: 'OPC'
# Authentication Management Field (AMF) value
amf: '8000'
# IMEI number of the device. It is used if no SUPI is provided
imei: '356938035643803'
# IMEISV number of the device. It is used if no SUPI and IMEI is provided
imeiSv: '0035609204079514'

# List of gNB IP addresses for Radio Link Simulation
gnbSearchList:
  - 192.168.70.141

# UAC Access Identities Configuration
uacAic:
  mps: false
  mcs: false

# UAC Access Control Class
uacAcc:
  normalClass: 0
  class11: false
  class12: false
  class13: false
  class14: false
  class15: false

# Initial PDU sessions to be established
sessions:
  - type: 'IPv4'
    apn: 'default'
    slice:
      sst: 222

# Configured NSSAI for this UE by HPLMN
configured-nssai:
  - sst: 222
    sd: 123

# Default Configured NSSAI for this UE
default-nssai:
  - sst: 222
    sd: 123

# Supported integrity algorithms by this UE
integrity:
  IA1: true
  IA2: true
  IA3: true

# Supported encryption algorithms by this UE
ciphering:
  EA1: true
  EA2: true
  EA3: true

# Integrity protection maximum data rate for user plane
integrityMaxRate:
  uplink: 'full'
  downlink: 'full'
                                    

将配置文件修改为如下:

# IMSI number of the UE. IMSI = [MCC|MNC|MSISDN] (In total 15 digits)
supi: 'imsi-208950000000031'
# Mobile Country Code value of HPLMN
mcc: '208'
# Mobile Network Code value of HPLMN (2 or 3 digits)
mnc: '95'

# Permanent subscription key
key: '0C0A34601D4F07677303652C0462535B'
# Operator code (OP or OPC) of the UE
op: '63bfa50ee6523365ff14c1f45f88737d'
# This value specifies the OP type and it can be either 'OP' or 'OPC'
opType: 'OPC'
# Authentication Management Field (AMF) value
amf: '8000'
# IMEI number of the device. It is used if no SUPI is provided
imei: '356938035643803'
# IMEISV number of the device. It is used if no SUPI and IMEI is provided
imeiSv: '0035609204079514'

# List of gNB IP addresses for Radio Link Simulation
gnbSearchList:
  - 127.0.0.1

# UAC Access Identities Configuration
uacAic:
  mps: false
  mcs: false

# UAC Access Control Class
uacAcc:
  normalClass: 0
  class11: false
  class12: false
  class13: false
  class14: false
  class15: false

# Initial PDU sessions to be established
sessions:
  - type: 'IPv4'
    apn: 'default'
    slice:
      sst: 222
      sd: 123

# Configured NSSAI for this UE by HPLMN
configured-nssai:
  - sst: 222
    sd: 123

# Default Configured NSSAI for this UE
default-nssai:
  - sst: 222
    sd: 123

# Supported integrity algorithms by this UE
integrity:
  IA1: true
  IA2: true
  IA3: true

# Supported encryption algorithms by this UE
ciphering:
  EA1: true
  EA2: true
  EA3: true

# Integrity protection maximum data rate for user plane
integrityMaxRate:
  uplink: 'full'
  downlink: 'full'

启动ue:

/ueransim/bin/nr-ue -c /ueransim/etc/custom-ue.yaml

启动成功日志:
在这里插入图片描述
接下来进行一直没有成功的跨世界测试:

root@lab-virtual-machine:/ueransim/etc# ping -I 12.1.1.2 192.168.70.1

在这里插入图片描述
芜湖!卡了近一个月的问题到此成功解决!

下面我们再来测试下其他:

TCP通信测试:
将TCPclient.py脚本与ueransim配置文件放在同一文件夹下;
在这里插入图片描述
将TCPclient.py克隆到容器内部:

lab@lab-virtual-machine:~/oai-cn5g-fed/docker-compose/ueransim yz$ sudo docker cp  TCPclient.py 2082b5aa7971:/ueransim/bin
Successfully copied 2.56kB to 2082b5aa7971:/ueransim/bin

在服务器A的核心网启动TCPsever.py:
在这里插入图片描述

python3 TCPsever.py

TCPsever.py代码如下:

import socket
from threading import Thread


def new_client_connect(new_client_socket, client_ip_port):
    while True:
        # 收发数据
        recv_data = new_client_socket.recv(1024)
        if len(recv_data) != 0:
            recv_text = recv_data.decode("gb2312")
            print("接收到[%s]的信息:%s" % (str(client_ip_port), recv_text))
        else:
            print("客户端断开连接")
            break

        # # 关闭连接
        # new_client_socket.close()  # 表示断开与当前的客户端的通信


def main():
    # 创建套接字
    tcp_server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

    # 绑定端口和ip
    tcp_server_socket.bind(("192.168.70.1", 8088))

    # 设置套接字为被动监听模式,不能主动发送数据,128为允许接收的最大连接数
    tcp_server_socket.listen(128)

    while True:
        # 接收客户端连接
        new_client_socket, client_ip_port = tcp_server_socket.accept()

        t1 = Thread(target=new_client_connect, args=(new_client_socket, client_ip_port))
        t1.start()

        # tcp_server_socket.close()  # 表示不再接受新客户端的连接,已经连接的可以继续服务


if __name__ == '__main__':
    main()

在主机B基站测ueransim容器内启动TCPclient.py:

./nr-binder 12.1.1.2 python3 TCPclient.py

在这里插入图片描述

成功发送!!!

在服务器A核心网侧成功接收:
在这里插入图片描述
** 测试使用curl访问网页:**

#安装curl
root@lab-virtual-machine:/ueransim/bin# apt-get install curl

确保服务器A核心网侧网络连接正常可以访问百度

root@lab-virtual-machine:/ueransim/bin# ./nr-binder 12.1.1.2 curl www.baidu.com

在这里插入图片描述

最后几个重要的问题:

  1. 如何将这个容器变为一键启动式,而不需要接入容器内部
  2. 这样的容器是否可以生成多个运行,而不是像之前直接在虚拟机中只能运行一个
  3. 在主机上是否可以直接看到uesimtun0,ueranism安全性的问题

答案如下:

  1. 在主机上是否可以直接看到uesimtun0,ueranism安全性的问题?
    是可以看见的:
    在这里插入图片描述
    容器内外网络完全一致

  2. 这样的容器是否可以生成多个运行,而不是像之前直接在虚拟机中只能运行一个
    构建其他的ueransim:

新发现:之前在容器内部进行各种更改,对gnb以及ue的配置文件进行更改,其实只需要对ueransim的docke-compose配置文件进行进一步更改,就可以直接启动:
更改后的ueransim文件:

version: '3.8'
services:
    ueransim1:
        container_name: ueransim
        image: ueransim:latest
        privileged: true
        environment:
            # GNB Congig Parameters
            - MCC=208
            - MNC=95
            - NCI=0x000000010
            - TAC=0xa000
            - LINK_IP=127.0.0.1
            - NGAP_IP=192.168.12.33
            - GTP_IP=192.168.12.33
            - NGAP_PEER_IP=192.168.70.132
            - SST=222
            - SD=123
            - IGNORE_STREAM_IDS=true
            # UE Config Parameters
            - NUMBER_OF_UE=1
            - IMSI=208950000000031
            - KEY=0C0A34601D4F07677303652C0462535B
            - OP=63bfa50ee6523365ff14c1f45f88737d
            - OP_TYPE=OPC
            - AMF_VALUE=8000
            - IMEI=356938035643803
            - IMEI_SV=0035609204079514
            - GNB_IP_ADDRESS=127.0.0.1
            - PDU_TYPE=IPv4
            - APN=default
            - SST_0=222
            - SD_0=123
            - SST_C=222
            - SD_C=123
            - SST_D=222
            - SD_D=123
        network_mode: "host"
           
        healthcheck:
            test: /bin/bash -c "ifconfig uesimtun0"
            interval: 10s
            timeout: 5s
            retries: 5
#networks:
#    hostnet:
#     external: true
#    name: host

启动该ueransim结果如下:

lab@lab-virtual-machine:~/oai-cn5g-fed/docker-compose/ueransim yz$ docker-compose -f docker-compose-ueransim-vpp1.yaml up
WARNING: Found orphan containers (4aab7c15a706_ueransim) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.
Creating ueransim ... done
Attaching to ueransim
ueransim     | Now setting these variables '@GTP_IP@ @IGNORE_STREAM_IDS@ @LINK_IP@ @MCC@ @MNC@ @NCI@ @NGAP_IP@ @NGAP_PEER_IP@ @SD_0@ @SD_1@ @SD_2@ @SST_0@ @SST_1@ @SST_2@ @TAC@'
ueransim     | Now setting these variables '@AMF_VALUE@ @APN@ @GNB_IP_ADDRESS@ @IMEI@ @IMEI_SV@ @IMSI@ @KEY@ @MCC@ @MNC@ @OP@ @OP_TYPE@ @PDU_TYPE@ @SD_C@ @SD_D@ @SD_R@ @SST_C@ @SST_D@ @SST_R@'
ueransim     | Done setting the configuration
ueransim     | ### Running ueransim ###
ueransim     | Running gnb
ueransim     | UERANSIM v3.2.5
ueransim     | [2023-04-24 06:28:49.614] [sctp] [info] Trying to establish SCTP connection... (192.168.70.132:38412)
ueransim     | [2023-04-24 06:28:49.616] [sctp] [info] SCTP connection established (192.168.70.132:38412)
ueransim     | [2023-04-24 06:28:49.616] [sctp] [debug] SCTP association setup ascId[6]
ueransim     | [2023-04-24 06:28:49.616] [ngap] [debug] Sending NG Setup Request
ueransim     | [2023-04-24 06:28:49.621] [ngap] [debug] NG Setup Response received
ueransim     | [2023-04-24 06:28:49.621] [ngap] [info] NG Setup procedure is successful
ueransim     | Running ue
ueransim     | UERANSIM v3.2.5
ueransim     | [2023-04-24 06:28:50.613] [nas] [info] UE switches to state [MM-DEREGISTERED/PLMN-SEARCH]
ueransim     | [2023-04-24 06:28:50.613] [rrc] [debug] UE[1] new signal detected
ueransim     | [2023-04-24 06:28:50.614] [rrc] [debug] New signal detected for cell[1], total [1] cells in coverage
ueransim     | [2023-04-24 06:28:50.614] [nas] [info] Selected plmn[208/95]
ueransim     | [2023-04-24 06:28:50.614] [rrc] [info] Selected cell plmn[208/95] tac[40960] category[SUITABLE]
ueransim     | [2023-04-24 06:28:50.615] [nas] [info] UE switches to state [MM-DEREGISTERED/PS]
ueransim     | [2023-04-24 06:28:50.615] [nas] [info] UE switches to state [MM-DEREGISTERED/NORMAL-SERVICE]
ueransim     | [2023-04-24 06:28:50.615] [nas] [debug] Initial registration required due to [MM-DEREG-NORMAL-SERVICE]
ueransim     | [2023-04-24 06:28:50.615] [nas] [debug] UAC access attempt is allowed for identity[0], category[MO_sig]
ueransim     | [2023-04-24 06:28:50.615] [nas] [debug] Sending Initial Registration
ueransim     | [2023-04-24 06:28:50.615] [nas] [info] UE switches to state [MM-REGISTER-INITIATED]
ueransim     | [2023-04-24 06:28:50.615] [rrc] [debug] Sending RRC Setup Request
ueransim     | [2023-04-24 06:28:50.616] [rrc] [info] RRC Setup for UE[1]
ueransim     | [2023-04-24 06:28:50.617] [rrc] [info] RRC connection established
ueransim     | [2023-04-24 06:28:50.617] [rrc] [info] UE switches to state [RRC-CONNECTED]
ueransim     | [2023-04-24 06:28:50.617] [nas] [info] UE switches to state [CM-CONNECTED]
ueransim     | [2023-04-24 06:28:50.617] [ngap] [debug] Initial NAS message received from UE[1]
ueransim     | [2023-04-24 06:28:50.638] [nas] [debug] Authentication Request received
ueransim     | [2023-04-24 06:28:50.655] [nas] [debug] Security Mode Command received
ueransim     | [2023-04-24 06:28:50.655] [nas] [debug] Selected integrity[1] ciphering[1]
ueransim     | [2023-04-24 06:28:50.663] [ngap] [debug] Initial Context Setup Request received
ueransim     | [2023-04-24 06:28:50.664] [nas] [debug] Registration accept received
ueransim     | [2023-04-24 06:28:50.664] [nas] [info] UE switches to state [MM-REGISTERED/NORMAL-SERVICE]
ueransim     | [2023-04-24 06:28:50.664] [nas] [debug] Sending Registration Complete
ueransim     | [2023-04-24 06:28:50.665] [nas] [info] Initial Registration is successful
ueransim     | [2023-04-24 06:28:50.665] [nas] [debug] Sending PDU Session Establishment Request
ueransim     | [2023-04-24 06:28:50.665] [nas] [debug] UAC access attempt is allowed for identity[0], category[MO_sig]
ueransim     | [2023-04-24 06:28:50.905] [ngap] [info] PDU session resource(s) setup for UE[1] count[1]
ueransim     | [2023-04-24 06:28:50.906] [nas] [debug] PDU Session Establishment Accept received
ueransim     | [2023-04-24 06:28:50.906] [nas] [info] PDU Session establishment is successful PSI[1]
ueransim     | [2023-04-24 06:28:50.923] [app] [info] Connection setup for PDU session[1] is successful, TUN interface[uesimtun0, 12.1.1.4] is up.


基于此,我们后见两个ueransim,分别问ueransim1与ueransim2:
配置文件如下:

version: '3.8'
services:
    ueransim1:
        container_name: ueransim1
        image: ueransim:latest
        privileged: true
        environment:
            # GNB Congig Parameters
            - MCC=208
            - MNC=95
            - NCI=0x000000010
            - TAC=0xa000
            - LINK_IP=127.0.0.1
            - NGAP_IP=192.168.12.33
            - GTP_IP=192.168.12.33
            - NGAP_PEER_IP=192.168.70.132
            - SST=222
            - SD=123
            - IGNORE_STREAM_IDS=true
            # UE Config Parameters
            - NUMBER_OF_UE=1
            - IMSI=208950000000031
            - KEY=0C0A34601D4F07677303652C0462535B
            - OP=63bfa50ee6523365ff14c1f45f88737d
            - OP_TYPE=OPC
            - AMF_VALUE=8000
            - IMEI=356938035643803
            - IMEI_SV=0035609204079514
            - GNB_IP_ADDRESS=127.0.0.1
            - PDU_TYPE=IPv4
            - APN=default
            - SST_0=222
            - SD_0=123
            - SST_C=222
            - SD_C=123
            - SST_D=222
            - SD_D=123
        network_mode: "host"
           
        healthcheck:
            test: /bin/bash -c "ifconfig uesimtun0"
            interval: 10s
            timeout: 5s
            retries: 5
#networks:
#    hostnet:
#     external: true
#    name: host
version: '3.8'
services:
    ueransim2:
        container_name: ueransim2
        image: ueransim:latest
        privileged: true
        environment:
            # GNB Congig Parameters
            - MCC=208
            - MNC=95
            - NCI=0x000000010
            - TAC=0xa000
            - LINK_IP=127.0.0.1
            - NGAP_IP=192.168.12.33
            - GTP_IP=192.168.12.33
            - NGAP_PEER_IP=192.168.70.132
            - SST=222
            - SD=123
            - IGNORE_STREAM_IDS=true
            # UE Config Parameters
            - NUMBER_OF_UE=1
            - IMSI=208950000000041
            - KEY=0C0A34601D4F07677303652C0462535B
            - OP=63bfa50ee6523365ff14c1f45f88737d
            - OP_TYPE=OPC
            - AMF_VALUE=8000
            - IMEI=356938035643803
            - IMEI_SV=0035609204079514
            - GNB_IP_ADDRESS=127.0.0.1
            - PDU_TYPE=IPv4
            - APN=default
            - SST_0=222
            - SD_0=123
            - SST_C=222
            - SD_C=123
            - SST_D=222
            - SD_D=123
        network_mode: "host"
           
        healthcheck:
            test: /bin/bash -c "ifconfig uesimtun0"
            interval: 10s
            timeout: 5s
            retries: 5
#networks:
#    hostnet:
#     external: true
#    name: host

启动第一个正常启动,启动第二个后失败,日志如下:

lab@lab-virtual-machine:~/oai-cn5g-fed/docker-compose/ueransim yz$ docker-compose -f docker-compose-ueransim-vpp2.yaml up
WARNING: Found orphan containers (4aab7c15a706_ueransim, ueransim) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.
Creating ueransim2 ... done
Attaching to ueransim2
ueransim2    | Now setting these variables '@GTP_IP@ @IGNORE_STREAM_IDS@ @LINK_IP@ @MCC@ @MNC@ @NCI@ @NGAP_IP@ @NGAP_PEER_IP@ @SD_0@ @SD_1@ @SD_2@ @SST_0@ @SST_1@ @SST_2@ @TAC@'
ueransim2    | Now setting these variables '@AMF_VALUE@ @APN@ @GNB_IP_ADDRESS@ @IMEI@ @IMEI_SV@ @IMSI@ @KEY@ @MCC@ @MNC@ @OP@ @OP_TYPE@ @PDU_TYPE@ @SD_C@ @SD_D@ @SD_R@ @SST_C@ @SST_D@ @SST_R@'
ueransim2    | Done setting the configuration
ueransim2    | ### Running ueransim ###
ueransim2    | Running gnb
ueransim2    | UERANSIM v3.2.5
ueransim2    | [2023-04-24 06:33:23.593] [rls-udp] [error] RLS failure [Socket bind failed: Address already in use]
ueransim2    | [2023-04-24 06:33:23.593] [sctp] [info] Trying to establish SCTP connection... (192.168.70.132:38412)
ueransim2    | [2023-04-24 06:33:23.593] [gtp] [error] GTP/UDP task could not be created. Socket bind failed: Address already in use
ueransim2    | [2023-04-24 06:33:23.596] [sctp] [info] SCTP connection established (192.168.70.132:38412)
ueransim2    | [2023-04-24 06:33:23.596] [sctp] [debug] SCTP association setup ascId[7]
ueransim2    | [2023-04-24 06:33:23.596] [ngap] [debug] Sending NG Setup Request
ueransim2    | [2023-04-24 06:33:23.601] [ngap] [debug] NG Setup Response received
ueransim2    | [2023-04-24 06:33:23.601] [ngap] [info] NG Setup procedure is successful
ueransim2    | Running ue
ueransim2    | UERANSIM v3.2.5
ueransim2    | [2023-04-24 06:33:24.592] [nas] [info] UE switches to state [MM-DEREGISTERED/PLMN-SEARCH]
ueransim2    | [2023-04-24 06:33:24.593] [rrc] [debug] New signal detected for cell[1], total [1] cells in coverage
ueransim2    | [2023-04-24 06:33:24.594] [nas] [info] Selected plmn[208/95]
ueransim2    | [2023-04-24 06:33:24.594] [rrc] [info] Selected cell plmn[208/95] tac[40960] category[SUITABLE]
ueransim2    | [2023-04-24 06:33:24.594] [nas] [info] UE switches to state [MM-DEREGISTERED/PS]
ueransim2    | [2023-04-24 06:33:24.594] [nas] [info] UE switches to state [MM-DEREGISTERED/NORMAL-SERVICE]
ueransim2    | [2023-04-24 06:33:24.594] [nas] [debug] Initial registration required due to [MM-DEREG-NORMAL-SERVICE]
ueransim2    | [2023-04-24 06:33:24.594] [nas] [debug] UAC access attempt is allowed for identity[0], category[MO_sig]
ueransim2    | [2023-04-24 06:33:24.594] [nas] [debug] Sending Initial Registration
ueransim2    | [2023-04-24 06:33:24.595] [nas] [info] UE switches to state [MM-REGISTER-INITIATED]
ueransim2    | [2023-04-24 06:33:24.595] [rrc] [debug] Sending RRC Setup Request
ueransim2    | [2023-04-24 06:33:24.596] [rrc] [info] RRC connection established
ueransim2    | [2023-04-24 06:33:24.596] [rrc] [info] UE switches to state [RRC-CONNECTED]
ueransim2    | [2023-04-24 06:33:24.596] [nas] [info] UE switches to state [CM-CONNECTED]
ueransim2    | [2023-04-24 06:33:24.618] [nas] [debug] Authentication Request received
ueransim2    | [2023-04-24 06:33:24.637] [nas] [debug] Security Mode Command received
ueransim2    | [2023-04-24 06:33:24.637] [nas] [debug] Selected integrity[1] ciphering[1]
ueransim2    | [2023-04-24 06:33:24.647] [nas] [debug] Registration accept received
ueransim2    | [2023-04-24 06:33:24.647] [nas] [info] UE switches to state [MM-REGISTERED/NORMAL-SERVICE]
ueransim2    | [2023-04-24 06:33:24.647] [nas] [debug] Sending Registration Complete
ueransim2    | [2023-04-24 06:33:24.647] [nas] [info] Initial Registration is successful
ueransim2    | [2023-04-24 06:33:24.647] [nas] [debug] Sending PDU Session Establishment Request
ueransim2    | [2023-04-24 06:33:24.647] [nas] [debug] UAC access attempt is allowed for identity[0], category[MO_sig]
ueransim2    | [2023-04-24 06:33:24.887] [nas] [debug] PDU Session Establishment Accept received
ueransim2    | [2023-04-24 06:33:24.887] [nas] [info] PDU Session establishment is successful PSI[1]
ueransim2    | RTNETLINK answers: File exists
ueransim2    | [2023-04-24 06:33:24.907] [app] [error] TUN configuration failure [Command execution failed. The command was: ip route add default dev uesimtun1 table rt_uesimtun1. The output is: 'Command execution failure]

不过查看网络发现还是生成了两个接口:
在这里插入图片描述
容器查看也有两个健康的容器:
在这里插入图片描述
在核心网侧amf查看接入了两个用户:
在这里插入图片描述

测试这两个容器:
uerasnim1测试结果

lab@lab-virtual-machine:~/oai-cn5g-fed/docker-compose/ueransim yz$ docker exec -it ueransim /bin/bash
WARNING: Error loading config file: /home/lab/.docker/config.json: open /home/lab/.docker/config.json: permission denied
root@lab-virtual-machine:/ueransim/bin# ls
entrypoint.sh  libdevbnd.so  nr-binder  nr-cli  nr-gnb  nr-ue
root@lab-virtual-machine:/ueransim/bin# ping -I 12.1.1.4 192.168.70.1
PING 192.168.70.1 (192.168.70.1) from 12.1.1.4 : 56(84) bytes of data.
64 bytes from 192.168.70.1: icmp_seq=1 ttl=62 time=5.95 ms
^C
--- 192.168.70.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 5.958/5.958/5.958/0.000 ms
root@lab-virtual-machine:/ueransim/bin# ping -I 12.1.1.4 192.168.70.132
PING 192.168.70.132 (192.168.70.132) from 12.1.1.4 : 56(84) bytes of data.
^C
--- 192.168.70.132 ping statistics ---
4 packets transmitted, 0 received, 100% packet loss, time 3058ms

root@lab-virtual-machine:/ueransim/bin# ping -I 12.1.1.4 192.168.70.132
PING 192.168.70.132 (192.168.70.132) from 12.1.1.4 : 56(84) bytes of data.
^C
--- 192.168.70.132 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 2041ms

root@lab-virtual-machine:/ueransim/bin# chmod 777 nr-binder
root@lab-virtual-machine:/ueransim/bin# ls
TCPclient.py  entrypoint.sh  libdevbnd.so  nr-binder  nr-cli  nr-gnb  nr-ue
root@lab-virtual-machine:/ueransim/bin# ./nr-binder 12.1.1.4 python3 TCPclient.py
./nr-binder: line 20: python3: command not found
root@lab-virtual-machine:/ueransim/bin# apt-get update
Err:1 http://security.ubuntu.com/ubuntu bionic-security InRelease        
  Temporary failure resolving 'security.ubuntu.com'
Err:2 http://archive.ubuntu.com/ubuntu bionic InRelease                  
  Temporary failure resolving 'archive.ubuntu.com'
Err:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease
  Temporary failure resolving 'archive.ubuntu.com'
0% [Connecting to archive.ubuntu.com]^C
root@lab-virtual-machine:/ueransim/bin# apt-get install python3
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following additional packages will be installed:
  file libexpat1 libmagic-mgc libmagic1 libmpdec2 libpython3-stdlib libpython3.6-minimal libpython3.6-stdlib libreadline7 libsqlite3-0 libssl1.1
  mime-support python3-minimal python3.6 python3.6-minimal readline-common xz-utils
Suggested packages:
  python3-doc python3-tk python3-venv python3.6-venv python3.6-doc binutils binfmt-support readline-doc
The following NEW packages will be installed:
  file libexpat1 libmagic-mgc libmagic1 libmpdec2 libpython3-stdlib libpython3.6-minimal libpython3.6-stdlib libreadline7 libsqlite3-0 libssl1.1
  mime-support python3 python3-minimal python3.6 python3.6-minimal readline-common xz-utils
0 upgraded, 18 newly installed, 0 to remove and 0 not upgraded.
Need to get 6668 kB of archives.
After this operation, 34.1 MB of additional disk space will be used.
Do you want to continue? [Y/n] y
Ign:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libssl1.1 amd64 1.1.1-1ubuntu2.1~18.04.17
Ign:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libpython3.6-minimal amd64 3.6.9-1~18.04ubuntu1.7
Err:1 http://security.ubuntu.com/ubuntu bionic-updates/main amd64 libssl1.1 amd64 1.1.1-1ubuntu2.1~18.04.17
  Temporary failure resolving 'archive.ubuntu.com'
Err:2 http://security.ubuntu.com/ubuntu bionic-updates/main amd64 libpython3.6-minimal amd64 3.6.9-1~18.04ubuntu1.7
  Temporary failure resolving 'archive.ubuntu.com'
Ign:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libexpat1 amd64 2.2.5-3ubuntu0.7        
Ign:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3.6-minimal amd64 3.6.9-1~18.04ubuntu1.7
Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-minimal amd64 3.6.7-1~18.04 [23.7 kB]
Err:3 http://security.ubuntu.com/ubuntu bionic-updates/main amd64 libexpat1 amd64 2.2.5-3ubuntu0.7
  404  Not Found [IP: 91.189.91.39 80]
Get:6 http://archive.ubuntu.com/ubuntu bionic/main amd64 mime-support all 3.60ubuntu1 [30.1 kB]
Err:4 http://security.ubuntu.com/ubuntu bionic-updates/main amd64 python3.6-minimal amd64 3.6.9-1~18.04ubuntu1.7
  404  Not Found [IP: 91.189.91.39 80]
Get:7 http://archive.ubuntu.com/ubuntu bionic/main amd64 libmpdec2 amd64 2.4.2-1ubuntu1 [84.1 kB]
Get:8 http://archive.ubuntu.com/ubuntu bionic/main amd64 readline-common all 7.0-3 [52.9 kB]
Get:9 http://archive.ubuntu.com/ubuntu bionic/main amd64 libreadline7 amd64 7.0-3 [124 kB]                                                             
Ign:10 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libsqlite3-0 amd64 3.22.0-1ubuntu0.5                                                 
Ign:11 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libpython3.6-stdlib amd64 3.6.9-1~18.04ubuntu1.7                                     
Ign:12 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3.6 amd64 3.6.9-1~18.04ubuntu1.7                                               
Get:13 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libpython3-stdlib amd64 3.6.7-1~18.04 [7240 B]                                       
Err:10 http://security.ubuntu.com/ubuntu bionic-updates/main amd64 libsqlite3-0 amd64 3.22.0-1ubuntu0.5                                                
  404  Not Found [IP: 91.189.91.39 80]
Get:14 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3 amd64 3.6.7-1~18.04 [47.2 kB]                                                
Err:11 http://security.ubuntu.com/ubuntu bionic-updates/main amd64 libpython3.6-stdlib amd64 3.6.9-1~18.04ubuntu1.7                                    
  404  Not Found [IP: 91.189.91.39 80]
Err:12 http://security.ubuntu.com/ubuntu bionic-updates/main amd64 python3.6 amd64 3.6.9-1~18.04ubuntu1.7                                              
  404  Not Found [IP: 91.189.91.39 80]
Get:15 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libmagic-mgc amd64 1:5.32-2ubuntu0.4 [184 kB]                                        
Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libmagic1 amd64 1:5.32-2ubuntu0.4 [68.6 kB]                                          
Get:17 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 file amd64 1:5.32-2ubuntu0.4 [22.1 kB]                                               
Get:18 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 xz-utils amd64 5.2.2-1.3ubuntu0.1 [83.8 kB]                                          
Fetched 727 kB in 1min 15s (9742 B/s)                                                                                                                  
E: Failed to fetch http://security.ubuntu.com/ubuntu/pool/main/o/openssl/libssl1.1_1.1.1-1ubuntu2.1~18.04.17_amd64.deb  Temporary failure resolving 'archive.ubuntu.com'
E: Failed to fetch http://security.ubuntu.com/ubuntu/pool/main/p/python3.6/libpython3.6-minimal_3.6.9-1~18.04ubuntu1.7_amd64.deb  Temporary failure resolving 'archive.ubuntu.com'
E: Failed to fetch http://security.ubuntu.com/ubuntu/pool/main/e/expat/libexpat1_2.2.5-3ubuntu0.7_amd64.deb  404  Not Found [IP: 91.189.91.39 80]
E: Failed to fetch http://security.ubuntu.com/ubuntu/pool/main/p/python3.6/python3.6-minimal_3.6.9-1~18.04ubuntu1.7_amd64.deb  404  Not Found [IP: 91.189.91.39 80]
E: Failed to fetch http://security.ubuntu.com/ubuntu/pool/main/s/sqlite3/libsqlite3-0_3.22.0-1ubuntu0.5_amd64.deb  404  Not Found [IP: 91.189.91.39 80]
E: Failed to fetch http://security.ubuntu.com/ubuntu/pool/main/p/python3.6/libpython3.6-stdlib_3.6.9-1~18.04ubuntu1.7_amd64.deb  404  Not Found [IP: 91.189.91.39 80]
E: Failed to fetch http://security.ubuntu.com/ubuntu/pool/main/p/python3.6/python3.6_3.6.9-1~18.04ubuntu1.7_amd64.deb  404  Not Found [IP: 91.189.91.39 80]
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
root@lab-virtual-machine:/ueransim/bin# apt-get update
Err:1 http://security.ubuntu.com/ubuntu bionic-security InRelease        
  Temporary failure resolving 'security.ubuntu.com'
Err:2 http://archive.ubuntu.com/ubuntu bionic InRelease                  
  Temporary failure resolving 'archive.ubuntu.com'
Err:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease
  Temporary failure resolving 'archive.ubuntu.com'
Get:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [83.3 kB]
Get:5 http://archive.ubuntu.com/ubuntu bionic-backports/main amd64 Packages [64.0 kB]
Get:6 http://archive.ubuntu.com/ubuntu bionic-backports/universe amd64 Packages [20.6 kB]
Fetched 168 kB in 1min 29s (1882 B/s)
Reading package lists... Done
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/bionic/InRelease  Temporary failure resolving 'archive.ubuntu.com'
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/bionic-updates/InRelease  Temporary failure resolving 'archive.ubuntu.com'
W: Failed to fetch http://security.ubuntu.com/ubuntu/dists/bionic-security/InRelease  Temporary failure resolving 'security.ubuntu.com'
W: Some index files failed to download. They have been ignored, or old ones used instead.
root@lab-virtual-machine:/ueransim/bin# apt-get install python3
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following additional packages will be installed:
  file libexpat1 libmagic-mgc libmagic1 libmpdec2 libpython3-stdlib libpython3.6-minimal libpython3.6-stdlib libreadline7 libsqlite3-0 libssl1.1
  mime-support python3-minimal python3.6 python3.6-minimal readline-common xz-utils
Suggested packages:
  python3-doc python3-tk python3-venv python3.6-venv python3.6-doc binutils binfmt-support readline-doc
The following NEW packages will be installed:
  file libexpat1 libmagic-mgc libmagic1 libmpdec2 libpython3-stdlib libpython3.6-minimal libpython3.6-stdlib libreadline7 libsqlite3-0 libssl1.1
  mime-support python3 python3-minimal python3.6 python3.6-minimal readline-common xz-utils
0 upgraded, 18 newly installed, 0 to remove and 0 not upgraded.
Need to get 6668 kB of archives.
After this operation, 34.1 MB of additional disk space will be used.
Do you want to continue? [Y/n] y
Ign:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libssl1.1 amd64 1.1.1-1ubuntu2.1~18.04.17
Ign:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libpython3.6-minimal amd64 3.6.9-1~18.04ubuntu1.7
Ign:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libexpat1 amd64 2.2.5-3ubuntu0.7
Ign:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3.6-minimal amd64 3.6.9-1~18.04ubuntu1.7
Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-minimal amd64 3.6.7-1~18.04 [23.7 kB]
Err:1 http://security.ubuntu.com/ubuntu bionic-updates/main amd64 libssl1.1 amd64 1.1.1-1ubuntu2.1~18.04.17
  404  Not Found [IP: 185.125.190.39 80]
Err:2 http://security.ubuntu.com/ubuntu bionic-updates/main amd64 libpython3.6-minimal amd64 3.6.9-1~18.04ubuntu1.7
  404  Not Found [IP: 185.125.190.39 80]
Err:3 http://security.ubuntu.com/ubuntu bionic-updates/main amd64 libexpat1 amd64 2.2.5-3ubuntu0.7
  404  Not Found [IP: 185.125.190.39 80]
Err:4 http://security.ubuntu.com/ubuntu bionic-updates/main amd64 python3.6-minimal amd64 3.6.9-1~18.04ubuntu1.7
  404  Not Found [IP: 185.125.190.39 80]
Get:6 http://archive.ubuntu.com/ubuntu bionic/main amd64 mime-support all 3.60ubuntu1 [30.1 kB]
Get:7 http://archive.ubuntu.com/ubuntu bionic/main amd64 libmpdec2 amd64 2.4.2-1ubuntu1 [84.1 kB]
Get:8 http://archive.ubuntu.com/ubuntu bionic/main amd64 readline-common all 7.0-3 [52.9 kB]
Get:9 http://archive.ubuntu.com/ubuntu bionic/main amd64 libreadline7 amd64 7.0-3 [124 kB]
Ign:10 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libsqlite3-0 amd64 3.22.0-1ubuntu0.5
Ign:11 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libpython3.6-stdlib amd64 3.6.9-1~18.04ubuntu1.7
Err:10 http://security.ubuntu.com/ubuntu bionic-updates/main amd64 libsqlite3-0 amd64 3.22.0-1ubuntu0.5
  404  Not Found [IP: 185.125.190.39 80]
Ign:12 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3.6 amd64 3.6.9-1~18.04ubuntu1.7
Err:11 http://security.ubuntu.com/ubuntu bionic-updates/main amd64 libpython3.6-stdlib amd64 3.6.9-1~18.04ubuntu1.7
  404  Not Found [IP: 185.125.190.39 80]
Err:12 http://security.ubuntu.com/ubuntu bionic-updates/main amd64 python3.6 amd64 3.6.9-1~18.04ubuntu1.7
  404  Not Found [IP: 185.125.190.36 80]
Get:13 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libpython3-stdlib amd64 3.6.7-1~18.04 [7240 B]
Get:14 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3 amd64 3.6.7-1~18.04 [47.2 kB]
Get:15 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libmagic-mgc amd64 1:5.32-2ubuntu0.4 [184 kB]                                        
Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libmagic1 amd64 1:5.32-2ubuntu0.4 [68.6 kB]                                          
Get:17 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 file amd64 1:5.32-2ubuntu0.4 [22.1 kB]                                               
Get:18 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 xz-utils amd64 5.2.2-1.3ubuntu0.1 [83.8 kB]                                          
Fetched 727 kB in 9s (80.6 kB/s)                                                                                                                       
E: Failed to fetch http://security.ubuntu.com/ubuntu/pool/main/o/openssl/libssl1.1_1.1.1-1ubuntu2.1~18.04.17_amd64.deb  404  Not Found [IP: 185.125.190.39 80]
E: Failed to fetch http://security.ubuntu.com/ubuntu/pool/main/p/python3.6/libpython3.6-minimal_3.6.9-1~18.04ubuntu1.7_amd64.deb  404  Not Found [IP: 185.125.190.39 80]
E: Failed to fetch http://security.ubuntu.com/ubuntu/pool/main/e/expat/libexpat1_2.2.5-3ubuntu0.7_amd64.deb  404  Not Found [IP: 185.125.190.39 80]
E: Failed to fetch http://security.ubuntu.com/ubuntu/pool/main/p/python3.6/python3.6-minimal_3.6.9-1~18.04ubuntu1.7_amd64.deb  404  Not Found [IP: 185.125.190.39 80]
E: Failed to fetch http://security.ubuntu.com/ubuntu/pool/main/s/sqlite3/libsqlite3-0_3.22.0-1ubuntu0.5_amd64.deb  404  Not Found [IP: 185.125.190.39 80]
E: Failed to fetch http://security.ubuntu.com/ubuntu/pool/main/p/python3.6/libpython3.6-stdlib_3.6.9-1~18.04ubuntu1.7_amd64.deb  404  Not Found [IP: 185.125.190.39 80]
E: Failed to fetch http://security.ubuntu.com/ubuntu/pool/main/p/python3.6/python3.6_3.6.9-1~18.04ubuntu1.7_amd64.deb  404  Not Found [IP: 185.125.190.36 80]
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
root@lab-virtual-machine:/ueransim/bin# apt-get update
Get:1 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB]
Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease                               
Get:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB]             
Get:4 http://security.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [1546 kB]
Hit:5 http://archive.ubuntu.com/ubuntu bionic-backports InRelease                  
Get:6 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [3270 kB]
Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [3691 kB]
Get:8 http://security.ubuntu.com/ubuntu bionic-security/multiverse amd64 Packages [23.8 kB]
Get:9 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [1618 kB]
Get:10 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [2392 kB]  
Get:11 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [30.8 kB]                                                             
Get:12 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [1587 kB]                                                             
Fetched 14.3 MB in 7s (2083 kB/s)                                                                                                                      
Reading package lists... Done
root@lab-virtual-machine:/ueransim/bin# apt-get install python3
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following additional packages will be installed:
  file libexpat1 libmagic-mgc libmagic1 libmpdec2 libpython3-stdlib libpython3.6-minimal libpython3.6-stdlib libreadline7 libsqlite3-0 libssl1.1
  mime-support python3-minimal python3.6 python3.6-minimal readline-common xz-utils
Suggested packages:
  python3-doc python3-tk python3-venv python3.6-venv python3.6-doc binutils binfmt-support readline-doc
The following NEW packages will be installed:
  file libexpat1 libmagic-mgc libmagic1 libmpdec2 libpython3-stdlib libpython3.6-minimal libpython3.6-stdlib libreadline7 libsqlite3-0 libssl1.1
  mime-support python3 python3-minimal python3.6 python3.6-minimal readline-common xz-utils
0 upgraded, 18 newly installed, 0 to remove and 21 not upgraded.
Need to get 6671 kB of archives.
After this operation, 34.1 MB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libssl1.1 amd64 1.1.1-1ubuntu2.1~18.04.21 [1304 kB]
Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libpython3.6-minimal amd64 3.6.9-1~18.04ubuntu1.12 [533 kB]
Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libexpat1 amd64 2.2.5-3ubuntu0.9 [82.8 kB]
Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3.6-minimal amd64 3.6.9-1~18.04ubuntu1.12 [1609 kB]
Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-minimal amd64 3.6.7-1~18.04 [23.7 kB]
Get:6 http://archive.ubuntu.com/ubuntu bionic/main amd64 mime-support all 3.60ubuntu1 [30.1 kB]
Get:7 http://archive.ubuntu.com/ubuntu bionic/main amd64 libmpdec2 amd64 2.4.2-1ubuntu1 [84.1 kB]
Get:8 http://archive.ubuntu.com/ubuntu bionic/main amd64 readline-common all 7.0-3 [52.9 kB]
Get:9 http://archive.ubuntu.com/ubuntu bionic/main amd64 libreadline7 amd64 7.0-3 [124 kB]
Get:10 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libsqlite3-0 amd64 3.22.0-1ubuntu0.7 [499 kB]
Get:11 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libpython3.6-stdlib amd64 3.6.9-1~18.04ubuntu1.12 [1713 kB]
Get:12 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3.6 amd64 3.6.9-1~18.04ubuntu1.12 [203 kB]
Get:13 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libpython3-stdlib amd64 3.6.7-1~18.04 [7240 B]                                       
Get:14 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3 amd64 3.6.7-1~18.04 [47.2 kB]                                                
Get:15 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libmagic-mgc amd64 1:5.32-2ubuntu0.4 [184 kB]                                        
Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libmagic1 amd64 1:5.32-2ubuntu0.4 [68.6 kB]                                          
Get:17 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 file amd64 1:5.32-2ubuntu0.4 [22.1 kB]                                               
Get:18 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 xz-utils amd64 5.2.2-1.3ubuntu0.1 [83.8 kB]                                          
Fetched 6671 kB in 9s (775 kB/s)                                                                                                                       
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package libssl1.1:amd64.
(Reading database ... 8226 files and directories currently installed.)
Preparing to unpack .../libssl1.1_1.1.1-1ubuntu2.1~18.04.21_amd64.deb ...
Unpacking libssl1.1:amd64 (1.1.1-1ubuntu2.1~18.04.21) ...
Selecting previously unselected package libpython3.6-minimal:amd64.
Preparing to unpack .../libpython3.6-minimal_3.6.9-1~18.04ubuntu1.12_amd64.deb ...
Unpacking libpython3.6-minimal:amd64 (3.6.9-1~18.04ubuntu1.12) ...
Selecting previously unselected package libexpat1:amd64.
Preparing to unpack .../libexpat1_2.2.5-3ubuntu0.9_amd64.deb ...
Unpacking libexpat1:amd64 (2.2.5-3ubuntu0.9) ...
Selecting previously unselected package python3.6-minimal.
Preparing to unpack .../python3.6-minimal_3.6.9-1~18.04ubuntu1.12_amd64.deb ...
Unpacking python3.6-minimal (3.6.9-1~18.04ubuntu1.12) ...
Setting up libssl1.1:amd64 (1.1.1-1ubuntu2.1~18.04.21) ...
debconf: unable to initialize frontend: Dialog
debconf: (No usable dialog-like program is installed, so the dialog based frontend cannot be used. at /usr/share/perl5/Debconf/FrontEnd/Dialog.pm line 76.)
debconf: falling back to frontend: Readline
debconf: unable to initialize frontend: Readline
debconf: (Can't locate Term/ReadLine.pm in @INC (you may need to install the Term::ReadLine module) (@INC contains: /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.26.1 /usr/local/share/perl/5.26.1 /usr/lib/x86_64-linux-gnu/perl5/5.26 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.26 /usr/share/perl/5.26 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base) at /usr/share/perl5/Debconf/FrontEnd/Readline.pm line 7.)
debconf: falling back to frontend: Teletype
Setting up libpython3.6-minimal:amd64 (3.6.9-1~18.04ubuntu1.12) ...
Setting up libexpat1:amd64 (2.2.5-3ubuntu0.9) ...
Setting up python3.6-minimal (3.6.9-1~18.04ubuntu1.12) ...
Selecting previously unselected package python3-minimal.
(Reading database ... 8483 files and directories currently installed.)
Preparing to unpack .../0-python3-minimal_3.6.7-1~18.04_amd64.deb ...
Unpacking python3-minimal (3.6.7-1~18.04) ...
Selecting previously unselected package mime-support.
Preparing to unpack .../1-mime-support_3.60ubuntu1_all.deb ...
Unpacking mime-support (3.60ubuntu1) ...
Selecting previously unselected package libmpdec2:amd64.
Preparing to unpack .../2-libmpdec2_2.4.2-1ubuntu1_amd64.deb ...
Unpacking libmpdec2:amd64 (2.4.2-1ubuntu1) ...
Selecting previously unselected package readline-common.
Preparing to unpack .../3-readline-common_7.0-3_all.deb ...
Unpacking readline-common (7.0-3) ...
Selecting previously unselected package libreadline7:amd64.
Preparing to unpack .../4-libreadline7_7.0-3_amd64.deb ...
Unpacking libreadline7:amd64 (7.0-3) ...
Selecting previously unselected package libsqlite3-0:amd64.
Preparing to unpack .../5-libsqlite3-0_3.22.0-1ubuntu0.7_amd64.deb ...
Unpacking libsqlite3-0:amd64 (3.22.0-1ubuntu0.7) ...
Selecting previously unselected package libpython3.6-stdlib:amd64.
Preparing to unpack .../6-libpython3.6-stdlib_3.6.9-1~18.04ubuntu1.12_amd64.deb ...
Unpacking libpython3.6-stdlib:amd64 (3.6.9-1~18.04ubuntu1.12) ...
Selecting previously unselected package python3.6.
Preparing to unpack .../7-python3.6_3.6.9-1~18.04ubuntu1.12_amd64.deb ...
Unpacking python3.6 (3.6.9-1~18.04ubuntu1.12) ...
Selecting previously unselected package libpython3-stdlib:amd64.
Preparing to unpack .../8-libpython3-stdlib_3.6.7-1~18.04_amd64.deb ...
Unpacking libpython3-stdlib:amd64 (3.6.7-1~18.04) ...
Setting up python3-minimal (3.6.7-1~18.04) ...
Selecting previously unselected package python3.
(Reading database ... 8941 files and directories currently installed.)
Preparing to unpack .../python3_3.6.7-1~18.04_amd64.deb ...
Unpacking python3 (3.6.7-1~18.04) ...
Selecting previously unselected package libmagic-mgc.
Preparing to unpack .../libmagic-mgc_1%3a5.32-2ubuntu0.4_amd64.deb ...
Unpacking libmagic-mgc (1:5.32-2ubuntu0.4) ...
Selecting previously unselected package libmagic1:amd64.
Preparing to unpack .../libmagic1_1%3a5.32-2ubuntu0.4_amd64.deb ...
Unpacking libmagic1:amd64 (1:5.32-2ubuntu0.4) ...
Selecting previously unselected package file.
Preparing to unpack .../file_1%3a5.32-2ubuntu0.4_amd64.deb ...
Unpacking file (1:5.32-2ubuntu0.4) ...
Selecting previously unselected package xz-utils.
Preparing to unpack .../xz-utils_5.2.2-1.3ubuntu0.1_amd64.deb ...
Unpacking xz-utils (5.2.2-1.3ubuntu0.1) ...
Setting up readline-common (7.0-3) ...
Setting up mime-support (3.60ubuntu1) ...
Setting up libreadline7:amd64 (7.0-3) ...
Setting up libmagic-mgc (1:5.32-2ubuntu0.4) ...
Setting up libmagic1:amd64 (1:5.32-2ubuntu0.4) ...
Setting up xz-utils (5.2.2-1.3ubuntu0.1) ...
update-alternatives: using /usr/bin/xz to provide /usr/bin/lzma (lzma) in auto mode
update-alternatives: warning: skip creation of /usr/share/man/man1/lzma.1.gz because associated file /usr/share/man/man1/xz.1.gz (of link group lzma) doesn't exist
update-alternatives: warning: skip creation of /usr/share/man/man1/unlzma.1.gz because associated file /usr/share/man/man1/unxz.1.gz (of link group lzma) doesn't exist
update-alternatives: warning: skip creation of /usr/share/man/man1/lzcat.1.gz because associated file /usr/share/man/man1/xzcat.1.gz (of link group lzma) doesn't exist
update-alternatives: warning: skip creation of /usr/share/man/man1/lzmore.1.gz because associated file /usr/share/man/man1/xzmore.1.gz (of link group lzma) doesn't exist
update-alternatives: warning: skip creation of /usr/share/man/man1/lzless.1.gz because associated file /usr/share/man/man1/xzless.1.gz (of link group lzma) doesn't exist
update-alternatives: warning: skip creation of /usr/share/man/man1/lzdiff.1.gz because associated file /usr/share/man/man1/xzdiff.1.gz (of link group lzma) doesn't exist
update-alternatives: warning: skip creation of /usr/share/man/man1/lzcmp.1.gz because associated file /usr/share/man/man1/xzcmp.1.gz (of link group lzma) doesn't exist
update-alternatives: warning: skip creation of /usr/share/man/man1/lzgrep.1.gz because associated file /usr/share/man/man1/xzgrep.1.gz (of link group lzma) doesn't exist
update-alternatives: warning: skip creation of /usr/share/man/man1/lzegrep.1.gz because associated file /usr/share/man/man1/xzegrep.1.gz (of link group lzma) doesn't exist
update-alternatives: warning: skip creation of /usr/share/man/man1/lzfgrep.1.gz because associated file /usr/share/man/man1/xzfgrep.1.gz (of link group lzma) doesn't exist
Setting up libsqlite3-0:amd64 (3.22.0-1ubuntu0.7) ...
Setting up libmpdec2:amd64 (2.4.2-1ubuntu1) ...
Setting up libpython3.6-stdlib:amd64 (3.6.9-1~18.04ubuntu1.12) ...
Setting up python3.6 (3.6.9-1~18.04ubuntu1.12) ...
Setting up file (1:5.32-2ubuntu0.4) ...
Setting up libpython3-stdlib:amd64 (3.6.7-1~18.04) ...
Setting up python3 (3.6.7-1~18.04) ...
running python rtupdate hooks for python3.6...
running python post-rtupdate hooks for python3.6...
Processing triggers for libc-bin (2.27-3ubuntu1.6) ...
root@lab-virtual-machine:/ueransim/bin# ./nr-binder 12.1.1.2 python3 TCPclient.py
Waitting for connecting...
Connected!
Sending message...
^CTraceback (most recent call last):
  File "TCPclient.py", line 14, in <module>
    time.sleep(5)
KeyboardInterrupt
root@lab-virtual-machine:/ueransim/bin# ./nr-binder 12.1.1.5 python3 TCPclient.py
Waitting for connecting...
^CTraceback (most recent call last):
  File "TCPclient.py", line 7, in <module>
    tcp_client_socket.connect(("192.168.70.1", 8088))
KeyboardInterrupt
root@lab-virtual-machine:/ueransim/bin# ./nr-binder 12.1.1.3 python3 TCPclient.py
Waitting for connecting...
Connected!
Sending message...
^CTraceback (most recent call last):
  File "TCPclient.py", line 14, in <module>
    time.sleep(5)
KeyboardInterrupt

ueransim2测试结果:

lab@lab-virtual-machine:~/oai-cn5g-fed/docker-compose/ueransim yz$ docker exec -it ueransim2 /bin/bash
WARNING: Error loading config file: /home/lab/.docker/config.json: open /home/lab/.docker/config.json: permission denied
root@lab-virtual-machine:/ueransim/bin# ls
entrypoint.sh  libdevbnd.so  nr-binder  nr-cli  nr-gnb  nr-ue
root@lab-virtual-machine:/ueransim/bin# ping -I 12.1.1.4 192.168.70.1
PING 192.168.70.1 (192.168.70.1) from 12.1.1.4 : 56(84) bytes of data.
64 bytes from 192.168.70.1: icmp_seq=1 ttl=62 time=4.23 ms
^C
--- 192.168.70.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 4.234/4.234/4.234/0.000 ms
root@lab-virtual-machine:/ueransim/bin# ping -I 12.1.1.4 192.168.70.132
PING 192.168.70.132 (192.168.70.132) from 12.1.1.4 : 56(84) bytes of data.
^C
--- 192.168.70.132 ping statistics ---
1 packets transmitted, 0 received, 100% packet loss, time 0ms

root@lab-virtual-machine:/ueransim/bin# chmod 777 nr-binder
root@lab-virtual-machine:/ueransim/bin# ifconfig
docker0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 172.17.0.1  netmask 255.255.0.0  broadcast 172.17.255.255
        ether 02:42:27:d3:5c:f4  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

ens33: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.2.106  netmask 255.255.255.0  broadcast 192.168.2.255
        inet6 fe80::f5f:7af1:a9b:2e90  prefixlen 64  scopeid 0x20<link>
        ether 00:0c:29:a6:b4:4b  txqueuelen 1000  (Ethernet)
        RX packets 70263  bytes 68285454 (68.2 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 24343  bytes 2168035 (2.1 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

ens39: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.14.33  netmask 255.255.255.0  broadcast 192.168.14.255
        inet6 fe80::20c:29ff:fea6:b47d  prefixlen 64  scopeid 0x20<link>
        ether 00:0c:29:a6:b4:7d  txqueuelen 1000  (Ethernet)
        RX packets 7798  bytes 688116 (688.1 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 1667  bytes 113211 (113.2 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

ens40: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.12.33  netmask 255.255.255.0  broadcast 192.168.12.255
        inet6 fe80::20c:29ff:fea6:b469  prefixlen 64  scopeid 0x20<link>
        ether 00:0c:29:a6:b4:69  txqueuelen 1000  (Ethernet)
        RX packets 96233  bytes 6076638 (6.0 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 5113  bytes 368788 (368.7 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

ens41: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.13.33  netmask 255.255.255.0  broadcast 192.168.13.255
        inet6 fe80::20c:29ff:fea6:b473  prefixlen 64  scopeid 0x20<link>
        ether 00:0c:29:a6:b4:73  txqueuelen 1000  (Ethernet)
        RX packets 1480  bytes 310927 (310.9 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 1670  bytes 113705 (113.7 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 11648  bytes 811979 (811.9 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 11648  bytes 811979 (811.9 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

uesimtun0: flags=369<UP,POINTOPOINT,NOTRAILERS,RUNNING,PROMISC>  mtu 1400
        inet 12.1.1.4  netmask 255.255.255.255  destination 12.1.1.4
        inet6 fe80::74a3:2948:10c9:3a71  prefixlen 64  scopeid 0x20<link>
        unspec 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00  txqueuelen 500  (UNSPEC)
        RX packets 2  bytes 168 (168.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 22  bytes 1528 (1.5 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

uesimtun1: flags=369<UP,POINTOPOINT,NOTRAILERS,RUNNING,PROMISC>  mtu 1400
        inet 12.1.1.5  netmask 255.255.255.255  destination 12.1.1.5
        inet6 fe80::c9fd:7b73:83a2:a1b5  prefixlen 64  scopeid 0x20<link>
        unspec 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00  txqueuelen 500  (UNSPEC)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

root@lab-virtual-machine:/ueransim/bin# apt-get update
Err:1 http://archive.ubuntu.com/ubuntu bionic InRelease                  
  Temporary failure resolving 'archive.ubuntu.com'
Err:2 http://security.ubuntu.com/ubuntu bionic-security InRelease        
  Temporary failure resolving 'security.ubuntu.com'
Err:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease          
  Temporary failure resolving 'archive.ubuntu.com'
Err:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease
  Temporary failure resolving 'archive.ubuntu.com'
Reading package lists... Done        
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/bionic/InRelease  Temporary failure resolving 'archive.ubuntu.com'
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/bionic-updates/InRelease  Temporary failure resolving 'archive.ubuntu.com'
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/bionic-backports/InRelease  Temporary failure resolving 'archive.ubuntu.com'
W: Failed to fetch http://security.ubuntu.com/ubuntu/dists/bionic-security/InRelease  Temporary failure resolving 'security.ubuntu.com'
W: Some index files failed to download. They have been ignored, or old ones used instead.
root@lab-virtual-machine:/ueransim/bin# apt-get update
Hit:1 http://archive.ubuntu.com/ubuntu bionic InRelease                                 
Get:2 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB]               
Get:3 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [83.3 kB]             
Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [1587 kB]                                                              
Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [2392 kB]                                                                
Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [3691 kB]                                                                    
Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [30.8 kB]                                                              
Get:8 http://archive.ubuntu.com/ubuntu bionic-backports/main amd64 Packages [64.0 kB]                                                                  
Get:9 http://archive.ubuntu.com/ubuntu bionic-backports/universe amd64 Packages [20.6 kB]                                                              
Err:10 http://security.ubuntu.com/ubuntu bionic-security InRelease                                                                                     
  Temporary failure resolving 'security.ubuntu.com'
Fetched 7957 kB in 33s (241 kB/s)     
Reading package lists... Done
W: Failed to fetch http://security.ubuntu.com/ubuntu/dists/bionic-security/InRelease  Temporary failure resolving 'security.ubuntu.com'
W: Some index files failed to download. They have been ignored, or old ones used instead.
root@lab-virtual-machine:/ueransim/bin# apt-get install python3
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following additional packages will be installed:
  file libexpat1 libmagic-mgc libmagic1 libmpdec2 libpython3-stdlib libpython3.6-minimal libpython3.6-stdlib libreadline7 libsqlite3-0 libssl1.1
  mime-support python3-minimal python3.6 python3.6-minimal readline-common xz-utils
Suggested packages:
  python3-doc python3-tk python3-venv python3.6-venv python3.6-doc binutils binfmt-support readline-doc
The following NEW packages will be installed:
  file libexpat1 libmagic-mgc libmagic1 libmpdec2 libpython3-stdlib libpython3.6-minimal libpython3.6-stdlib libreadline7 libsqlite3-0 libssl1.1
  mime-support python3 python3-minimal python3.6 python3.6-minimal readline-common xz-utils
0 upgraded, 18 newly installed, 0 to remove and 21 not upgraded.
Need to get 6671 kB of archives.
After this operation, 34.1 MB of additional disk space will be used.
Do you want to continue? [Y/n] y
Err:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libssl1.1 amd64 1.1.1-1ubuntu2.1~18.04.21
  Temporary failure resolving 'archive.ubuntu.com'
Err:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libpython3.6-minimal amd64 3.6.9-1~18.04ubuntu1.12
  Temporary failure resolving 'archive.ubuntu.com'
Err:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libexpat1 amd64 2.2.5-3ubuntu0.9
  Temporary failure resolving 'archive.ubuntu.com'
Err:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3.6-minimal amd64 3.6.9-1~18.04ubuntu1.12
  Temporary failure resolving 'archive.ubuntu.com'
Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-minimal amd64 3.6.7-1~18.04 [23.7 kB]
Get:6 http://archive.ubuntu.com/ubuntu bionic/main amd64 mime-support all 3.60ubuntu1 [30.1 kB]
Get:7 http://archive.ubuntu.com/ubuntu bionic/main amd64 libmpdec2 amd64 2.4.2-1ubuntu1 [84.1 kB]
Get:8 http://archive.ubuntu.com/ubuntu bionic/main amd64 readline-common all 7.0-3 [52.9 kB]                                                           
Get:9 http://archive.ubuntu.com/ubuntu bionic/main amd64 libreadline7 amd64 7.0-3 [124 kB]                                                             
Get:10 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libsqlite3-0 amd64 3.22.0-1ubuntu0.7 [499 kB]                                        
Get:11 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libpython3.6-stdlib amd64 3.6.9-1~18.04ubuntu1.12 [1713 kB]                          
Get:12 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3.6 amd64 3.6.9-1~18.04ubuntu1.12 [203 kB]                                     
Get:13 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libpython3-stdlib amd64 3.6.7-1~18.04 [7240 B]                                       
Get:14 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3 amd64 3.6.7-1~18.04 [47.2 kB]                                                
Get:15 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libmagic-mgc amd64 1:5.32-2ubuntu0.4 [184 kB]                                        
Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libmagic1 amd64 1:5.32-2ubuntu0.4 [68.6 kB]                                          
Get:17 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 file amd64 1:5.32-2ubuntu0.4 [22.1 kB]                                               
Get:18 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 xz-utils amd64 5.2.2-1.3ubuntu0.1 [83.8 kB]                                          
Fetched 3142 kB in 2min 13s (23.6 kB/s)                                                                                                                
E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/o/openssl/libssl1.1_1.1.1-1ubuntu2.1~18.04.21_amd64.deb  Temporary failure resolving 'archive.ubuntu.com'
E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/p/python3.6/libpython3.6-minimal_3.6.9-1~18.04ubuntu1.12_amd64.deb  Temporary failure resolving 'archive.ubuntu.com'
E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/e/expat/libexpat1_2.2.5-3ubuntu0.9_amd64.deb  Temporary failure resolving 'archive.ubuntu.com'
E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/p/python3.6/python3.6-minimal_3.6.9-1~18.04ubuntu1.12_amd64.deb  Temporary failure resolving 'archive.ubuntu.com'
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
root@lab-virtual-machine:/ueransim/bin# ls
TCPclient.py  entrypoint.sh  libdevbnd.so  nr-binder  nr-cli  nr-gnb  nr-ue
root@lab-virtual-machine:/ueransim/bin# ./nr-binder 12.1.1.5 python3 TCPclient.py
./nr-binder: line 20: python3: command not found
root@lab-virtual-machine:/ueransim/bin# apt-get install python3
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following additional packages will be installed:
  file libexpat1 libmagic-mgc libmagic1 libmpdec2 libpython3-stdlib libpython3.6-minimal libpython3.6-stdlib libreadline7 libsqlite3-0 libssl1.1
  mime-support python3-minimal python3.6 python3.6-minimal readline-common xz-utils
Suggested packages:
  python3-doc python3-tk python3-venv python3.6-venv python3.6-doc binutils binfmt-support readline-doc
The following NEW packages will be installed:
  file libexpat1 libmagic-mgc libmagic1 libmpdec2 libpython3-stdlib libpython3.6-minimal libpython3.6-stdlib libreadline7 libsqlite3-0 libssl1.1
  mime-support python3 python3-minimal python3.6 python3.6-minimal readline-common xz-utils
0 upgraded, 18 newly installed, 0 to remove and 21 not upgraded.
Need to get 3529 kB/6671 kB of archives.
After this operation, 34.1 MB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libssl1.1 amd64 1.1.1-1ubuntu2.1~18.04.21 [1304 kB]
Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libpython3.6-minimal amd64 3.6.9-1~18.04ubuntu1.12 [533 kB]
Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libexpat1 amd64 2.2.5-3ubuntu0.9 [82.8 kB]
Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3.6-minimal amd64 3.6.9-1~18.04ubuntu1.12 [1609 kB]
Fetched 3529 kB in 3s (1164 kB/s)           
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package libssl1.1:amd64.
(Reading database ... 8226 files and directories currently installed.)
Preparing to unpack .../libssl1.1_1.1.1-1ubuntu2.1~18.04.21_amd64.deb ...
Unpacking libssl1.1:amd64 (1.1.1-1ubuntu2.1~18.04.21) ...
Selecting previously unselected package libpython3.6-minimal:amd64.
Preparing to unpack .../libpython3.6-minimal_3.6.9-1~18.04ubuntu1.12_amd64.deb ...
Unpacking libpython3.6-minimal:amd64 (3.6.9-1~18.04ubuntu1.12) ...
Selecting previously unselected package libexpat1:amd64.
Preparing to unpack .../libexpat1_2.2.5-3ubuntu0.9_amd64.deb ...
Unpacking libexpat1:amd64 (2.2.5-3ubuntu0.9) ...
Selecting previously unselected package python3.6-minimal.
Preparing to unpack .../python3.6-minimal_3.6.9-1~18.04ubuntu1.12_amd64.deb ...
Unpacking python3.6-minimal (3.6.9-1~18.04ubuntu1.12) ...
Setting up libssl1.1:amd64 (1.1.1-1ubuntu2.1~18.04.21) ...
debconf: unable to initialize frontend: Dialog
debconf: (No usable dialog-like program is installed, so the dialog based frontend cannot be used. at /usr/share/perl5/Debconf/FrontEnd/Dialog.pm line 76.)
debconf: falling back to frontend: Readline
debconf: unable to initialize frontend: Readline
debconf: (Can't locate Term/ReadLine.pm in @INC (you may need to install the Term::ReadLine module) (@INC contains: /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.26.1 /usr/local/share/perl/5.26.1 /usr/lib/x86_64-linux-gnu/perl5/5.26 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.26 /usr/share/perl/5.26 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base) at /usr/share/perl5/Debconf/FrontEnd/Readline.pm line 7.)
debconf: falling back to frontend: Teletype
Setting up libpython3.6-minimal:amd64 (3.6.9-1~18.04ubuntu1.12) ...
Setting up libexpat1:amd64 (2.2.5-3ubuntu0.9) ...
Setting up python3.6-minimal (3.6.9-1~18.04ubuntu1.12) ...
Selecting previously unselected package python3-minimal.
(Reading database ... 8483 files and directories currently installed.)
Preparing to unpack .../0-python3-minimal_3.6.7-1~18.04_amd64.deb ...
Unpacking python3-minimal (3.6.7-1~18.04) ...
Selecting previously unselected package mime-support.
Preparing to unpack .../1-mime-support_3.60ubuntu1_all.deb ...
Unpacking mime-support (3.60ubuntu1) ...
Selecting previously unselected package libmpdec2:amd64.
Preparing to unpack .../2-libmpdec2_2.4.2-1ubuntu1_amd64.deb ...
Unpacking libmpdec2:amd64 (2.4.2-1ubuntu1) ...
Selecting previously unselected package readline-common.
Preparing to unpack .../3-readline-common_7.0-3_all.deb ...
Unpacking readline-common (7.0-3) ...
Selecting previously unselected package libreadline7:amd64.
Preparing to unpack .../4-libreadline7_7.0-3_amd64.deb ...
Unpacking libreadline7:amd64 (7.0-3) ...
Selecting previously unselected package libsqlite3-0:amd64.
Preparing to unpack .../5-libsqlite3-0_3.22.0-1ubuntu0.7_amd64.deb ...
Unpacking libsqlite3-0:amd64 (3.22.0-1ubuntu0.7) ...
Selecting previously unselected package libpython3.6-stdlib:amd64.
Preparing to unpack .../6-libpython3.6-stdlib_3.6.9-1~18.04ubuntu1.12_amd64.deb ...
Unpacking libpython3.6-stdlib:amd64 (3.6.9-1~18.04ubuntu1.12) ...
Selecting previously unselected package python3.6.
Preparing to unpack .../7-python3.6_3.6.9-1~18.04ubuntu1.12_amd64.deb ...
Unpacking python3.6 (3.6.9-1~18.04ubuntu1.12) ...
Selecting previously unselected package libpython3-stdlib:amd64.
Preparing to unpack .../8-libpython3-stdlib_3.6.7-1~18.04_amd64.deb ...
Unpacking libpython3-stdlib:amd64 (3.6.7-1~18.04) ...
Setting up python3-minimal (3.6.7-1~18.04) ...
Selecting previously unselected package python3.
(Reading database ... 8941 files and directories currently installed.)
Preparing to unpack .../python3_3.6.7-1~18.04_amd64.deb ...
Unpacking python3 (3.6.7-1~18.04) ...
Selecting previously unselected package libmagic-mgc.
Preparing to unpack .../libmagic-mgc_1%3a5.32-2ubuntu0.4_amd64.deb ...
Unpacking libmagic-mgc (1:5.32-2ubuntu0.4) ...
Selecting previously unselected package libmagic1:amd64.
Preparing to unpack .../libmagic1_1%3a5.32-2ubuntu0.4_amd64.deb ...
Unpacking libmagic1:amd64 (1:5.32-2ubuntu0.4) ...
Selecting previously unselected package file.
Preparing to unpack .../file_1%3a5.32-2ubuntu0.4_amd64.deb ...
Unpacking file (1:5.32-2ubuntu0.4) ...
Selecting previously unselected package xz-utils.
Preparing to unpack .../xz-utils_5.2.2-1.3ubuntu0.1_amd64.deb ...
Unpacking xz-utils (5.2.2-1.3ubuntu0.1) ...
Setting up readline-common (7.0-3) ...
Setting up mime-support (3.60ubuntu1) ...
Setting up libreadline7:amd64 (7.0-3) ...
Setting up libmagic-mgc (1:5.32-2ubuntu0.4) ...
Setting up libmagic1:amd64 (1:5.32-2ubuntu0.4) ...
Setting up xz-utils (5.2.2-1.3ubuntu0.1) ...
update-alternatives: using /usr/bin/xz to provide /usr/bin/lzma (lzma) in auto mode
update-alternatives: warning: skip creation of /usr/share/man/man1/lzma.1.gz because associated file /usr/share/man/man1/xz.1.gz (of link group lzma) doesn't exist
update-alternatives: warning: skip creation of /usr/share/man/man1/unlzma.1.gz because associated file /usr/share/man/man1/unxz.1.gz (of link group lzma) doesn't exist
update-alternatives: warning: skip creation of /usr/share/man/man1/lzcat.1.gz because associated file /usr/share/man/man1/xzcat.1.gz (of link group lzma) doesn't exist
update-alternatives: warning: skip creation of /usr/share/man/man1/lzmore.1.gz because associated file /usr/share/man/man1/xzmore.1.gz (of link group lzma) doesn't exist
update-alternatives: warning: skip creation of /usr/share/man/man1/lzless.1.gz because associated file /usr/share/man/man1/xzless.1.gz (of link group lzma) doesn't exist
update-alternatives: warning: skip creation of /usr/share/man/man1/lzdiff.1.gz because associated file /usr/share/man/man1/xzdiff.1.gz (of link group lzma) doesn't exist
update-alternatives: warning: skip creation of /usr/share/man/man1/lzcmp.1.gz because associated file /usr/share/man/man1/xzcmp.1.gz (of link group lzma) doesn't exist
update-alternatives: warning: skip creation of /usr/share/man/man1/lzgrep.1.gz because associated file /usr/share/man/man1/xzgrep.1.gz (of link group lzma) doesn't exist
update-alternatives: warning: skip creation of /usr/share/man/man1/lzegrep.1.gz because associated file /usr/share/man/man1/xzegrep.1.gz (of link group lzma) doesn't exist
update-alternatives: warning: skip creation of /usr/share/man/man1/lzfgrep.1.gz because associated file /usr/share/man/man1/xzfgrep.1.gz (of link group lzma) doesn't exist
Setting up libsqlite3-0:amd64 (3.22.0-1ubuntu0.7) ...
Setting up libmpdec2:amd64 (2.4.2-1ubuntu1) ...
Setting up libpython3.6-stdlib:amd64 (3.6.9-1~18.04ubuntu1.12) ...
Setting up python3.6 (3.6.9-1~18.04ubuntu1.12) ...
Setting up file (1:5.32-2ubuntu0.4) ...
Setting up libpython3-stdlib:amd64 (3.6.7-1~18.04) ...
Setting up python3 (3.6.7-1~18.04) ...
running python rtupdate hooks for python3.6...
running python post-rtupdate hooks for python3.6...
Processing triggers for libc-bin (2.27-3ubuntu1.6) ...
root@lab-virtual-machine:/ueransim/bin# ./nr-binder 12.1.1.2 python3 TCPclient.py
Waitting for connecting...
Connected!
Sending message...
^CTraceback (most recent call last):
  File "TCPclient.py", line 14, in <module>
    time.sleep(5)
KeyboardInterrupt
root@lab-virtual-machine:/ueransim/bin# ifconfig
docker0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 172.17.0.1  netmask 255.255.0.0  broadcast 172.17.255.255
        ether 02:42:27:d3:5c:f4  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

ens33: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.2.106  netmask 255.255.255.0  broadcast 192.168.2.255
        inet6 fe80::f5f:7af1:a9b:2e90  prefixlen 64  scopeid 0x20<link>
        ether 00:0c:29:a6:b4:4b  txqueuelen 1000  (Ethernet)
        RX packets 99639  bytes 109478116 (109.4 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 34952  bytes 3079711 (3.0 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

ens39: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.14.33  netmask 255.255.255.0  broadcast 192.168.14.255
        inet6 fe80::20c:29ff:fea6:b47d  prefixlen 64  scopeid 0x20<link>
        ether 00:0c:29:a6:b4:7d  txqueuelen 1000  (Ethernet)
        RX packets 7900  bytes 700746 (700.7 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 1700  bytes 115275 (115.2 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

ens40: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.12.33  netmask 255.255.255.0  broadcast 192.168.12.255
        inet6 fe80::20c:29ff:fea6:b469  prefixlen 64  scopeid 0x20<link>
        ether 00:0c:29:a6:b4:69  txqueuelen 1000  (Ethernet)
        RX packets 96868  bytes 6121208 (6.1 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 5212  bytes 377456 (377.4 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

ens41: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.13.33  netmask 255.255.255.0  broadcast 192.168.13.255
        inet6 fe80::20c:29ff:fea6:b473  prefixlen 64  scopeid 0x20<link>
        ether 00:0c:29:a6:b4:73  txqueuelen 1000  (Ethernet)
        RX packets 1582  bytes 323557 (323.5 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 1703  bytes 115769 (115.7 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 14097  bytes 962209 (962.2 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 14097  bytes 962209 (962.2 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

uesimtun0: flags=369<UP,POINTOPOINT,NOTRAILERS,RUNNING,PROMISC>  mtu 1400
        inet 12.1.1.4  netmask 255.255.255.255  destination 12.1.1.4
        inet6 fe80::74a3:2948:10c9:3a71  prefixlen 64  scopeid 0x20<link>
        unspec 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00  txqueuelen 500  (UNSPEC)
        RX packets 2  bytes 168 (168.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 23  bytes 1576 (1.5 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

uesimtun1: flags=369<UP,POINTOPOINT,NOTRAILERS,RUNNING,PROMISC>  mtu 1400
        inet 12.1.1.5  netmask 255.255.255.255  destination 12.1.1.5
        inet6 fe80::c9fd:7b73:83a2:a1b5  prefixlen 64  scopeid 0x20<link>
        unspec 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00  txqueuelen 500  (UNSPEC)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

root@lab-virtual-machine:/ueransim/bin# ./nr-binder 12.1.1.4 python3 TCPclient.py
Waitting for connecting...
Connected!
Sending message...
^CTraceback (most recent call last):
  File "TCPclient.py", line 14, in <module>
    time.sleep(5)
KeyboardInterrupt
root@lab-virtual-machine:/ueransim/bin# ./nr-binder 12.1.1.5 python3 TCPclient.py
Waitting for connecting...
^CTraceback (most recent call last):
  File "TCPclient.py", line 7, in <module>
    tcp_client_socket.connect(("192.168.70.1", 8088))
KeyboardInterrupt
root@lab-virtual-machine:/ueransim/bin# ./nr-binder 12.1.1.3 python3 TCPclient.py
Waitting for connecting...
Connected!
Sending message...
^CTraceback (most recent call last):
  File "TCPclient.py", line 14, in <module>
    time.sleep(5)
KeyboardInterrupt
root@lab-virtual-machine:/ueransim/bin# 

测试总结:生成两个容器,都健康,在核心网看见一个基站接入,两个ue接入,生成了两个uesimtun0接口,这些接口在容器内都可以使用,成功通过接口进行TCP传输,但是ifconfig发现明明是12.1.1.4和12.1.1.5,但在使用时却发现12.1.1.2与12.1.1.3还有12.1.1.4可以进行TCP传输,而12.1.1.5不行。

再次进行测试,使用这些接口绑定不同的端口号进行传输,之前是都绑定:192.168.70.1:8088进行传输,这次对不同的接口绑定不同的端口进行传输,验证不同应用传输的可行性:

在服务器A核心网侧创建两个TCPsever,两者的差别是端口号不一样:

TCPsever1.py:

import socket
from threading import Thread


def new_client_connect(new_client_socket, client_ip_port):
    while True:
        # 收发数据
        recv_data = new_client_socket.recv(1024)
        if len(recv_data) != 0:
            recv_text = recv_data.decode("gb2312")
            print("接收到[%s]的信息:%s" % (str(client_ip_port), recv_text))
        else:
            print("客户端断开连接")
            break

        # # 关闭连接
        # new_client_socket.close()  # 表示断开与当前的客户端的通信


def main():
    # 创建套接字
    tcp_server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

    # 绑定端口和ip
    tcp_server_socket.bind(("192.168.70.1", 8088))

    # 设置套接字为被动监听模式,不能主动发送数据,128为允许接收的最大连接数
    tcp_server_socket.listen(128)

    while True:
        # 接收客户端连接
        new_client_socket, client_ip_port = tcp_server_socket.accept()

        t1 = Thread(target=new_client_connect, args=(new_client_socket, client_ip_port))
        t1.start()

        # tcp_server_socket.close()  # 表示不再接受新客户端的连接,已经连接的可以继续服务


if __name__ == '__main__':
    main()

TCPsever2.py:

import socket
from threading import Thread


def new_client_connect(new_client_socket, client_ip_port):
    while True:
        # 收发数据
        recv_data = new_client_socket.recv(1024)
        if len(recv_data) != 0:
            recv_text = recv_data.decode("gb2312")
            print("接收到[%s]的信息:%s" % (str(client_ip_port), recv_text))
        else:
            print("客户端断开连接")
            break

        # # 关闭连接
        # new_client_socket.close()  # 表示断开与当前的客户端的通信


def main():
    # 创建套接字
    tcp_server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

    # 绑定端口和ip
    tcp_server_socket.bind(("192.168.70.1", 8087))

    # 设置套接字为被动监听模式,不能主动发送数据,128为允许接收的最大连接数
    tcp_server_socket.listen(128)

    while True:
        # 接收客户端连接
        new_client_socket, client_ip_port = tcp_server_socket.accept()

        t1 = Thread(target=new_client_connect, args=(new_client_socket, client_ip_port))
        t1.start()

        # tcp_server_socket.close()  # 表示不再接受新客户端的连接,已经连接的可以继续服务


if __name__ == '__main__':
    main()

同时运行这两个TCPsever:

python3 TCPserve1.py
python3 TCPserve2.py

然后在主机Bueransim侧的两个ueranism容器内分别绑定不同的uesimtun0,运行两个TCPclient:
TCPclient1:

import socket
import time
# 创建套接字
tcp_client_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
print("Waitting for connecting...")
# 建立连接
tcp_client_socket.connect(("192.168.70.1", 8088))
print("Connected!")

# 发送数据
print("Sending message...")
while 1:
    tcp_client_socket.send("你好".encode("gb2312"))
    time.sleep(5)
# 接收数据
recv_data = tcp_client_socket.recv(1024).decode("gb2312")
print(recv_data)

# 关闭套接字
tcp_client_socket.close()

TCPclient2:

import socket
import time
# 创建套接字
tcp_client_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
print("Waitting for connecting...")
# 建立连接
tcp_client_socket.connect(("192.168.70.1", 8087))
print("Connected!")

# 发送数据
print("Sending message...")
while 1:
    tcp_client_socket.send("你好".encode("gb2312"))
    time.sleep(5)
# 接收数据
recv_data = tcp_client_socket.recv(1024).decode("gb2312")
print(recv_data)

# 关闭套接字
tcp_client_socket.close()

分别复制到两个容器中:

lab@lab-virtual-machine:~/oai-cn5g-fed/docker-compose/ueransim yz$sudo docker cp  TCPclient1.py ueransim:/ueransim/bin
lab@lab-virtual-machine:~/oai-cn5g-fed/docker-compose/ueransim yz$sudo docker cp  TCPclient2.py ueransim2:/ueransim/bin

在两个容器中使用两个不同uesimtun接口绑定运行这两个TCPclient:

#在ueransim1容器内
root@lab-virtual-machine:/ueransim/bin# ./nr-binder 12.1.1.2 python3 TCPclient1.py
#在ueransim2容器内
root@lab-virtual-machine:/ueransim/bin# ./nr-binder 12.1.1.3 python3 TCPclient2.py

结果两个TCP服务都传输成功了!

服务1:
在这里插入图片描述
在这里插入图片描述

服务2:
在这里插入图片描述
在这里插入图片描述
到此,我们可以确定,用这种方式,至少可以在一个虚拟机中使用多个uesimtun接口进行多个业务的传输。

结论

使用host容器网络模式部署完成,可以实现单ueransim容器分立部署接入OAI核心网,并测试传输数据

问题提出:修改官方ueransim容器为自定义容器

最后一个问题:如何将这个容器变为一键启动式,而不需要接入容器内部

首先尝试基于已经构建完成的容器,能否使用docker commit命令,使其生成新的镜像:

ueransim2在运行时进行commit提交:

lab@lab-virtual-machine:~/oai-cn5g-fed/docker-compose/ueransim yz$ docker container commit -a "@yinzhao" -m"add python3" ueransim2 ueransimyz2
WARNING: Error loading config file: /home/lab/.docker/config.json: open /home/lab/.docker/config.json: permission denied
sha256:f64be76d1d92b0785ee366b69ed6394e98c281e001e2282a1a017bb70269e535

生成了新的image:
在这里插入图片描述
使用该image构建docker-compose启动文件:
在这里插入图片描述
启动:

在这里插入图片描述
结果是这样直接commit提交的镜像是无法启动的,会直接退出
又到了这个老问题

ueransim容器自定义构建的方法探索

为了让ueransim无需每次启动都接入容器进行操作,在此探索可行的方法:

https://blog.csdn.net/a772304419/article/details/123199579
根据此博客所言,使用commite构建新镜像是会让镜像因为过多无关操作以及以往层级无法修改使镜像非常臃肿,且更改的内容是黑盒操作其他人无从得知进行了那些修改,即使是镜像生成者在一段时间后也可能忘掉,使维护变得非常困难。所以更推荐从dockerfile开始构建镜像。

如下是官方的ueransim+OAI教程,其中有两种方法使用ueransim镜像,其中第一种就是自己构建镜像的方法:
https://gitlab.eurecom.fr/oai/cn5g/oai-cn5g-fed/-/blob/master/docs/DEPLOY_SA5G_WITH_UERANSIM.md#6-getting-a-ueransim-docker-image

在这里插入图片描述
ueransim构建clone地址

从官网clone代码到本地:

git clone -b docker_support https://github.com/orion-belt/UERANSIM.git

如果不行可以直接进入ueransim构建clone地址下载代码后复制到Ubuntu中
在这里插入图片描述
然后进入文件夹进行构建:

lab@lab-virtual-machine:~/oai-cn5g-fed/docker-compose/ueransim yz/UERANSIM$ sudo docker build --target ueransim --tag ueransim:latest1 -f docker/Dockerfile.ubuntu.18.04 .

这里为与之前已存在的ueransim镜像进行区别,将进行tag由ueransim:latest改为了ueransim:latest1

构建非常消耗时间:

在这里插入图片描述

lab@lab-virtual-machine:~/oai-cn5g-fed/docker-compose/ueransim yz/UERANSIM$ sudo docker build --target ueransim --tag ueransimyz:latestyz -f docker/Dockerfile.ubuntu.18.04 .
[sudo] password for lab: 
[+] Building 154.1s (18/18) FINISHED                                            
 => [internal] load .dockerignore                                          0.0s
 => => transferring context: 2B                                            0.0s
 => [internal] load build definition from Dockerfile.ubuntu.18.04          0.0s
 => => transferring dockerfile: 45B                                        0.0s
 => [internal] load metadata for docker.io/library/ubuntu:bionic           0.0s
 => [internal] load build context                                          0.0s
 => => transferring context: 333B                                          0.0s
 => [ueransim 1/7] FROM docker.io/library/ubuntu:bionic                    0.0s
 => CACHED [ueransim 2/7] RUN apt update && apt install -y iproute2 iputi  0.0s
 => CACHED [ueransim 3/7] WORKDIR /ueransim/etc                            0.0s
 => CACHED [ueransim-builder 2/7] RUN apt-get update -y && apt-get upgrad  0.0s
 => CACHED [ueransim-builder 3/7] RUN wget -O - https://apt.kitware.com/k  0.0s
 => CACHED [ueransim-builder 4/7] RUN cmake --version                      0.0s
 => CACHED [ueransim-builder 5/7] RUN git clone https://github.com/aligun  0.0s
 => CACHED [ueransim-builder 6/7] COPY docker/ /UERANSIM/docker/           0.0s
 => [ueransim-builder 7/7] RUN make -C UERANSIM                          153.2s
 => [ueransim 4/7] COPY --from=ueransim-builder /UERANSIM/docker/custom*.  0.0s 
 => [ueransim 5/7] WORKDIR /ueransim/bin                                   0.0s 
 => [ueransim 6/7] COPY --from=ueransim-builder /UERANSIM/build/*  /ueran  0.0s 
 => [ueransim 7/7] COPY --from=ueransim-builder /UERANSIM/docker/entrypoi  0.0s 
 => exporting to image                                                     0.4s 
 => => exporting layers                                                    0.4s 
 => => writing image sha256:270824a92f2db50362f9e6fdf60a4eb44e73808788c22  0.0s
 => => naming to docker.io/library/ueransimyz:latestyz                     0.0s

在这里插入图片描述
测试该镜像是否可以正常使用:

在yaml文件内将使用镜像设置为此镜像:
在这里插入图片描述

无法直接启动:
在这里插入图片描述
https://www.coder.work/article/7362041
https://bbs.csdn.net/topics/390847016 是关于这个报错的原因:入口点有问题
可以直接docker run运行容器:

lab@lab-virtual-machine:~/oai-cn5g-fed/docker-compose/ueransim yz$ sudo docker run -it ueransimyz:latestyz /bin/bash

检查dockerfile,对其进行修改:
原dockerfile:

#---------------------------------------------------------------------
# BUILDER IMAGE
#---------------------------------------------------------------------
FROM ubuntu:bionic as ueransim-builder

RUN apt-get update -y && apt-get upgrade -y &&\
    apt install -y make gcc g++ libsctp-dev lksctp-tools\
    iproute2 wget software-properties-common git 

RUN wget -O - https://apt.kitware.com/keys/kitware-archive-latest.asc 2>/dev/null\
    | gpg --dearmor - | tee /etc/apt/trusted.gpg.d/kitware.gpg >/dev/null\
    && apt-add-repository 'deb https://apt.kitware.com/ubuntu/ bionic main'\
    && apt update && apt install -y cmake
    
RUN cmake --version

RUN git clone https://github.com/aligungr/UERANSIM.git && cd UERANSIM && git checkout -f v3.2.5
COPY docker/ /UERANSIM/docker/

RUN make -C UERANSIM

#---------------------------------------------------------------------
# TARGET IMAGE
#---------------------------------------------------------------------
FROM ubuntu:bionic as ueransim

RUN apt update && apt install -y iproute2 iputils-ping net-tools\
    iperf3 libsctp-dev lksctp-tools 

WORKDIR /ueransim/etc
COPY --from=ueransim-builder /UERANSIM/docker/custom*.yaml  /ueransim/etc/

WORKDIR /ueransim/bin
COPY --from=ueransim-builder /UERANSIM/build/*  /ueransim/bin/
COPY --from=ueransim-builder /UERANSIM/docker/entrypoint.sh . 


ENTRYPOINT ["/ueransim/bin/entrypoint.sh"]

修改后:

版本由3.2.5改为3.2.6,多装了vim python3 curl

#---------------------------------------------------------------------
# BUILDER IMAGE
#---------------------------------------------------------------------
FROM ubuntu:bionic as ueransim-builder

RUN apt-get update -y && apt-get upgrade -y &&\
    apt install -y make gcc g++ libsctp-dev lksctp-tools\
    iproute2 wget software-properties-common git 

RUN wget -O - https://apt.kitware.com/keys/kitware-archive-latest.asc 2>/dev/null\
    | gpg --dearmor - | tee /etc/apt/trusted.gpg.d/kitware.gpg >/dev/null\
    && apt-add-repository 'deb https://apt.kitware.com/ubuntu/ bionic main'\
    && apt update && apt install -y cmake
    
RUN cmake --version

RUN git clone https://github.com/aligungr/UERANSIM.git && cd UERANSIM && git checkout -f v3.2.6
COPY docker/ /UERANSIM/docker/

RUN make -C UERANSIM

#---------------------------------------------------------------------
# TARGET IMAGE
#---------------------------------------------------------------------
FROM ubuntu:bionic as ueransim

RUN apt update && apt install -y iproute2 iputils-ping net-tools\
    iperf3 libsctp-dev lksctp-tools vim python3 curl

WORKDIR /ueransim/etc
COPY --from=ueransim-builder /UERANSIM/docker/custom*.yaml  /ueransim/etc/

WORKDIR /ueransim/bin
COPY --from=ueransim-builder /UERANSIM/build/*  /ueransim/bin/
COPY --from=ueransim-builder /UERANSIM/docker/entrypoint.sh . 


ENTRYPOINT ["/ueransim/bin/entrypoint.sh"]

构建镜像完成后镜像依旧无法正常使用docker-compose启动,使用docker run单独启动发现里面正常安装了vim 与python3
在这里插入图片描述

在这里插入图片描述
报错是 no command specified但明明启动接入了entrypoiny
在这里插入图片描述

猜你喜欢

转载自blog.csdn.net/qq_41224270/article/details/130194668