Cyborg/TestReport/IntelNic
< Cyborg | TestReport
目录
Intel x710 网卡 Cyborg 驱动测试报告
此测试报告显示当前的 intel 网卡驱动在本地环境中得到了充分的测试，该环境包含 Intel x710 网卡。 替代第三方 CI，此报告可以作为证明。
报告部分
在 cyborg agent 部分启用 intel 网卡驱动
- 在 enabled_drivers 中添加 intel_nic_driver
- 将 "enabled_nic_types" 设置为 "x710_static"，这意味着支持预编程的网卡。
- 在 "x710_static" 部分设置 "physical_device_mappings" 和 "function_device_mappings"，指示 physnet/设备、function/设备之间的映射关系。
$ vim /etc/cyborg/cyborg.conf [agent] enabled_drivers = fake_driver,intel_nic_driver
[nic_devices] enabled_nic_types = x710_static
[x710_static] physical_device_mappings = physnet1:eth2|eth3 function_device_mappings = GTPv1:eth3|eth2
- 重启 cyborg-agent
$ systemctl restart devstack@cyborg-agent
报告日志
cyborg-conductor: we can see cyborg conductor call placement API to create releated resource providerFeb 02 03:10:04 newbuy1217.maas cyborg-conductor[1836836]: 2021-02-02 03:10:04.476 1836836 INFO cyborg.common.placement_client [req-27d71744-8a79-4fb6-a2ca-2595b464051c - - - - -] Creating resource provider newbuy1217.maas_0000:86:02.3 Feb 02 03:10:04 newbuy1217.maas cyborg-conductor[1836836]: 2021-02-02 03:10:04.485 1836836 INFO cyborg.common.placement_client [req-27d71744-8a79-4fb6-a2ca- 2595b464051c - - - - -] [req-b6cbe2bb-201a-4c86-9789-02c0a4648be1] Created resource provider record via placement API for resource provider with UUID 5fe904cf-5d52-3dff-9a25- 635841deed1a and name newbuy1217.maas_0000:86:02.3. Feb 02 03:10:04 newbuy1217.maas cyborg-conductor[1836836]: 2021-02-02 03:10:04.520 1836836 INFO cyborg.common.placement_client [req-27d71744-8a79-4fb6-a2ca- 2595b464051c - - - - -] Trait CUSTOM_VF already existed Feb 02 03:10:04 newbuy1217.maas cyborg-conductor[1836836]: 2021-02-02 03:10:04.530 1836836 INFO cyborg.common.placement_client [req-27d71744-8a79-4fb6-a2ca- 2595b464051c - - - - -] Trait CUSTOM_GTPV1 already existed Feb 02 03:10:04 newbuy1217.maas cyborg-conductor[1836836]: 2021-02-02 03:10:04.540 1836836 INFO cyborg.common.placement_client [req-27d71744-8a79-4fb6-a2ca- 2595b464051c - - - - -] Trait CUSTOM_PHYSNET1 already existed Feb 02 03:10:04 newbuy1217.maas cyborg-conductor[1836836]: 2021-02-02 03:10:04.580 1836836 INFO cyborg.common.placement_client [req-27d71744-8a79-4fb6-a2ca- 2595b464051c - - - - -] Resource class CUSTOM_NIC already exists Feb 02 03:10:04 newbuy1217.maas cyborg-conductor[1836836]: 2021-02-02 03:10:04.587 1836836 INFO cyborg.common.placement_client [req-27d71744-8a79-4fb6-a2ca- 2595b464051c - - - - -] Creating resource provider newbuy1217.maas_0000:86:02.1
检查 placement 记录
$ openstack resource provider list +--------------------------------------+------------------------------+------------+ | uuid | name | generation | +--------------------------------------+------------------------------+------------+ | 5977653e-12df-319f-a420-17716b7e5123 | newbuy1217.maas_0000:86:00.1 | 2 | | e71b41e9-da06-3b85-8131-b753c01c224b | newbuy1217.maas_0000:86:02.2 | 2 | | 84cd0653-049d-36c7-8ad3-d02f9bad4a8c | newbuy1217.maas_0000:86:02.0 | 2 | | 5fe904cf-5d52-3dff-9a25-635841deed1a | newbuy1217.maas_0000:86:02.3 | 2 | | e95f79d4-8ac1-3b0d-b1ce-002cb073fc9c | newbuy1217.maas_0000:86:02.1 | 2 | +--------------------------------------+------------------------------+------------+
创建了 5 个 RP，因为在此环境中，有 2 个 NIC PF，其中一个虚拟化为 6 个 VF，另一个未虚拟化。
$ openstack resource provider inventory list 5fe904cf-5d52-3dff-9a25-635841deed1a +----------------+------------------+----------+----------+----------+-----------+-------+ | resource_class | allocation_ratio | min_unit | max_unit | reserved | step_size | total | +----------------+------------------+----------+----------+----------+-----------+-------+ | CUSTOM_NIC | 1.0 | 1 | 1 | 0 | 1 | 1 | +----------------+------------------+----------+----------+----------+-----------+-------+
启动带有 NIC VF 附加的虚拟机
创建设备 profile 和 flavor
$ export GRP=[{\"resources\:CUSTOM_NIC\"\:\"1\"\,\"trait:CUSTOM_GTPV1\":\"required\"\,\"trait:CUSTOM_VF\":\"required\"}]
$ openstack accelerator device profile create sriovdp $GRP
$ openstack accelerator device profile show d30efbf9-bdf5-45d1-9d34-74d720121971
+------------+--------------------------------------------------------------------------------------------------+
| Field | Value |
+------------+--------------------------------------------------------------------------------------------------+
| created_at | 2021-01-14T08:11:32+00:00 |
| updated_at | None |
| uuid | d30efbf9-bdf5-45d1-9d34-74d720121971 |
| name | sriovdp |
| groups | [{'resources:CUSTOM_NIC': '1', 'trait:CUSTOM_GTPV1': 'required', 'trait:CUSTOM_VF': 'required'}] |
+------------+--------------------------------------------------------------------------------------------------+
$ openstack port create test --network public --device-profile new-sriovdp --vnic-type=accelerator-direct
$ neutron port-show 299e665f-502c-42f8-b744-058d8ead5a55
neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
+-----------------------+--------------------------------------------------------------------------------------+
| Field | Value |
+-----------------------+--------------------------------------------------------------------------------------+
| admin_state_up | True |
| allowed_address_pairs | |
| binding:host_id | |
| binding:profile | {"physical_network": "physnet1"} |
| binding:vif_details | {} |
| binding:vif_type | unbound |
| binding:vnic_type | accelerator-direct |
| created_at | 2021-01-14T08:13:05Z |
| description | |
| device_id | |
| device_owner | |
| device_profile | new-sriovdp |
| extra_dhcp_opts | |
| fixed_ips | {"subnet_id": "26921a61-a89b-44d9-ba3a-6135cfe2ffec", "ip_address": "172.24.4.173"} |
| | {"subnet_id": "82914187-5c21-4a78-a787-a7befc10c8d1", "ip_address": "2001:db8::3aa"} |
| id | 299e665f-502c-42f8-b744-058d8ead5a55 |
| mac_address | fa:16:3e:3e:28:7b |
| name | test |
| network_id | db1a7132-6469-4b75-965e-dc103b1b04cb |
| project_id | 6aa3db65287949cfab355baff912a5f3 |
| revision_number | 143 |
| security_groups | 2fcaedb7-b148-4597-9bc4-6a5173034125 |
| status | DOWN |
| tags | |
| tenant_id | 6aa3db65287949cfab355baff912a5f3 |
| updated_at | 2021-01-29T07:41:41Z |
+-----------------------+--------------------------------------------------------------------------------------+
启动带有 x710 网卡 VF 附加的虚拟机
$ export ID=299e665f-502c-42f8-b744-058d8ead5a55 $ nova boot testbackend --flavor m1.tiny --image cirros-0.5.1-x86_64-disk --nic port-id=$ID
Nova 获取 DP 详情
Feb 03 03:24:30 newbuy1217.maas devstack@cyborg-api.service[1836730]: [pid: 1836744|app: 0|req: 8/181] 192.168.0.186 () {60 va
rs in 1207 bytes} [Wed Feb 3 03:24:30 2021] GET /accelerator/v2/device_profiles => generated 738 bytes in 126 msecs (HTTP/1.1
200) 8 headers in 286 bytes (2 switches on core 0)
Feb 03 03:29:34 newbuy1217.maas devstack@cyborg-api.service[1836730]: 2021-02-03 03:29:34.389 1836748 INFO cyborg.api.controll
ers.v2.device_profiles [req-31b7d118-2df2-403e-8d5d-205be1de2145 50fe054b2deb4669abccf2f3ef980eb9 6aa3db65287949cfab355baff912
a5f3 - default default] [device_profiles] get_all returned: <cyborg.api.controllers.v2.device_profiles.DeviceProfileCollection
object at 0x7f915b0e6e10>
Feb 03 03:29:34 newbuy1217.maas devstack@cyborg-api.service[1836730]: [pid: 1836748|app: 0|req: 8/182] 192.168.0.186 () {60 va
rs in 1375 bytes} [Wed Feb 3 03:29:34 2021] GET /accelerator/v2/device_profiles?name=new-sriovdp => generated 414 bytes in 22
msecs (HTTP/1.1 200) 8 headers in 286 bytes (1 switches on core 0)
ARQ 创建
Feb 03 03:29:34 newbuy1217.maas devstack@cyborg-api.service[1836730]: 2021-02-03 03:29:34.409 1836750 INFO cyborg.api.controll
ers.v2.arqs [req-a7f6c123-f415-49d9-8aba-ba21582a41e5 50fe054b2deb4669abccf2f3ef980eb9 6aa3db65287949cfab355baff912a5f3 - defa
ult default] [arq] post req = ({'device_profile_name': 'new-sriovdp'})
ARQ 绑定
Feb 03 03:29:34 newbuy1217.maas devstack@cyborg-api.service[1836730]: 2021-02-03 03:29:34.926 1836747 INFO cyborg.api.controll
ers.v2.arqs [req-dc455c20-9b91-46b9-a3a5-076a8e5f3688 50fe054b2deb4669abccf2f3ef980eb9 6aa3db65287949cfab355baff912a5f3 - defa
ult default] [arqs] patch. list=({'b9bf6023-f1c2-40ec-ab0d-04c3c4958e94': [{'path': '/hostname', 'op': 'add', 'value': 'newbuy
1217.maas'}, {'path': '/device_rp_uuid', 'op': 'add', 'value': 'e95f79d4-8ac1-3b0d-b1ce-002cb073fc9c'}, {'path': '/instance_uu
id', 'op': 'add', 'value': '1fd31114-40cb-4f0f-af49-794545ba8e38'}]})
获取绑定的 ARQ
Feb 03 03:29:35 newbuy1217.maas devstack@cyborg-api.service[1836730]: [pid: 1836755|app: 0|req: 8/186] 192.168.0.186 () {58 vars in 1248 bytes} [Wed Feb 3 03:29:35 2021]
GET /accelerator/v2/accelerator_requests?instance=1fd31114-40cb-4f0f-af49-794545ba8e38 => generated 680 bytes in 18 msecs (HTTP/1.1 200) 8 headers in 286 bytes (2 switches
on core 0)
检查虚拟机
登录到虚拟机，运行`lspci`
它将显示一个 pci 设备，这意味着 x710 网卡 vf 已成功传递到虚拟机。
$ lspci 00:01.2 Class 0c03: 8086:7020 00:01.0 Class 0601: 8086:7000 00:04.0 Class 0200: 8086:154c 00:00.0 Class 0600: 8086:1237 00:01.3 Class 0680: 8086:7113 ...
我们可以看到 8086:154c，这是一个 x710 网卡的 VF。
Nova 会拒绝服务器迁移或挂起
$ openstack server shelve testbackend Forbidden with instances that have accelerators. (HTTP 403) (Request-ID: req-9b988fb9-159f-43df-8f70-ef358f6b1672)
$ openstack server resize testbackend --flavor 3 Forbidden with instances that have accelerators. (HTTP 403) (Request-ID: req-9779d326-647d-4ad4-80b9-a5e1bdaaa436)
]$ openstack server suspend testbackend Forbidden with instances that have accelerators. (HTTP 403) (Request-ID: req-c827af77-c37b-4d66-833b-741781a7e03c)
$ openstack server evacuate testbackend Forbidden with instances that have accelerators. (HTTP 403) (Request-ID: req-21fdf02b-32a0-45b1-b424-8b54614943dc)
$ openstack server migrate --live-migration testbackend Forbidden with instances that have accelerators. (HTTP 403) (Request-ID: req-75591909-29f2-465a-9d16-a31e955e498a)
