Cyborg/TestReport/InspurFPGA
< Cyborg | TestReport
Inspur FPGA Cyborg 驱动测试报告
此测试报告显示当前的 Inspur FPGA 驱动在 Inspur FPGA 卡所在的本地环境中经过了充分测试。作为第三方 CI 的替代，此报告可以作为证明。
报告部分
在 cyborg agent 部分启用 Inspur FPGA 驱动
[root@node11 cyborg]$ cat /etc/kolla/cyborg/cyborg.conf | grep inspur enabled_drivers = inspur_fpga_driver
报告日志
Aug 20 17:06:48 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:48.909 9441 INFO cyborg.conductor.manager [req-e9c7b9ec-eff4-
4480-8178-134037b60524 - - - - -] Start differing devices.
Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.501 9441 INFO cyborg.common.placement_client [req-e9c7b9ec-
eff4-4480-8178-134037b60524 - - - - -] Creating resource provider
Inspur Electronic Information Industry Co., Ltd. Device_0000:60:00.0
Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.553 9441 INFO cyborg.common.placement_client [req-e9c7b9ec-
eff4-4480-8178-134037b60524 - - - - -] [req-c5566486-40b5-4a20-b4f4-b0b96b7812c9] Created resource provider record via placement
API for resource provider with UUID bcedfee6-c1f0-349c-acfa-7d39033b7b8c and name
Inspur Electronic Information Industry Co., Ltd. Device_0000:60:00.0.
Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.715 9441 INFO cyborg.common.placement_client [req-e9c7b9ec-
eff4-4480-8178-134037b60524 - - - - -] Created trait CUSTOM_FPGA_INSPUR
Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.788 9441 INFO cyborg.common.placement_client [req-e9c7b9ec-
eff4-4480-8178- 134037b60524 - - - - -] Created trait CUSTOM_FPGA_PRODUCT_ID_A115
Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.950 9441 INFO cyborg.conductor.manager [req-e9c7b9ec-eff4-
4480-8178-134037b60524 - - - - -] Start differing deploybles.
Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.955 9441 INFO cyborg.conductor.manager [req-e9c7b9ec-eff4-
4480-8178-134037b60524 - - - - -] Start differing attributes.
Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.972 9441 INFO cyborg.conductor.manager [req-e9c7b9ec-eff4-
4480-8178-134037b60524 - - - - -] Start differing attach_handles.
Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.973 9441 INFO cyborg.conductor.manager [req-e9c7b9ec-eff4-
4480-8178-134037b60524 - - - - -] new info list ['{"bus": "60", "device": "00", "domain": "0000", "function": "0"}']
Aug 20 17:06:49 ubuntu cyborg-conductor[9441]: 2020-08-20 17:06:49.973 9441 INFO cyborg.conductor.manager [req-e9c7b9ec-eff4-
4480-8178-134037b60524 - - - - -] old info list ['{"bus": "60", "device": "00", "domain": "0000", "function": "0"}']
...
检查 cyborg 记录
root@ubuntu:~# openstack accelerator device list
+--------------------------------------+------+--------+----------+-----------------------------------------------------------------+
| uuid | type | vendor | hostname | std_board_info |
+--------------------------------------+------+--------+----------+-----------------------------------------------------------------+
| 462a37e7-3654-44ab-b5bf-c462cd10f8a4 | FPGA | 1bd4 | ubuntu | {"product_id": "a115", "controller": "Processing accelerators"} |
+--------------------------------------+------+--------+----------+-----------------------------------------------------------------
root@ubuntu:~# openstack accelerator deployable list +--------------------------------------+----------------------------------------------------------------------+-----------+ | uuid | name | device_id | +--------------------------------------+----------------------------------------------------------------------+-----------+ | 3355050d-6a21-4129-bab8-63280598684e | Inspur Electronic Information Industry Co., Ltd. Device_0000:60:00.0 | 2 | +--------------------------------------+----------------------------------------------------------------------+-----------+
检查 placement 记录
[root@ubuntu cyborg]$ openstack resource provider list +--------------------------------------+----------------------------+------------+ | uuid | name | generation | +--------------------------------------+----------------------------+------------+ | bcedfee6-c1f0-349c-acfa-7d39033b7b8c | Inspur Electronic Information Industry Co., Ltd. Device_0000:60:00.0 | 2 | | 1b3419ee-f45c-4b4e-ba91-7715b9070da8 | ubuntu | 60 |
[root@ubuntu cyborg]$ openstack resource provider inventory list bcedfee6-c1f0-349c-acfa-7d39033b7b8c +----------------+------------------+----------+----------+----------+-----------+-------+ | resource_class | allocation_ratio | min_unit | max_unit | reserved | step_size | total | +----------------+------------------+----------+----------+----------+-----------+-------+ | FPGA | 1.0 | 1 | 1 | 0 | 1 | 1 | +----------------+------------------+----------+----------+----------+-----------+-------+
启动带有 Inspur FPGA 的虚拟机
创建设备 profile 和 flavor
root@ubuntu:/opt/stack/cyborg# openstack accelerator device profile create inspur-fpga '[{"resources:FPGA":1, "trait:CUSTOM_FPGA_PRODUCT_ID_A115": "required", "trait:CUSTOM_FPGA_INSPUR": "required"}]'
+------------+--------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+---------------+-----------------------------------------------------------------------------------------------------------------+
| created_at | 2020-08-21 10:47:06+00:00 |
| updated_at | None |
| uuid | 63cac6f6-1122-46d1-825e-b288886ebdf5 |
| name | inspur-fpga |
| groups | [{'resources:FPGA': '1', 'trait:CUSTOM_FPGA_PRODUCT_ID_A115': 'required',
'trait:CUSTOM_FPGA_INSPUR': 'required'}] |
+---------------+------------------------------------------------------------------------------------------------------------- ---+
root@ubuntu:/opt/stack/cyborg# openstack accelerator device profile list
+--------------------------------------+-------------+--------------------------------------------------------------------------------------------------------------------+
| uuid | name | groups |
+--------------------------------------+-------------+--------------------------------------------------------------------------------------------------------------------+
| 63cac6f6-1122-46d1-825e-b288886ebdf5 | inspur-fpga | [{'resources:FPGA': '1', 'trait:CUSTOM_FPGA_PRODUCT_ID_A115': 'required',
'trait:CUSTOM_FPGA_INSPUR': 'required'}] |
root@ubuntu:/opt/stack/cyborg# nova flavor-key 3 set accel:device_profile='inspur-fpga'
root@ubuntu:/opt/stack/cyborg# nova flavor-show 3
+----------------------------+-------------------------------------------------------------------+
| Property | Value |
+----------------------------+-------------------------------------------------------------------+
| OS-FLV-DISABLED:disabled | False |
| OS-FLV-EXT-DATA:ephemeral | 0 |
| description | - |
| disk | 40 |
| extra_specs | {"accel:device_profile": "inspur-fpga", "hw_rng:allowed": "True"} |
| id | 3 |
| name | m1.medium |
| os-flavor-access:is_public | True |
| ram | 4096 |
| rxtx_factor | 1.0 |
| swap | 0 |
| vcpus | 2 |
+----------------------------+-------------------------------------------------------------------+
启动带有 Inspur FPGA 的虚拟机
在此步骤中，我提供了一些 cyborg-api 日志输出，以展示 Nova 如何与 Cyborg 交互。
Nova 获取 DP 详情
Aug 21 17:33:48 ubuntu devstack@cyborg-api.service[9353]: 2020-08-21 17:33:48.594 9365 INFO
cyborg.api.controllers.v2.device_profiles
[req-e89db88c-80f1-420f-b4b3-a38998f41f98 613b4ba7e86a4058b458e4e9496a0507
fef0949e2ce6415ab5cd001ba254e563 - default default] [device_profiles] get_all returned:
{'device_profiles': [{'name': 'inspur-fpga', 'description': None, 'uuid': '63cac6f6-1122-46d1-825e-
b288886ebdf5', 'groups': [{'resources:FPGA': '1', 'trait:CUSTOM_FPGA_PRODUCT_ID_A115': 'required',
'trait:CUSTOM_FPGA_INSPUR': 'required'}], 'created_at': '2020-08-21 09:13:29+00:00',
'updated_at': 'None', 'links': [{'href': 'http://10.180.249.199/accelerator/v2/device
_profiles/63cac6f6-1122-46d1-825e-b288886ebdf5', 'rel': 'self'}]}]}
ARQ 创建
Aug 18 02:03:10 vca-server devstack@cyborg-api.service[2243671]: 2020-08-18 02:03:10.052 2243674 INFO
cyborg.api.controllers.v2.arqs [req-73c06ce1-b9df-4d94-8760-e4ab5f595328 1f86481df3024d8bb74e213e23dcc2e4
7cd6c62783934a148605e271b4f6f922 - default default] [arq] post req = ({'device_profile_name': 'inspur-fpga'})
ARQ 绑定
Aug 21 17:33:49 ubuntu devstack@cyborg-api.service[9353]: 2020-08-21 17:33:49.980 9360 INFO
cyborg.api.controllers.v2.arqs [req-f92f6792-4730-4cb7-9932-9ca4f393500a 613b4ba7e86a4058b458e4e9496a0507
fef0949e2ce6415ab5cd001ba254e563 - default default] [arqs] patch. list=({'549f40f9-ff45-4b55-8d7d-89f0f6822203': [
{'path': '/hostname', 'op': 'add', 'value': 'ubuntu'}, {'path': '/device_rp_uuid', 'op': 'add',
'value': 'bcedfee6-c1f0-349c-acfa-7d39033b7b8c'}, {'path': '/instance_uuid', 'op': 'add',
'value': '45716b59-7eaf-4663-bd7e- f83416f7ab48'}]})
获取绑定的 ARQ
Aug 21 17:33:51 ubuntu devstack@cyborg-api.service[9353]: 2020-08-21 17:33:51.025 9366 INFO cyborg.api.controllers.v2.arqs [req-d094f906-3969-49d5-88e2-8459a1b72158 613b4ba7e86a4058b458e4e9496a0507 fef0949e2ce6415ab5cd001ba254e563 - default default] [arqs:get_all] Returned:<cyborg.api.controllers.v2.arqs.ARQCollection object at 0x7fc2deb187b8>
检查虚拟机
登录到虚拟机，运行`lspci | grep Inspur`
它将显示一个 pci 设备，这意味着 Inspur FPGA 已成功传递到虚拟机。
[root@host-10-180-249-159 ~]# lspci|grep Inspur 00:05.0 Processing accelerators: Inspur Electronic Information Industry Co., Ltd. Device a115 (rev 04)
删除带有 Inspur FPGA 的虚拟机
在此步骤中，我提供了一些 cyborg 和 nova 日志的输出，以展示 Nova 如何与 Cyborg 交互。
Nova api 删除虚拟机请求
Aug 22 09:54:48 ubuntu devstack@n-api.service[1808]: [pid: 1916|app: 0|req: 25/296] 10.180.249.199 ()
{70 vars in 1494 bytes} [Sat Aug 22 09:54:47 2020]
DELETE /compute/v2.1/servers/45716b59-7eaf-4663-bd7e-f83416f7ab48 => generated 0 bytes
in 404 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0)
Nova compute 开始删除虚拟机
Aug 22 09:54:48 ubuntu nova-compute[20353]: #033[00;32mDEBUG nova.compute.manager [#033[01;36m None req-a307b6cf-101a-4e9c-923f-a6fd4d8f6e6f #033[00;36madmin admin#033[00;32m] #033[01;35m [instance: 45716b59-7eaf-4663-bd7e-f83416f7ab48] #033[00;32mStart destroying the instance on the hypervisor. #033[00m #033[00;33mTemplate:(pid=20353) shutdown instance /opt/stack/nova/nova/compute/manager.py:2837#033[00m#033[00m
Nova compute 开始删除 arq
Aug 22 09:54:50 ubuntu nova-compute[20353]: #033[00;32mDEBUG nova.compute.utils [#033[01;36mNone
req-a307b6cf-101a-4e9c-923f-a6fd4d8f6e6f
#033[00;36madmin admin#033[00;32m] #033[01;35m#033[00;32mCalling Cyborg to delete ARQs for instance
45716b59-7eaf-4663-bd7e-f83416f7ab48#033[00m #033[00;33m{{(pid=20353) delete_arqs_if_needed
/opt/stack/nova/nova/compute/utils.py:1554}}
Cyborg api 开始删除 arq
Aug 22 09:54:50 ubuntu devstack@cyborg-api.service[9353]: 2020-08-22 09:54:50.466 9364 INFO cyborg.api.controllers.v2.arqs [req-3b7aca17-b96c-4d08-9c99-7df8093fc927 613b4ba7e86a4058b458e4e9496a0507 fef0949e2ce6415ab5cd001ba254e563 - default default] [arqs] delete. instance=(45716b59-7eaf-4663-bd7e-f83416f7ab48)
Cyborg api 删除 arq
Aug 22 09:54:50 ubuntu devstack@cyborg-api.service[9353]: 2020-08-22 09:54:50.487 9364 INFO cyborg.objects.ext_arq [req-3b7aca17-b96c-4d08-9c99-7df8093fc927 613b4ba7e86a4058b458e4e9496a0507 fef0949e2ce6415ab5cd001ba254e563 - default default] Deleting obj_extarq uuid 549f40f9-ff45-4b55-8d7d-89f0f6822203 for instance 45716b59-7eaf-4663-bd7e-f83416f7ab48#033[00m
Arq 删除成功
Aug 22 09:54:50 2020] DELETE /accelerator/v2/accelerator_requests?instance= 45716b59-7eaf-4663-bd7e-f83416f7ab48 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 7 headers in 260 bytes (1 switches on core 0)
