PROCESSOR-SDK-AM62X: Cryptodev硬件加速器不生效

1. 确保Cryptodev硬件加速器已经正确安装在AM62X处理器上。检查硬件连接和驱动程序安装是否正确。

2. 检查Cryptodev硬件加速器的配置和设置。确保硬件加速器已经正确配置并启用。

3. 检查软件应用程序的代码和配置。确保应用程序正确地使用Cryptodev硬件加速器进行加密和解密操作。

4. 检查系统日志和错误信息。查看系统日志和错误信息以了解是否有任何与Cryptodev硬件加速器相关的问题。

1、可以确定Cryptodev驱动程序配置和加载无误，应用程序配置也没有问题，因为openssl可以正常使用devcrypto引擎，目前运行openssl测速命令中也没有任何报错。

2、目前怀疑是SA2UL驱动的问题，虽然该驱动存在并加载，但是该驱动里的算法似乎没有实现，在proc/crypto里无法找到SA2UL驱动里实现的算法，所以查看

硬件引擎支持的算法也是无可用的：

root@starcharge:/# openssl engine -t -c

(dynamic) Dynamic engine loading support

    [ unavailable ]

(devcrypto) /dev/crypto engine

    [ available ]

你好，感谢您的回复。

1、系统日志里没有任何与加密相关的错误或警告信息。

2、proc/crypto中无法找到SA2UL驱动实现的算法请问是否是SA2UL驱动的问题，如果认为是硬件引擎没有正确配置和初始化的问题，请问硬件引擎的配置文件需要怎么配置才能确保它们正确加载和初始化？

目前我的疑问就是为什么使用贵司的SDK参考贵司的安全硬件加速驱动的用户指南，硬件加速不生效？3.2.2.3. Crypto — 处理器SDK AM62x文档 (ti.com)

我们的软件SDK是ti-processor-sdk-linux-am62xx-evm-08.06.00.42，请问是否可以帮忙排查明确问题点？

为您咨询了产品线工程师，这需要一些时间

您可以在“time -v”的帮助下测试详细输出

# with sa2ul
time -v openssl speed -evp aes-128-cbc -engine devcrypto

# without sa2ul
time -v openssl speed -evp aes-128-cbc

作为参考，以下是在我的 TI AM62X HSFS 板上完成的测试的日志。日志显示，当 openssl 使用 sa2ul 引擎时，CPU 使用率降低了近 50%。

root@am62xx-evm:~# time -v openssl speed -evp aes-128-cbc -engine devcrypto
Engine "devcrypto" set.
Doing AES-128-CBC for 3s on 16 size blocks: 63085 AES-128-CBC's in 0.10s
Doing AES-128-CBC for 3s on 64 size blocks: 66531 AES-128-CBC's in 0.08s
Doing AES-128-CBC for 3s on 256 size blocks: 59120 AES-128-CBC's in 0.07s
Doing AES-128-CBC for 3s on 1024 size blocks: 49130 AES-128-CBC's in 0.02s
Doing AES-128-CBC for 3s on 8192 size blocks: 22171 AES-128-CBC's in 0.04s
Doing AES-128-CBC for 3s on 16384 size blocks: 12943 AES-128-CBC's in 0.03s
version: 3.0.12
built on: Tue Oct 24 13:48:41 2023 UTC
options: bn(64,64)
compiler: aarch64-oe-linux-gcc  -mbranch-protection=standard --sysroot=recipe-sysroot -O2 -pipe -g -feliminate-unused-debug-types -fmacro-prefix-map=                      -fdebug-prefix-map=                      -fdebug-prefix-map=                      -fdebug-prefix-map=  -DOPENSSL_USE_NODELETE -DOPENSSL_PIC -DOPENSSL_BUILDING_OPENSSL -DNDEBUG
CPUINFO: OPENSSL_armcap=0xbd
The 'numbers' are in 1000s of bytes per second processed.
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes  16384 bytes
AES-128-CBC      10093.60k    53224.80k   216210.29k  2515456.00k  4540620.80k  7068603.73k
        Command being timed: "openssl speed -evp aes-128-cbc -engine devcrypto"
        User time (seconds): 0.37
        System time (seconds): 7.49
        Percent of CPU this job got: 43%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 0m 18.16s
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 23744
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 0
        Minor (reclaiming a frame) page faults: 314
        Voluntary context switches: 273070
        Involuntary context switches: 29
        Swaps: 0
        File system inputs: 0
        File system outputs: 0
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 0
root@am62xx-evm:~# time -v openssl speed -evp aes-128-cbc
Doing AES-128-CBC for 3s on 16 size blocks: 22747996 AES-128-CBC's in 2.99s
Doing AES-128-CBC for 3s on 64 size blocks: 16963282 AES-128-CBC's in 2.99s
Doing AES-128-CBC for 3s on 256 size blocks: 8155871 AES-128-CBC's in 3.00s
Doing AES-128-CBC for 3s on 1024 size blocks: 2753497 AES-128-CBC's in 2.99s
Doing AES-128-CBC for 3s on 8192 size blocks: 382388 AES-128-CBC's in 3.00s
Doing AES-128-CBC for 3s on 16384 size blocks: 192330 AES-128-CBC's in 2.99s
version: 3.0.12
built on: Tue Oct 24 13:48:41 2023 UTC
options: bn(64,64)
compiler: aarch64-oe-linux-gcc  -mbranch-protection=standard --sysroot=recipe-sysroot -O2 -pipe -g -feliminate-unused-debug-types -fmacro-prefix-map=                      -fdebug-prefix-map=                      -fdebug-prefix-map=                      -fdebug-prefix-map=  -DOPENSSL_USE_NODELETE -DOPENSSL_PIC -DOPENSSL_BUILDING_OPENSSL -DNDEBUG
CPUINFO: OPENSSL_armcap=0xbd
The 'numbers' are in 1000s of bytes per second processed.
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes  16384 bytes
AES-128-CBC     121728.41k   363093.66k   695967.66k   943003.65k  1044174.17k  1053891.21k
        Command being timed: "openssl speed -evp aes-128-cbc"
        User time (seconds): 17.97
        System time (seconds): 0.00
        Percent of CPU this job got: 99%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 0m 18.02s
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 23536
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 0
        Minor (reclaiming a frame) page faults: 307
        Voluntary context switches: 1
        Involuntary context switches: 243
        Swaps: 0
        File system inputs: 0
        File system outputs: 0
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 0
root@am62xx-evm:~#

您好，

测试的完整输出如下，使用引擎和不使用引擎的输出基本一致，并没有降低CPU使用率。

root@starcharge:/# time -v openssl speed -evp aes-128-cbc
Doing AES-128-CBC for 3s on 16 size blocks: 16880834 AES-128-CBC's in 2.98s
Doing AES-128-CBC for 3s on 64 size blocks: 13149722 AES-128-CBC's in 2.99s
Doing AES-128-CBC for 3s on 256 size blocks: 6723700 AES-128-CBC's in 2.99s
Doing AES-128-CBC for 3s on 1024 size blocks: 2380492 AES-128-CBC's in 2.98s
Doing AES-128-CBC for 3s on 8192 size blocks: 339638 AES-128-CBC's in 2.99s
Doing AES-128-CBC for 3s on 16384 size blocks: 171439 AES-128-CBC's in 2.99s
version: 3.1.4
built on: Mon Dec 11 08:59:12 2023 UTC
options: bn(64,64)
compiler: aarch64-buildroot-linux-gnu-gcc -fPIC -pthread -Wa,--noexecstack -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -Os -g0 -D_FORTIFY_SOURCE=1 -DOPENSSL_USE_NODELETE -DOPENSSL_PIC -DOPENSSL_BUILDING_OPENSSL -DZLIB -DZLIB_SHARED -DNDEBUG -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64
CPUINFO: OPENSSL_armcap=0xbd
The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes 16384 bytes
AES-128-CBC 90635.35k 281465.62k 575674.65k 817994.57k 930539.97k 939416.92k
Command being timed: "openssl speed -evp aes-128-cbc"
User time (seconds): 17.93
System time (seconds): 0.01
Percent of CPU this job got: 99%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0m 18.04s
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 22464
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 13
Minor (reclaiming a frame) page faults: 316
Voluntary context switches: 9
Involuntary context switches: 382
Swaps: 0
File system inputs: 0
File system outputs: 0
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 0
root@starcharge:/# lsmod |grep sa2ul
sa2ul 40960 0 - Live 0xffff800008b90000
sha512_generic 16384 1 sa2ul, Live 0xffff800008b8b000
authenc 16384 1 sa2ul, Live 0xffff800008b60000
root@starcharge:/# insmod media/mmcblk0p1/cryptodev.ko
root@starcharge:/# time -v openssl speed -evp aes-128-cbc -engine devcrypto
Engine "devcrypto" set.
Doing AES-128-CBC for 3s on 16 size blocks: 17034714 AES-128-CBC's in 2.99s
Doing AES-128-CBC for 3s on 64 size blocks: 13238269 AES-128-CBC's in 2.99s
Doing AES-128-CBC for 3s on 256 size blocks: 6746281 AES-128-CBC's in 2.98s
Doing AES-128-CBC for 3s on 1024 size blocks: 2381812 AES-128-CBC's in 2.99s
Doing AES-128-CBC for 3s on 8192 size blocks: 339715 AES-128-CBC's in 2.99s
Doing AES-128-CBC for 3s on 16384 size blocks: 171454 AES-128-CBC's in 2.98s
version: 3.1.4
built on: Mon Dec 11 08:59:12 2023 UTC
options: bn(64,64)
compiler: aarch64-buildroot-linux-gnu-gcc -fPIC -pthread -Wa,--noexecstack -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -Os -g0 -D_FORTIFY_SOURCE=1 -DOPENSSL_USE_NODELETE -DOPENSSL_PIC -DOPENSSL_BUILDING_OPENSSL -DZLIB -DZLIB_SHARED -DNDEBUG -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64
CPUINFO: OPENSSL_armcap=0xbd
The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes 16384 bytes
AES-128-CBC 91155.66k 283360.94k 579546.29k 815710.87k 930750.93k 942651.79k
Command being timed: "openssl speed -evp aes-128-cbc -engine devcrypto"
User time (seconds): 17.93
System time (seconds): 0.00
Percent of CPU this job got: 96%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0m 18.67s
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 23024
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 3
Minor (reclaiming a frame) page faults: 331
Voluntary context switches: 46
Involuntary context switches: 419
Swaps: 0
File system inputs: 0
File system outputs: 0
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 0
root@starcharge:/#

另外opnessl是有命令可以查看引擎支持的硬件算法的，我们AM62X开发板显示没有支持的硬件算法，都是软件算法，如下：

root@starcharge:/# openssl engine -pre DUMP_INFO devcrypto
(devcrypto) /dev/crypto engine
Information about ciphers supported by the /dev/crypto engine:
Cipher DES-CBC, NID=31, /dev/crypto info: id=1, driver=cbc(des-generic) (software)
Cipher DES-EDE3-CBC, NID=44, /dev/crypto info: id=2, driver=cbc(des3_ede-generic) (software)
Cipher BF-CBC, NID=91, /dev/crypto info: id=3, CIOCGSESSION (session open call) failed
Cipher CAST5-CBC, NID=108, /dev/crypto info: id=4, CIOCGSESSION (session open call) failed
Cipher AES-128-CBC, NID=419, /dev/crypto info: id=11, driver=cbc-aes-ce (software)
Cipher AES-192-CBC, NID=423, /dev/crypto info: id=11, driver=cbc-aes-ce (software)
Cipher AES-256-CBC, NID=427, /dev/crypto info: id=11, driver=cbc-aes-ce (software)
Cipher RC4, NID=5, /dev/crypto info: id=12, CIOCGSESSION (session open call) failed
Cipher AES-128-CTR, NID=904, /dev/crypto info: id=21, driver=ctr-aes-ce (software)
Cipher AES-192-CTR, NID=905, /dev/crypto info: id=21, driver=ctr-aes-ce (software)
Cipher AES-256-CTR, NID=906, /dev/crypto info: id=21, driver=ctr-aes-ce (software)
Cipher AES-128-ECB, NID=418, /dev/crypto info: id=23, driver=ecb-aes-ce (software)
Cipher AES-192-ECB, NID=422, /dev/crypto info: id=23, driver=ecb-aes-ce (software)
Cipher AES-256-ECB, NID=426, /dev/crypto info: id=23, driver=ecb-aes-ce (software)
Cipher CAMELLIA-128-CBC, NID=751, /dev/crypto info: id=101, CIOCGSESSION (session open call) failed
Cipher CAMELLIA-192-CBC, NID=752, /dev/crypto info: id=101, CIOCGSESSION (session open call) failed
Cipher CAMELLIA-256-CBC, NID=753, /dev/crypto info: id=101, CIOCGSESSION (session open call) failed

Information about digests supported by the /dev/crypto engine:
Digest MD5, NID=4, /dev/crypto info: id=13, driver=md5-generic (software), CIOCCPHASH capable
Digest SHA1, NID=64, /dev/crypto info: id=14, driver=sha1-ce (software), CIOCCPHASH capable
Digest RIPEMD160, NID=117, /dev/crypto info: id=102, driver=unknown. CIOCGSESSION (session open) failed
Digest SHA224, NID=675, /dev/crypto info: id=103, driver=sha224-ce (software), CIOCCPHASH capable
Digest SHA256, NID=672, /dev/crypto info: id=104, driver=sha256-ce (software), CIOCCPHASH capable
Digest SHA384, NID=673, /dev/crypto info: id=105, driver=sha384-generic (software), CIOCCPHASH capable
Digest SHA512, NID=674, /dev/crypto info: id=106, driver=sha512-generic (software), CIOCCPHASH capable

[Success]: DUMP_INFO

已跟进

看起来SA2UL硬件加速不起作用。

我们可以暂时从openssl转移到cryptodev驱动程序(board-support/extra-drivers/cryptodev-module-*/tests)中提供的更简单的密码测试。所以只需将以下两个文件复制到目标文件系统:1)cipher.c & 2) testhelper.h。然后，执行如下操作:

root@am62xx-evm:~# ls
cipher.c  testhelper.h
root@am62xx-evm:~# gcc -o cipher cipher.c
root@am62xx-evm:~# modprobe -r sa2ul
root@am62xx-evm:~# ./cipher -v
requested cipher CRYPTO_AES_CBC, got cbc(aes) with driver cbc-aes-ce
AES Test passed
requested cipher CRYPTO_AES_CBC, got cbc(aes) with driver cbc-aes-ce
requested cipher CRYPTO_AES_CBC, got cbc(aes) with driver cbc-aes-ce
Test passed
root@am62xx-evm:~# modprobe sa2ul
root@am62xx-evm:~# ./cipher -v
requested cipher CRYPTO_AES_CBC, got cbc(aes) with driver cbc-aes-sa2ul
AES Test passed
requested cipher CRYPTO_AES_CBC, got cbc(aes) with driver cbc-aes-sa2ul
requested cipher CRYPTO_AES_CBC, got cbc(aes) with driver cbc-aes-sa2ul
Test passed
root@am62xx-evm:~#

从上面的日志可以看出，去使能SA2UL模块时使用软件算法进行测试，加载SA2UL模块时使用硬件加速器进行测试。

Hi Gary

根据你给出的测试操作如下：

root@starcharge:/media/mmcblk0p1/crypto# modprobe -r sa2ul
root@starcharge:/media/mmcblk0p1/crypto# ./cipher -v
-sh: ./cipher: Permission denied
root@starcharge:/media/mmcblk0p1/crypto# chmod +x cipher
root@starcharge:/media/mmcblk0p1/crypto# ./cipher -v
open(/dev/crypto): No such file or directory
root@starcharge:/media/mmcblk0p1/crypto# insmod cryptodev.ko
root@starcharge:/media/mmcblk0p1/crypto# ./cipher -v
requested cipher CRYPTO_AES_CBC, got cbc(aes) with driver cbc-aes-ce
AES Test passed
requested cipher CRYPTO_AES_CBC, got cbc(aes) with driver cbc-aes-ce
requested cipher CRYPTO_AES_CBC, got cbc(aes) with driver cbc-aes-ce
Test passed
root@starcharge:/media/mmcblk0p1/crypto# modprobe sa2ul
root@starcharge:/media/mmcblk0p1/crypto# ./cipher -v
requested cipher CRYPTO_AES_CBC, got cbc(aes) with driver cbc-aes-ce
AES Test passed
requested cipher CRYPTO_AES_CBC, got cbc(aes) with driver cbc-aes-ce
requested cipher CRYPTO_AES_CBC, got cbc(aes) with driver cbc-aes-ce
Test passed

从上面日志可以看出，无论是否加载SA2UL模块都只有软件算法，这正如我之前和您说的里无法找到SA2UL驱动里实现的算法，如下

root@starcharge:/# lsmod |grep sa2ul
sa2ul 40960 0 - Live 0xffff800008d41000
sha512_generic 16384 1 sa2ul, Live 0xffff800008d36000
authenc 16384 1 sa2ul, Live 0xffff800008b51000
root@starcharge:/# cat proc/crypto |grep sa2ul
root@starcharge:/#

已跟进，由于美国假期，产品线工程师回复可能会推迟到1月2日。谢谢您的耐心等待。

Hello，

看起来主要是SA2UL驱动程序本身的问题。

您是否修改了SA2UL驱动程序(board-support/ti-linux-kernel/drivers/crypto/ SA2UL .c)或原样使用它吗?如果需要修改，请确保正确填写支持的算法字段。

static struct sa_match_data am64_match_data = {
	.priv = 0,
	.priv_id = 0,
	.supported_algos = BIT(SA_ALG_CBC_AES) |
			   BIT(SA_ALG_EBC_AES) |
			   BIT(SA_ALG_SHA256) |
			   BIT(SA_ALG_SHA512) |
			   BIT(SA_ALG_AUTHENC_SHA256_AES),
};

static const struct of_device_id of_match[] = {
	{ .compatible = "ti,j721e-sa2ul", .data = &am654_match_data, },
	{ .compatible = "ti,am654-sa2ul", .data = &am654_match_data, },
	{ .compatible = "ti,am64-sa2ul", .data = &am64_match_data, },
	{ .compatible = "ti,am62-sa3ul", .data = &am64_match_data, },
	{},
};

Hi Gary

我们使用SA2UL的驱动就是SDK 08.06.00.42的原版驱动，并没有改变，如附件。sa2ul.c

已跟进

您好，

假设正在PSDK v8.6上工作。我在想我们在讨论PSDK v9，所以我在做同样的测试。现在我测试了PSDK v8.6预构建的WIC映像，我确实看到SA2UL不起作用，这意味着问题可能会重现。

然后，我开始研究PSDK v8.6和PSDK v9之间的差异，并偶然发现了加密节点中的以下差异

首先，我简单地将加密节点从PSDK v9复制粘贴到PSDK v8.6。这导致SA2UL驱动程序工作，这意味着密码测试使用cbc-aes-sa2ul驱动程序进行测试。那么PSDK v8.6 DTS中最小的变化也是有效的

diff --git a/board-support/linux-5.10.168+gitAUTOINC+2c23e6c538-g2c23e6c538/arch/arm64/boot/dts/ti/k3-am62-main.dtsi b/board-support/linux-5.10.168+gitAUTOINC+2c23e6c538-g2c23e6c538/arch/arm64/boot/dts/ti/k3-am62-main.dtsi
index 510775e9a..1689b8278 100644
--- a/board-support/linux-5.10.168+gitAUTOINC+2c23e6c538-g2c23e6c538/arch/arm64/boot/dts/ti/k3-am62-main.dtsi
+++ b/board-support/linux-5.10.168+gitAUTOINC+2c23e6c538-g2c23e6c538/arch/arm64/boot/dts/ti/k3-am62-main.dtsi
@@ -179,7 +179,6 @@
 	crypto: crypto@40900000 {
 		compatible = "ti,am62-sa3ul";
 		reg = <0x00 0x40900000 0x00 0x1200>;
-		power-domains = <&k3_pds 70 TI_SCI_PD_SHARED>;
 		#address-cells = <2>;
 		#size-cells = <2>;
 		ranges = <0x00 0x40900000 0x00 0x40900000 0x00 0x30000>;

Hi

感谢回复。我在加密节点中尝试了上述更改，SA2UL可以正常工作。我在内核社区也发现了这个修改https://lore.kernel.org/all/20230417133308.1990057-1-kamlesh@ti.com/，原因是”With latest firmware update, we can no longer control power of SA3UL from main domain.Remove power-domains property from the crypto node.”

我目前还有个疑问，根据AM62X的data sheet，AM62X支持的hash硬件加速算法有SHA224、256、384和512，但是sa2ul驱动中没有SHA224和SHA384，crypto的集成文档中也没有SHA224和SHA384，请问SHA224和SHA384是否支持硬件加速？