The timeout threshold of the hpre cluster is 16ms. When the CPU
and device share virtual address, page fault processing time may
exceed the threshold.
In the current test, there is a high probability that the
cluster times out. However, the cluster is waiting for the
completion of memory access, which is not an error, the device
does not need to be reset. If an error occurs in the cluster,
qm also reports the error. Therefore, the cluster timeout
error of hpre can be masked.
Fixes: d90fab0deb ("crypto: hisilicon/qm - get error type from hardware registers")
Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Before the device is enabled again, the device may still
store the previously processed data. If an error occurs in
the previous task, the device may fail to be enabled again.
Therefore, before enabling device, reset the device to restore
the initial state.
Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Apply for a lock before the qp send operation to ensure no
resource race in multi-concurrency situations.
This modification has almost no impact on performance.
Signed-off-by: Chenghai Huang <huangchenghai2@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Some information showed by the dump function is invalid. Mask
the unnecessary information from the dump file.
Signed-off-by: Chenghai Huang <huangchenghai2@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
There is a scenario where the file directory is created but the
file memory is not set. In this case, if a user accesses the
file, an error occurs.
So during the creation process of debugfs, memory should be
allocated first before creating the directory. In the release
process, the directory should be deleted first before releasing
the memory to avoid the situation where the memory does not
exist when accessing the directory.
In addition, the directory released by the debugfs is a global
variable. When the debugfs of an accelerator fails to be
initialized, releasing the directory of the global variable
affects the debugfs initialization of other accelerators.
The debugfs root directory released by debugfs init should be a
member of qm, not a global variable.
Signed-off-by: Chenghai Huang <huangchenghai2@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Switch to raw_smp_processor_id() to prevent a number of
warnings from kernel debugging. We do not care about
preemption here, as the CPU number is only used as a
poor mans load balancing or device selection. If preemption
happens during an encrypt/decrypt operation a small performance
hit will occur but everything will continue to work, so just
ignore it. This commit is similar to e7a9b05ca4
("crypto: cavium - Fix smp_processor_id() warnings").
[ 7538.874350] BUG: using smp_processor_id() in preemptible [00000000] code: af_alg06/8438
[ 7538.874368] caller is debug_smp_processor_id+0x1c/0x28
[ 7538.874373] CPU: 50 PID: 8438 Comm: af_alg06 Kdump: loaded Not tainted 5.10.0.pc+ #18
[ 7538.874377] Call trace:
[ 7538.874387] dump_backtrace+0x0/0x210
[ 7538.874389] show_stack+0x2c/0x38
[ 7538.874392] dump_stack+0x110/0x164
[ 7538.874394] check_preemption_disabled+0xf4/0x108
[ 7538.874396] debug_smp_processor_id+0x1c/0x28
[ 7538.874406] sec_create_qps+0x24/0xe8 [hisi_sec2]
[ 7538.874408] sec_ctx_base_init+0x20/0x4d8 [hisi_sec2]
[ 7538.874411] sec_aead_ctx_init+0x68/0x180 [hisi_sec2]
[ 7538.874413] sec_aead_sha256_ctx_init+0x28/0x38 [hisi_sec2]
[ 7538.874421] crypto_aead_init_tfm+0x54/0x68
[ 7538.874423] crypto_create_tfm_node+0x6c/0x110
[ 7538.874424] crypto_alloc_tfm_node+0x74/0x288
[ 7538.874426] crypto_alloc_aead+0x40/0x50
[ 7538.874431] aead_bind+0x50/0xd0
[ 7538.874433] alg_bind+0x94/0x148
[ 7538.874439] __sys_bind+0x98/0x118
[ 7538.874441] __arm64_sys_bind+0x28/0x38
[ 7538.874445] do_el0_svc+0x88/0x258
[ 7538.874447] el0_svc+0x1c/0x28
[ 7538.874449] el0_sync_handler+0x8c/0xb8
[ 7538.874452] el0_sync+0x148/0x180
Signed-off-by: Wenkai Lin <linwenkai6@hisilicon.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Pre-store the valid value of hpre alg support related capability
register in hpre_qm_init(), which will be called by hpre_probe().
It can reduce the number of capability register queries and avoid
obtaining incorrect values in abnormal scenarios, such as reset
failed and the memory space disabled.
Fixes: f214d59a06 ("crypto: hisilicon/hpre - support hpre capability")
Signed-off-by: Zhiqi Song <songzhiqi1@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
When the Kunpeng accelerator executes tasks such as encryption
and decryption have minimum requirements on the number of device
queues. If the number of queues does not meet the requirement,
the process initialization will fail. Therefore, the driver checks
the number of queues on the device before registering the algorithm.
If the number does not meet the requirements, the driver does not register
the algorithm to crypto subsystem, the device is still added to the
qm_list.
Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
If the queue isolation feature is enabled, the number of queues
supported by the device changes. When PF is enabled using the
current default number of queues, the default number of queues may
be greater than the number supported by the device. As a result,
the PF fails to be bound to the driver.
After modification, if queue isolation feature is enabled, when
the default queue parameter is greater than the number supported
by the device, the number of enabled queues will be changed to
the number supported by the device, so that the PF and driver
can be properly bound.
Fixes: 8bbecfb402 ("crypto: hisilicon/qm - add queue isolation support for Kunpeng930")
Signed-off-by: Longfang Liu <liulongfang@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This error handling looks really strange.
Check if the string has been truncated instead.
Fixes: 02ab994635 ("crypto: hisilicon - Fixed some tiny bugs of HPRE")
Signed-off-by: Christophe JAILLET <christophe.jaillet@wanadoo.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Enable sva error interrupt event. When an error occurs on
the sva module, the device reports an abnormal interrupt to
the driver.
Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The private key of the curve key size generated by stdrng, which maybe
not less than n. Therefore, the private key with the curve key size
minus 1 is generated to ensure that the private key is less than n.
Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This driver has been implicitly relying on kmalloc alignment
to be sufficient for DMA. This may no longer be the case with
upcoming arm64 changes.
This patch changes it to explicitly request DMA alignment from
the Crypto API.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The last register logic and different register logic are combined.
Use "u32" instead of 'int' in the regs function input parameter to
simplify some checks.
Signed-off-by: Kai Ye <yekai13@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
In hpre_remove(), when the disable operation of qm sriov failed,
the following logic should continue to be executed to release the
remaining resources that have been allocated, instead of returning
directly, otherwise there will be resource leakage.
Signed-off-by: Zhiqi Song <songzhiqi1@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The value of qm algorithm can change dynamically according to the
value of the capability register.
Add xxx_set_qm_algs() function to obtain the algs that the
hardware device supported from the capability register and set
them into usr mode attribute files.
Signed-off-by: Zhiqi Song <songzhiqi1@huawei.com>
Signed-off-by: Wenkai Lin <linwenkai6@hisilicon.com>
Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Use table to store the different ecdh curve configuration,
making the registration of ecdh clearer and expansion more
convenient.
Signed-off-by: Zhiqi Song <songzhiqi1@huawei.com>
Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Hardware V3 and later versions support get error type from
registers. To be compatible with later hardware versions,
get error type from registers instead of fixed marco.
Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Hardware V3 and later versions can obtain qp num and depth supported
by the hardware from registers. To be compatible with later hardware
versions, get qp num and depth from registers instead of fixed marcos.
Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Before hardware V3, hardwares do not provide the feature registers,
driver resolves hardware differences based on the hardware version.
As a result, the driver does not support the new hardware.
Hardware V3 and later versions support to obtain hardware features,
such as power-gating management and doorbell isolation, through
the hardware registers. To be compatible with later hardware versions,
the features of the current device is obtained by reading the
hardware registers instead of the hardware version.
Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The hpre encryption driver may be used to encrypt and decrypt packets
during the rx softirq, it is not allowed to use GFP_KERNEL.
Fixes: c8b4b47707 ("crypto: hisilicon - add HiSilicon HPRE accelerator")
Signed-off-by: Zhengchao Shao <shaozhengchao@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
1. Add some debugging registers.
2. Add last word dumping function during hpre engine controller reset.
Signed-off-by: Kai Ye <yekai13@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The value of the register is changed after the task running. A debugfs
file node is added to help users to check the change of register values.
Signed-off-by: Longfang Liu <liulongfang@huawei.com>
Signed-off-by: Kai Ye <yekai13@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
hpre_curve25519_src_init() allocates memory for 'ptr' before calling
memcmp(). If memcmp() returns 0, the function will return '-EINVAL'
without freeing memory.
Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The accelerator devices support runtime PM, when device is in suspended, an
exception will occur if reading registers. Therefore, this patch uses
'debugfs_create_file' instead of 'debugfs_create_regset32' to create
debugfs file, and then the driver can get the device status before
reading the register.
Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Kunpeng930 hpre device supports dynamic clock gating. When doing tasks,
the algorithm core is opened, and when idle, the algorithm core is closed.
This patch enables hpre dynamic clock gating by writing hardware registers.
Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The HPRE driver support configure each function's QoS in the Host
for Kunpeng930. The HPRE driver needs to configure the maximum shaper
type rate.
Signed-off-by: Kai Ye <yekai13@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The result of gx modulo p is zero if gx is equal to p, so return
error immediately if gx is equal to p.
Signed-off-by: Hui Tang <tanghui20@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
When the key length is zero, use stdrng to generate private key
to pass the crypto ecdh-nist-p256 self test on vector 2.
Signed-off-by: Hui Tang <tanghui20@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Kunpeng930 hardware supports address prefetching to improve performance
before doing tasks in SVA scenario.
This patch enables this function in device initialization by writing
hardware registers. In the process of reset, address prefetching is
disabled to avoid the failure of interaction between accelerator device
and SMMU.
Signed-off-by: Longfang Liu <liulongfang@huawei.com>
Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Kunpeng930 could be able to close master ooo when NFE occurs, which will
disable memory accessing from device and execute tasks. This ensures that
errors do not spread.
This patch enables the hardware to close master ooo when an error occurs
by writing hardware registers, and ensures that the driver will not drain
qp because the hardware will empty the tasks automatically.
Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
In order to avoid reporting an exception but the error type is not
configured, the driver needs to configure the error type first, and then
enable the error interrupt. Before executing the task, hardware error
initialization is needed so that the hardware can detect the error in time.
Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Functional macro lacks type checking, which is not as strict as function
call checking.
Signed-off-by: Hui Tang <tanghui20@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
