Gao Xiang
0005e01e1e
erofs: fix out-of-bound access when z_erofs_gbuf_growsize() partially fails
...
If z_erofs_gbuf_growsize() partially fails on a global buffer due to
memory allocation failure or fault injection (as reported by syzbot [1]),
new pages need to be freed by comparing to the existing pages to avoid
memory leaks.
However, the old gbuf->pages[] array may not be large enough, which can
lead to null-ptr-deref or out-of-bound access.
Fix this by checking against gbuf->nrpages in advance.
[1] https://lore.kernel.org/r/000000000000f7b96e062018c6e3@google.com
Reported-by: syzbot+242ee56aaa9585553766@syzkaller.appspotmail.com
Fixes: d6db47e571stable@vger.kernel.org > # 6.10+
Reviewed-by: Chunhai Guo <guochunhai@vivo.com >
Reviewed-by: Sandeep Dhavale <dhavale@google.com >
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com >
Link: https://lore.kernel.org/r/20240820085619.1375963-1-hsiangkao@linux.alibaba.com
2024-08-21 08:12:05 +08:00
Gao Xiang
7dc5537c3f
erofs: fix race in z_erofs_get_gbuf()
...
In z_erofs_get_gbuf(), the current task may be migrated to another
CPU between `z_erofs_gbuf_id()` and `spin_lock(&gbuf->lock)`.
Therefore, z_erofs_put_gbuf() will trigger the following issue
which was found by stress test:
<2>[772156.434168] kernel BUG at fs/erofs/zutil.c:58!
..
<4>[772156.435007]
<4>[772156.439237] CPU: 0 PID: 3078 Comm: stress Kdump: loaded Tainted: G E 6.10.0-rc7+ #2
<4>[772156.439239] Hardware name: Alibaba Cloud Alibaba Cloud ECS, BIOS 1.0.0 01/01/2017
<4>[772156.439241] pstate: 83400005 (Nzcv daif +PAN -UAO +TCO +DIT -SSBS BTYPE=--)
<4>[772156.439243] pc : z_erofs_put_gbuf+0x64/0x70 [erofs]
<4>[772156.439252] lr : z_erofs_lz4_decompress+0x600/0x6a0 [erofs]
..
<6>[772156.445958] stress (3127): drop_caches: 1
<4>[772156.446120] Call trace:
<4>[772156.446121] z_erofs_put_gbuf+0x64/0x70 [erofs]
<4>[772156.446761] z_erofs_lz4_decompress+0x600/0x6a0 [erofs]
<4>[772156.446897] z_erofs_decompress_queue+0x740/0xa10 [erofs]
<4>[772156.447036] z_erofs_runqueue+0x428/0x8c0 [erofs]
<4>[772156.447160] z_erofs_readahead+0x224/0x390 [erofs]
..
Fixes: f36f3010f6stable@vger.kernel.org > # 6.10+
Reviewed-by: Chunhai Guo <guochunhai@vivo.com >
Reviewed-by: Sandeep Dhavale <dhavale@google.com >
Reviewed-by: Chao Yu <chao@kernel.org >
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com >
Link: https://lore.kernel.org/r/20240722035110.3456740-1-hsiangkao@linux.alibaba.com
2024-07-26 18:47:33 +08:00
Sandeep Dhavale
9d01f6f6d8
erofs: fix possible memory leak in z_erofs_gbuf_exit()
...
Because we incorrectly reused of variable `i` in `z_erofs_gbuf_exit()`
for inner loop, we may exit early from outer loop resulting in memory
leak. Fix this by using separate variable for iterating through inner loop.
Fixes: f36f3010f6dhavale@google.com >
Reviewed-by: Gao Xiang <hsiangkao@linux.alibaba.com >
Reviewed-by: Chao Yu <chao@kernel.org >
Link: https://lore.kernel.org/r/20240624220206.3373197-1-dhavale@google.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com >
2024-06-26 17:02:15 +08:00
Chunhai Guo
0f6273ab46
erofs: add a reserved buffer pool for lz4 decompression
...
This adds a special global buffer pool (in the end) for reserved pages.
Using a reserved pool for LZ4 decompression significantly reduces the
time spent on extra temporary page allocation for the extreme cases in
low memory scenarios.
The table below shows the reduction in time spent on page allocation for
LZ4 decompression when using a reserved pool. The results were obtained
from multi-app launch benchmarks on ARM64 Android devices running the
5.15 kernel with an 8-core CPU and 8GB of memory. In the benchmark, we
launched 16 frequently-used apps, and the camera app was the last one in
each round. The data in the table is the average time of camera app for
each round.
After using the reserved pool, there was an average improvement of 150ms
in the overall launch time of our camera app, which was obtained from
the systrace log.
+--------------+---------------+--------------+---------+
| | w/o page pool | w/ page pool | diff |
+--------------+---------------+--------------+---------+
| Average (ms) | 3434 | 21 | -99.38% |
+--------------+---------------+--------------+---------+
Based on the benchmark logs, 64 pages are sufficient for 95% of
scenarios. This value can be adjusted with a module parameter
`reserved_pages`. The default value is 0.
This pool is currently only used for the LZ4 decompressor, but it can be
applied to more decompressors if needed.
Signed-off-by: Chunhai Guo <guochunhai@vivo.com >
Reviewed-by: Gao Xiang <hsiangkao@linux.alibaba.com >
Link: https://lore.kernel.org/r/20240402131523.2703948-1-guochunhai@vivo.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com >
2024-05-08 17:12:51 +08:00
Chunhai Guo
d6db47e571
erofs: do not use pagepool in z_erofs_gbuf_growsize()
...
Let's use alloc_pages_bulk_array() for simplicity and get rid of
unnecessary pagepool.
Signed-off-by: Chunhai Guo <guochunhai@vivo.com >
Reviewed-by: Gao Xiang <hsiangkao@linux.alibaba.com >
Link: https://lore.kernel.org/r/20240402092757.2635257-1-guochunhai@vivo.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com >
2024-05-08 17:12:50 +08:00
Chunhai Guo
f36f3010f6
erofs: rename per-CPU buffers to global buffer pool and make it configurable
...
It will cost more time if compressed buffers are allocated on demand for
low-latency algorithms (like lz4) so EROFS uses per-CPU buffers to keep
compressed data if in-place decompression is unfulfilled. While it is kind
of wasteful of memory for a device with hundreds of CPUs, and only a small
number of CPUs concurrently decompress most of the time.
This patch renames it as 'global buffer pool' and makes it configurable.
This allows two or more CPUs to share a common buffer to reduce memory
occupation.
Suggested-by: Gao Xiang <xiang@kernel.org >
Reviewed-by: Gao Xiang <hsiangkao@linux.alibaba.com >
Signed-off-by: Chunhai Guo <guochunhai@vivo.com >
Link: https://lore.kernel.org/r/20240402100036.2673604-1-guochunhai@vivo.com
Signed-off-by: Sandeep Dhavale <dhavale@google.com >
Link: https://lore.kernel.org/r/20240408215231.3376659-1-dhavale@google.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com >
2024-05-08 17:12:49 +08:00
Chunhai Guo
cacd5b04e2
erofs: rename utils.c to zutil.c
...
Currently, utils.c is only useful if CONFIG_EROFS_FS_ZIP is on.
So let's rename it to zutil.c as well as avoid its inclusion if
CONFIG_EROFS_FS_ZIP is explicitly disabled.
Signed-off-by: Chunhai Guo <guochunhai@vivo.com >
Reviewed-by: Gao Xiang <hsiangkao@linux.alibaba.com >
Link: https://lore.kernel.org/r/20240401135550.2550043-1-guochunhai@vivo.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com >
2024-05-08 17:12:49 +08:00
