Song Liu 20b6cc34ea bpf: Avoid hashtab deadlock with map_locked ...

If a hashtab is accessed in both non-NMI and NMI context, the system may
deadlock on bucket->lock. Fix this issue with percpu counter map_locked.
map_locked rejects concurrent access to the same bucket from the same CPU.
To reduce memory overhead, map_locked is not added per bucket. Instead,
8 percpu counters are added to each hashtab. buckets are assigned to these
counters based on the lower bits of its hash.

Signed-off-by: Song Liu <songliubraving@fb.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://lore.kernel.org/bpf/20201029071925.3103400-3-songliubraving@fb.com

2020-10-30 13:03:29 -07:00

..

preload

bpf: Move prog->aux->linked_prog and trampoline into bpf_link on attach

2020-09-29 13:09:23 -07:00

arraymap.c

bpf: Allow for map-in-map with dynamic inner array map entries

2020-10-11 10:21:04 -07:00

bpf_inode_storage.c

bpf: Allow specifying a BTF ID per argument in function protos

2020-09-21 15:00:40 -07:00

bpf_iter.c

bpf: Permit cond_resched for some iterators

2020-10-28 14:54:31 -07:00

bpf_local_storage.c

bpf: Use hlist_add_head_rcu when linking to local_storage

2020-09-19 01:12:35 +02:00

bpf_lru_list.c

treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 206

2019-05-30 11:29:53 -07:00

bpf_lru_list.h

bpf: Fix a typo "inacitve" -> "inactive"

2020-04-06 21:54:10 +02:00

bpf_lsm.c

bpf: Change bpf_sk_storage_*() to accept ARG_PTR_TO_BTF_ID_SOCK_COMMON

2020-09-25 13:58:01 -07:00

bpf_struct_ops_types.h

bpf: tcp: Support tcp_congestion_ops in bpf

2020-01-09 08:46:18 -08:00

bpf_struct_ops.c

bpf: Move btf_resolve_size into __btf_resolve_size

2020-08-25 15:37:41 -07:00

btf.c

bpf: Introduce bpf_per_cpu_ptr()

2020-10-02 15:00:49 -07:00

cgroup.c

Merge branch 'work.set_fs' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs

2020-10-22 09:59:21 -07:00

core.c

Merge tag 'net-next-5.10' of git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next

2020-10-15 18:42:13 -07:00

cpumap.c

bpf, cpumap: Remove rcpu pointer from cpu_map_build_skb signature

2020-09-28 23:30:42 +02:00

devmap.c

bpf: {cpu,dev}map: Change various functions return type from int to void

2020-09-01 15:45:58 +02:00

disasm.c

treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 295

2019-06-05 17:36:38 +02:00

disasm.h

treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 295

2019-06-05 17:36:38 +02:00

dispatcher.c

bpf: Remove bpf_image tree

2020-03-13 12:49:52 -07:00

hashtab.c

bpf: Avoid hashtab deadlock with map_locked

2020-10-30 13:03:29 -07:00

helpers.c

bpf: Introducte bpf_this_cpu_ptr()

2020-10-02 15:00:49 -07:00

inode.c

Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net

2020-09-22 16:45:34 -07:00

local_storage.c

bpf/local_storage: Fix build without CONFIG_CGROUP

2020-07-25 20:16:36 -07:00

lpm_trie.c

bpf: Add map_meta_equal map ops

2020-08-28 15:41:30 +02:00

Makefile

bpf: Implement bpf_local_storage for inodes

2020-08-25 15:00:04 -07:00

map_in_map.c

bpf: Relax max_entries check for most of the inner map types

2020-08-28 15:41:30 +02:00

map_in_map.h

bpf: Add map_meta_equal map ops

2020-08-28 15:41:30 +02:00

map_iter.c

bpf: Implement link_query callbacks in map element iterators

2020-08-21 14:01:39 -07:00

net_namespace.c

bpf: Add support for forced LINK_DETACH command

2020-08-01 20:38:28 -07:00

offload.c

bpf, offload: Replace bitwise AND by logical AND in bpf_prog_offload_info_fill

2020-02-17 16:53:49 +01:00

percpu_freelist.c

bpf: Use raw_spin_trylock() for pcpu_freelist_push/pop in NMI

2020-10-06 00:04:11 +02:00

percpu_freelist.h

bpf: Use raw_spin_trylock() for pcpu_freelist_push/pop in NMI

2020-10-06 00:04:11 +02:00

prog_iter.c

bpf: Refactor bpf_iter_reg to have separate seq_info member

2020-07-25 20:16:32 -07:00

queue_stack_maps.c

bpf: Add map_meta_equal map ops

2020-08-28 15:41:30 +02:00

reuseport_array.c

bpf, net: Rework cookie generator as per-cpu one

2020-09-30 11:50:35 -07:00

ringbuf.c

bpf: Add map_meta_equal map ops

2020-08-28 15:41:30 +02:00

stackmap.c

bpf: Allow specifying a BTF ID per argument in function protos

2020-09-21 15:00:40 -07:00

syscall.c

bpf: Remove unneeded break

2020-10-19 20:40:21 +02:00

sysfs_btf.c

bpf: Fix sysfs export of empty BTF section

2020-09-21 21:50:24 +02:00

task_iter.c

bpf: Permit cond_resched for some iterators

2020-10-28 14:54:31 -07:00

tnum.c

bpf: Verifier, do explicit ALU32 bounds tracking

2020-03-30 14:59:53 -07:00

trampoline.c

bpf: Move prog->aux->linked_prog and trampoline into bpf_link on attach

2020-09-29 13:09:23 -07:00

verifier.c

bpf: Enforce id generation for all may-be-null register type

2020-10-19 15:57:42 -07:00