diff options
author | Eric Dumazet <edumazet@google.com> | 2024-03-29 18:42:23 +0300 |
---|---|---|
committer | David S. Miller <davem@davemloft.net> | 2024-04-01 13:28:32 +0300 |
commit | 36b83ffcf209a2e6099dae1070df6a2001dfab27 (patch) | |
tree | 5a99feee312c8da948ac413e797827ceb321397d /include/net | |
parent | f7efd01fe21431a98677e5505e5de46649121ca7 (diff) | |
download | linux-36b83ffcf209a2e6099dae1070df6a2001dfab27.tar.xz |
net: rps: change input_queue_tail_incr_save()
input_queue_tail_incr_save() is incrementing the sd queue_tail
and save it in the flow last_qtail.
Two issues here :
- no lock protects the write on last_qtail, we should use appropriate
annotations.
- We can perform this write after releasing the per-cpu backlog lock,
to decrease this lock hold duration (move away the cache line miss)
Also move input_queue_head_incr() and rps helpers to include/net/rps.h,
while adding rps_ prefix to better reflect their role.
v2: Fixed a build issue (Jakub and kernel build bots)
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Diffstat (limited to 'include/net')
-rw-r--r-- | include/net/rps.h | 23 |
1 files changed, 23 insertions, 0 deletions
diff --git a/include/net/rps.h b/include/net/rps.h index 7660243e905b..10ca25731c1e 100644 --- a/include/net/rps.h +++ b/include/net/rps.h @@ -122,4 +122,27 @@ static inline void sock_rps_record_flow(const struct sock *sk) #endif } +static inline u32 rps_input_queue_tail_incr(struct softnet_data *sd) +{ +#ifdef CONFIG_RPS + return ++sd->input_queue_tail; +#else + return 0; +#endif +} + +static inline void rps_input_queue_tail_save(u32 *dest, u32 tail) +{ +#ifdef CONFIG_RPS + WRITE_ONCE(*dest, tail); +#endif +} + +static inline void rps_input_queue_head_incr(struct softnet_data *sd) +{ +#ifdef CONFIG_RPS + sd->input_queue_head++; +#endif +} + #endif /* _NET_RPS_H */ |