todo

[wanghenshui]

No description provided.

[wanghenshui]

https://www.scattered-thoughts.net/writing/smolderingly-fast-btrees/

[wanghenshui]

https://www3.cs.stonybrook.edu/~bender/newpub/BenderHu07-TODS.pdf

pma

[wanghenshui]

Designing A Fast Concurrent Hash Table
https://ibraheem.ca/posts/designing-papaya/
ibraheemdev papaya
Snapshot-Free, Transparent, and Robust Memory
Reclamation for Lock-Free Data Structures https://arxiv.org/pdf/1905.07903

[wanghenshui]

https://15721.courses.cs.cmu.edu/spring2019/papers/17-hashjoins/richter-vldb2015.pdf

A Seven-Dimensional Analysis of Hashing Methods and its
Implications on Query Processing

[wanghenshui]

https://graphallthethings.com/posts/better-buf-read

https://fgiesen.wordpress.com/2011/11/21/buffer-centric-io/

[wanghenshui]

https://adriacabeza.github.io/2024/07/12/caffeine-cache.html

[wanghenshui]

https://the-mikedavis.github.io/posts/german-string-optimizations-in-spellbook/

[wanghenshui]

https://mrsuh.com/articles/2024/sqlite-index-visualization-structure/

[wanghenshui]

Hyrum's Law in Golang

https://abenezer.org/blog/hyrum-law-in-golang

任何接口一单定下来就会被依赖上。ABI stable

[wanghenshui]

https://www.datadoghq.com/blog/engineering/formal-modeling-and-simulation/

没看懂这个TLA咋搞的

[wanghenshui]

https://tokio.rs/blog/2019-10-scheduler

inko/issues/337

[wanghenshui]

bhumikamittal7/graphstore_doc

[wanghenshui]

https://arxiv.org/pdf/2408.05625v2

[wanghenshui]

Memento_Filter

[wanghenshui]

https://lobste.rs/s/ck7dtw/adaptive_packed_memory_array

[wanghenshui]

https://core.ac.uk/download/pdf/301638222.pdf

[wanghenshui]

https://avi.im/blag/2024/s3-log/

[wanghenshui]

https://www.coonote.com/brpc-note/brpc-channel-init.html

[wanghenshui]

https://seekstar.github.io/2022/05/15/linux-kernel-rcu/

[wanghenshui]

https://whiteccinn.github.io/2021/03/27/%E6%95%B0%E6%8D%AE%E5%BA%93%E5%BC%80%E5%8F%91%E7%9F%A5%E8%AF%86/msource/

[wanghenshui]

https://portrait.gitee.com/sun_grad/docs-cn/blob/master/grafana-tikv-dashboard.md?skip_mobile=true

[wanghenshui]

https://asktug.com/t/topic/933127

[wanghenshui]

https://gitee.com/mallog/pingcap-docs-cn/blob/master/grafana-tikv-dashboard.md

[wanghenshui]

https://szza.github.io/2022/02/13/rocksdb/WritePath/write_thread_4/

[wanghenshui]

https://hub.baai.ac.cn/view/31611

[wanghenshui]

https://juejin.cn/post/7194404671014830117#heading-7

[wanghenshui]

mgtv-tech/redis-GunYu

[wanghenshui]

NUMA 背景

CPU概念

Socket(芯片插槽) > NUMA node(一个插槽可能有一个或多个NUMA node) > CPU core (一个NUMA node下有多个CPU core) > thread (一个CPU core可以有一个或两个thread(超线程))

前端总线共享 内存存在距离 所以就有了所谓的跨NUMA 节点 分配内存的问题

检查清单
lscpu | grep -i numa

NUMA node(s): 2
NUMA node0 CPU(s): 0-23,48-71
NUMA node1 CPU(s): 24-47,72-95

如果只有一个就不用看了

numactl -H
available: 1 nodes (0)
node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
node 0 size: 128769 MB
node 0 free: 25772 MB
node distances:
node 0
0: 10

更详细一些

numastat
node0 node1
numa_hit 103088096825 89269643558
numa_miss 891068056 1027344082
numa_foreign 1027338134 891045360
interleave_hit 28545 28827
local_node 103088209277 89270366494
other_node 891316004 1028095857

重点关注numa miss和numa_foreign 分别代表缓存失效和跨节点访问内存

可以简单计算cache miss rate 2% 影响有限

numactl --hardware

查看核心以及对应的cpu

是否均衡？ sysctl -w kernel.numa_balancing=0

规避网卡绑核 ethtool -i eth0

https://tidb.net/book/tidb-monthly/2022/2022-04/usercase/NUMA-node

网卡影响案例一例 https://tech.meituan.com/2018/11/15/docker-architecture-and-evolution-practice.html

选择绑定策略

numactl --show，常见策略内存分配：

strict/default:
仅在进程运行的Numa node上分配内存

interleave:
在所有的numa node上交叉分配 提高内存利用率更均衡
提高吞吐而不是延迟
场景: 随机内存大内存大带宽场景

numactl --interleave=all
bind
numactl --cpunodebind=0 --membind=0 bin

场景 最小化跨节点的内存访问，提高缓存命中率和内存访问性能
绑核和绑内存通常是配对的

preferred:
在进程运行的numa node上分配，不足时在去其他node分配。

preferred相比membind不会失败，可以和membind组合，实现既要有要

physcpubind

最小化上下文切换场景 需要确认是CPU cache影响还是NUMA 跨内存影响来选择cpunodebind还是这个

组合使用

numactl --cpunodebind=0 --preferred=0 优先0内存

numactl --cpunodebind=0 --membind=0 --preferred=0 --physcpubind=0,1,2,3

存在上层宿主机numa配置影响容器案例

https://cloud.tencent.com/developer/article/1543313

宿主机配了，但容器没配

宿主机没配，容器可以配，也能生效

taskset -pc $$ #pid 可以查看容器使用的CPU亲和情况

上层可以通过k8s kubelet 或者docker自身也可以配置

k8s numa.zone-topology
docker run -d -i -t --cpus=8 --cpuset-cpus=4-11 --cpuset-mems=0 -m 16g --name 8u16g_01 --privileged=true centos:latest /bin/bash

[wanghenshui]

percpu_arena:percpu 和numactl冲突

[wanghenshui]

MALLOC_CONF="background_thread:true,dirty_decay_ms:4000,thp:always,metadata_thp:always"

[wanghenshui]

https://juejin.cn/post/7139755599952085022

[wanghenshui]

https://www.reddit.com/r/cpp/comments/1hqj6ve/feeing_hard_to_understand_coroutine_in_c20_and/

[wanghenshui]

facebook /rocksdb/pull/12540/files#diff-85aa8bd0327318a639a4c8e84b7890713f09e675017b6e9c06f044c2b0dc512a

另一种思路，跟踪mannifest变动同步文件

[wanghenshui]

blobdb问题

facebook /rocksdb/issues/11570 批量写gc策略错误

10657 问题相同

[wanghenshui]

https://www.synacktiv.com/publications/exploring-android-heap-allocations-in-jemalloc-new

[wanghenshui]

https://szza.github.io/2022/02/13/rocksdb/WritePath/write_thread_4/

[wanghenshui]

https://juejin.cn/post/7239516348434907194

[wanghenshui]

https://godbolt.org/z/46sbx1811

[wanghenshui]

https://www.sigarch.org/fast-memcpy-a-system-design/
https://mp.weixin.qq.com/s/jCVULqDz8dDyqrkV8Msbbg
http://0x80.pl/notesen/2018-04-19-simd-parsing-int-sequences.html

https://github.com/WojciechMula/parsing-int-series

https://github.com/KholdStare/qnd-integer-parsing-experiments
https://johnnysswlab.com/crash-course-introduction-to-parallelism-the-algorithms/

[wanghenshui]

https://blog.bkryza.com/posts/generating-uml-diagrams-from-moderncpp-using-clang/

[wanghenshui]

stas00/ml-engineering

intel/ScalableVectorSearch