0%

NUMA 非一致性内存访问

Posted on Edited on Valine:

NUMA（Non-Uniform Memory Access，非一致性内存访问）是一种多 CPU 计算机架构。非一致性，表示 CPU 访问不同内存的是时间上的不一致，访问本地内存快于非本地内存。

背景

早期多 CPU 访问内存采用 SMP （Symmetric Multi-Processor）架构，即：所有 CPU 通过同一条总线共享访问相同的内存，所有 CPU 访问内存的速度都是相同的。

image-20250418000257117

但随着 CPU 数量的增加，使用相同总线和资源必然导致访问冲突，从而造成 CPU 利用率降低。

NUMA

NUMA（Non-Uniform Memory Access，非一致性内存访问）将 CPU 划分到多个 NUMA Node，每个 Node 都有独立的 Bus 和 Memory。Node 之间通过 QPI 总线（高速互联总线）互通。

image-20250418002625551

CPU 访问不同的 Memory 速度不同，访问本地 Memory 的速度快于非本地 Memory。

Socket

Socket 是 CPU 的物理插槽，一个 Socket 可以插入多个 CPU，通常一个 Socket 就是一个 NUMA Node。

查看命令

  • 查看 CPU 相关信息

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    root@ubuntu2404:/# lscpu
    Architecture:             aarch64
      CPU op-mode(s):         64-bit
      Byte Order:             Little Endian
    CPU(s):                   3
      On-line CPU(s) list:    0-2
    Vendor ID:                Apple
      BIOS Vendor ID:         Apple
      Model name:             -
        BIOS Model name:      Apple silicon  CPU @ 0.0GHz
        BIOS CPU family:      280
        Model:                0
        Thread(s) per core:   1
        Core(s) per socket:   3  # 每个 socket 里有 3 个 CPU
        Socket(s):            1  # 一共 1 个 socket
        Stepping:             0x0
        BogoMIPS:             48.00
        Flags:                fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop s
                              ha3 asimddp sha512 asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 flagm2 frint svei8mm svebf1
                              6 smei16i64 smef64f64 smei8i32 smef16f32 smeb16f32 smef32f32 sme2 smei16i32 smebi32i32
    Caches (sum of all):
      L1d:                    384 KiB (3 instances)
      L1i:                    576 KiB (3 instances)
      L2:                     48 MiB (3 instances)
    NUMA:
      NUMA node(s):           1    # 一共 1 个 NUMA node
      NUMA node0 CPU(s):      0-2  # 0～2 号 CPU 在 NUMA node0 中
    Vulnerabilities:
      Gather data sampling:   Not affected
      Itlb multihit:          Not affected
      L1tf:                   Not affected
      Mds:                    Not affected
      Meltdown:               Not affected
      Mmio stale data:        Not affected
      Reg file data sampling: Not affected
      Retbleed:               Not affected
      Spec rstack overflow:   Not affected
      Spec store bypass:      Vulnerable
      Spectre v1:             Mitigation; __user pointer sanitization
      Spectre v2:             Not affected
      Srbds:                  Not affected
      Tsx async abort:        Not affected

  • 查看 NUMA 信息

    1
    2
    3
    4
    5
    6
    7
    8
    root@ubuntu2404:/# numactl --hardware
    available: 1 nodes (0)
    node 0 cpus: 0 1 2
    node 0 size: 3902 MB
    node 0 free: 72 MB
    node distances:
    node   0
      0:  10

  • 查看 CPU 以及其对应的 NODE、SOCKET、CORE

    1
    2
    3
    4
    5
    6
    7
    8
    root@ubuntu2404:/# lscpu -p=CPU,NODE,SOCKET,CORE
    # The following is the parsable format, which can be fed to other
    # programs. Each different item in every column has an unique ID
    # starting usually from zero.
    # CPU,Node,Socket,Core
    0,0,0,0  # 逻辑CPU0 -> NUMA node0 -> Socket0 -> 物理核心0
    1,0,0,1
    2,0,0,2