計算機結構與作業系統

Regarding LLM inference, quantization, and memory hierarchy, consider running inference for a large language model on a CPU system with the following specifications:

• L1 cache: 32 KB per core, 4 cycles latency
• L2 cache: 256 KB per core, 12 cycles latency
• L3 cache: 32 MB shared, 40 cycles latency
• Main memory: 64 GB DDR5, 200 cycles latency, 50 GB/s bandwidth
• The model has 7 billion parameters

The following code performs a single matrix-vector multiplication for one transformer layer:

// Weight matrix W: 4096 x 4096, activation vector x: 4096
// FP16: 2 bytes per element, INT8: 1 byte per element, INT4: 0.5 bytes per element
void matvec(void *W, float *x, float *y, int N, int precision) {
    for (int i = 0; i < N; i++) {
        float sum = 0;
        for (int j = 0; j < N; j++) {
            float w = dequantize(W, i, j, precision);
            sum += w * x[j];
        }
        y[i] = sum;
    }
}

Select correct description(s).