[dev-alloc] Pattern Replication API

[kmd2zjw]

I'm basing my MLP implementation off of the batched GEMM application, but before I begin to migrate the updated batched code I just wanted to make sure that it was fully implemented. Within the batched GEMM code, there is a TODO that requests a pattern replication feature:

// Concatenate matrices. Needs to be done once
std::vectorstd::vector batchedSrcMatA;
for (uint64_t i = 0; i < colA; ++i)
{
std::vector temp;
for (uint64_t j = 0; j < batchSize; ++j) {
// TODO: Add replicate a pattern API in the simulator
for (uint64_t k = 0; k < row; ++k) {
temp.push_back(srcMatrixAT[i][k]);
}
}
batchedSrcMatA.push_back(temp);
}

So I was wondering if this is worth waiting for or if I should proceed with updating MLP?

[deyuan]

Hi @kmd2zjw,

Thanks for bringing this up. From your example code, the pattern replication API means that we need to replicate a pattern of row number of elements for batchSize times.

Could you provide more details, such as if row and batchSize are determined by PIM hardware properties or can be arbitrary numbers? We need to think how this can be accelerated with any hardware support.

An alternative approach is to use the ranged PIM copy APIs. By providing idxBegin and idxEnd, you can directly copy a pattern into a specific range of a PIM object, to avoid constructing a big vector in host memory.

[kmd2zjw]

I think Farzana had determined how to replicate the pattern based on hardware properties. Here are the preceding lines of the code I provided in the original discussion post:

PimDeviceProperties deviceProp;
PimStatus status = pimGetDeviceProperties(&deviceProp);
if (status != PIM_OK)
{
std::cout << "Abort" << std::endl;
return;
}
// How many GEMVs can be processed in parallel?
// Following calculation is done based on bit serial data layout
uint64_t totalAvailableBitsPerItr = (uint64_t)deviceProp.numRanks * (uint64_t)deviceProp.numBankPerRank * (uint64_t)deviceProp.numSubarrayPerBank * (uint64_t)deviceProp.numRowPerSubarray * (uint64_t)deviceProp.numColPerSubarray;
uint64_t totalBitsRequired = colB * row * 32 * 4; // assuming 32bits INT
uint64_t batchSize = std::min(colB, (uint64_t)std::ceil(totalAvailableBitsPerItr * 1.0 / totalBitsRequired));

The file is located at "misc-bench/cpp-gemm-batch/PIM/gemm.cpp" if you need more information.

[fasiddique]

@kmd2zjw it may take a while to design and implement this API with performance and energy modeling. So, I would recommend not waiting on this.