You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In addition to the eval<...> functions, OpenFHE also provides eval<...>InPlace which overrides one of the operands,
There's also eval<...>Mutable where it will do any necessary preprocessing (e.g., rescaling/etc) of the operands in-place,
and eval<...>MutableInPlace, where it will do both the preprocessing and the operation in-place.
The plain eval<...>always forces a copy, eval<...>InPlace can still force a copy if pre-processing is necessary (as it's only allowed to modify/overwrite one operand), while eval<...>MutableInPlace will never cause copies. Since we know if an SSA value has other uses*, we should be able to determine fairly easily in printEvalMethod if a given op can be lowered to one of the more efficient versions.
For small "toy" programs, the performance differences are very much negligible, but for larger real-world workloads, memory tends to become a significant bottleneck, so doing this "correctly" should be beneficial. Filing this as an issue, rather than putting together a PR, as it's just not very critical at the moment 😉 |
*Thinking about this again, hasOneUse() is correct but too conservative, as we don't care about prior uses. But MLIR's Liveness analysis has isDeadAfter which sounds like exactly what we want.
The text was updated successfully, but these errors were encountered:
+1000! I've wanted to do this, and didn't know about isDeadAfter - which is such a cool feature.
If for some reason someone ever writes an interpreter, that would also be a really useful function for it. For e.g. jaxite IR in a python interpreter @code-perspective
You may see me tackle this at the end of the week for demo speed purposes, but I will assign myself / update the issue when I do.
I thought about those Inplace optimizations as well (Lattigo prefers in-place API). I was thinking about converting value semantic to memref semantic at some, maybe lwe, stage, then use canonicalization rule to rewrite copy op into inplace op.
I think converting to memref semantic is needed to correctly model the memory effect behavior.
I think converting to memref semantic is needed to correctly model the memory effect behavior.
That might be overkill for this specific issue - even a "conservative" estimate using Liveness should give some pretty good results. But then again we will eventually need to consider bufferization for non-llvm targets anyway, and doing it on the lwe.ctxt/lwe.ptxt level would mean no duplication.
In addition to the
eval<...>
functions, OpenFHE also provideseval<...>InPlace
which overrides one of the operands,There's also
eval<...>Mutable
where it will do any necessary preprocessing (e.g., rescaling/etc) of the operands in-place,and
eval<...>MutableInPlace
, where it will do both the preprocessing and the operation in-place.The plain
eval<...>
always forces a copy,eval<...>InPlace
can still force a copy if pre-processing is necessary (as it's only allowed to modify/overwrite one operand), whileeval<...>MutableInPlace
will never cause copies. Since we know if an SSA value has other uses*, we should be able to determine fairly easily inprintEvalMethod
if a given op can be lowered to one of the more efficient versions.For small "toy" programs, the performance differences are very much negligible, but for larger real-world workloads, memory tends to become a significant bottleneck, so doing this "correctly" should be beneficial. Filing this as an issue, rather than putting together a PR, as it's just not very critical at the moment 😉 |
*Thinking about this again,
hasOneUse()
is correct but too conservative, as we don't care about prior uses. But MLIR's Liveness analysis hasisDeadAfter
which sounds like exactly what we want.The text was updated successfully, but these errors were encountered: