A little logic inside ordinary memory
At the Hot Chips conference this year, Samsung described LPDDR5X-PIM: a version of the familiar low-power memory standard with a small logic unit built in beside the memory cells. PIM stands for processing-in-memory. Rather than sending every value across the bus to the processor, waiting for the arithmetic and sending the answer back, the memory performs simple calculations itself and hands over only what the processor needs.
The idea is not new at Samsung. The company has been demonstrating it since 2021, first inside stacked high-bandwidth memory (HBM), and introduced HBM-PIM in 2023. What has changed, according to reports from the conference, is that the low-power version has reached validation as an actual product, no longer a lab demonstration. It keeps the standard package of ordinary LPDDR5X, a sign that it is meant to slot into existing board designs, and Samsung would like the JEDEC standards body to publish a first specification for its successor, LPDDR6X-PIM, before the year is out.
The numbers are Samsung’s own, described as preliminary, and should be read that way. On its benchmarks, doing the arithmetic in place raised effective bandwidth roughly eightfold, from the standard part’s 76.8 gigabytes per second peak to 614. In AI inference it reported a 2.28 times improvement in model run time and 3.01 times more tokens per second. The reporting is also clear about the limits: PIM does not bring this memory anywhere near the bandwidth of HBM, so training accelerators are not the target. Servers, client machines and phones are, all places where power is scarce.
An old rule at a new layer
Anyone who has run a large analytical database will recognize the principle, because the discipline was built on it. Moving data is expensive; computing on it is comparatively cheap. So the work goes to the data. Shared-nothing warehouses spread rows across nodes so that each node filters and aggregates its own slice before anything crosses the network. Query engines push predicates down to the storage layer so that rows destined to be discarded are never read. Columnar formats carry statistics that let whole blocks be skipped unopened.
Processing-in-memory applies the same rule one layer further down, at the boundary between memory and processor. In many AI workloads that boundary is now the constraint, because large volumes of model data have to cross it for every step of output. If the memory can do part of the arithmetic in place, less has to cross.
What this could eventually mean for enterprise workloads is plausible rather than proven: some inference running on less expensive, lower-power memory instead of the stacked kind; scans and aggregations that reduce data in memory before the processor sees it; more capable models on client machines. Only the inference figures appear in Samsung’s material. The rest is where the principle points, not a product claim.
Honest timing
This is not something to buy this year. The part is in validation. The figures come from the vendor’s own preliminary tests. The next-generation specification is a hope, not an agreed standard. And a capability inside the memory is only useful once the software above it, from compilers and runtimes to database engines and model-serving frameworks, knows it is there and how to use it. Hardware features that software ignores sit idle.
Memory technologies tend to travel from proprietary trick to industry standard, then into mainstream parts, then up the software stack over several product generations. A data leader does not need to predict when that journey ends. What they need is an architecture that benefits if it does, and loses nothing if it does not.
What pays off either way
- Keep work pushable. Favor data layouts and query patterns that let filtering and aggregation run where the data sits: partitioning aligned with common predicates, columnar formats with usable statistics, queries that reduce early instead of pulling raw rows into an application to sum them. This pays off on today’s warehouse and on any future memory-side compute.
- Avoid designs that assume all data must move to one place. Architectures that copy everything into a single engine, or into one accelerator’s memory, before doing any work are betting against the direction of travel. Prefer designs where compute can be placed near each store.
- Measure movement, not just compute. Bytes scanned, bytes shipped and bytes returned are the numbers that show whether work is really being pushed down. Few teams track them; the ones that do usually find waste quickly.
- Watch the software, not the silicon. The signal worth waiting for is support in the engines and frameworks you already run. When it appears in their release notes, that is the moment for a measured pilot on your own workload.
In our work, the estates that gain most from new hardware are rarely the ones that bought it first. They are the ones whose data was already arranged so that the work could go to it.
