perf: more efficient _sparse_nanmean

[Reovirus]

Fixes #1894. Reduced redundant data copying; the original matrix is now copied once instead of twice. One copy remains necessary to replace NaNs with zeros without modifying the original matrix.​

• Closes #1894
• Tests included
• Release notes not necessary because

[Reovirus]

@flying-sheep Can you assign the reviewer please? Fall test looks like testing bug

[Reovirus]

And I can rewrite logic without any coping using numba, but it can be slowly that implemented methods

[Reovirus]

I know that I have "No milestone failure" but I haven't permissions to set milestone, probably I need help of maintainers to set it

[Zethson]

@Reovirus I restarted the CI as there should hopefully be no flaky tests at the moment.

Don't worry about the milestone, please.

[Reovirus]

failure is scipy.sparse.csr_matrix.count_nonzero which has axis argument since scipy 1.15.0. I'll rewrite with numba

[Reovirus]

@Zethson cund you please restart CI? I catch very strange bug with http, it's not mine code

[Reovirus]

CI pased, can somebody review in some time pls

[flying-sheep]

I made the benchmarks run, let’s see how well this works!

Could you please add a release note fragment?

hatch run towncrier:create 3570.performance.md

[Reovirus]

Yes, l'll make a note, thanks!)

[scverse-benchmark]

Benchmark changes

Change	Before [9bc2c1e]	After [24f042e]	Ratio	Benchmark (Parameter)
+	290M	338M	1.16	tools.ToolsSuite.peakmem_score_genes
-	19.4±0.2ms	17.5±0.1ms	0.9	tools.ToolsSuite.time_score_genes

Comparison: https://github.com/scverse/scanpy/compare/9bc2c1ed1bd1cf6f6c06cec71cce99916d048163..24f042ecdcc124a1ebcc526606373b9b3a940024
Last changed: Tue, 7 Apr 2026 15:34:15 +0000

More details: https://github.com/scverse/scanpy/pull/3570/checks?check_run_id=70256269762

[flying-sheep]

according to the benchmarks, this is actually slower than before.

the benchmarks are a bit flaky, so it could be wrong, but usually a factor of 2.5 like here is real.

[Reovirus]

Understand. Try to change and speed up

[Reovirus]

@flying-sheep
We have a small win (23.4±2ms | 21.8±0.8ms | 0.93 | tools.time_score_genes, too high p-value to display cause stderr is high, implementation is lessly effective on smal-amount-of-zeros data) in score_genes time and small lose in memory. Should I optimize memory usage? (it can be caused by mask=np.isnan(data), instead of thic I can backgroundly recalc mask in np.nan_to_num and reduce memory on 1 array)))

[Intron7]

I think these should be parallel numba kernels with mask arguments for major and minor axis. I have a working implementation in rapids-singlecell that doesnt need to copy at all. Starting from _get_mean_var is I think the best way forward

[Intron7]

Refactor kernels to be 0 copy

[Reovirus]

@Intron7 I rewrite logics. But my local benchmarking gives another result (I just measure peak memory by ), I'm trying to fix it. And I have no idea about some metrics like preprocessing_counts.FastSuite.time_log1p('pbmc3k', 'counts-off-axis'). It change sign randomly, is it normal?

[Intron7]

The kernels already look good. However my main point I was trying to make is that you can create the means without subsetting. That means you wouldnt need to use the get subset function but just use masks within your kernel.

[Zethson]

Just typos I think. These should be renamed.

[codecov]

Codecov Report

❌ Patch coverage is 77.77778% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 78.49%. Comparing base (9bc2c1e) to head (24f042e).
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
src/scanpy/tools/_score_genes.py	77.77%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3570      +/-   ##
==========================================
- Coverage   78.51%   78.49%   -0.02%     
==========================================
  Files         117      117              
  Lines       12753    12751       -2     
==========================================
- Hits        10013    10009       -4     
- Misses       2740     2742       +2     

Flag	Coverage Δ
hatch-test.low-vers	77.78% <77.77%> (-0.02%)	⬇️
hatch-test.pre	77.45% <77.77%> (+12.61%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
src/scanpy/tools/_score_genes.py	85.45% <77.77%> (-2.05%)	⬇️

[flying-sheep]

I merged upstream changes. Do you still plan to work in the changes @Intron7 requested?