Evolutions, matings, infection variance, bugfixes

[erolm-a]

This PR fixes the wrong evolution rates and deeply changes the way matings are made.

Evolutions

We decided to drop poisson distributions for age generation as they would encourage ages way lower than the threshold resulting in very few lice evolving.
The new formula is extracted from the Aldrin paper and internally emulates an age-distribution, then manually generates quantiles from the median development rate from a Weibull distribution and calculates average evolution rates from such event expectations. This is meant to be used in the following cases: L1->L2, L3->L4, L4->L5(m/f).

This resulted extremely effective especially as it would rise the chances of evolving between levels, but comes to a (slight) disadvantage of being somewhat slower. Therefore, sampling is performed no more than 1000 times to get a fairly reasonable expectation. In theory one could numerically compute thee expected development rate between stages (a constant) and apply a Gaussian over it, but for now this approach suffices.

Matings

The original spirit (Mendeleev-based matching) is preserved but the implementation has been changed to be O(1) (see #154 ). Similarly, a number of functions like dam/sire selection has been changed to be done in one go with a hypergeometric distribution. See #154

Furthermore, we spotted that currently mating code correctly calculates N_E eggs to extrude but actually generates O(N_E^2) eggs because each single "mating" will thus produce N_E eggs rather than being distributed.

The proposed solution fixes both problems at once.

This fix alone makes the simulation hundreds of times faster. For example, in a pandemic-like scenario with millions of eggs a single day update could take seconds while now the speed is consistently above 5 iterations per second (see below). Profiling data shows that after 80 days before this patch (runtime of ~1 minute) up to 22 seconds would be (cumulatively) spent on generate_eggs_discrete. Now this number dropped to 0.334 s with get_num_matings taking 0.4807 s overall, meaning the average time is 108 us /153 us .

Variance, treatment mortality etc.

All such functions have in common that they would sample on some quantity depending on lice population rather than being constant time. Due to a slip it was mentioned in the comments but not done ultimately.

Treatment mortality used to take overall 12.47s or ~4ms s per iteration, thus in a total of 6 it/s. Now it takes 3.12s, average runtime less than 1ms, thus a 4x speedup. The total runtime to emulate one year reduced from one minute to 36 seconds (~2x speedup).

Note that, when serialisation is disabled we get a speedup from 35 seconds to 15 seconds - as to be expected.

I believe we have collected evidence to close #145 .

Multiple testing environments

A new farm environment, test_Fyne, has been introduced. Users are now free to make experiments in Fyne without caring to break test cases. This has been possible for a while.

• Fix a glitch in cage averaging in the visualiser Visualiser tracking issue #103
• Implement a weibull-based evolution approach (closes Replace age expectancy function, clean up evolution code #152 )
• Make variance calculation O(1) (closes Infection variance and mating calculation have linear complexity on the population #153)
• Make mating calculation O(1) (closes generate_eggs_discrete is an IMB approach and does not scale #154 )
• Make treatment calculation O(1)
• Fix test cases

[codecov-commenter]

Codecov Report

Merging #155 (78b9a4b) into master (b05196f) will increase coverage by 1.55%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #155      +/-   ##
==========================================
+ Coverage   81.09%   82.64%   +1.55%     
==========================================
  Files          10       10              
  Lines        1465     1406      -59     
  Branches      221      212       -9     
==========================================
- Hits         1188     1162      -26     
+ Misses        255      222      -33     
  Partials       22       22              

Flag	Coverage Δ
unittests	82.64% <100.00%> (+1.55%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
refactor/src/Config.py	93.61% <ø> (+23.53%)	⬆️
refactor/src/Cage.py	98.62% <100.00%> (+0.13%)	⬆️
refactor/src/Farm.py	96.35% <100.00%> (ø)
refactor/src/LicePopulation.py	89.74% <100.00%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b05196f...78b9a4b. Read the comment docs.