Sum functions by x value (potential solution: inverse function; solution to simpler version: ordering lists)

[leopoldaschenbrenner]

Ex:

• project1MarginalValueOfDollar(dollars) = [value of marginal dollar to this project, given "dollars" total spend)
• project2MarginalValueOfDollar(dollars) = [value of marginal dollar to this project, given "dollars" total spend)
• ideal output: project1andproject2MarginalValueOfDollar(dollars) = [value of marginal dollar, given portfolio that is allocated optimally across project1 and project2]

A simpler version could be not having DMR curves for each grant, but just a dollar size and (average) cost-effectiveness attached. Then, you'd just need need to order a list of ["projectname", cost-effectiveness] by cost-effectiveness.

I'm not sure how to best achieve this. For the simpler version, we'd just need the ability to order lists, but I don't think Squiggle has that yet?

For the complex thing, maybe there are neat solutions to this with some programming creativity.

--

Idea for complex thing:

• invert project1 and project2, to get dollars(marginalvalue) for both.
• get project1andproject2Dollars(marginalvalue) = dollarsProject1(marginalvalue) + dollarsProject1(marginalvalue)
• invert project1andproject2Dollars

I'm not sure how to best achieve the inversion.

• perhaps some manual approach that iterates through x values over some interval with some epsilon, to get the y value that's closest?

--

This would get even more complicated if project1MarginalValueOfDollar(dollars) is a distribution (or cost-effectiveness of each project, in the simpler case). These distributions would also be correlated.

(Might be easiest to start by making this for numeric values, just using the means.)

--

This would be a common use case for me.

• Essentially, I have diminishing return curves for a bunch of projects (or just average cost-effectiveness of each of a bunch of projects, in simpler case)
• I would like to get the aggregate returns curve, i.e. the value of the first x dollars to all the projects overall.

[leopoldaschenbrenner]

There might be neat things you guys are already doing for distributions that we could maybe port over to this use case (e.g. inverting cdfs)

maybe a hacky solution is forcing the function into a degenerate distribution

[leopoldaschenbrenner]

another potential solution to the more complex/more general case:

• get value of function on a bunch of intervals with epsilon length (like integration)
• then you have a list of x y pairs
• and it reduces to the simpler case
• and you just need list ordering

so my guess is we could get a hacky version of the general case with just list ordering?

[OAGr]

Example use case: you have a bunch of estimates for different projects. You want to

Seems like it got cut off at the end.

[OAGr]

I feel like to do this well, directly in Squiggle, would probably want for loops and mutation. This would probably require a lot of changes. In the meantime, we could implement functions directly in Squiggle. Like, Danger.accumulateCurves or something.

[leopoldaschenbrenner]

Yeah makes sense!

haha, I like Danger.accumulateCurves. My guess is that this is not-performance-light, so probably better to have directly integrate anyway.

[leopoldaschenbrenner]

Here's a hacky quick implementation for doing this for two grant marg returns curves in squiggle:

`
getCombinedDMRCurve(firstMarginalBpsBn, secondMarginalBpsBn) = {

marginalBpsBn(dollars) = {

    potential_fraction_grant1 = [0.02, 0.05, 0.1, 0.15, 0.2, 0.25, 0.3, 0.35, 0.4, 0.45, 0.5, 0.55, 0.6, 0.65, 0.7, 0.75, 0.8, 0.85, 0.95, 0.98]
    // get each potential percent allocation
    // potential_fraction_grant1 = List.upTo(1, 99) -> map({|x| x*0.01})


    potential_allocations = List.map(potential_fraction_grant1, {
        |fraction_grant_1|

        marg_bpsbn_grant1 = mean(firstMarginalBpsBn(dollars*fraction_grant_1))
        marg_bpsbn_grant2 = mean(secondMarginalBpsBn(dollars*(1-fraction_grant_1)))

        // want to minimize diff in expected cost-effectiveness
        // to optimize returns
        diff_marginal_bpsbn = abs(marg_bpsbn_grant2 - marg_bpsbn_grant1)
        [fraction_grant_1, marg_bpsbn_grant1, marg_bpsbn_grant2, diff_marginal_bpsbn]
    })


    potential_allocation_min_diff_bpsbn = List.map(potential_allocations, {|x| x[3]}) -> min
    best_allocation = filter(potential_allocations, {|x| x[3]==potential_allocation_min_diff_bpsbn})

    best_fraction_grant1  = best_allocation[0][0]

    // marginal expected bps/bn should be roughly the same
    // to get distribution, use mx of the two distributions
    marginal_bps_bn = mx(firstMarginalBpsBn(dollars*best_fraction_grant1), secondMarginalBpsBn(dollars*(1-best_fraction_grant1)))

    marginal_bps_bn


}

marginalBpsBn

}
`

[leopoldaschenbrenner]

This is actually working pretty decently in the new branch with faster .map!

[NunoSempere]

Following up on Ozzie's point, we can:

1. Try to implement this in Squiggle userland
2. Have a function in the squiggle library
3. Import the distributions to outside of Squiggle, process it then, and feed it bad to Squiggle.

I'm feeling curious about how 2 & 3 go, and I'd been meaning to go back to writing some squiggle code, so might assign this task to myself to work on over the next few days.

[leopoldaschenbrenner]

Cool!

[leopoldaschenbrenner]

btw, re "import the distributions to outside of squiggle" - maybe you don't even need to export distributions, vs. mean (since portfolio optimization, i.e. how much dollars to each project, would be via expected value)

[OAGr]

I imagine the ideal solution would be to make more advanced Squiggle primatives, so that this is fairly elegant to do in Squiggle. For example, an "XYLine" type or some other native geometric type could be really nice.

That said, that's probably a few months away (maybe less with the Danger prefix though)

[leopoldaschenbrenner]

Another, more complex version:

• DMR curve to grant 1
• DMR curve to grant 2
• you have some distribution over how many grants like grant 1 and grant 2 you'll find (e.g., 1M to 5M for 1, and 2M to 20M for 2).

the gnarly thing here is that really for each sample of the distribution over how much of each you'd find, you'd want to redo the portfolio optimization (which gets performance impossible currently)

Also, re the above hacky implementation, that obviously gets slow if you try to do for more than 2 DMR curves by nesting.

[leopoldaschenbrenner]

Ok, one more different use case that isn't the same as above, but also uses function inversion:

here it's very simple: have f(x), want f^-1(x)

• you have DMR curve to a grant/ to a program
• you want optimal spend for the grant/program, based on "how many dollars could we spend above the last dollar bar"