Budget allocation not sum up to 1

[nieszkodzi]

Hi!
I cannot find that issue so maybe it's my problem, but still cannot find solution.
I try to use allocate_budget_to_maximize_response() function.

response_30 = mmm.allocate_budget_to_maximize_response(
    budget=5230,
    num_days=30,
    time_granularity="daily",
    budget_bounds=budget_bounds,
)

According to example notebook the optimal_allocation_dict should sum up to total_budget, but it is scaled:

optimal_budget_allocation_dict_30 = mmm.optimal_allocation_dict

# Round values in the dictionary to 4 decimal places
rounded_optimal_budget_allocation_dict_30 = {key: round(value, 4) for key, value in optimal_budget_allocation_dict_30.items()}
rounded_optimal_budget_allocation_dict_30

output:

{'source_apple_brand': 0.1319,
 'source_apple_non_brand': 0.0,
 'source_criteo': 0.0,
 'source_facebook': 0.0,
 'source_google': 0.0396,
 'source_tiktok': 0.0,
 'source_branding_campaigns_spends': 0.0}

ok, scale is not problem, but it does not sum up to 1.

sum(mmm.optimal_allocation_dict.values())

output:

0.17155497855384777

after un-scaling:

allocate_spend_30 = (
                np.array(list(mmm.optimal_allocation_dict.values()))
                * mmm.channel_transformer["scaler"].scale_
            )
np.round(allocate_spend_30)

output:

array([838.,   0.,   0.,   0., 533.,   0.,   0.])

can someone tell me what happened. Is it my bug or library?

[cetagostini-wise]

Hello @nieszkodzi

Can you share a notebook to be able to observe the results in detail?

On the other hand, the problem could be in budget_bounds. Can you check what limits are you passing to the model? If the limits are small, the result not necessarily should sum 1.

Best,

[nieszkodzi]

I made many different tests, it never sums up to the total budget. Here, no bounds, 1000 euro, sum... 263 euro.

[AlfredoJF]

Hi @nieszkodzi

Have a look here since this might be the same case for you

Hi @YT19 That means you do not need to spend 500k in the following 4 weeks to maximize your response and only less than 250k is enough :slight_smile:

[nieszkodzi]

Hi @AlfredoJF :)
The problem is definitely the same but I am sure that it is not the answer :) if there will be not point to spend more then if I increase the hole budget, the budget allocation should not increase. In fact - it does.
Second thing - sample notebook suggest it should allocate hole budget as the next step is limiting the budget based on saturation curves to show that it almost doesn't affect outcome.
Also in library code it is impossible to not allocate hole budget, from budget_allocation.py

constraints = {"type": "eq", "fun": lambda x: np.sum(x) - total_budget}

that means that default constraints ensure us that budget will be sum up to 1. So somewhere in between something is going on badly.
I will put it also in the discourse page to be sure the thread will remain open :)

[AlfredoJF]

Hi @nieszkodzi, I've been thinking about it and made some tests too. Yes, the allocation changes when the budget changes. This makes sense when the channels are not saturated but not when it's saturated. I agree with the constraint too. If the type is eq then the sum of the optimized budget should be equal to total_budgetbut it's not the case.

@cetagostini-wise could you please explain the reasoning behind the objective function?

pymc-marketing/pymc_marketing/mmm/budget_optimizer.py

Line 78 in 9755d3b

def objective(self, budgets: list[float]) -> float:

I do not think the purpose of the allocate_budget function is happening as described but I can be wrong. Happy to hear your input and thoughts :)

pymc-marketing/pymc_marketing/mmm/budget_optimizer.py

Line 120 in 9755d3b

def allocate_budget(

The optimization is done using the Sequential Least Squares Quadratic Programming (SLSQP) method
and it's constrained such that:

1. The sum of budgets across all channels equals the total available budget.
2. The budget allocated to each individual channel lies within its specified range.

[AlfredoJF]

Correct me if I'm wrong but shouldn't this be rescaled using the max scales as in self.channel_transformer["scaler"].scale_.max() as in when scaling the budget and budget bounds?

pymc-marketing/pymc_marketing/mmm/delayed_saturated_mmm.py

Lines 2156 to 2165 in 49ac689

	inverse_scaled_channel_spend = self.channel_transformer.inverse_transform(
	np.array([list(self.optimal_allocation_dict.values())])
	)
	original_scale_allocation_dict = dict(
	zip(
	self.optimal_allocation_dict.keys(),
	inverse_scaled_channel_spend[0],
	strict=False,
	)
	)

[cetagostini]

Hey @AlfredoJF, indeed.

Yesterday, I was checking the process and I think the issue is in the scaler, most probably, I need a few extra days to validate. But after spending some time on it, my intuition says that it is there. That's why when @nieszkodzi rescales with the other scaler the numbers don't add up. If the optimizer did not follow the constraints we should get an alert or error.

[AlfredoJF]

Hey @cetagostini

I think the _create_synth_dataset method is doing some funny things, especially the allocation_strategy. Although there's some randomness, it is not the best way to create the synthetic dataset to sample the posterior as it allocates the optimized budget (I'm testing self.channel_transformer["scaler"].scale_.max()) to each day. + some random number and not an informed fraction of it. Because of this, the response is almost flat throughout this period.

I'm not quite sure if that's why the max channel scaler was used before but seems a bit odd to me.

pymc-marketing/pymc_marketing/mmm/delayed_saturated_mmm.py

Lines 2064 to 2071 in 93fc9d9

	**{
	channel: allocation_strategy.get(channel, 0)
	+ np.random.normal(0, 0.1 * allocation_strategy.get(channel, 0))
	for channel in channels
	},
	**{control: 0 for control in _controls},
	target_col: 0,
	}

Have a look and let me know if you find something interesting. :)

[cetagostini]

Hey @AlfredoJF indeed the noise was implicit to try mimic real life situation. Even with a fixed spending level, the actual spending (Or impressions) can fluctuate based on factors such as the number of people bidding on your ad or viewing ads on a given day. To account for this unpredictable variation, we have included a parameter called noise_level that allows you to introduce white noise into the projection.

You can check the PR here.

[AlfredoJF]

Hey @cetagostini

My only concern with using the allocation_strategy dictionary is that the entire spend allocation by channel + the noise will be used for each period in the synthetic dataset.

Say the budget to optimise is 6.2M for the following 13 weeks, and the total spend in the synth_dataset is around 81M (accounting for noise too) because of what I mentioned above.

Shouldn't the allocation per week be a fraction of the total allocation per channel and the total spend in the synth_dataset be closer to the original budget to optimise?

I'm thinking of using seasonality if it's part of the model or a historical spend share per week but happy to hear your thoughts.

[AlfredoJF]

Me again @cetagostini :)

I've been playing with the Dirichlet distribution in Numpy to create some random distributions of the optimized budgets and is working well. I didn't want to make too many changes so I'm just iterating over the numpy array and multiplying that random allocation by the allocation_stratety + noise_level for now but there should be a more efficient way to do it.

Let me know your thoughts on this

random_allocation = np.random.dirichlet(alpha=np.ones(time_length), size=len(channels)).T

        new_rows = [
            {
                self.date_column: pd.to_datetime(new_date),
                **{
                    channel: random_allocation[row_index, col_index]
                    * (allocation_strategy.get(channel, 0)
                       + np.random.normal(0, noise_level * allocation_strategy.get(channel, 0)))
                    for col_index, channel in enumerate(channels)
                },
                **{control: 0 for control in _controls},
                target_col: 0,
            }
            for row_index, new_date in enumerate(new_dates)
        ]

[cetagostini]

Say the budget to optimise is 6.2M for the following 13 weeks, and the total spend in the synth_dataset is around 81M (accounting for noise too) because of what I mentioned above.

I think there is confusion here. The budget used in the function parameter is supposed to be the budget for each time unit. Meaning, if you say budget=10 and num_periods=5 then your total budget is 50 -10*5-.

If you want your total budget to be 6.2M then you would say 6.2/period, and that will give the time budget you are willing to use in each unit of time with the granularity from your model.

[cetagostini]

I'm thinking of using seasonality if it's part of the model or a historical spend share per week but happy to hear your thoughts.

I'm not sure if I follow you here. We're predicting the total marketing contribution, so why would you want to add seasonality to the mix? And how would that change your decision? Seasonality is not something we can control.