Framing and periodicity

[sbenthall]

The PerfForesightConsumerType has two poststate_vars: aNrmNow and pLvlNow
https://github.com/econ-ark/HARK/blob/master/HARK/ConsumptionSaving/ConsIndShockModel.py#L1450

However pLvlNow is set in getStates():
https://github.com/econ-ark/HARK/blob/master/HARK/ConsumptionSaving/ConsIndShockModel.py#L1665

...and not in getPostStates(). Rather, aLvlNow is updated as a poststate in the code:
https://github.com/econ-ark/HARK/blob/master/HARK/ConsumptionSaving/ConsIndShockModel.py#L1706

Correcting the poststate_var field to list aLvlNow instead of pLvlNow raises an AttributeError.
I guess this is because the class is depending on the automatic initialization of the poststates, to create the attribute.

[mnwhite]

Yes. Attributes named in poststate_vars are the ones that are initialized at birth, and thus also when initializeSim() is called. They are the variables that describe the agents' state at the moment that time t becomes time t+1: they're the end of time t and the beginning of time t+1, and thus the variables that are mandated to exist for a new agent who comes into existence at some time. The permanent income level `pLvlNow` is a state variable in the sense that its value in time t is a function of the post-state variable from t-1 and the shock variables from time t. For an IndShockConsumerType, `pLvlNow` is only trivially used to make a consumption decision, as the problem is normalized by permanent income. For a GenIncProcessConsumerType, `pLvlNow` is a direct argument into the consumption function in a non-trivial way. But it's *also* a post-state variable in the sense that it exists at the moment of intertemporal transition. Non-trivial post-state variables are determined as a function of period t state variables and period t control variables, e.g. end-of-period assets are market resources less consumption. But permanent income as a post-state variable is a trivial (identity) function of its value as a state variable: permanent income at t equals permanent income at t.

…

On Thu, Aug 13, 2020 at 5:36 PM Sebastian Benthall ***@***.***> wrote: The PerfForesightConsumerType has two poststate_vars: aNrmNow and pLvlNow https://github.com/econ-ark/HARK/blob/master/HARK/ConsumptionSaving/ConsIndShockModel.py#L1450 However pLvlNow is set in getStates(): https://github.com/econ-ark/HARK/blob/master/HARK/ConsumptionSaving/ConsIndShockModel.py#L1665 ...and not in getPostStates(). Rather, aLvlNow is updated as a poststate in the code: https://github.com/econ-ark/HARK/blob/master/HARK/ConsumptionSaving/ConsIndShockModel.py#L1706 Correcting the poststate_var field to list aLvlNow instead of pLvlNow raises an AttributeError. I guess this is because the class is depending on the automatic initialization of the poststates, to create the attribute. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#798>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADKRAFMQIYJMPEF7MN2WOK3SARMHXANCNFSM4P632Z4A> .

[sbenthall]

Hmmm.

For context, I bumped into this while working on #761.
I'm trying to functionalize out some of the behavior and that requires consistency as to how the functions used in simulations and variable designations are used.

If I understand correctly, you are saying:

• pLvlNow is a state variable because it is a function of past state and stochastic shocks and is an input to the determination of a period's control values.
• It is also an intertemporal transition, so is in that respect a 'poststate'

I find this confusing. If the variable is being used intertemporally, wouldn't its value in the previous period be pLvlPrev, or $p_{t-1}$, and the new value be pLvlNow, or p? It seems like some notational slippage was introduced into the code here.

I.e., why not:

• in getStates, assign a value to state variable pLvlNow based on pLvlPrev and the stochastic shocks
• in getPostStates, assign to poststate variable pLvlPrev that value of pLvlNow.

(I expect that once the state variables are all properly listed out, it should be possible to automatically store the previous value for all of them. But that would be a separate issue.)

Also, what about aLvlNow? Can you explain why that is being set in getPostStates(), but not listed as a poststate_var?

[mnwhite]

I think I do that exact thing in some of the models, where there's a line of code in getStates: `pLvlPrev = self.pLvlNow` `pLvlNow = pLvlPrev*self.PermShkNow` The span of time in the code where what *was* pLvlNow is now pLvlPrev is extremely short. That can be moved to `getPostStates` if you want. In some sense it would make things more explicit, *but* it would introduce the bad feature that if you added `pLvlPrev` to track_vars, hoping it would track p_{t-1}, it would instead track p_t. `aLvlNow` isn't in poststate_vars for an IndShockConsumerType because it isn't needed at simulation initialization; note that it *is* in poststate_vars for GenIncProcessConsumerType. It's calculated here merely for user convenience. I guess it's also a slight code speedup for when the simulation is being run for a AggShockConsumerType as part of a Market being solved, and thus might be computed in parallel (to other instances of AggShockConsumerType).

…

On Fri, Aug 14, 2020 at 11:56 AM Sebastian Benthall < ***@***.***> wrote: Hmmm. For context, I bumped into this while working on #761 <#761>. I'm trying to functionalize out some of the behavior and that requires consistency as to how the functions used in simulations and variable designations are used. If I understand correctly, you are saying: - pLvlNow is a state variable because it is a function of past state and stochastic shocks and is an input to the determination of a period's control values. - It is also an intertemporal transition, so is in that respect a 'poststate' I find this confusing. If the variable is being used intertemporally, wouldn't its value in the previous period be pLvlPrev, or $p_{t-1}$, and the new value be pLvlNow, or p? It seems like some notational slippage was introduced into the code here. I.e., why not: - in getStates, assign a value to state variable pLvlNow based on pLvlPrev and the stochastic shocks - in getPostStates, assign to poststate variable pLvlPrev that value of pLvlNow. (I expect that once the state variables are all properly listed out, it should be possible to automatically store the previous value for all of them. But that would be a separate issue.) Also, what about aLvlNow? Can you explain why that is being set in getPostStates(), but not listed as a poststate_var? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#798 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADKRAFKUIZQGPVPTAFUPYCDSAVNCBANCNFSM4P632Z4A> .

[sbenthall]

Ok. As I said, the main thing that's needed for refactoring is consistency.

That's interesting about the tracking issue.
I suppose my ideal implementation would look like this:

• At the beginning of a period, move all the state variables into storage in a state_prev dictionary
• Use those 't - 1' values as an input to the transition function, which would
  • evaluate each state and control variable in the order determined by their equations
• track all the state and control variables in every run by default

In the short term, I'd like to go ahead with having the previous state stored in pLvlPrev consistently across the library, since I don't think I'm going to be able to convince you to drop the 'poststate' construct without a very compelling implementation in hand.

[llorracc]

This is related to the discussion Seb and I had with @albop last week.

I believe we agreed to move to a system in which there was a clearly defined point at which each exogenous shock was realized and at which each endogenous decision was made.

The transition equations would define the state variables. But we would need to have a name for every state variable that exists at any point -- including some that might not have names in the mathematical formulation of the problem.

For example, in the portfolio problem:

Subperiod	State	Shock	Operation
0	b_{t} = a_{t-1} \RPort	\Risky	Transition
1	p_{t}=p_{t-1}\PermShk_{t}	\PermShk_{t}	Transition
2	y_{t} = p_{t}\TranShk_{t}	TranShk	Transition
3	m_{t} = b_{t} + y_{t}		Transition
4	a_{t} = m_{t} - c_{t}		c_{t} is chosen
5	\PortShare		\PortShare chosen optimally

There are several virtues of disarticulating things in this way:

1. There are no more "post-state" and "pre-state" variables. There are just the states at each subperiod.
2. The state variables are all defined by the transition equations -- there's no need to have any separate articulation of what is a state and what is a control
3. All the variables have a unique value within the period. So we don't need to have extra notation to decide whether the value in question is a "pre" or a "post" value
4. In the case of models where we are really thinking of shocks as being realized simultaneously (like the risky return, the transitory shock, and the permanent shock, the order in which we lay out the draws of those shocks in sequence does not matter for the mathematical solution, but might matter sometimes for computational purposes.

[mnwhite]

I support what Chris just wrote, with the extension that I'd like to make *every single thing* its own subperiod or step or whatever we call it. Then there are only three basic kinds of operations: 1) Assign: Determining a variable as a *function* of other variables. 2) Draw: Determining variable(s) by drawing from a distribution. 3) Choose: Determine a variable by selecting among alternatives. In some sense, Choose is a subset of Assign: `cNrm = cFunc(mNrm)`. When "acted out", assigning consumption based on the consumption function evaluated at market resources is just another kind of assignment. However, `cFunc` is not part of the agent's problem, but is instead determined as part of its solution. All Assign and Draw steps could be executed once the *problem* has been specified, but any Choose step only has a value when the model has been "solved" in some sense. This is also the way to indicate to the future "magic solver" that *this thing* is to be optimized, and *these* are the variables in the information set at the time the choice is made. Note that I specifically wrote Draw as potentially multivariate, whereas the other ones have one variable on the LHS. Any multivariable distribution can be written as a sequence of conditional multivariate distributions, but that can add work for the user if we mandate that. Somewhat related, a Choose step might have a multivariate LHS purely for computational speed reasons.

…

On Mon, Aug 17, 2020 at 1:40 PM Christopher Llorracc Carroll < ***@***.***> wrote: This is related to the discussion Seb and I had with @albop <https://github.com/albop> last week. I believe we agreed to move to a system in which there was a clearly defined point at which each exogenous shock was realized and at which each endogenous decision was made. The transition equations would define the state variables. But we would need to have a name for every state variable that exists at any point -- including some that might not have names in the mathematical formulation of the problem. For example, in the portfolio problem: Subperiod State Shock Operation 0 b_{t} = a_{t-1} \RPort \Risky Transition 1 p_{t}=p_{t-1}\PermShk_{t} \PermShk_{t} Transition 2 y_{t} = p_{t}\TranShk_{t} TranShk Transition 3 m_{t} = b_{t} + y_{t} Transition 4 a_{t} = m_{t} - c_{t} c_{t} is chosen 5 \PortShare \PortShare chosen optimally There are several virtues of disarticulating things in this way: 1. There are no more "post-state" and "pre-state" variables. There are just the states at each subperiod. 2. The state variables are all defined by the transition equations -- there's no need to have any separate articulation of what is a state and what is a control 3. All the variables have a unique value within the period. So we don't need to have extra notation to decide whether the value in question is a "pre" or a "post" value 4. In the case of models where we are really thinking of shocks as being realized simultaneously (like the risky return, the transitory shock, and the permanent shock, the order in which we lay out the draws of those shocks in sequence does not matter for the mathematical solution, but might matter sometimes for computational purposes. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#798 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADKRAFO43SXEECQXLHQPCTLSBFTP5ANCNFSM4P632Z4A> .

[sbenthall]

Ok cool. I think this is something we are getting consensus around then.

I'm working towards making this happen.

I'm trying to start with some more basic refactoring: #760 #761 #660
These all have to do with make sure each model variable (exogenous shock, control variable, state variable) is getting accounted explicitly in each model an attached unambiguously to a bit of functionality for how it's getting assigned.

When everything has been standardized around that, it should be possible to write a "magic simulator" that uses them, along with a decision rule for the control variables.

Then "magic solvers" can be written that algorithmically find good decision rules.

[llorracc]

@mnwhite makes a good point about multivariate statistical draws, and that for computational efficiency we might want to have several LHS variables rather than one.

Let me expound a point that I forgot to make earlier, then elaborate on it in light of Matt's response.

The point was that we should refer to subperiods by the name of the product(s) returned therein, rather than by a numerical order (0, 1, etc). So the earlier figure should have been

Subperiod	State	Shock	Operation
b	b_{t} = a_{t-1} \RPort	\Risky	Transition
p	p_{t}=p_{t-1}\PermShk_{t}	\PermShk_{t}	Transition
y	y_{t} = p_{t}\TranShk_{t}	TranShk	Transition
m	m_{t} = b_{t} + y_{t}		Transition
a	a_{t} = m_{t} - c_{t}		c_{t} is chosen
portshare	\PortShare		\PortShare chosen optimally

If (counterfactually) the most efficient way to solve this problem were a simultaneous method that produced both c and portshare, the last line could be modified as below. And if it were necessary to draw the transitory and permanent shocks jointly (because they are mutually correlated, we could modify the representation as below

Subperiod	State	Shock	Operation
b	b_{t} = a_{t-1} \RPort	\Risky	Transition
PermShk_TranShk	$\psi,\theta$	\PermShkTranShkMatrix	Draw
p	p_{t}=p_{t-1}\PermShk_{t}	\PermShk_{t}	Transition
y	y_{t} = p_{t}\TranShk_{t}	TranShk	Transition
m	m_{t} = b_{t} + y_{t}		Transition
c_portshare	\PortShare		c and \PortShare chosen optimally
a	a_{t} = m_{t} - c_{t}		Transition

Then we would be able to write sentences like "In the period labeled c_portshare, both the level of consumption and the share of the portfolio invested in risky assets are jointly determined."

[mnwhite]

I'd prefer more descriptive indexing/naming, but I agree they shouldn't be absolutely numbered (at least by us; the code will look at their order and number them).

…

On Mon, Aug 17, 2020 at 5:11 PM Christopher Llorracc Carroll < ***@***.***> wrote: @mnwhite <https://github.com/mnwhite> makes a good point about multivariate statistical draws, and that for computational efficiency we might want to have several LHS variables rather than one. Let me expound a point that I forgot to make earlier, then elaborate on it in light of Matt's response. The point was that we should refer to subperiods by the name of the product(s) returned therein, rather than by a numerical order (0, 1, etc). So the earlier figure should have been Subperiod State Shock Operation b b_{t} = a_{t-1} \RPort \Risky Transition p p_{t}=p_{t-1}\PermShk_{t} \PermShk_{t} Transition y y_{t} = p_{t}\TranShk_{t} TranShk Transition m m_{t} = b_{t} + y_{t} Transition a a_{t} = m_{t} - c_{t} c_{t} is chosen portshare \PortShare \PortShare chosen optimally If (counterfactually) the most efficient way to solve this problem were a simultaneous method that produced both c and portshare, the last line could be modified as below. And if it were necessary to draw the transitory and permanent shocks jointly (because they are mutually correlated, we could modify the representation as below Subperiod State Shock Operation b b_{t} = a_{t-1} \RPort \Risky Transition PermShk_TranShk $\psi,\theta$ \PermShkTranShkMatrix Draw p p_{t}=p_{t-1}\PermShk_{t} \PermShk_{t} Transition y y_{t} = p_{t}\TranShk_{t} TranShk Transition m m_{t} = b_{t} + y_{t} Transition c_portshare \PortShare c and \PortShare chosen optimally a a_{t} = m_{t} - c_{t} Transition Then we would be able to write sentences like "In the period labeled c_portshare, both the level of consumption and the share of the portfolio invested in risky assets are jointly determined." — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#798 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADKRAFJ2CQ4O4F6VUGUJJOTSBGMIHANCNFSM4P632Z4A> .

[sbenthall]

The equations/rules for assignments will specify a partial order, but not necessarily a total order, on the variables.

In my view, this means we should not consider them to be numbered.

It also means that the exogenous shocks can be both "simultaneous" and at differentiated. What's important is that they can be evaluated prior to state or control variables.

[sbenthall]

I might not get to the full refactor of post-post-state glory in the next PR.

For the transitional work, @mnwhite could you tell me if aLvlNow is a state or a post-state in AggShockConsumerType ?

[albop]

I'm getting back to this discussion by pure chance. There is one argument I didn't convey properly. In my view, the definition of a subperiod does not result from a succession of operations, but rather by a change n the state-space that does not correspond to a change in the information set (information set being indexed by t). It defines the succession of functional spaces that you are going to go through when doing your time-iteration. When you do $m_{t} = b_{t} + y_{t}$ you move to a new subperiod, not because there is a new subperiod (there isn't) but because the state-space has changed from $b_t$ to $m_t$. It defines how you iterate "backward": i..e you start by functions of m_t (`c(m_t)`, then functions of `b_t` (psi(b_t), p(b_t), ...). Conceptually, all these function could be computed as a function of m_t or b_t, but we would loose useful information. So this is not the same as defining an order in which you compute the transitions (b then p then y then m). It is possible to keep this order to compute transitions from one subperiod to the next. When doing time-iteration, if there is a system of several functions to solve for, which have the same state-space, one can also look for the right succession of operation (either provide it or autodetect it). Now, the idea of labeling implicitly subperiod (in the sense I introduced), with the name of the last computed variable does make some sense: after you have computed `m` then `m` is the new state-space where functions are defined. But what if you have more than one state ? Which variables do you keep to define the state-space in a given subperiod ? I suppose when could name subperiod, $(m,p)$ the period obtained just after m and p have both been computed. I'm not saying it doesn't work but it requires a bit more some thought compared to the 0,1,2,3... version which certainly does.

…

On Mon, Sep 14, 2020 at 10:20 PM Sebastian Benthall < ***@***.***> wrote: I might not get to the full refactor of post-post-state glory in the next PR. For the transitional work, @mnwhite <https://github.com/mnwhite> could you tell me if aLvlNow is a state or a post-state in AggShockConsumerType ? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#798 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACDSKJBBP2VQLTXC5UMYQLSFZ3HTANCNFSM4P632Z4A> .

[sbenthall]

In my view, there is not a linear series of subperiods.

Rather, over a period at time t, there is a directed acyclic graph of variables.

In simulation, a decision rule induces the choice variables into a normal random variable with conditional dependencies on its inputs. State variables are like this as well. Exogenous shocks are the roots (no parents) of the DAG structure. Simulation proceeds down the DAG. In principle, the evaluation of two nodes at the same "depth" can be parallelized at the computational level. Nothing makes it sequential, necessarily.

When doing the solution, I expect you can work in reverse in a similar way. Start with the "deepest" nodes in the DAG, solve them, then go by backwards induction to "back" of the graph.

So I think I agree with what @albop is saying, in that what matters about a "period" is which variable's states are being sampled/solved. But calling them "subperiods" imposes a linearity on this process needlessly.

[sbenthall]

I've renamed this issue to make it clear that it is now about the new proposed architecture.
I will slate it for HARK 1.0 though that may be ambitious.

[albop]

Actually I find the term subperiod confusing. I prefer post-state which is less ambiguous. How to generalize it ? Post-post-state or post-k-state or just k-state ?

[albop]

On Fri, Sep 18, 2020 at 4:00 PM Sebastian Benthall ***@***.***> wrote: The definitions still work for natural numbers. You could define 1,2,3,4 to be a period then 1,2 and 3,4 would be subperiods. And, anyway, it would be nice if naming conventions for a subcategory of problems (discrete time) matched ones for the broader class (continuous time), especially when the latter matches physical experience. I would certainly not deny that intervals are well defined on the natural numbers. There is, in the natural numbers, a minimum non-degenerate interval size, 1, which cannot be subdivided. Maybe this is what you meant by an "atomic period". Speaking strictly from the point of convention: - The MDP formalism defines the relationship between States and Actions ordered by their atomic periods. Because of how this formalism is defined, the problem at t is isomorphic to the problem at t+1, and there is not much redundant information. It's a natural way to break the problem down. - It may be possible to transform an MDP into a different "time-scale" by aggregating N atomic periods into larger intervals. It's not clear why anybody would do this. - Time at the sub-atomic level simply does not work in the same way that it works at the atomic level. It is different because: (a) it is not totally ordered, and (b) it has no periodic structure--i.e., it is not repeating itself frame by frame. I hesitate to assert the following, as it may muddy the waters, but: "Physical experience" is a bit of an oxymoron. The phenomenology of time is quite different from the social consensus around it, and indeed if we were *truly* physicists we concede time to be much more complex than the real number system (i.e., "space-time"). So I don't think "physical experience" is solid ground for choice of mathematical formalism in this case. I just suspected that if we look at "frames" coming from the splitting of one max operation over several argument into several max operators we are naturally getting ordered frames. Considering a DAG of variables, as determined by the transition function, the following will be true: - There exists at least one valid linear ordering of the variables based on topological sorting <https://en.wikipedia.org/wiki/Topological_sorting>. - If we consider "framings" to be a partitioning of the variables and each subset in the partition a "frame", there will exist a framing such - that the frames will have a unique topological order. Problem is not whether frames are clearly and uniquely defined by

transitions but which set of frames are useful.

…

Do you have one easy characterization of one if the models you are working on ? Not yet. Thank you for the prompt to derive one. The difficulty I have had is that I have not been able to find the mathematical definition of the classes of models that "Economists find interesting", in this context. I am now digging for a formal description of the scope of heterogeneous agent models (HAM), as opposed to ABMs, and MDPs, but have not found one yet. My hope is that as the formal constraints crystalize in the software, it will be expressive enough for me construct such a model. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#798 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACDSKM7COACBA2KLMATVDTSGNRXVANCNFSM4P632Z4A> .

[albop]

On Fri, Sep 18, 2020 at 4:00 PM Sebastian Benthall ***@***.***> wrote: The definitions still work for natural numbers. You could define 1,2,3,4 to be a period then 1,2 and 3,4 would be subperiods. And, anyway, it would be nice if naming conventions for a subcategory of problems (discrete time) matched ones for the broader class (continuous time), especially when the latter matches physical experience. I would certainly not deny that intervals are well defined on the natural numbers. There is, in the natural numbers, a minimum non-degenerate interval size, 1, which cannot be subdivided. Maybe this is what you meant by an "atomic period".

I defined "atomic period" earlier in this discussion as a continuous interval that doesn't contain any new information. I'm not pushing for this vocable by the way, just clarifying what I meant. With this definitions, frames can live within one atomic period.

Speaking strictly from the point of convention: - The MDP formalism defines the relationship between States and Actions ordered by their atomic periods. Because of how this formalism is defined, the problem at t is isomorphic to the problem at t+1, and there is not much redundant information. It's a natural way to break the problem down. There is an alternative formulation of the Markov property where the

future distribution of of a process can be forecast using (X_t, X_{t-1}, ... X_{t-k}) with some finite k. By redefining the states, it can be recast as a one-period memory process but when formulating a model it is sometimes easier to refer to states that are not in the immediate past.

- It may be possible to transform an MDP into a different "time-scale" by aggregating N atomic periods into larger intervals. It's not clear why anybody would do this. seasonality for instance. As a modeller you might want to refer to t+1 as

the next quarter, while t+4 would be the next year.

- Time at the sub-atomic level simply does not work in the same way that it works at the atomic level. It is different because: (a) it is not totally ordered, and (b) it has no periodic structure--i.e., it is not repeating itself frame by frame. I hesitate to assert the following, as it may muddy the waters, but: "Physical experience" is a bit of an oxymoron. The phenomenology of time is quite different from the social consensus around it, and indeed if we were *truly* physicists we concede time to be much more complex than the real number system (i.e., "space-time"). So I don't think "physical experience" is solid ground for choice of mathematical formalism in this case.

This is a quite useless comment. By physical experience, I mean day to day life, not general relativity or Planck units. I just suspected that if we look at "frames" coming from the splitting of

…

one max operation over several argument into several max operators we are naturally getting ordered frames. Considering a DAG of variables, as determined by the transition function, the following will be true: - There exists at least one valid linear ordering of the variables based on topological sorting <https://en.wikipedia.org/wiki/Topological_sorting>. - If we consider "framings" to be a partitioning of the variables and each subset in the partition a "frame", there will exist a framing such that the frames will have a unique topological order. Do you have one easy characterization of one if the models you are working on ? Not yet. Thank you for the prompt to derive one. The difficulty I have had is that I have not been able to find the mathematical definition of the classes of models that "Economists find interesting", in this context. I am now digging for a formal description of the scope of heterogeneous agent models (HAM), as opposed to ABMs, and MDPs, but have not found one yet. My hope is that as the formal constraints crystalize in the software, it will be expressive enough for me construct such a model. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#798 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACDSKM7COACBA2KLMATVDTSGNRXVANCNFSM4P632Z4A> .

[sbenthall]

1. d. in discrete time, one can define natural ("atomic") periods as intervals [t1, t2[ such that F_t = F_t1 for any t in [t1,t2[ and F_t1 ⊄F_t2 . With these notations, t+1 is "one unit of time later", but does not necessarily denote the "next" period, as in "the next time information will be released.

What is "F" in this notation?

There is an alternative formulation of the Markov property where the
future distribution of of a process can be forecast using (X_t, X_{t-1}, ... X_{t-k}) with some finite k.

True.

seasonality for instance. As a modeller you might want to refer to t+1 as
the next quarter, while t+4 would be the next year.

I see. Yes, seasonality. This case has been quite tricky.

My impression of this discussion so far is that there are many equivalent ways to model the seasonality problem, and that the sticking point is computational tractability.

My own preference would be, in terms of setting up the model:

• reserve t (time, time step) as the natural numbers, scoped such that the k=1 Markov property holds
• a different variable, such as c, be used to represent the number of time steps within a cyclical superperiod (i.e, c = 4 for four seasons, where one time step (t) is a season.)
• a different variable, such as s, advances deterministically as t mod c.

But then have the solver and simulator know how to handle these cases intelligently and not waste time on, say, computing what happens if s_(t+1) = s_(t) + 2 because that never happens.

I.e., separate the model definition and computational levels of abstraction cleverly.

This is a quite useless comment.

I will wait at least a decade before trying to debate this point with you further.
But at that point, interplanetary macroeconomics may be viable and I would like to revisit the topic.

Problem is not whether frames are clearly and uniquely defined by
transitions but which set of frames are useful.

Ah. This gets to the heart of the matter.

Is there a way to formulate what "usefulness" is?

If so, maybe there's a way to programmatically discover the best framings of a particular model.

[sbenthall]

And I wanted to have an example in mind where this is not the case. I can see why it would be the case for some limited information problem though. In the basic consumption-savings model it is not the case. In fact there another condition is met that a frame does not loose information w.r.t. its parent frame. Do you have one easy characterization of one if the models you are working on ?

Trying to get at this more directly... suppose you have a problem with several control variables:

• consumption, informed by current market assets
• endogenous labor quantity, informed by an exogenous physical energy level
• risky asset allocation share, informed by an exogenous and intermittent selective attention to stock prices.

I wonder if it matters whether:

• (a) these variables are determined in sequential order, each made with knowledge of the preceding choices, or
• (b) if the variables are determined only in light of the state at t-1.

Maybe the consumer can only investigate their asset allocations on days that they don't labor.

[albop]

On Fri, Sep 18, 2020, 4:47 PM Sebastian Benthall ***@***.***> wrote: 1. d. in discrete time, one can define natural ("atomic") periods as intervals [t1, t2[ such that F_t = F_t1 for any t in [t1,t2[ and F_t1 ⊄F_t2 . With these notations, t+1 is "one unit of time later", but does not necessarily denote the "next" period, as in "the next time information will be released. What is "F" in this notation?

The infamous filtration ;-) in other words a sigma-algebra used to measure information up to date t. In France we usually call it a "tribe" and there is a recurring joke that most annoying questions in a math seminar can be answered by "it is a custom of my tribe". It is the standard mathematical apparatus to define random processes both in discrete and continuous time. There is an alternative formulation of the Markov property where the

future distribution of of a process can be forecast using (X_t, X_{t-1}, ... X_{t-k}) with some finite k. True. seasonality for instance. As a modeller you might want to refer to t+1 as the next quarter, while t+4 would be the next year. I see. Yes, seasonality. This case has been quite tricky. My impression of this discussion so far is that there are many equivalent ways to model the seasonality problem, and that the sticking point is computational tractability.

Not necessarily. There is a difference between the modeling language, and the convention we take and the internal representation. The internal representation can be a markov chain while the user-facing formalism could be expressed differently.

My own preference would be, in terms of setting up the model: - reserve *t* (time, time step) as the natural numbers, scoped such that the k=1 Markov property holds - a different variable, such as *c*, be used to represent the number of time steps within a cyclical superperiod (i.e, c = 4 for four seasons, where one time step (t) is a season. - a different variable, such as *s*, advances deterministically as *t* mod *c*. But then have the solver and simulator know how to handle these cases intelligently and not waste time on, say, computing what happens if s_(t+1) = s_(t) + 2 because that never happens. I.e., separate the model definition and computational levels of abstraction cleverly.

Yes it is the kind of of things we need to differentiate physical time and computational ticks.

This is a quite useless comment. I will wait at least a decade before trying to debate this point with you further.

But at that point, interplanetary macroeconomics may be viable and I would

like to revisit the topic.

By then quantum computing might be a thing which will probably allow for conflicting opinions to be simultaneously true.

Problem is not whether frames are clearly and uniquely defined by transitions but which set of frames are useful. Ah. This gets to the heart of the matter. Is there a way to formulate what "usefulness" is?

No that is to be defined. It is probably not about simulating a model though but rather about specifying additional informations to solve it.

…

If so, maybe there's a way to programmatically discover the best framings of a particular model. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#798 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACDSKO3M4X7BOCS3WSU4QTSGNXI7ANCNFSM4P632Z4A> .no

[albop]

Yes it is this kind of example which could make a difference. Some quick comments : - if it's about changing the order of arrival of new information or about taking all decisions at date t, to me it is a problem of type 1, i.e. about cycles and about language shortcuts to define and redefine them - if it's about ignoring selectively some shocks when taking some decisions it becomes about "decision frames" (I actually like the addition of "decision" there)

…

On Fri, Sep 18, 2020, 5:22 PM Sebastian Benthall ***@***.***> wrote: And I wanted to have an example in mind where this is not the case. I can see why it would be the case for some limited information problem though. In the basic consumption-savings model it is not the case. In fact there another condition is met that a frame does not loose information w.r.t. its parent frame. Do you have one easy characterization of one if the models you are working on ? Trying to get at this more directly... suppose you have a problem with several control variables: - consumption, informed by current market assets - endogenous labor quantity, informed by an exogenous physical energy level - risky asset allocation share, informed by an exogenous and intermittent selective attention to stock prices. I wonder if it matters whether: - (a) these variables are determined in sequential order, each made with knowledge of the preceding choices, or - (b) if the variables are determined only in light of the state at t-1. Maybe the consumer can only investigate their asset allocations on days that they don't labor. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#798 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACDSKKKXRPM66736CAYPCLSGN3L3ANCNFSM4P632Z4A> .

[sbenthall]

It is getting to the point where it will be possible to implement a general framework for this. I expect it will be viable to work on it soon after #836 is pulled.

I'm wondering, from a software design standpoint, how we would like to support this in HARK.

My preference would be for this architecture to ultimately be in HARK.core.

My understanding is that the status quo is to support the overwriting of the core functionality for custom models in cases like this. See the work of @Mv77 on the new multi-stage portfolio model, this comment from @mnwhite, and the work in PR #832

Recall that the goal for this work is twofold:

• to support the frame-based organization of a model
• to allow the user to write a model configuration that is not "hard coded" as a model class.

I'm somewhat ambivalent between trying to modify the core first and then refactoring all the existing models to fit, vs. the perhaps more conservative strategy of making minimal changes to the core while subclassing AgentType to create a class with the desired API. This latter approach was what I was aiming for when working on #696.

Speaking of #696, I wonder how its experimental approach to dictionary-based model configuration looks in light of the more recent discussions in this thread. While in that experimental PR each state has its own individual transition function--because the dictionary of transition functions uses single variable names as keys--it would also be possible for it to use tuples of variables names as keys, allowing for transition functions that determine multiple variables 'at once'.

[sbenthall]

I'm now trying to figure out a generic data structure for the frame architecture (this is informed by work on #862). @Mv77 wondering if you could weigh in on this especially.

I think we have consensus about what a frame is. The question is how to design the implementation so that it elegantly does all the things we want it to do.

This is a list of desiderata for what a Frame implementation might have:
(1) A Frame should know what model variables are updated when it is simulated. Name suggestion: target
(2) A Frame should know what model variables it depends on when it is simulated.
(3) A Frame should know the transition function. This is used in simulations.
(4) A Frame should ideally know how to differentiate its own transition function. This is used in solutions. (I could be wrong here ... could somebody please confirm? @llorracc @mnwhite )

I have been playing with implementations in terms of the basic data structures that come with Python. But now I'm wondering if we should have a dedicated Frame class that can capture all this information.

The other question that's coming up for me is that when listing the variables that a transition function depends on, it needs to distinguish between current time values and previous time values for any referenced model variables.

This can be done structurally (i.e., separate lists of variable names for 'now' and previous values) or notationally (i.e., adding 'prev' to indicate a previous value). I can see pros or cons to either approach.

I have a prototype implementation ready for your review at PR #865 but it sidesteps these issues, which I do think will matter moving forward.

[Mv77]

On the data-structure question, my own uninformed prior is that the most future-proof way to do things would be for frames to have their own class. They seem like a complex enough object that at some point in the future you might want to add properties or attributes to it that might not be accommodated by primitive structures without substantial changes. It might also make it more user friendly in the sense of, e.g., knowing that the 'target' is at frame.target instead of having to remember/look up that it is on the third entry of some tuple named frame.

[sbenthall]

Thanks @Mv77 that does sound right.

The implementations in #865 and #866 don't yet use a dedicated class, but I will move them in that direction.

I think we can get the best of both worlds with a sensible constructor. I.e. initialize the Frame with something like

Frame(target, scope, transition)

[sbenthall]

Just a broader example of what the Frame initialization fragment may look like

Frame(
    target=('pLvl', 'PlvlAgg', 'bNrm', 'mNrm'),
    scope=('aNrm','Risky',"Adjust'),
    transition=transitionEquation,
    default=(0,1),
    # anything else that's needed for the solver? such as...
    arbitrage=arbitrageEquation
)

[llorracc]

One point to make here is that "arbitrage equation" is a legacy from the Representative Agent world in which one of the chief cheats is that all optimization solutions are interior solutions for which a first order condition holds. That often will not be true. (Like, in our dcegm tool). So I think I'd advocate replacing "arbitrage" with "optimality" in the above, which allows for the possibility of dcegm-like tools.

[sbenthall]

Roger that.
What does 'optimality' refer to in general? A function to be optimized?

[llorracc]

Definition of an objective function that should be maximized by the agent. Maybe the category should be "objective" or something like that rather than "optimality". We should consult Pablo on this.

…

On Tue, Nov 17, 2020 at 6:50 PM Sebastian Benthall ***@***.***> wrote: Roger that. What does 'optimality' refer to in general? A function to be optimized? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#798 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAKCK75T7KNHVAHBPFHYFPDSQMD5LANCNFSM4P632Z4A> .

-- - Chris Carroll

[sbenthall]

I have another question about the design of frames.

Control variables may be subject to bounds that functions of other state variables.

https://dolo.readthedocs.io/en/latest/model_specification.html#boundaries

In @llorracc 's examples in this thread, which I'm working from, it's not clear where the bounds are specified.

My intuition is:

• A frame with a control variable may also have bounds for that variable
• Any state variables that are inputs to the functions computing the bounds of the control variable should be included in the 'scope' of the frame

What I'm unclear about is how a frame-specific constraint would impact the complexity of a frame's solution problem, and whether these constraints are any impediment to the composability of frame solutions in backwards induction.