Agent communication based on networks

[meiertgrootes]

In (many) ABMs (even with a spatial component) communication/interaction between agents is best described via networks. For example, for infectious disease modelling, agents are members in several networks, incl. household, profession, and a larger random encounter network. Such a model is potentially also well suited to a more general class of models investigating the impacts/outcomes of information spread.

Does FLAMEGPU2 currently support such models, and/or what could an implementation look like? This issue has been raised in #68 and is partially being addressed in #883. However, #883 only covers the situation where the model is described as a (static) network, rather than the agent communication being described as a (dynamic) network.

#883 also mentions that the currently implemented bucket communication scheme might suffice, but I'm not sure this is the case.
As an example, consider an infectious disease model. An agent has interactions with a number of other agents defined by a network. One possibility would be for each agent to write its infectious state to a bucket corresponding to a unique id associated with it. Then other agents would have to read a list of buckets. It is unclear to me whether this could be handled concurrently, or how most efficiently. It seems that each bucket would need to be read separately by its own agent function. The iteration through the buckets might be feasible in a submodel, but the question of efficiency looms large. In addition one would be left with synchronizing exposure times between two agents. Defining interaction lists and exposure time might be possible via environment variables, but overall the question is can this be implemented in an efficient fashion In FlameGPU2 at the moment or in the foreseeable future.

The alternative would be to have agents write to a bucket with a unique id corresponding to the agent they interact with. This would then only require the interrogating agent to read messages from one bucket, but require an iteration over all the buckets and agent would have to write to (as defined by a interaction list which would have to be upated via an environment variable). The same issues as above seem to apply.

We'd (@meiertgrootes, @cpranav93, @vmgaribay, @debraj001) be very interested in hearing your thoughts on this @mondus and @ptheywood and discussing if and how FLAMEGPU2 might be a suitable tool. Thanks

[mondus]

Hi @meiertgrootes . The FLAME GPU team had a chat about this. There are a few different approaches for what you want to do but it would depend very much on the details of your use case. I suggested when we spoke by email to consider using an agent array variable so that each agent could store a list of indices of agents that it are in its network (and that it want to receive information from). Each agent could then write an array (1D) message to a unique position (it's index) and agents could query their network by traversing their list of stored neighbours and then querying the 1D array message list using the stored indices.

The obvious limitations to the above would be that this would require agents to build their networks during initialization or using some other form of communication (e.g. spatial or perhaps bucket if they add friends when say visiting a building/website).

If you needed to be able to query friends of friends then the network would have to be stored in the environment (macro environment property). If you used the same 1D message strategy then you could traverse friends and then lookup friends of friends in the same way. This would however have to be static.

Some case would be needed regarding IDs. This would work best with agents having contiguous IDs defined at initialization as the message array would become sparse otherwise. Some degree of sparsity would be fine but if your model was very dynamic then another approach may be required.

Do any of the above fit your use case?

[meiertgrootes]

Hi @mondus, thanks for the suggestions. Indeed, 1d array messages seem promising for large networks with relatively low connectivity, while bucket messaging would seem well suited for smaller, fully connected networks.

Overall, all agents are nodes/vertices in several networks, at least one small fully connected network as well as several larger low-connectivity networks. For these latter networks, although the vertices remain constant (barring the death of an agent) the edges will change between each time step. I'd imagine that defining these edges could be performed by a host function, with the results (connection lists per agent) being stored as an environment variable that is updated. Would it be advisable/feasible to have multiple message arrays (e.g. 1 per network) or would lookup be more efficient if one common array were used?

While we currently are not considering friends of friends, I'd also imagine this approach would be amenable to that situation, provided the FoF list could also be passed as an environment variable. I guess this is quasi-static from the perspective of the agents.

Regarding you point about the IDs: could one simply use the agent IDs directly, or would it make sense to create a lookup with contiguous indices mapped to the agent IDS? The latter would like be necessary if multiple arrays (1 per network) were to be used, I'd think. Is the former

My apologies for not being more concrete, but we are in the process of both identifying the models shape and requirements as well as considering different possible implementation frameworks.

[mondus]

Hi @mondus, thanks for the suggestions. Indeed, 1d array messages seem promising for large networks with relatively low connectivity, while bucket messaging would seem well suited for smaller, fully connected networks.

Overall, all agents are nodes/vertices in several networks, at least one small fully connected network as well as several larger low-connectivity networks. For these latter networks, although the vertices remain constant (barring the death of an agent) the edges will change between each time step. I'd imagine that defining these edges could be performed by a host function, with the results (connection lists per agent) being stored as an environment variable that is updated. Would it be advisable/feasible to have multiple message arrays (e.g. 1 per network) or would lookup be more efficient if one common array were used?

An environment variable would certainly work for this purpose. I would be inclined to logically separate each of these per message rather than keep a common list.

While we currently are not considering friends of friends, I'd also imagine this approach would be amenable to that situation, provided the FoF list could also be passed as an environment variable. I guess this is quasi-static from the perspective of the agents.

Yes this would allow more complex traversal.

Regarding you point about the IDs: could one simply use the agent IDs directly, or would it make sense to create a lookup with contiguous indices mapped to the agent IDS? The latter would like be necessary if multiple arrays (1 per network) were to be used, I'd think. Is the former

You could use the agent IDs but if there is a high turn over of birth and death then the list would become quite sparse. Creating a list of contiguous addresses effectively requires a memory management system (think malloc) which would introduce quite a bit of serialization in a host function.

My apologies for not being more concrete, but we are in the process of both identifying the models shape and requirements as well as considering different possible implementation frameworks.

[LangLEvoI]

I've recently been trying to develop a static social network model using Agent Python. I ran into two problems when I implemented it:

a) As mentioned above, each agent stores neighbour IDs in a 1D array. But it seems all of IDs arrays have the same length (neighbours_size). So how can I assign 1D arrays with different lengths to respective agents when each agent has a different number of neighbours?

model = pyflamegpu.ModelDescription('network')
agent = model.newAgent('user')
agent.newVariableArrayInt('neighbours', neighbours_size)

# Is it possible here to assign 1D arrays with different lengths one by one?
t_pop = FLAMEGPU.agent('speaker')
for _ in range(AGENT_COUNT):
    t.setVariableArrayInt('neighbours', neighbours_list)

b) How to implement the np.random.choice() method or loop a list in an agent function? It looks agent function cannot support these types. In my model, each agent uses a 1D array to store neighbour's IDs and a 1D array to store selection probability (indegree_list/indegree.sum()). Each agent picks up a neighbour based on the neighbour's in-degree at each round/step. Higher in-degree, higher probability of being selected. The agent then replicates this neighbour's emoji.

import numpy as np

AGENT_COUNT = 5

message = model.newMessageArray('emoji')
message.setLength(AGENT_COUNT)
message.newVariableInt('emoji')

# 0~7, integer
agent.newVariableInt('emoji')
# A probability array that neighbours being selected according to their in-degree
agent.newVariableArrayFloat('prob', neighbours_size)


@pyflamegpu.agent_function
def output_message(message_in: pyflamegpu.MessageNone, message_out: pyflamegpu.MessageArray):
    # Return agent's ID and emoji
    message_out.setIndex(pyflamegpu.getID())
    message_out.setVariableInt('emoji', pyflamegpu.getVariableInt('emoji'))

    return pyflamegpu.ALIVE


@pyflamegpu.agent_function
def input_message(message_in: pyflamegpu.MessageArray, message_out: pyflamegpu.MessageNone):
    # Pick up a neighbour: higher in-degree, higher probability to be selected
    neighbours = pyflamegpu.getVariableIntArray('neighbours')
    probs = pyflamegpu.getVariableFloatArray('prob')

    neighbour_id = np.random.choice(a=neighbours, p=probs)
    
    msg = message_in.at(neighbour_id)
    pyflamegpu.setVariableInt('emoji', msg.getVariableInt('emoji'))

    return pyflamegpu.ALIVE

[Robadob]

@LangLEvoI

a) As mentioned above, each agent stores neighbour IDs in a 1D array. But it seems all of IDs arrays have the same length (neighbours_size). So how can I assign 1D arrays with different lengths to respective agents when each agent has a different number of neighbours?

That's not currently possible. You would need to declare the agent variable array with a maximum length, and then add a second variable to the agent to denote how many elements within the array for any particular agent are in use.

I'm not actually sure of the best way we could achieve variable length agent variable arrays, it's not a feature I recall our team considering. It would possibly be considered if/when we work on agent networks (#1121).

b) How to implement the np.random.choice() method or loop a list in an agent function? It looks agent function cannot support these types. In my model, each agent uses a 1D array to store neighbour's IDs and a 1D array to store selection probability (indegree_list/indegree.sum()). Each agent picks up a neighbour based on the neighbour's in-degree at each round/step. Higher in-degree, higher probability of being selected. The agent then replicates this neighbour's emoji.

Again not a feature I believe we've considered. You'd need to implement such an algorithm manually with the random primitives available (e.g. sum the probabilities(p), generate a random number in the range [0, p), iterate the probabilities calculating the cumulative sum to find the selected index.

If you think it should be considered, and have an idea for how it's API may look I would suggest creating an issue so the whole team can look at it and feedback, though I can't promise that it would be implemented as a priority.

---

Sadly, working with variable length arrays in C++/CUDA is not as trivial as Python lists.