Skip to content

Permanent Memory

Jump to bottom Edit New page
Innocent Bystander edited this page Sep 7, 2015 · 7 revisions

Otherwise known as permamem, this framework feature stores data on all conversations and users the bot encounters.

It is intended as a workaround for some of the known limitations (forgotten users/conversations) of the current version of the hangups library.

Table of Contents

Internal Data Structure

Permanent Conversation Memory

Other names: convmem

The bot stores a persistent memory of all conversations in memory.json under the key convmem.

The memory of each conversation is updated each time the bot is run and for every event that occurs during the bot's operation. The structure of the "convmem" is directly accessible via bot.conversations.catalog, and is as follows: 

bot.conversations.catalog = bot.memory["convmem"] = {
  '<conversation id>' : {
    "source": "init", # or event
    "title": "<conversation title>",
    "type": "GROUP", # or ONE_TO_ONE
    "updated": "20150101000000",
    "participants": [
      "<user chat id>",
      "<user chat id>",
      "<user chat id>"
  },
  '<conversation id>' : { ... }
}

Permanent User Memory

The hangups User object is also cached in the user_data key in memory.json. These records are updated each time a conversation event occurs, if there are any changes in the user list and/or individual user attributes. 

bot.memory["user_data"]["<user chat id>"]["_hangups"] = {
  "chat_id": "<user chat id>",
  "gaia_id": "<user gaia id>",
  "first_name": "<user first name>",
  "full_name": "<user full name>",
  "emails": [
    "<user email 1>"
  ], # or []
  "photo_url": "<user photo url>", # or null
  "is_self": false, # true for bot account
  "is_definitive": false, # true if canonical record from API query or hangups
  "updated": "20150101000000",
}

Helper Functions

accessing conversations list

bot.conversations.get([<convfilter>])

Hint: You can use convfilter from CONVID Commands to test this function
  • <convfilter> can be:
    • id:<convid> or <convid> - returns single matching conversation
    • text:<text fragment> - return conversations with matching fragment in title
    • "" (or not supplied) - returns all conversations (no filter)
    • chat_id:<chatid> - returns conversations where chatid is a participant
    • [ver>=2.7] tag:<conversation tag> - returns conversations with matching tag
    • [ver>=2.7] type:ONE_TO_ONE or type:GROUP - returns conversations matching type (case-insensitive)
    • [ver>=2.7] minusers:<num> - returns conversations with at least <num> users
    • [ver>=2.7] maxusers:<num> - returns conversations with at most <num> users
    • [ver>=2.7] random:<float> - returns conversations based on random selection criteria: random.random() generates numbers in range [0.0, 1.0), if the generated number is lower or equal to <float> the conversation will be added
  • [ver>=2.7] supports multiple filter terms with sequential boolean operations
    • each term must be enclosed in brackets
    • supported boolean operators: and, or
  • returns a filtered conversation catalog based on <convfilter>`
    • the return is always a {dictionary} indexed by `convid`

example: conversations retrieval 

# data retrieval examples

## single conversation returned
[(convid, convdata)] = bot.conversations.get(<conversation id>).items()
[convdata] = bot.conversations.get(<conversation id>).values()

## multiple conversations returned
for convid, convdata in bot.conversations.get("text:something").items():
    # ...do something

example: boolean operations 

bot.conversations.get("(text:bot)and(tag:test-tag)")
# returns conversations with "bot" in title AND tagged "test-tag"

bot.conversations.get("(text:bot)or(chat_id:111111)")<br />
# returns conversations with "bot" in title OR has user 111111

conversation titles

bot.conversations.get_name(<conv|conv id>, truncate=False, fallback_string=None)

  • drop-in replacement for hangups.ui.utils.get_conv_name
  • truncate will always be False and serves no purpose except to provide compatibility for hangups.ui.utils.get_conv_name(conv, truncate=True) usage - internally, no truncation will occur
  • returns the conversation name for the given conversation object or id
  • if conv (hangups Conversation object) is supplied:
    1. try to get title from from permanent memory; otherwise,
    2. get it via permamem.name_from_hangups_conversation(), which is a custom adaptation of hangups.ui.utils.get_conv_name without the presentation logic
  • if conv id (conversation id) is supplied:
    1. try and get title from permanent memory; otherwise,
    2. return fallback_string if provided, or raise a ValueError exception

list unique users from one or more conversations

bot.get_users_in_conversation(<conversation id|[list of conversation ids]>)

  • returns a list of unique hangups.user.User for the given conversation id(s)
  • the user list is derived from two sources:
  • bot.conversations.catalog
    • always complete
    • basic fields only: id_.chat_id, id_.gaia_id, full_name, is_self
  • bot._user_list._user_dict
    • if available, overwrites the incomplete entry derived from bot.conversations.catalog
    • incomplete for less active users
    • additional fields: emails, photo url, first_name
  • each user object in the list will always have the following values set properly: id_.chat_id, id_.gaia_id, full_name, is_self
  • never returns the bot's own user details, please see bot.user_self()

retrieve most-complete single user record

bot.get_hangups_user(<string chat_id|namedtuple hangups.user.UserID>)

  • implements workaround for unreliable hangups user list (related to forgotten conversation bug)
  • returns a single hangups.user.User for the supplied parameter(s)
  • tries hangups-based bot._user_list first and will re-construct the hangups.user.User object from permanent conversation memory if not found. if the user still cannot be found, will return a user with name "unknown user"
  • the user object will always have the following values set properly: id_.chat_id, id_.gaia_id, full_name, is_self
  • depending on the source the user object may also have the following fields set correctly: emails, photo url, first_name

retrieve most-complete conversation record

bot.get_hangups_conversation(<string conv_id|hangups.conversation.Conversation>)

  • implements workaround for unreliable hangups conversations list (related to forgotten conversation bug)
  • returns a single HangupsConversation instance that inherits from hangups.conversation.Conversation
  • the returned object is re-built from permanent conversation memory, with additional data cloned from the hangups-based bot._conv_list when available - it is intended to be a reliable substitute for an actual instance of hangups.conversation.Conversation via duck-typing
    • the _user_list of this derivative object is instantiated as a blank list - the users property has been overridden to directly support permanent user memory. Thus this object has a more complete user list than hangups itself
  • it is highly-recommended to use this functionality instead of attempting to bot._conv_list.get(), which usually results in general pain and confusion ;)

Permamem Diagnostics & Repair

[ver<2.7] Requires plugin: unittest_convmem, [ver>=2.7] no plugin dependency

Helper commands for permamem that can be used to perform some diagnostics and permanent memory manipulation.

All these commands are admin-only.

show definitive unknown users 

/bot dumpunknownusers
  • prints cached users marked as unknown but definitive
  • command maintained mostly for historical purposes - the bot will mark unknown or fallback users as indefinite

make unknown users not definitive 

/bot resetunknownusers
  • removes the is_definitive flag from cached but unknown users
  • command maintained mostly for historical purposes - the bot will mark unknown or fallback users as indefinite

force user data refresh 

/bot refreshusermemory <chat id> [<chat id> [...]]

remove conversation record 

/bot removeconvrecord <convid> [<convid> [...]]
  • permanently removes conv record in memory (until next update)
  • use if you a stray conversation data hanging around in memory.json even though the bot has previously left the conversation

makes bot refresh every user record 

/bot makeallusersindefinite
  • makes all cached user records indefinite (sets is_definite = False)
  • command maintained mostly for historical purposes - the bot will mark unknown or fallback users as indefinite
  • use with care - it will take time to rebuild the user list again

Plugin List | Developer Reference: [ Intro | Plugins | Sinks | In-built Functionality | [Configuration] (Configuration) ]

# ## ###

Clone this wiki locally