Memory Module

The bayes_hdc.memory module provides content-addressable memory structures.

SparseDistributedMemory

class bayes_hdc.memory.SparseDistributedMemory(locations, contents, dimensions, radius)[source]

Bases: object

Sparse Distributed Memory (SDM) for content-addressable storage.

Parameters:

  • locations (Array)

  • contents (Array)

  • dimensions (int)

  • radius (float)

locations: Array
contents: Array
dimensions: int
radius: float
static create(num_locations, dimensions, radius=0.0, key=None)[source]
Parameters:

  • num_locations (int)

  • dimensions (int)

  • radius (float)

  • key (Array | None)

Return type:

SparseDistributedMemory

write(address, value)[source]
Parameters:

  • address (Array)

  • value (Array)

Return type:

SparseDistributedMemory

read(address)[source]
Parameters:

address (Array)

Return type:

Array

__init__(locations, contents, dimensions, radius)
Parameters:

  • locations (Array)

  • contents (Array)

  • dimensions (int)

  • radius (float)

Return type:

None

HopfieldMemory

class bayes_hdc.memory.HopfieldMemory(patterns, dimensions, beta=1.0)[source]

Bases: object

Modern continuous Hopfield network (Ramsauer et al. 2020).

One-step softmax-attention retrieval over stored patterns. Distinct from the classical sign-thresholded recurrent Hopfield network (Hopfield 1982) — which settles via repeated application of a sign update — and from the spiking-neuron cleanup memories of Stewart, Tang & Eliasmith (2010), which run on populations of leaky- integrate-and-fire neurons via the Neural Engineering Framework. Retrieval here is a single feed-forward softmax over cosine similarities to the stored patterns; no recurrent settling.

References: Ramsauer, H. et al. (2020). Hopfield Networks is All You Need. arXiv:2008.02217. Hopfield, J. J. (1982). Neural networks and physical systems with emergent collective computational abilities. PNAS 79(8): 2554-2558. Stewart, T. C., Tang, Y., Eliasmith, C. (2010). A Biologically Realistic Cleanup Memory: Autoassociation in Spiking Neurons. Cognitive Systems Research 12: 84-92.

Parameters:

  • patterns (Array)

  • dimensions (int)

  • beta (float)

patterns: Array
dimensions: int
beta: float = 1.0
static create(dimensions, beta=1.0)[source]
Parameters:
Return type:

HopfieldMemory

add(pattern)[source]
Parameters:

pattern (Array)

Return type:

HopfieldMemory

retrieve(query)[source]
Parameters:

query (Array)

Return type:

Array

__init__(patterns, dimensions, beta=1.0)
Parameters:

  • patterns (Array)

  • dimensions (int)

  • beta (float)

Return type:

None

AttentionMemory

class bayes_hdc.memory.AttentionMemory(keys, values, dimensions, temperature=1.0, num_heads=1)[source]

Bases: object

Attention-based retrieval with key-value storage and multi-head support.

Parameters:

  • keys (Array)

  • values (Array)

  • dimensions (int)

  • temperature (float)

  • num_heads (int)

keys: Array
values: Array
dimensions: int
temperature: float = 1.0
num_heads: int = 1
static create(dimensions, temperature=1.0, num_heads=1)[source]
Parameters:
Return type:

AttentionMemory

write(key, value)[source]
Parameters:

  • key (Array)

  • value (Array)

Return type:

AttentionMemory

write_batch(keys, values)[source]
Parameters:

  • keys (Array)

  • values (Array)

Return type:

AttentionMemory

retrieve(query)[source]
Parameters:

query (Array)

Return type:

Array

retrieve_with_weights(query)[source]
Parameters:

query (Array)

Return type:

tuple

__init__(keys, values, dimensions, temperature=1.0, num_heads=1)
Parameters:

  • keys (Array)

  • values (Array)

  • dimensions (int)

  • temperature (float)

  • num_heads (int)

Return type:

None