Memory Module¶

The bayes_hdc.memory module provides content-addressable memory structures.

SparseDistributedMemory¶

class bayes_hdc.memory.SparseDistributedMemory(locations, contents, dimensions, radius)[source]¶

Bases: object

Sparse Distributed Memory (SDM) for content-addressable storage.

Parameters:

locations (Array)
contents (Array)
dimensions (int)
radius (float)

locations: Array¶

contents: Array¶

dimensions: int¶

radius: float¶

static create(num_locations, dimensions, radius=0.0, key=None)[source]¶

Parameters:

num_locations (int)
dimensions (int)
radius (float)
key (Array | None)

Return type:

SparseDistributedMemory

write(address, value)[source]¶

Parameters:

address (Array)
value (Array)

Return type:

SparseDistributedMemory

read(address)[source]¶

Parameters:: address (Array)
Return type:: Array

__init__(locations, contents, dimensions, radius)¶

Parameters:

locations (Array)
contents (Array)
dimensions (int)
radius (float)

Return type:

None

HopfieldMemory¶

class bayes_hdc.memory.HopfieldMemory(patterns, dimensions, beta=1.0)[source]¶

Bases: object

Modern continuous Hopfield network (Ramsauer et al. 2020).

One-step softmax-attention retrieval over stored patterns. Distinct from the classical sign-thresholded recurrent Hopfield network (Hopfield 1982) — which settles via repeated application of a sign update — and from the spiking-neuron cleanup memories of Stewart, Tang & Eliasmith (2010), which run on populations of leaky- integrate-and-fire neurons via the Neural Engineering Framework. Retrieval here is a single feed-forward softmax over cosine similarities to the stored patterns; no recurrent settling.

References: Ramsauer, H. et al. (2020). Hopfield Networks is All You Need. arXiv:2008.02217. Hopfield, J. J. (1982). Neural networks and physical systems with emergent collective computational abilities. PNAS 79(8): 2554-2558. Stewart, T. C., Tang, Y., Eliasmith, C. (2010). A Biologically Realistic Cleanup Memory: Autoassociation in Spiking Neurons. Cognitive Systems Research 12: 84-92.

Parameters:

patterns (Array)
dimensions (int)
beta (float)

patterns: Array¶

dimensions: int¶

beta: float = 1.0¶

static create(dimensions, beta=1.0)[source]¶

Parameters:

dimensions (int)
beta (float)

Return type:

HopfieldMemory

add(pattern)[source]¶

Parameters:: pattern (Array)
Return type:: HopfieldMemory

retrieve(query)[source]¶

Parameters:: query (Array)
Return type:: Array

__init__(patterns, dimensions, beta=1.0)¶

Parameters:

patterns (Array)
dimensions (int)
beta (float)

Return type:

None

AttentionMemory¶

class bayes_hdc.memory.AttentionMemory(keys, values, dimensions, temperature=1.0, num_heads=1)[source]¶

Bases: object

Attention-based retrieval with key-value storage and multi-head support.

Parameters:

keys (Array)
values (Array)
dimensions (int)
temperature (float)
num_heads (int)

keys: Array¶

values: Array¶

dimensions: int¶

temperature: float = 1.0¶

num_heads: int = 1¶

static create(dimensions, temperature=1.0, num_heads=1)[source]¶

Parameters:

dimensions (int)
temperature (float)
num_heads (int)

Return type:

AttentionMemory

write(key, value)[source]¶

Parameters:

key (Array)
value (Array)

Return type:

AttentionMemory

write_batch(keys, values)[source]¶

Parameters:

keys (Array)
values (Array)

Return type:

AttentionMemory

retrieve(query)[source]¶

Parameters:: query (Array)
Return type:: Array

retrieve_with_weights(query)[source]¶

Parameters:: query (Array)
Return type:: tuple

__init__(keys, values, dimensions, temperature=1.0, num_heads=1)¶

Parameters:

keys (Array)
values (Array)
dimensions (int)
temperature (float)
num_heads (int)

Return type:

None