hierarchical-predictive-coding-transformer
Research codebase exploring hierarchical predictive coding as a training principle for transformers (HERT and variants).
Research codebase exploring hierarchical predictive coding as a training principle for transformers (HERT and variants).
Karpathy-style nanoGPT fork exploring adaptive-compute transformers with per-block evidence accumulation.
How do patterns emerge from experience? An ongoing exploration of local learning in spiking neural networks.
Training a recurrent network to settle into its attractor state faster - LISSOM, but differentiable.
Learning latent-variable probability distributions directly, as histograms over quantized bins.
What if a variational autoencoder were used as a convolution kernel?
Information-density normalization - adjust histogram bins so precision follows where the data is.