Skip to content

Some experiments with linear RNNs with MLPs interleaved.

License

Notifications You must be signed in to change notification settings

adrian-valente/lru_experiments

Repository files navigation

LRU pytorch experiments

An experimental framework to train LRU networks in pytorch (linear recurrent unit networks, see Orvieto et al., Resurrecting Recurrent Neural Networks for Long Sequences, 2023), and fit dynamical systems with them, among other tasks.

Also supporting material for my blog post RNNs strike back in the notebook ds_exps.ipynb.

Code adapted from the jax/flax implementation of N. Zucchet minimal-LRU.

Still unstable and may change significantly.

About

Some experiments with linear RNNs with MLPs interleaved.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published