COLLAPSED VARIATIONAL BAYESIAN INFERENCE FOR
LATENT DIRICHLET ALLOCATION (CVB-LDA)
Version 0.1

David Andrzejewski ([email protected])
Department of Computer Sciences
University of Wisconsin-Madison, USA

This software implements Collapsed Variational Bayesian (CVB)
inference [1] for the LDA model [2] of discrete count data.  The code
is based on the Teh et al paper [1], but also uses some practical
implementation details kindly provided by the authors on the extremely
helpful topic-models mailing list:

https://lists.cs.princeton.edu/mailman/listinfo/topic-models

The code implements the collapsed variational inference for LDA as a
Python C extension module.


DISCLAIMER

Importantly, I wrote this code as an educational exercise for *my own
benefit only*. This software has no connection with [1] or its authors
whatsoever. Any errors contained herein are my own.


BUILD/INSTALL

Building this module requires installation of Python and NumPy, as
well as a C compiler.  From the command-line, do:

% python setup.py install

(Note that if NumPy or Python are installed to non-standard locations,
you may need to make the appropriate changes in setup.py)

You can then test the installation with:

% python test/testCvbLDA.py -v

There is also a simple example scipt showing how to use the software:

% python example.py


LOCAL INSTALL

If you do not have write access to your Python installation directories,
you will need to tell setup.py to install this module somewhere else.
For example:

% python setup.py install --prefix=~/local

will install the module under a subdirectory of your home directory called 
"local".

It may then be necessary to let Python know where that is by setting
the PYTHONPATH environment variable (e.g., in .bashrc or .cshrc).  For
our example this might involve adding something like the line:

setenv PYTHONPATH ~/local/lib/python2.5/site-packages


HOW TO USE

The commenting in the example.py script explains the meanings and
types of all input and return arguments.  For more detailed examples
of good/bad inputs, you can examine the tests in test/testCvbLDA.py.


LICENSE

This software is open-source, released under the terms of the GNU
General Public License version 3, or any later version of the GPL (see
LICENSE.txt).


REFERENCES

[1] 
A Collapsed Variational Bayesian Inference Algorithm for Latent Dirichlet Allocation 
Teh Y.W., Newman D., and Welling, M.
Advances in Neural Information Processing Systems (NIPS) 19, 2007.

[2] 
Latent Dirichlet Allocation
Blei, D. M., Ng, A. Y., and Jordan, M. I. 
Journal of Machine Learning Research (JMLR), 3, Mar. 2003, 993-1022.


VERSION HISTORY
0.1     Initial release
0.1.1   Return (phi,theta,gamma) instead of just (phi,theta)
        gamma is a length D List of T x Md NumPy arrays