SpectralLDA

Note: This is the single-host version, for the up-to-date and distributed version please refer to [https://github.com/Mega-DatA-Lab/SpectralLDA-Spark].

This code implements a Spectral (third order tensor decomposition) learning method for the Latent Dirichlet Allocation model in Python.

The Spectral learning method works with empirical counts of word pair or word triplet from any document in the dataset. We average the counts and put them in tensors. We then perform tensor decomposition to learn the Latent Dirichlet Allocation model. For more details, please refer to report.pdf in the repository.

Usage

Invoke spectral_lda with the doc-term count matrix. At output we'd learn alpha for the Dirichlet prior parameter, beta for the topic-word-distribution, with one topic per column.

# docs is the doc-term count matrix
# alpha0 is the sum of the Dirichlet prior parameter
# k is the rank aka number of topics
from spectral_lda import spectral_lda
alpha, beta = spectral_lda(docs, alpha0=<alpha0>, k=<k>, l1_simplex_proj=False)

# alpha is the learnt Dirichlet prior
# beta is the topic-word-distribution matrix
# with one column per topic

By default each column in beta may not sum to one, set l1_simplex_proj=True to perform post-processing that projects beta into the l1-simplex.

References

Anandkumar, Animashree, Rong Ge, Daniel Hsu, Sham M. Kakade, and Matus Telgarsky, Tensor Decompositions for Learning Latent Variable Models.