Pitman-Yor Mixture (PYM) Entropy Estimator

Do you desperately need to estimate the entropy of a discrete distribution from data? You’re in luck.

PYMentropy is a MATLAB package providing routines to compute the Bayes’ least squares estimate (i.e., posterior mean) of the entropy of a discrete distribution from samples under a Pitman-Yor Mixture (PYM) prior. The PYM prior is a mixture of Pitman-Yor (or 2-parameter Poisson-Dirichlet) distributions over countably infinite discrete distributions, with mixing weights set to produce an approximately flat prior over entropy. Also computes the posterior variance for quantifying uncertainty.

You can visit the GitHub project page, view the README, or just download the code as a zip file directly.

References:

  1. Archer E, Park IM, Pillow JW (2014). Bayesian entropy estimation for countable discrete distributions. Journal of Machine Learning Research (JMLR), 15 (Oct): 2833-2868.
    [bibtex|pdf]
  2. Archer E, Park I, Pillow JW (2012). Bayesian estimation of discrete entropy with mixtures of stick-breaking priors. Advances in Neural Information Processing Systems (NIPS) 25.
    [bibtex|poster|pdf]

  2 comments for “Pitman-Yor Mixture (PYM) Entropy Estimator

  1. Andy
    November 18, 2014 at 6:25 am

    Hi Evan, I very much appreciate your work and I’m really interested in applying this entropy estimator. Still I’m facing a problem as I want to make use of this estimator in a C# application without the need for using Matlab. Therefore it would be nice to have a compiled version of this implementation in terms of a dll file. Using the compiler toolbax Matlab provides this possibility. Unfortunately I don’t have access to this toolbox. Alternatively, also a c, c++, c# or java implementation would be useful. Please let me know. Thanks a lot! Andy

    • Evan
      November 18, 2014 at 11:04 am

      Hi Andy,

      Thanks for your interest! I’d love to make the code available in C# (or other compiled languages) but it’s probably not something I’ll undertake in the near future. I’ve looked a bit into the MATLAB compiler toolbox, but I don’t seem to have access to it currently, and I’m not sure how easy it would be to distribute the MATLAB-compiled versions of the code. I’d be very happy to give my input if you were interested in developing your own code.

      Evan

Leave a Reply

Your email address will not be published. Required fields are marked *