Default value for the gamma parameter of the extended BIC

The extended BIC (eBIC) score was originally introduced by Foygel and Drton in this NIPS paper for Gaussian graphical models. Its construction extends the classical BIC score with an additional penalty term and an associated penalty coefficient γ. Their simulation studies suggest that γ = 0.5 works well enough in different scenarios, which makes it an ideal default value for Gaussian Bayesian networks (score = "ebic-g"). But is γ = 0.5 a good default for discrete Bayesian networks as well (score = "ebic")?

Using hc() to perform structure learning for 11 reference networks from the Bayesian network repository across sample sizes between 0.1 * nparams(bn) and 5 * nparams(bn), we evaluated γ = {0, 0.125, 0.25, 0.5, 0.75, 1}. The value γ = 0 corresponds to the classical BIC score, which we can use to normalise SHD as SHD(eBIC(gamma)) / SHD(BIC) to have a common scale for all the plots. The lower the normalized SHD values, the more eBIC improves on BIC.

In the small sample size regime between 0.1 * nparams(bn) and 0.5 * nparams(bn) the value γ = 0.5 sometimes gives the most improvement, sometimes it is neither the best nor the worst γ, and it is never the worst performer of the values under consideration.

plot of chunk unnamed-chunk-2

The same is true in the large sample size regime between 1 * nparams(bn) and 5 * nparams(bn).

plot of chunk unnamed-chunk-3

In conclusion, γ = 0.5 seems a reasonable default choice that can perform well and never performs the worst among γ = {0, 0.125, 0.25, 0.5, 0.75, 1}. If nothing else, it ensures better performance than BIC in the vast majority of simulations in the small sample size regime.

Last updated on Tue Nov 29 18:24:11 2022 with bnlearn 4.9-20221107 and R version 4.2.2 Patched (2022-11-10 r83330).