How to scale SHD and SID to a common scale, for instance, for plotting?
Causal discovery algorithms are commonly benchmarked using the Structural Hamming Distance (SHD) and the Structural Interventional Distance (SID). SHD measures structural differences between the equivalence classes of two networks and treats them as probabilistic graphical models. SID measures structural differences in terms of differential intervention outcomes. The question is: how to plot SHD and SID against each other in a way that makes visual sense across networks with very different numbers of nodes?
A simulation study based on tabu search and BIC ran over a wide range (37–724 nodes) of networks from the Bayesian network repository suggests that SHD and SID can be rescaled to vary in a common range as
SHD / #ARCS ~ 1.5 * SID / (#ARCS / 2)^2
where #ARCS is the number of arcs in the reference network.
Behold:
Even splitting the plot into different panels works well.
Note that the 1.5 multiplier is purely empirical, so it may not center SID with respect to SHD perfectly
in every simulation study.
Sat Mar 14 11:28:09 2026 with bnlearn
5.2-20251203
and R version 4.5.2 (2025-10-31).
