Comments on the HOI repository

[brovelli]

I have a few general comments about the repository.

1. Theoretical background
https://brainets.github.io/hoi/theory.html

• I would add a paragraph that explains that the goal of the toolbox is to infer HOI from time series. And that this is a open problem in biology and neuroscience, where you don't have access to the HOI dataset (as in authorship or social contact patterns). This is an important point that must be clearly stated and I think the introduciton is a good place. You can use as examples social contacts and neural time series. I would also introduce right from the beginning some disclaimer about the distinction between functional HOI and structural HOI. Structural HOI are something else that we don't cover here. You can cite the mechaninsms vs beh paper as well.
• In addition, I would prefer to say from the beginning that we are talling about functional higher-order interactions related to the collective behavior of a system, and which we try to infer from the analysis of the node activity in a multivariate manner.
• The references do not start from 1, correct to the right order
• Equations are not numbered, can you add them?
• The equation of the Entropy is missing the i index in the summation term and in the p(x_i)
• For the parametric estimation you can cite Ince et al. Gaussian copula paper [13] in addition to [11]
• The equation of the MI, normally in the papers we use I(X;Y) rather than MI(X;Y), but it's ok to keep MI
• The "From pairwise to higher-order interactions" is very important, because it defines the "network behavior" and "network encoding". I would add a clearer description and definition of the two categories, maybe using a bullet point list. The "Network behavior " category contians measures that quantify collective higher-order behaviors from multivariate time series. These information theoretical measures quantify the degree of higher-order functional interactions between time series. The "Network encoding" category contains measures that quantify the information carried by higher-order functional interactions about a set target variables.
• In the same lines, I would add two simpe examples of "Network encoding" and "Network behavior" measures. For example, a measure of network would quantify the degree of HOI among multivariate time series, whereas a measure of network encoding would quantify the amount of information about a sensory or behavioral variable is carried by HOIs.
• In the TC equation the definition of $X^n$. It is present but later in the text.
• In the DTC equation, the definition of $X_-j$ is missing. Remind what -j is
• The definition of the Oinfo in terms of "balance between higher-order and lower-order constraints among the set of variables of interest." is not clear. I never found it clear myself actually, and the sentence is unclear. Either remove it or clarify.
• In the Topological information section, the definition of cardinality is not given. And the summation term is a bit misterious. What is X(I)?
• In the Oinfo equation you use -i whereas you used -j in the previous equations. You should keep the same format.
• Make the titles "Network behavior" and "Network encoding" larger and more visible
• Maybe I would repeat the definition of the two categories "Network behavior" and "Network encoding" at the beginning of each section
• In the Gradient of O-information you use "target" instead of the "Y" variable. You should probably stick to Y because the code uses is.
• In the Synergy and redundancy (MMI), you should define what is MMI. You can use the same equations of the MEG paper page 33. The definition of Redundancy with i<n is not clear. In addition, the equation of the synergy uses -i whereas you used -j in the previous equations
• Correct "Synergy and redundancy integraed..."
• I think the equation with the red and syn should contain the tagert variable. In the MEG paper I used R_MMI(X_n;Y) and S_MMI(X_n;Y) which are eqs. 11 and 12. I would suggest to use this formatlism.
• The paragraph on IID is not very clear, it lacks some key information such as the need to generalise the PID to multiple targets and the forward and backward PID. Then , we should define the DoubleRedundancy function within the MMI frameworks etc. Finally, we should be careful with the definitions and equations of the Syn and Red for the IID, because they look too similar to those of the PID, but they are completely different theoretically. Overall, we need to work more on the details of the IID. See for example the section "3.6.3. Integrated Information Decomposition" in this paper, I think it is clear (although too long): https://www.mdpi.com/1099-4300/26/2/108 . Otherwise from Luppi's TINS paper, there should be a summary of the IID https://www.cell.com/trends/cognitive-sciences/fulltext/S1364-6613(23)00284-X
• You can take the same formalism for the equations from this paper as well.
• The Bibliography contains some strange numbering.... check

2. Glossary

• The definition of HOI is not precise. I would remove the sentence "which cannot be explained by simply considering pairwise interactions." by a sentence in which it makes reference to the lower order. Something like "which cannot be explained by lower order interactions".
• See also my previous point on the need to clarify that we search for functional HOI.
• The definition of redundancy is not very clear. I would additionally use the definion from the PID litterature, such "shared information about a target variable". Now the definition is too general. For clarity, I would stick to the def from PID.
• I would probably add a glossary on the multvariate gaussian model? Or at least a short definition?

[brovelli]

3. Quickstart

• Add at the beginnig that the repo contains tools for the inference of HOI from multivariate variables (e.g. time series from brain areas) at the node level.

[brovelli]

4. Tutorial. How to simulate redundancy and synergy

• Check some typos in the "Redundant and synergistic encoding" of https://brainets.github.io/hoi/auto_examples/tutorials/plot_sim_red_syn.html#sphx-glr-auto-examples-tutorials-plot-sim-red-syn-py
• In the tutorial, it would be better if you use capital X and Y as in the Theoretical page, instead of lower case x and y. Not in the code, I mean in the main markdown text. In fact, I would do the same as in the ML tutorial section "Data simulation" https://brainets.github.io/hoi/auto_examples/tutorials/plot_ml_vs_it.html#sphx-glr-auto-examples-tutorials-plot-ml-vs-it-py
• I would make clearer titles specifying the Sections "Network behavior" and "Network encoding"

[brovelli]

5. Example - Introduction to core information theoretical metrics
https://brainets.github.io/hoi/auto_examples/it/plot_tutorial_core.html#sphx-glr-auto-examples-it-plot-tutorial-core-py

• I would use the capital X and Y as in the Theoretical page amd ML tutorial, section "Data simulation" https://brainets.github.io/hoi/auto_examples/tutorials/plot_ml_vs_it.html#sphx-glr-auto-examples-tutorials-plot-ml-vs-it-py

6. Example - Comparison of mutual-information estimators
https://brainets.github.io/hoi/auto_examples/it/plot_mi.html#sphx-glr-auto-examples-it-plot-mi-py

• In section https://brainets.github.io/hoi/auto_examples/it/plot_mi.html#mi-of-data-sampled-from-normal-distribution the covariance varia could be written a sigma_xy instead of C. Actually, it's up to you.

7. Example - Comparison of entropy estimators for a multivariate normal
https://brainets.github.io/hoi/auto_examples/it/plot_entropies_mvar.html

• Revise a bit the text of the first paragraph, clean it up a bit. You acn use chatGPT, it's made for this :). Here is an example:
  When calculating Higher-Order Interactions using entropy-based metrics, the process includes estimating the entropy of multivariate variables. Nevertheless, certain entropy estimators may not function optimally as the number of dimensions increases. This can be demonstrated through an example where various entropy estimators are compared using data sampled from a multivariate normal distribution with increasing dimensionality.
• The equation of the theoretical entropy seems different from the one in the previous example. Check
• Comment on the last figure of the example. The biaas for GC in red seems higher at 4-dim and then it decrease with 6- and 8-dim data. Isn't that strange?

8. Example - Comparison of entropy estimators for various distributions
https://brainets.github.io/hoi/auto_examples/it/plot_entropies.html

• The equation of the theoretical entropy seems different from the one in the previous example. Check

9. Example - Topological Information : conditioning on orders
https://brainets.github.io/hoi/auto_examples/metrics/plot_infotopo.html#sphx-glr-auto-examples-metrics-plot-infotopo-py

• The first paragraph is not verry clear, GPT it! For example:
  The following example highlights a metric known as "Topological Information," which was introduced by Baudot et al. in 2019 (ref. [4]). This metric, founded on disparities in entropies, enables estimation of Higher Order Interactions while possessing the notable characteristic of being able to segregate specific orders. Specifically, when simulating redundancies at the third order utilizing the O-information metric (hoi.metrics.Oinfo), any quadruplets containing the superfluous triplet will yield redundancy. Conversely, the InfoTopo metric (hoi.metrics.InfoTopo) has the capability of pinpointing exclusively the redundant triplet.
• For the InfoTopo you can cite and link the webpage of Baudot and the tutorials, they were complete https://infotopo.readthedocs.io/en/latest/

10. Example - Redundancy-Synergy Index
https://brainets.github.io/hoi/auto_examples/metrics/plot_rsi.html

• Why do you use S instead of X in the equation for the RSI?
• Again use the latex capital X and Y in the main text

11. Example - Integrated Information Decomposition
https://brainets.github.io/hoi/auto_examples/metrics/plot_syn_phiID.html

• The refs for IID of Mediano et al are missing. You can also cite the latest from Luppi TINS 2024.
• I think we should be clearer on the way the terms are computed. I guess we should briefly specify the terms of the IID and those that we can compute such as the Red and Syn. In fact, there could be some confusion with the classical Red and Syn from the PID with a single target from the other examples. So, we should explain better that these IID Red and Syn are different. Maybe in the theoretical backgroun we could make a longer paragraph with some details.

12. Example - Redundancy and Synergy MMI
https://brainets.github.io/hoi/auto_examples/metrics/plot_syn_red_mmi.html

• Variable S instead of X? You should clarify more why we need S and X
• Add a link to the Theoretical introduction

13. Example - O-information and its derivatives for network behavior and encoding
https://brainets.github.io/hoi/auto_examples/metrics/plot_oinfo.html

• The equation of the TC has j subscript, but the rest of the repo uses i for the nodes.
• Same for the TDC, O-info and S-information equations later in the text
• Then for the gradient O-info, the i indedx is used instead...

[brovelli]

Additional comments

1. I would like to link HOI to other packages for higher-order analysis. At least some suggestions. Maybe we could have a section dedicated to this, because we could create a network of packages that intergate. For example:

• Software for higher-order networks
• https://networkx.org/ ?
• https://github.com/pyt-team/TopoNetX ?
• https://github.com/HGX-Team/hypergraphx

2. Acknowledgements section. EBRAINS, Amidex, ANR for example should be added

[brovelli]

GREAT WORK GUYS!!!!

[EtienneCmb]

Hi @brovelli,

Thank you for your comment. I went through the examples and considered most of your comments. Please check the following new sections :

• Funding
• List of related software
• List of contributors

[EtienneCmb]

@Mattehub can you please commit your changes to the theoretical part today? Also, @brovelli made some comments to the example 11. Example - Integrated Information Decomposition. This the example you pushed, can you take care of Andrea's suggestions for this specific example please?

[Mattehub]

Hi @EtienneCmb, of course, I can take care of those comments. I am sorry if I still have not started a pull request, but I have problems compiling the docs with sphinx on my computer... do you have ideas about how to deal with the extension error of sphinx_gallery.gen_gallery ? It never happened to you?

This is the error it gives me and I still did not menage to solve it this morning.

Extension error (sphinx_gallery.gen_gallery):
Handler <function fill_gallery_conf_defaults at 0x0000022A585172E0> for event 'config-inited' threw an exception (exception: process PID not found (pid=20620))

If I do not menage I will push the commits anyway and I see if there are problems later...

[EtienneCmb]

@Mattehub , no problem, you can make your PR today and we can fix those issues later