id.json

[
    {
        "title": "SaNN: Simple Yet Powerful Simplicial-aware Neural Networks"
    },
    {
        "review": {
            "id": "4sGhXacqSF",
            "forum": "eUgS9Ig8JG",
            "replyto": "eUgS9Ig8JG",
            "signatures": [
                "ICLR.cc/2024/Conference/Submission9491/Reviewer_4crX"
            ],
            "nonreaders": [],
            "readers": [
                "everyone"
            ],
            "writers": [
                "ICLR.cc/2024/Conference",
                "ICLR.cc/2024/Conference/Submission9491/Reviewer_4crX"
            ],
            "content": {
                "summary": {
                    "value": "This paper considers the design of neural networks for simplicial complexes, which are more general combinatorial structures than graphs, but less general than hypergraphs. The authors propose to use multihop aggregation schemes to build an architecture that is more expressive than the simplicial Weisfeiler-Lehman isomorphism test, while satisfying useful invariance, equivariance, and expressivity properties. They also demonstrate the efficientcy of their proposed method, and its performance for a few different tasks in simplicial data processing."
                },
                "soundness": {
                    "value": "3 good"
                },
                "presentation": {
                    "value": "3 good"
                },
                "contribution": {
                    "value": "3 good"
                },
                "strengths": {
                    "value": "1. For the most part, this paper is well-written, and is easy to digest for someone who is familiar with graph neural networks. I don't think the intended audience of this paper includes someone not familiar with GNNs, but this is fine in my opinion.\n\n2. The proposed method is demonstrated to be quite efficient in comparison to existing ones, with similar performance as well."
                },
                "weaknesses": {
                    "value": "1. Certain definitions regarding the types of operators and features are not laid out clearly enough, which leads to ambiguity in the paper on a technical level. As noted in the list of questions and suggestions, the claimed properties of the proposed models are not clearly true, possibly due to this misunderstanding."
                },
                "questions": {
                    "value": "My most important concern is summarized in point 1 -- in particular, the ambiguities around orientation equivariance and the use of oriented operators built from the incidence matrices are what cause me to suggest this paper be rejected. If the authors are to focus on either of the two points in order to change my mind on this paper, it should be the first one.\n\n1. There are some details missing regarding the type of data being handled. In particular, the incidence matrices are not defined in a way sufficient for the discussion following in the paper. Normally, the incidence matrices have values of +-1 depending on a chosen reference orientation (usually given by some ordering of the nodes). Coupled to this, the signs of the features on the simplices are determined relative to the same reference orientation -- this gives meaning to the notion of orientation equivariance. Without discussing these things, orientation equivariance is not a meaningful concept within the context of the paper. \n\na. This calls into question the validity of the example in Section 4.1. You say that all simplices are given a feature value given by some scalar $a$ -- yet, the matrices acting on these feature vectors/matrices have an orientation associated to them. It seems as if you are using an *oriented operator* to act on *unoriented features*. Property 1 in this example is thus difficult to claim, as the property of orientation equivariance is one describing the action of *oriented operators* acting on *oriented features*, and how the choice of orientation to begin with is irrelevant to the computation. \n\nb. Furthermore, this problem yields a comparison for isomorphism testing incorrect, as the erroneous imposition of differently-oriented features relative to the chosen orientations could be used by SaNN to yield a \"false negative,\" i.e., saying that two isomorphic complexes are different. \n\nc. A more minor comment in this direction comes from the **Insights** section of Section 5.1. It is not correct to say that \"the superior performance of SaNN also proves the orientation equivariance of SaNN experimentally.\" Orientation equivariance is a simple mathematical property, and does not guarantee good performance, nor are all performant architectures on a given dataset orientation equivariant. These properties are possibly linked, but the claim that one proves the other in some way is not justified. \n\nd. Moreover, based on my reading of the appendix, many of their experimental setups for tasks other than trajectory prediction use \"unoriented data\" by simply assigning scalar values to high-order simplices, which is again incompatible with the use of oriented operators. Perhaps something in the implementation of SaNN in these examples does not use oriented operators such as the incidence matrices, but this is not clear to me. \n\nPlease either justify, clarify, or revise the paper's discussion regarding orientation equivariance.\n\n2. Related to the above point, the claims in Section 4.2 seem reasonable at first glance, but are not explained well enough. Permutation equivariance is easily seen to hold, so is not much of a concern. Orientation equivariance is subject to the problems noted above, so more clarification on the type of simplicial features and relevant operators needs to be made. That is not to say that the result proved in the appendix is wrong, but it needs to be clarified in order to be understood in a way that acts on oriented features. Simplicial awareness is more subtle than the other two, based on the definition from (Roddenberry et. al., 2021). For instance, some of the existing convolutional-type SNNs in the literature fail to satisfy simplicial awareness if they are implemented without nonlinear activation functions, due to the fact that the square of the (co)boundary operator is the zero operator. Perhaps it is the case that the assumptions of Theorem 4.2 are sufficient to exclude such methods, but a clearer connection is needed. It would be very helpful for the authors to briefly survey some of the methods they compare to, and clarify whether Theorems 4.1 and 4.2 apply or don't apply to them.\n\n---\n\nThank you for addressing my questions -- I have raised my suggested score."
                },
                "flag_for_ethics_review": {
                    "value": [
                        "No ethics review needed."
                    ]
                },
                "rating": {
                    "value": "6: marginally above the acceptance threshold"
                },
                "confidence": {
                    "value": "4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work."
                },
                "code_of_conduct": {
                    "value": "Yes"
                },
                "first_time_reviewer": {
                    "readers": [
                        "ICLR.cc/2024/Conference/Program_Chairs",
                        "ICLR.cc/2024/Conference/Submission9491/Senior_Area_Chairs",
                        "ICLR.cc/2024/Conference/Submission9491/Area_Chairs",
                        "ICLR.cc/2024/Conference/Submission9491/Reviewer_4crX"
                    ]
                }
            },
            "number": 1,
            "invitations": [
                "ICLR.cc/2024/Conference/Submission9491/-/Official_Review",
                "ICLR.cc/2024/Conference/-/Edit"
            ],
            "domain": "ICLR.cc/2024/Conference",
            "tcdate": 1698633222904,
            "cdate": 1698633222904,
            "tmdate": 1700612825353,
            "mdate": 1700612825353,
            "license": "CC BY 4.0",
            "version": 2
        },
        "responses": [
            {
                "id": "lVtTiAmxxy",
                "forum": "eUgS9Ig8JG",
                "replyto": "4sGhXacqSF",
                "signatures": [
                    "ICLR.cc/2024/Conference/Submission9491/Authors"
                ],
                "readers": [
                    "everyone"
                ],
                "writers": [
                    "ICLR.cc/2024/Conference",
                    "ICLR.cc/2024/Conference/Submission9491/Authors"
                ],
                "content": {
                    "comment": {
                        "value": "We sincerely appreciate your positive feedback and constructive remarks on our paper. We acknowledge that the unclear definitions may have led to some confusion regarding the properties of the proposed models. Upon reflection, we agree that the orientations of signals and simplices were not sufficiently defined, potentially leading to misunderstandings. We have made revisions to the paper, as detailed below, to enhance clarity and readability."
                    }
                },
                "number": 25,
                "invitations": [
                    "ICLR.cc/2024/Conference/Submission9491/-/Official_Comment"
                ],
                "domain": "ICLR.cc/2024/Conference",
                "tcdate": 1700466218293,
                "cdate": 1700466218293,
                "tmdate": 1700466218293,
                "mdate": 1700466218293,
                "license": "CC BY 4.0",
                "version": 2
            },
            {
                "id": "w7AyJJzGF1",
                "forum": "eUgS9Ig8JG",
                "replyto": "4sGhXacqSF",
                "signatures": [
                    "ICLR.cc/2024/Conference/Submission9491/Authors"
                ],
                "readers": [
                    "everyone"
                ],
                "writers": [
                    "ICLR.cc/2024/Conference",
                    "ICLR.cc/2024/Conference/Submission9491/Authors"
                ],
                "content": {
                    "title": {
                        "value": "Clarifying the Orientations of Simplices and Features"
                    },
                    "comment": {
                        "value": "We appreciate your feedback and recognize that our initial description, which stated \"The $(i, j)$th entry of $\\\\mathbf{B}_k$ is non-zero if the $i$th $(k\u22121)$-simplex is a boundary simplex of the $j$th $k$-simplex\", could have been clearer. \n\nWe have clarified in the revised manuscript that the $(i,j)$th entry of $\\mathbf{B}_k$ can be either $+1$ or $-1$, depending on the relative orientations of the $i$th $(k\u22121)$-simplex and the $j$th $k$-simplex. Even though in the introduction we mentioned about orientations of higher-order simplices as \u201cHigher-order simplices are usually oriented, ensuring a consistent node arrangement within each simplex, facilitating tasks like determining information flow directions along the edges.\u201d, we did not formally define the orientations of features on the simplicial complex in relation to the reference node ordering. Therefore, we have now specified in the revised manuscript that signs of features on simplices are determined based on the reference node arrangement within each simplex. Additionally, when features are unoriented, we clarify that there is no need for a reference orientation of simplices. In such cases, we consider unoriented incidence matrices.\n\nTo this end, we have incorporated the following points in Section 2 (Background) at appropriate locations:\n\n\"Each simplex has an orientation defined by a standardized vertex order, typically ascending or descending, establishing a standard node arrangement within each simplex.\"\n\n\"The non-zero entries of an oriented incidence matrix $\\mathbf{B}_k$ can be either $+1$ or $-1$, reflecting the relative orientations of the $i$th $(k\u22121)$-simplex and the $j$th $k$-simplex.\"\n\n\"The sign of feature $\\mathbf{X}_k[\\sigma_k]$ is determined based on the reference orientation of $\\sigma_k$.\"\n\n\"For unoriented features, there is no need for a reference orientation of simplices. In such cases, we consider unoriented incidence matrices. Specifically, the $(i, j)$th entry of an unoriented incidence matrix  $\\mathbf{B}_k$ is $1$ if the $i$th $(k\u22121)$-simplex is a boundary simplex of the $j$th $k$-simplex, and $0$ otherwise.\"\n\nWe believe that these revisions will enhance the comprehension of the orientation equivariance concept within our paper."
                    }
                },
                "number": 26,
                "invitations": [
                    "ICLR.cc/2024/Conference/Submission9491/-/Official_Comment"
                ],
                "domain": "ICLR.cc/2024/Conference",
                "tcdate": 1700466683794,
                "cdate": 1700466683794,
                "tmdate": 1700466720187,
                "mdate": 1700466720187,
                "license": "CC BY 4.0",
                "version": 2
            },
            {
                "id": "gR9uDjEuz9",
                "forum": "eUgS9Ig8JG",
                "replyto": "4sGhXacqSF",
                "signatures": [
                    "ICLR.cc/2024/Conference/Submission9491/Authors"
                ],
                "readers": [
                    "everyone"
                ],
                "writers": [
                    "ICLR.cc/2024/Conference",
                    "ICLR.cc/2024/Conference/Submission9491/Authors"
                ],
                "content": {
                    "title": {
                        "value": "Clarification on Orientations of Simplices in Experiments."
                    },
                    "comment": {
                        "value": "In our implementation of SaNN, we use oriented incidence matrices for feature aggregation when the features are oriented. On the other hand, we use unoriented incidence matrices when the features are unoriented. \n\nIn the numerical experiments for trajectory prediction, the input features are oriented flows on simplicial complexes, prompting us to use oriented incidence matrices for aggregation. For other experiments such as graph classification and simplicial closure prediction, no input features were given on the simplicial complexes. We constructed initial features on simplices as the sum of upper and lower adjacent simplices. As these features are unoriented, we utilized unoriented incidence matrices for feature aggregation.\n\nTo further clarify, we have included the following text in the respective subsections of Section 5 in the revised manuscript:\n\n1. In the Trajectory Prediction section: \"Trajectory prediction involves predicting the next node in a sequence formed by a series of nodes connected by edges, with oriented flows on the edges. As the features are oriented, we use oriented incidence matrices for aggregation.\"\n\n2. In the Simplicial-Closure Prediction section: \"In all simplicial closure prediction experiments, we consider the initial features on simplices as the cumulative count of their lower and upper adjacent simplices. Given that these are unoriented, we use unoriented incidence matrices for aggregation.\"\n\n3. In the Graph Classification section: \"For all graph classification experiments, we consider the initial features on simplices as the cumulative count of their lower and upper adjacent simplices.  As these are unoriented, we use unoriented incidence matrices for aggregation.\""
                    }
                },
                "number": 30,
                "invitations": [
                    "ICLR.cc/2024/Conference/Submission9491/-/Official_Comment"
                ],
                "domain": "ICLR.cc/2024/Conference",
                "tcdate": 1700468047382,
                "cdate": 1700468047382,
                "tmdate": 1700468160624,
                "mdate": 1700468160624,
                "license": "CC BY 4.0",
                "version": 2
            }
        ]
    },
    {
        "review": {
            "id": "tkcYyLyiYm",
            "forum": "eUgS9Ig8JG",
            "replyto": "eUgS9Ig8JG",
            "signatures": [
                "ICLR.cc/2024/Conference/Submission9491/Reviewer_LALK"
            ],
            "nonreaders": [],
            "readers": [
                "everyone"
            ],
            "writers": [
                "ICLR.cc/2024/Conference",
                "ICLR.cc/2024/Conference/Submission9491/Reviewer_LALK"
            ],
            "content": {
                "summary": {
                    "value": "The authors present a Simplicial Graph Neural Network, which considers higher-order structures in the input graphs. In comparison to previous work, the features from k-simplices are precomputed without trainable parameters and only then fed into a GNN. This leads to lower runtime during training since features can be reused in each epoch, which is validated by the authors theoretically and empirically.\n\nThe authors prove that their method is more powerful than the WL test and as powerful as the Simplicial WL (SWL) test, when it comes to distinguishing non-isomorphic subgraphs. Further, they prove permutation equivariance, orientation equivariance, and simplicial-awareness.\n\nThe method is evaluated on trajectory prediction, simplicial closure prediction, and graph classification, where it is on par/slightly outperforms previous works with better training runtimes."
                },
                "soundness": {
                    "value": "4 excellent"
                },
                "presentation": {
                    "value": "3 good"
                },
                "contribution": {
                    "value": "2 fair"
                },
                "strengths": {
                    "value": "- The goal of the work, achieving better scalability of expressive networks by using non-parametric simplicial encoders makes sense.\n- The authors thoroughly analyze their method theoretically and provide proofs for all relevant properties.\n- The presented method seems to find a good trade-off between expressiveness, runtime and empirical quality.\n- There is theoretical value in the non-parametric encoder for simplices that keeps equivariant properties and simplicial-awareness\n- The paper is mostly well written"
                },
                "weaknesses": {
                    "value": "- Runtime and asymptotic comparisons in this work are done by excluding the precomputation of features. I think this is misleading, since in practice, the precomputation is certainly part of the computation, especially during inference. Thus, the presented gains seem to be only valid during training, when the features need to be computed only once for many iterations of training. \n- At the same time, the method only performs on par with previous work, with small gains on some datasets.\n- The method requires many hyper parameter choices like hops, T, k, which seem to have different optimal settings on different datasets. The result quality differs substantially depending on the configuration.\n- I am skeptical regarding the practical relevance of the presented method due to above reasons.\n\n- The method lacks conceptual novelty. The main idea of precomputing features by non-learnable functions has been seen in other areas, e.g. non-parametric GNNs. The general structure of the work follows a long line of work about GNN expressiveness (higher order and WL-level) without presenting many novel insights."
                },
                "questions": {
                    "value": "- I wonder how the method compares to previous methods in inference runtime, when feature precomputation needs to be included.\n\n-----------\nI thank the authors for proving the precomputation times explicitly and for replying to other concerns I have - this is certainly helpful to evaluate the differences to previous work.\n\nIn general, I am still on the fence and still doubt the significance of the contribution. However, I acknowledge that other reviewers find value in it and, thus, slightly raise my score. I am not opposing this paper to be accepted, as I think it is a thorough and well executed work."
                },
                "flag_for_ethics_review": {
                    "value": [
                        "No ethics review needed."
                    ]
                },
                "rating": {
                    "value": "6: marginally above the acceptance threshold"
                },
                "confidence": {
                    "value": "3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked."
                },
                "code_of_conduct": {
                    "value": "Yes"
                },
                "first_time_reviewer": {
                    "readers": [
                        "ICLR.cc/2024/Conference/Program_Chairs",
                        "ICLR.cc/2024/Conference/Submission9491/Senior_Area_Chairs",
                        "ICLR.cc/2024/Conference/Submission9491/Area_Chairs",
                        "ICLR.cc/2024/Conference/Submission9491/Reviewer_LALK"
                    ]
                }
            },
            "number": 2,
            "invitations": [
                "ICLR.cc/2024/Conference/Submission9491/-/Official_Review",
                "ICLR.cc/2024/Conference/-/Edit"
            ],
            "domain": "ICLR.cc/2024/Conference",
            "tcdate": 1698692893981,
            "cdate": 1698692893981,
            "tmdate": 1700756110360,
            "mdate": 1700756110360,
            "license": "CC BY 4.0",
            "version": 2
        },
        "responses": [
            {
                "id": "sHwVeekuse",
                "forum": "eUgS9Ig8JG",
                "replyto": "tkcYyLyiYm",
                "signatures": [
                    "ICLR.cc/2024/Conference/Submission9491/Authors"
                ],
                "readers": [
                    "everyone"
                ],
                "writers": [
                    "ICLR.cc/2024/Conference",
                    "ICLR.cc/2024/Conference/Submission9491/Authors"
                ],
                "content": {
                    "comment": {
                        "value": "We sincerely appreciate your positive feedback on our work's objectives, theoretical analysis, trade-off balance, the value of our non-parametric encoder, and the overall quality of our writing."
                    }
                },
                "number": 19,
                "invitations": [
                    "ICLR.cc/2024/Conference/Submission9491/-/Official_Comment"
                ],
                "domain": "ICLR.cc/2024/Conference",
                "tcdate": 1700464829876,
                "cdate": 1700464829876,
                "tmdate": 1700464829876,
                "mdate": 1700464829876,
                "license": "CC BY 4.0",
                "version": 2
            },
            {
                "id": "a81LbbuZxv",
                "forum": "eUgS9Ig8JG",
                "replyto": "tkcYyLyiYm",
                "signatures": [
                    "ICLR.cc/2024/Conference/Submission9491/Authors"
                ],
                "readers": [
                    "everyone"
                ],
                "writers": [
                    "ICLR.cc/2024/Conference",
                    "ICLR.cc/2024/Conference/Submission9491/Authors"
                ],
                "content": {
                    "title": {
                        "value": "Addressing the Inclusion of Precomputation Times in Runtime Comparisons"
                    },
                    "comment": {
                        "value": "We appreciate your feedback regarding the inclusion of precomputation times in our overall computation assessment. While we agree that precomputation is indeed part of the overall process, it generally constitutes a relatively insignificant portion compared to the cumulative time spent on training and testing. \n\nIt is important to note that our method requires precomputation of features just once before the training begins. On the other hand, conventional SNNs require this computation for every layer, where the number of layers correspond to the neighborhood depth from which features are aggregated, and for each epoch and across all orders of simplices. In Appendix B, we do provide a theoretical comparison of overall computational complexities of SaNN and the existing SNN models. The overall time complexity of the existing SNN models is about $\\mathcal{O}\\left(T\\left(\\left(2 N_k^2 D_k+N_k N_{k-1} D_{k-1}+N_k N_{k+1} D_{k+1}\\right)+\\left(3 N_k D_k^2+N_k D_{k-1}^2+N_k D_{k+1}^2\\right)\\right)\\right)$, which is dominated by the first term containing $2 N_k^2 D_k, N_k N_{k-1} D_{k-1}$, and $N_k N_{k+1} D_{k+1}$, corresponding to feature aggregation in every layer. Since feature aggregation happens only once before training as a pecomputation step, an SaNN model capturing information from $0, \\ldots, T$-hop neighborhood of $k$-simplices has a time complexity of $\\mathcal{O}\\left(T\\left(3 N_k D_k^2+N_k D_{k-1}^2+N_k D_{k+1}^2\\right)\\right)$, which is negligibly small when compared to the computational complexity of the existing SNN models. \n\nWe acknowledge that the time savings during inference may not be as pronounced as those during training, but we assert that the substantial enhancements during the training phase are quite significant, given that training time is the primary contributor to overall computation time, which is the sum of precomputation time, training time and inference time. \nNonetheless, in response to your suggestion, we have incorporated precomputation times into our analysis   to provide a more comprehensive view of our method's performance. We present here the precomputation and per epoch training time values of SaNN and the per epoch training time values of baseline SNN models for the three tasks at hand. The first terms in the runtime values of SaNN correspond to the precomputation times and the second terms to the per epoch training times.\n\nTrajectory prediction:\n| Dataset | Ocean | Synthetic | Planar | Mesh |\n|---------|-------|-----------|--------|------|\n| ScoNe   | 0.4   | 3.2       | 30.2   | 18.9 |\n| SCNN    | 1.9   | 4.2       | 36.3   | 29.8 |\n| SaNN    | 0.01,0.1 | 0.06,2.5 | 1.22,26.8 | 0.39,8.1 |\n\n\nSimplicial closure prediction:\n| Dataset      | Enron   | High-school | Primary-school | NDC-classes | Math-sx     |\n|--------------|---------|-------------|----------------|-------------|-------------|\n| MPSN         | 255     | 413         | 3499           | -           | -           |\n| SCNN         | 17      | 401         | 1891           | -           | -           |\n| SaNN         | 0.01, 3 | 0.05, 112   | 0.76, 916      | 0.26, 13    | 95.91, 52883|\n\nGraph classification:\n| Dataset  | Proteins | NCI1 | IMDB-B | Reddit-B | Reddit-M |\n|----------|----------|------|--------|----------|----------|\n| MPSN     | 33       | 292  | 46     | 242      | 1119     |\n| SaNN     | 3.6, 0.4 | 6, 58| 4, 8   | 6, 45    | 34, 104  |\n\n\nThe values provided represent the training time per epoch. Out-of-memory results are indicated by \u2212. The total training time for SaNN is calculated as follows:\nTotal training time = Precomputation time + (Number of epochs * Training time per epoch). This total is substantially less than the precomputation times. Furthermore, it is also worth noting that for any neural network model, the larger computational burden lies in the training phase rather than the inference phase. In our study, we have demonstrated both theoretically and experimentally that our training times are significantly reduced compared to other models. Therefore, the overall computational efficiency of our method remains superior."
                    }
                },
                "number": 20,
                "invitations": [
                    "ICLR.cc/2024/Conference/Submission9491/-/Official_Comment"
                ],
                "domain": "ICLR.cc/2024/Conference",
                "tcdate": 1700465368957,
                "cdate": 1700465368957,
                "tmdate": 1700468901260,
                "mdate": 1700468901260,
                "license": "CC BY 4.0",
                "version": 2
            }
        ]
    },
    {
        "review": {
            "id": "i5Pj9In9p1",
            "forum": "eUgS9Ig8JG",
            "replyto": "eUgS9Ig8JG",
            "signatures": [
                "ICLR.cc/2024/Conference/Submission9491/Reviewer_7LnN"
            ],
            "nonreaders": [],
            "readers": [
                "everyone"
            ],
            "writers": [
                "ICLR.cc/2024/Conference",
                "ICLR.cc/2024/Conference/Submission9491/Reviewer_7LnN"
            ],
            "content": {
                "summary": {
                    "value": "The authors propose a class of simple simplicial neural network models, referred to as simplicial-aware neural\nnetworks (SaNNs), which leverage precomputation of simplicial features. The authors theoretically demonstrate that under certain conditions, SaNNs are better discriminators of non-isomorphic graphs than the WL and SWL test. Empirically, SaNNs are shown to perform competitively against other SNNs and GNNs on tasks such as trajectory prediction, simplicial closure prediction, and several graph classification tasks over various datasets."
                },
                "soundness": {
                    "value": "3 good"
                },
                "presentation": {
                    "value": "4 excellent"
                },
                "contribution": {
                    "value": "3 good"
                },
                "strengths": {
                    "value": "1. The theoretical results are intriguing. Indeed, a competitor to the WL and SWL tests would be a valuable contribution to the graph ML community. \n\n2. A wide variety of benchmarks over several tasks and datasets are conducted to demonstrate the efficacy and efficiency of SaNNs. \n\n3. SaNNs inherit several valuable invariance properties of other SNNs including permutation invariance, orientation invariance, and simplicial-awareness. \n\n4. Compared to MPSNs, consideration of higher-order simplices does not blow up computation complexity."
                },
                "weaknesses": {
                    "value": "1. It is unclear for a research with limited expertise in this rather niche area to conclude the strength of the conditions prescribed in Theorems 4.1 and 4.2. (See questions.) \n\n2. There do not appear to be any results describing the pre-computation time which should be included in any run-time comparisons which I imagine should scale near-exponentially with graph size and order of simplices considered. \n\n3. SaNNs are often not outright the winner in terms of prediction accuracies for the tasks displayed in Tables 1 and 3. For example, in Table 1, the SaNN is outcompeted by Projection and Scone on 3/4 of the datasets and the run-time savings of SaNN are not significant enough to justify usage of the SaNN. In Table 3, SaNN is not the leader in 4/5 of the datasets and it is not even the fastest. On the other hand, the time savings against MPSN are quite significant, but since many practitioners of graph learning expect training to take significant amounts of time, accuracy is the topmost priority, so there wouldn't be a strong enough justification to go with a SaNN."
                },
                "questions": {
                    "value": "1. Is assuming the learnable transformation functions $g_k^{(t)}\\cdot)$ are injective too strong? Although the MLPs will be injective, appealing to the Universal Approximation Theorem to declare that $g_k$ can be injectively-approximated is probably not practical. \n\n2. I may have missed this but are pre-computation times explicitly indicated in the results?"
                },
                "flag_for_ethics_review": {
                    "value": [
                        "No ethics review needed."
                    ]
                },
                "rating": {
                    "value": "8: accept, good paper"
                },
                "confidence": {
                    "value": "2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked."
                },
                "code_of_conduct": {
                    "value": "Yes"
                },
                "first_time_reviewer": {
                    "value": "Yes",
                    "readers": [
                        "ICLR.cc/2024/Conference/Program_Chairs",
                        "ICLR.cc/2024/Conference/Submission9491/Senior_Area_Chairs",
                        "ICLR.cc/2024/Conference/Submission9491/Area_Chairs",
                        "ICLR.cc/2024/Conference/Submission9491/Reviewer_7LnN"
                    ]
                }
            },
            "number": 3,
            "invitations": [
                "ICLR.cc/2024/Conference/Submission9491/-/Official_Review",
                "ICLR.cc/2024/Conference/-/Edit"
            ],
            "domain": "ICLR.cc/2024/Conference",
            "tcdate": 1699255892259,
            "cdate": 1699255892259,
            "tmdate": 1699637193033,
            "mdate": 1699637193033,
            "license": "CC BY 4.0",
            "version": 2
        },
        "responses": [
            {
                "id": "ik5WElVjQw",
                "forum": "eUgS9Ig8JG",
                "replyto": "i5Pj9In9p1",
                "signatures": [
                    "ICLR.cc/2024/Conference/Submission9491/Authors"
                ],
                "readers": [
                    "everyone"
                ],
                "writers": [
                    "ICLR.cc/2024/Conference",
                    "ICLR.cc/2024/Conference/Submission9491/Authors"
                ],
                "content": {
                    "comment": {
                        "value": "We sincerely appreciate your positive feedback and recognition of the potential impact of our work in the graph ML community. Your acknowledgment of the strengths of our SaNN model, its invariance properties, and its computational efficiency is highly encouraging. Thank you."
                    }
                },
                "number": 15,
                "invitations": [
                    "ICLR.cc/2024/Conference/Submission9491/-/Official_Comment"
                ],
                "domain": "ICLR.cc/2024/Conference",
                "tcdate": 1700462477326,
                "cdate": 1700462477326,
                "tmdate": 1700462477326,
                "mdate": 1700462477326,
                "license": "CC BY 4.0",
                "version": 2
            },
            {
                "id": "MYvfUBDMhZ",
                "forum": "eUgS9Ig8JG",
                "replyto": "i5Pj9In9p1",
                "signatures": [
                    "ICLR.cc/2024/Conference/Submission9491/Authors"
                ],
                "readers": [
                    "everyone"
                ],
                "writers": [
                    "ICLR.cc/2024/Conference",
                    "ICLR.cc/2024/Conference/Submission9491/Authors"
                ],
                "content": {
                    "title": {
                        "value": "Regarding Inclusion of Precomputation Times"
                    },
                    "comment": {
                        "value": "We appreciate your feedback regarding the inclusion of precomputation times in our overall computation assessment. While we agree that precomputation is indeed part of the overall process, it generally constitutes a relatively insignificant portion compared to the cumulative time spent on training and testing. \nNonetheless, in response to your suggestion, we incorporated precomputation times into our analysis to provide a more comprehensive view of our method's performance. We present here the precomputation and per epoch training time values of SaNN and the per epoch training time values of baseline SNN modesl for the three tasks at hand. The first terms in the runtime values of SaNN correspond to the precomputation times and the second terms to the per epoch training times.\nTrajectory prediction:\n| Dataset | Ocean | Synthetic | Planar | Mesh |\n|---------|-------|-----------|--------|------|\n| ScoNe   | 0.4   | 3.2       | 30.2   | 18.9 |\n| SCNN    | 1.9   | 4.2       | 36.3   | 29.8 |\n| SaNN    | 0.01,0.1 | 0.06,2.5 | 1.22,26.8 | 0.39,8.1 |\n\nSimplicial closure prediction:\n| Dataset      | Enron   | High-school | Primary-school | NDC-classes | Math-sx     |\n|--------------|---------|-------------|----------------|-------------|-------------|\n| MPSN         | 255     | 413         | 3499           | -           | -           |\n| SCNN         | 17      | 401         | 1891           | -           | -           |\n| SaNN         | 0.01, 3 | 0.05, 112   | 0.76, 916      | 0.26, 13    | 95.91, 52883|\n\nGraph classification:\n| Dataset  | Proteins | NCI1 | IMDB-B | Reddit-B | Reddit-M |\n|----------|----------|------|--------|----------|----------|\n| MPSN     | 33       | 292  | 46     | 242      | 1119     |\n| SaNN     | 3.6, 0.4 | 6, 58| 4, 8   | 6, 45    | 34, 104  |\n\nThe values provided represent the training time per epoch. Out-of-memory results are indicated by \u2212. The total training time for SaNN is calculated as follows:\nTotal training time = Precomputation time + (Number of epochs * Training time per epoch). This total is substantially less than the precomputation times."
                    }
                },
                "number": 17,
                "invitations": [
                    "ICLR.cc/2024/Conference/Submission9491/-/Official_Comment"
                ],
                "domain": "ICLR.cc/2024/Conference",
                "tcdate": 1700464470743,
                "cdate": 1700464470743,
                "tmdate": 1700468923386,
                "mdate": 1700468923386,
                "license": "CC BY 4.0",
                "version": 2
            },
            {
                "id": "11C0wmUq4m",
                "forum": "eUgS9Ig8JG",
                "replyto": "Xw68Ztxc3L",
                "signatures": [
                    "ICLR.cc/2024/Conference/Submission9491/Reviewer_7LnN"
                ],
                "readers": [
                    "everyone"
                ],
                "writers": [
                    "ICLR.cc/2024/Conference",
                    "ICLR.cc/2024/Conference/Submission9491/Reviewer_7LnN"
                ],
                "content": {
                    "title": {
                        "value": "Thanks for the response"
                    },
                    "comment": {
                        "value": "I'd like to thank the authors for addressing my questions and providing pre-computation times which are indeed not significant. I am maintaining my score."
                    }
                },
                "number": 37,
                "invitations": [
                    "ICLR.cc/2024/Conference/Submission9491/-/Official_Comment"
                ],
                "domain": "ICLR.cc/2024/Conference",
                "tcdate": 1700543223800,
                "cdate": 1700543223800,
                "tmdate": 1700543223800,
                "mdate": 1700543223800,
                "license": "CC BY 4.0",
                "version": 2
            }
        ]
    },
    {
        "review": {
            "id": "6iCqDrP0HV",
            "forum": "eUgS9Ig8JG",
            "replyto": "eUgS9Ig8JG",
            "signatures": [
                "ICLR.cc/2024/Conference/Submission9491/Reviewer_jAzK"
            ],
            "nonreaders": [],
            "readers": [
                "everyone"
            ],
            "writers": [
                "ICLR.cc/2024/Conference",
                "ICLR.cc/2024/Conference/Submission9491/Reviewer_jAzK"
            ],
            "content": {
                "summary": {
                    "value": "The paper describes an efficient, and effective approach for learning representations for simplices in a simplicial complex. The central idea is that of using injective functions for aggregating simplicial features, as it ensures that the embeddings are unique. The simplicial features are aggregated over upper, lower, boundary and co-boundary adjacencies. The paper provides precise definitions and theorems and statements on the properties of the networks. The proofs are summarized in the main body and provided in full detail in the appendices. The method is further experimentally validated and shows that the proposed model (SaNN) is both efficients (significantly faster than any of the other baselines) and effective (performance within the uncertainty intervals on accurcies, or above the baselines)."
                },
                "soundness": {
                    "value": "4 excellent"
                },
                "presentation": {
                    "value": "3 good"
                },
                "contribution": {
                    "value": "4 excellent"
                },
                "strengths": {
                    "value": "1. I am impressed by the clarity of presentation in the paper. I find talking and reading about simplicial complex often a messy business given all the types of simplices and adjacencies, and the abstract notion in the first place. It is clear that the authors though well about how to present the math. This includes proper use of figures.\n2. The goal of the paper itself -efficiency whilst not compromising on expressivity- is relevant and important, and it is great to see the authors succeeding in reaching this goal.\n3. I appreciate the summary of the proofs after the formal statements.\n4. Next to a sound theoretical exposition, the experiments are thorough as well and include many ablation studies that are used to distill insightful take home messages."
                },
                "weaknesses": {
                    "value": "I only have 1 important concern:\n\n1. Although the main principles are clear, I am still confused about the actual architecture/predictive models. In the end we have equation 8, but it describes a representation for each of the $N$ sets of $k$-simplices, each consisting of the $N_k$ simplices. It is unclear how to distill a global prediction out of all these representations, as would be needed for e.g. the classification tasks. Details on how the architectural design for each of the benchmarks is missing."
                },
                "questions": {
                    "value": "Could you respond to the above concern, and additionally address the following questions/comments?\n\n2. On several occasions the notion of \"non-deep baselines\" is used. What is meant by this. Could you clarify what non-deep means here, which methods are these?\n\n3. In section 2 when presenting the symbols it is mentioned that $k=1,2,\\dots,N+1$. Does $k$ always run up all the way to $N+1$?\n\n4. In section 4. The sentence that starts with \"The theorem implies that any arbitrary ...\" is extremely long and hard to comprehend. I suggest to split it 2 or 3 sentence to improve readability.\n\n5. Just above property 1 it is mentioned \"other commonly used sum, mean, or max read-out functions are not injective\" I am not fully sure I understand it correctly. The paragraph above explains that sum aggregation is the best injective aggregator, in contrast to mean aggregation. I think the statement that I just quoted is about aggregating over the different $\\mathbf{Y}$'s? Perhaps this can be clarified.\n\n6. In the tables: since colors red and blue are used you might as well color the text in the caption as well. I.e. \"The {\\color{red}first} and {\\color{blue}second} best performances ...\"\n\n7. The insights section says \"The deep models are observed to perform exceptionally better than logistic regression\", where do I see this? Logistic regression taking what as input? Could this be clarified.\n\nThank you for considering my comments and questions."
                },
                "flag_for_ethics_review": {
                    "value": [
                        "No ethics review needed."
                    ]
                },
                "rating": {
                    "value": "8: accept, good paper"
                },
                "confidence": {
                    "value": "4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work."
                },
                "code_of_conduct": {
                    "value": "Yes"
                }
            },
            "number": 4,
            "invitations": [
                "ICLR.cc/2024/Conference/Submission9491/-/Official_Review",
                "ICLR.cc/2024/Conference/-/Edit"
            ],
            "domain": "ICLR.cc/2024/Conference",
            "tcdate": 1699474420570,
            "cdate": 1699474420570,
            "tmdate": 1699637192920,
            "mdate": 1699637192920,
            "license": "CC BY 4.0",
            "version": 2
        },
        "responses": [
            {
                "id": "unE6WvIz2L",
                "forum": "eUgS9Ig8JG",
                "replyto": "6iCqDrP0HV",
                "signatures": [
                    "ICLR.cc/2024/Conference/Submission9491/Authors"
                ],
                "readers": [
                    "everyone"
                ],
                "writers": [
                    "ICLR.cc/2024/Conference",
                    "ICLR.cc/2024/Conference/Submission9491/Authors"
                ],
                "content": {
                    "comment": {
                        "value": "We deeply appreciate your kind words regarding the clarity of our presentation and the balance between efficiency and expressivity in our work. Thank you for acknowledging the thoroughness of our theoretical and experimental sections."
                    }
                },
                "number": 7,
                "invitations": [
                    "ICLR.cc/2024/Conference/Submission9491/-/Official_Comment"
                ],
                "domain": "ICLR.cc/2024/Conference",
                "tcdate": 1700460845595,
                "cdate": 1700460845595,
                "tmdate": 1700460845595,
                "mdate": 1700460845595,
                "license": "CC BY 4.0",
                "version": 2
            },
            {
                "id": "lsKXUSNLdG",
                "forum": "eUgS9Ig8JG",
                "replyto": "unE6WvIz2L",
                "signatures": [
                    "ICLR.cc/2024/Conference/Submission9491/Reviewer_jAzK"
                ],
                "readers": [
                    "everyone"
                ],
                "writers": [
                    "ICLR.cc/2024/Conference",
                    "ICLR.cc/2024/Conference/Submission9491/Reviewer_jAzK"
                ],
                "content": {
                    "title": {
                        "value": "Thank you for the clarification"
                    },
                    "comment": {
                        "value": "I checked appendix H which indeed gives sufficient details on the experiments. I also appreciate the given responses to my questions. Although the mentioned minor changes to the paper should be easy to incorporate,  I would appreciate seeing an updated pdf on openreview. Is it possible to do this? \n\nThank you!"
                    }
                },
                "number": 33,
                "invitations": [
                    "ICLR.cc/2024/Conference/Submission9491/-/Official_Comment"
                ],
                "domain": "ICLR.cc/2024/Conference",
                "tcdate": 1700474933711,
                "cdate": 1700474933711,
                "tmdate": 1700474933711,
                "mdate": 1700474933711,
                "license": "CC BY 4.0",
                "version": 2
            },
            {
                "id": "MWzVU6pDpZ",
                "forum": "eUgS9Ig8JG",
                "replyto": "W0bf1Fchrw",
                "signatures": [
                    "ICLR.cc/2024/Conference/Submission9491/Reviewer_jAzK"
                ],
                "readers": [
                    "everyone"
                ],
                "writers": [
                    "ICLR.cc/2024/Conference",
                    "ICLR.cc/2024/Conference/Submission9491/Reviewer_jAzK"
                ],
                "content": {
                    "comment": {
                        "value": "Thank you, much appreciated. I maintain my score."
                    }
                },
                "number": 38,
                "invitations": [
                    "ICLR.cc/2024/Conference/Submission9491/-/Official_Comment"
                ],
                "domain": "ICLR.cc/2024/Conference",
                "tcdate": 1700667714022,
                "cdate": 1700667714022,
                "tmdate": 1700667714022,
                "mdate": 1700667714022,
                "license": "CC BY 4.0",
                "version": 2
            }
        ]
    }
]