reflection symmetries using $(\times,\times,\times,\phantom{\times})$ is
most effective.
It solves one instances more than the remaining settings and reduces the
running time in comparison to not detecting and handling reflection
symmetries by 4.7 on all instances and 13.5
on the solvable instances.
The setting $(\times,\times,\times,\times)$, which disables lexicographic
reduction and instead applies (<ref>) when it is applicable,
still improves on the setting in which no symmetries are handled.
But in comparison to the other symmetry handling approaches, it is the least
Since the motivation of this setting was to compare the effect
of (<ref>) with lexicographic reduction, we also separately
considered the 25 instances for which (<ref>) is applicable
to see whether it has a positive effect there.
None of these instances could be solved within the time limit though.

§.§ Results for Benchmarking Instances

Besides the structured instances discussed in the previous section, we also
conducted experiments on general benchmarking instances.
The test sets that we considered are all instances from MIPLIB2017 [17] and
MINLPLIB [37], as well as the submitted instances of the SAT 2002 Competition [46].
To evaluate the impact of handling reflection symmetries, we removed all
instances from these test sets for which no reflection symmetries could be
We refer to the corresponding test sets as miplib2017, minlplib, and
sat2002, respectively.

In contrast to the structured instances, we cannot evaluate whether our
framework reliably detects reflection symmetries for benchmarking instances.
Our expectation was that reflection symmetries are rare for linear problems
(miplib2017) and arise frequently for nonlinear problems (minlplib) and SAT
Indeed, as Table <ref> shows, for 282 of the 819 instances
from sat2002, we could detect reflection symmetries, whereas we could find
only 60 instances from miplib2017 admitting reflection symmetries.
Among the 486 instances from minlplib, however, our framework could only
detect 6 instances admitting reflection symmetries.
This came as surprise to us, since MINLPLIB also contains instances corresponding to
geometric packing problems (instances whose names start with “kall_”).
Inspecting these instances revealed two explanations for not detecting
the reflection symmetries.
On the one hand, these instances already contain symmetry handling
On the other hand, in contrast to Example <ref>, the box in
which the objects need to be placed is not fixed.
Instead, one is looking for a box of minimal dimensions that can fit all
This is modeled asymmetrically by fixing the lower coordinate value and
introducing a variable to model the upper coordinate value of each
That is, although the real world problem admits reflection symmetries, the
corresponding MINLP model is asymmetric.

In the following, we will therefore focus on the miplib2017 and sat2002
instances containing reflection symmetries, since the minlplib test set is
too small to draw reliable conclusions.
The running times are summarized in Table <ref>.
Note that the table reports only on 59 instances although
Table <ref> shows that there are 60 instances with
reflection symmetries.
To ensure a fair comparison of the different methods, however, we removed
the instance “tokyometro” since all but one setting reached the memory

Discussion of MIPLIB2017
For miplib2017, we observe that the
$(\times,\times,\times,\phantom{\times})$ setting performs best w.r.t. the
number of solved instances.
It can solve 17 instances, while just handling permutation symmetries can
only solve 14 instances, and handling no symmetries at all solves 15
Regarding the running time, however,
$(\times,\times,\times,\phantom{\times})$ and the settings only handling
permutation symmetries perform equally and are on all instances 4.7
(on the solvable instances 17.1) slower than not handling
It thus seems that handling reflection symmetries can help
solving more instances, on average, however, it slows down the solving
As such, it is not a surprise that the mean running time of
$(\times,\times,\times,\times)$ is better than the one of

To understand why not handling symmetries performs better than handling
symmetries, we compared the results for the 17 solvable instances for the
setting in which no symmetries are handled
and $(\times,\times,\times,\phantom{\times})$.
The following three observations could be made:

* some instances are rather easy such that an improvement in running
time is negligible;

* for the two instances that cannot be solved when not handling
symmetries, also $(\times, \times, \times, \phantom{\times})$ needed
about 5900 and 6400, respectively.
That is, also when handling symmetries, the instances remain hard.

* The dual bound after presolving is (almost) optimal, i.e.,
it is sufficient to find an optimal solution.
While the power of symmetry handling lies in pruning symmetric
subproblems, which allows to more quickly improve the dual bound, it
seems to hinder in finding feasible or optimal solutions.

We conclude that, although handling symmetries on benchmarking instances
has a positive effect in general [40], the characteristics
of instances from MIPLIB2017 that admit reflection symmetries make symmetry
handling less suited to enhance branch-and-bound for these instances.

The second question that arises is why the
setting $(\times,\times,\times,\phantom{\times})$ has the same mean running
time as $(\times,\times,\phantom{\times},\phantom{\times})$ although it
solves three more instances.
Inspecting the symmetries that are found by the two different settings, we
observed that the number of generators varies a lot between only detecting
permutation symmetries and also reflection symmetries.
For example, although the detected symmetry group for the instance
is larger when detecting reflection symmetries
($\approx 10^{91.5}$ group elements in comparison to $\approx 10^{90.9}$
for permutation symmetries), the number of generators we get from
bliss is 35 for reflection symmetries and 64 for permutation symmetries.
When handling symmetries via lexicographic reduction, we thus lose a lot of
potential reductions when computing reflection symmetries.
Moreover, for the instance , we obtain the same number
of generators corresponding to permutation symmetries; when handling
reflection symmetries, however, we detect less column/row symmetries.
That is, we miss the potential of specialized algorithms for column/row

For the three additionally solved instances when
handling reflection symmetries, we either find more generators (instance
) or we detect more row/column symmetries
(instances  and ).
The explanation for the same mean running time thus indeed seems to be the
variability in the generators returned by bliss.

Discussion of SAT2002

On the sat2002 test set, the most effective setting is
It solves 169 instances, and thus almost all solvable instances, within the
time limit and improves upon only handling permutation symmetries
by 17.8.
Taking Table <ref> into account, this behavior is not
surprising as at most 125 of the 292 reflection symmetric sat2002 instances
contain permutation symmetries.
That is, if reflection symmetries are not handled, a lot of instances
become asymmetric.

Comparison of running times and number of detected row/column symmetries for solvable sat2002 instances containing permutation symmetries.

sym.     row+col     refl.     simp.     # solved     time     # row/column symmetries

7@lall instances (76):

72       63.10        1.00

$\times$                                             74       39.67       14.87

$\times$     $\times$                                74       39.80       14.87

$\times$     $\times$     $\times$                   74       50.57        7.01

$\times$     $\times$     $\times$     $\times$      74       53.28        7.01

7@lfeasible instances (27):

27       24.06        1.00

$\times$                                             25       27.90        4.59

$\times$     $\times$                                25       27.93        4.59

$\times$     $\times$     $\times$                   27       26.88        0.44

$\times$     $\times$     $\times$     $\times$      27       26.93        0.44

7@linfeasible instances (49):

45      106.55        1.00

$\times$                                             49       48.09       20.53

$\times$     $\times$                                49       48.31       20.53

$\times$     $\times$     $\times$                   47       71.38       10.63

$\times$     $\times$     $\times$     $\times$      47       77.27       10.63

To allow for a fair comparison between the different symmetry handling
settings, we therefore also considered the subset of all solvable sat2002
instances that contain proper permutation symmetries.
This results in 76 instances and the corresponding results are summarized
in Table <ref>.
On these instances, we observe that handling reflection symmetries on top
of permutation symmetries decreases the performance by 27.5,
and this effect is even more pronounced on the infeasible instances, for
which the running time increases by 48.4.
A possible explanation for this unexpected behavior is again the variance
in the generators of the symmetry groups reported by bliss.
While the mean number of row/column symmetries that are detected per instance
are about 20.5 when only detecting permutation symmetries, the number of
row/column symmetries drops to 10.6 when detecting reflection symmetries.
That is, when detecting reflection symmetries, the potential of handling
row/column symmetries by dedicated techniques cannot be exploited.

§.§ Conclusion and Outlook

In the introduction, we have formulated four main
goals (<ref>)–(<ref>), which also could be achieved in this
Our abstract framework of symmetry detection graphs turned out to be a
flexible mechanism for detecting reflection symmetries in
MINLP and beyond, cf. Goal (<ref>).
Our open-source implementation could be used to detect reflection
symmetries in many applications, cf. Goal (<ref>), and the numerical
experiments showed that handling reflection symmetries can be crucial to
accelerate branch-and-bound for specific applications, cf.Goal (<ref>).
Although we devised methods for handling reflection symmetries, cf.Goal (<ref>), we noted that the performance improvement due to
handling symmetries heavily depends on the structure of the detected symmetry
Handling reflection symmetries thus might slow
down the solving process if this prevents our heuristics to detect
row/column symmetries.

This latter observation opens directions for future research.
As we noted in our experiments, the generators of symmetry groups returned
by symmetry detection tools such as bliss heavily depend on the
structure of the symmetry detection graphs.
Thus, based on the returned generators, our heuristics can fail to detect
row and column symmetries.
To circumvent this issue, it might be promising to develop alternative
approaches for detecting row and column symmetries that depend less on the
structure of generators.
Ideally, one would use an exact mechanism to detect row/column symmetries,
but detecting such symmetries is as hard as the graph isomorphism
problem [7].
A possible future direction could thus be to exploit the specific structure
of the symmetry detection graphs to solve the graph isomorphism problem.

Moreover, for MIPLIB2017, we noted that some problems benefit from not
handling symmetries, because symmetry handling can hinder heuristics to
find feasible solutions.
A naive strategy for feasibility problems is thus to completely disable
symmetry handling.
For infeasible instances, however, this arguably slows down the solving
process, since a larger search space needs to be explored until
infeasibility is detected.
Instead, it could be interesting to investigate means to benefit from
handling symmetries in branch-and-bound, while removing the symmetry-based
restrictions in heuristics.

The author thanks Marc E. Pfetsch for very valuable discussions on the
choice of the data structure for encoding symmetry detection graphs in
as well as a thorough code review.
This publication is part of the project
“Local Symmetries for Global  Success” with project number
OCENW.M.21.299 which is financed by the Dutch Research Council (NWO).

[1]
Anders, M., Schweitzer, P.: Parallel computation of combinatorial symmetries.
In: 29th Annual European Symposium on Algorithms, ESA 2021,
September 6-8, 2021, Lisbon, Portugal (Virtual Conference), LIPIcs,
vol. 204, pp. 6:1–6:18. Schloss Dagstuhl - Leibniz-Zentrum für
Informatik (2021).

[2]
Anders, M., Schweitzer, P., Stieß, J.: Engineering a preprocessor for
symmetry detection.
CoRR abs/2302.06351 (2023).

[3]
Babai, L., Luks, E.M.: Canonical labeling of graphs.
In: Proceedings of the fifteenth annual ACM symposium on Theory of
computing - STOC '83. ACM Press (1983).

[4]
Belotti, P., Kirches, C., Leyffer, S., Linderoth, J., Luedtke, J., Mahajan, A.:
Mixed-integer nonlinear optimization.
Acta Numerica 22, 1–131 (2013).

[5]
Bendotti, P., Fouilhoux, P., Rottner, C.: Orbitopal fixing for the full
(sub-)orbitope and application to the unit commitment problem.
Mathematical Programming 186, 337–372 (2021).

[6]
Berthold, T.: Measuring the impact of primal heuristics.
Operations Research Letters 41(6), 611–614 (2013).

[7]
Berthold, T., Pfetsch, M.E.: Detecting orbitopal symmetries.
In: B. Fleischmann, K.H. Borgwardt, R. Klein, A. Tuma (eds.)
Operations Research Proceedings 2008, pp. 433–438. Springer Berlin
Heidelberg, Berlin, Heidelberg (2009)

[8]
Bessiere, C., Hebrard, E., Hnich, B., Walsh, T.: The complexity of global
In: Proceedings of the 19th National Conference on AI (2004)

[9]
Bödi, R., Herr, K., Joswig, M.: Algorithms for highly symmetric linear
and integer programs.
Mathematical Programming 137(1), 65–90 (2013).

[10]
Bolusani, S., Besançon, M., Bestuzheva, K., Chmiela, A., Dionísio, J.,
Donkiewicz, T., van Doornmalen, J., Eifler, L., Ghannam, M., Gleixner, A.,
Graczyk, C., Halbig, K., Hedtke, I., Hoen, A., Hojny, C., van der Hulst, R.,
Kamp, D., Koch, T., Kofler, K., Lentz, J., Manns, J., Mexi, G., Mühmer, E.,
Pfetsch, M.E., Schlösser, F., Serrano, F., Shinano, Y., Turner, M.,
Vigerske, S., Weninger, D., Xu, L.: The SCIP Optimization Suite 9.0

[11]
Cohen, J.S.: Computer algebra and symbolic computation: mathematical methods.
AK Peters, Natick, Massachusetts (2003)

[12]
Costa, A., Hansen, P., Liberti, L.: On the impact of symmetry-breaking
constraints on spatial branch-and-bound for circle packing in a square.
Discrete Applied Mathematics 161(1), 96–106 (2013).

[13]
van Doornmalen, J., Hojny, C.: A unified framework for symmetry handling

[14]
van Doornmalen, J., Hojny, C.: Efficient propagation techniques for handling
cyclic symmetries in binary programs.
INFORMS Journal on Computing 0(0), null (2024).

[15]
Flener, P., Frisch, A.M., Hnich, B., Kiziltan, Z., Miguel, I., Pearson, J.,
Walsh, T.: Breaking row and column symmetries in matrix models.
In: P. Van Hentenryck (ed.) Principles and Practice of Constraint
Programming - CP 2002, pp. 462–477. Springer Berlin Heidelberg, Berlin,
Heidelberg (2002)

[16]
Friedman, E.J.: Fundamental domains for integer programs with symmetries.
In: A. Dress, Y. Xu, B. Zhu (eds.) Combinatorial Optimization and
Applications, Lecture Notes in Computer Science, vol. 4616, pp.
146–153. Springer Berlin Heidelberg (2007).

[17]
Gleixner, A., Hendel, G., Gamrath, G., Achterberg, T., Bastubbe, M., Berthold,
T., Christophel, P.M., Jarck, K., Koch, T., Linderoth, J., Lübbecke, M.,
Mittelmann, H.D., Ozyurt, D., Ralphs, T.K., Salvagnin, D., Shinano, Y.:
MIPLIB 2017: Data-Driven Compilation of the 6th Mixed-Integer Programming
Mathematical Programming Computation pp. 443–490 (2021).

[18]
Hojny, C.: Supplementary material for the article “Detecting and handling
reflection symmetries in mixed-integer (nonlinear) programming”.

[19]
Hojny, C.: Packing, partitioning, and covering symresacks.
Discrete Applied Mathematics 283, 689–717 (2020).

[20]
Hojny, C.: Polynomial size IP formulations of knapsack may require
exponentially large coefficients.
Operations Research Letters 48(5), 612–618 (2020).

[21]
Hojny, C., Pfetsch, M.E.: Polytopes associated with symmetry handling.
Mathematical Programming 175, 197–240 (2019).

[22]
Junttila, T., Kaski, P.: Conflict propagation and component recursion for
canonical labeling.
In: A. Marchetti-Spaccamela, M. Segal (eds.) Theory and Practice of
Algorithms in (Computer) Systems – First International ICST Conference,
TAPAS 2011, Rome, Italy, April 18–20, 2011. Proceedings, Lecture
Notes in Computer Science, vol. 6595, pp. 151–162. Springer (2011).

[23]
Junttila, T., Kaski, P.: Conflict propagation and component recursion for
canonical labeling.
In: A. Marchetti-Spaccamela, M. Segal (eds.) Theory and Practice of
Algorithms in (Computer) Systems – First International ICST Conference,
TAPAS 2011, Rome, Italy, April 18–20, 2011. Proceedings, Lecture
Notes in Computer Science, vol. 6595, pp. 151–162. Springer (2011).

[24]
Kaibel, V., Peinhardt, M., Pfetsch, M.E.: Orbitopal fixing.
Discrete Optimization 8(4), 595–610 (2011).

[25]
Kaibel, V., Pfetsch, M.E.: Packing and partitioning orbitopes.
Mathematical Programming 114(1), 1–36 (2008).

[26]
Khajavirad, A.: Packing circles in a square: a theoretical comparison of
various convexification techniques (2017).

[27]
Korte, B., Vygen, J.: Combinatorial Optimization: Theory and Algorithms, 6 edn.
Springer, Heidelberg (2018)

[28]
Liberti, L.: Automatic generation of symmetry-breaking constraints.
In: Combinatorial optimization and applications, Lecture Notes
in Computer Science, vol. 5165, pp. 328–338. Springer, Berlin (2008).

[29]
Liberti, L.: Symmetry in mathematical programming.
In: J. Lee, S. Leyffer (eds.) Mixed Integer Nonlinear Programming,
IMA Series, vol. 154, pp. 263–283. Springer New York (2011).

[30]
Liberti, L.: Reformulations in mathematical programming: automatic symmetry
detection and exploitation.
Mathematical Programming 131(1-2), 273–304 (2012).

[31]
Liberti, L., Ostrowski, J.: Stabilizer-based symmetry breaking constraints for
mathematical programs.
Journal of Global Optimization 60, 183–194 (2014)

[32]
Linderoth, J., Núñez Ares, J., Ostrowski, J., Rossi, F., Smriglio, S.:
Orbital conflict: Cutting planes for symmetric integer programs.
INFORMS Journal on Optimization 3(2), 139–153 (2021).

[33]
Margot, F.: Pruning by isomorphism in branch-and-cut.
Mathematical Programming 94(1), 71–90 (2002).

[34]
Margot, F.: Exploiting orbits in symmetric ILP.
Mathematical Programming 98(1–3), 3–21 (2003).

[35]
Margot, F.: Symmetry in integer linear programming.
In: M. Jünger, T.M. Liebling, D. Naddef, G.L. Nemhauser, W.R.
Pulleyblank, G. Reinelt, G. Rinaldi, L.A. Wolsey (eds.) 50 Years of Integer
Programming, pp. 647–686. Springer (2010)

[36]
McKay, B.D., Piperno, A.: Practical graph isomorphism, II.
Journal of Symbolic Computation 60, 94–112 (2014).

[37]
Minlplib: A library of mixed-integer and continuous nonlinear programming

[38]
Ostrowski, J.: Symmetry in integer programming.
PhD dissertation, Lehigh University (2009)

[39]
Ostrowski, J., Linderoth, J., Rossi, F., Smriglio, S.: Orbital branching.
Mathematical Programming 126(1), 147–178 (2011).

[40]
Pfetsch, M.E., Rehn, T.: A computational comparison of symmetry handling
methods for mixed integer programs.
Mathematical Programming Computation 11(1), 37–93 (2019).

[41]
Puget, J.F.: Automatic Detection of Variable and Value Symmetries, pp.
Springer Berlin Heidelberg, Berlin, Heidelberg (2005).

[42]
Saff, E., Kuijlaars, A.: Distributing many points on a sphere.
The Mathematical Intelligencer 19, 5–11 (1997)

[43]
Sakallah, K.A.: Handbook of Satisfiability, Editors: Armin Biere, Marijn Heule,
Hans van Maaren, and Toby Walsh, chap. Symmetry and Satisfiability.
IOS Press (2021)

[44]
Salvagnin, D.: A dominance procedure for integer programming.
Master’s thesis, University of Padova, Padova, Italy  (2005)

[45]
Salvagnin, D.: Symmetry breaking inequalities from the Schreier-Sims table.
In: W.J. van Hoeve (ed.) Integration of Constraint Programming,
Artificial Intelligence, and Operations Research, pp. 521–529. Springer
International Publishing (2018).

[46]
Sat 2002 competition: problem instances.
<https://www.cs.ubc.ca/ hoos/SATLIB/Benchmarks/SAT/New/Competition-02/sat-2002-beta.tgz>

[47]
Szabó, P.G., Markót, M.C., Csendes, T.: Global Optimization in Geometry
— Circle Packing into the Square, pp. 233–265.
Springer US, Boston, MA (2005).

[48]
Wächter, A., Biegler, L.T.: On the implementation of an interior-point filter
line-search algorithm for large-scale nonlinear programming.
Mathematical Programming 106, 25–57 (2006).

[49]
Zhu, W.: Unsolvability of some optimization problems.
Applied Mathematics and Computation 174(2), 921–926 (2006).

§ OVERVIEW OF IMPORTANT FUNCTIONS TO APPLY OUR SYMMETRY DETECTION FRAMEWORK

This appendix provides an overview of the most important functions needed
to extend an SDG within a symmetry detection callback.
Since our implementation of SDGs allows for four different types of nodes,
we have different functions for adding these nodes:

SCIPaddSymgraphOpnode()     adds an operator node to an SDG;

SCIPaddSymgraphValnode()     adds a numerical value node to an SDG;

SCIPaddSymgraphConsnode()     adds a constraint node to an SDG.

Recall that we do not allow to add variable nodes to an SDG, because ensures that every SDG contains all necessary variable nodes.
Instead, the indices of variable nodes can be accessed via the functions

SCIPgetSymgraphVarnodeidx()     returns the index of the node
corresponding to a given variable;

SCIPgetSymgraphNegatedVarnodeidx()     returns the index of the node
corresponding to a negated/reflected variable.

To add edges to a graph, the function

SCIPaddSymgraphEdge()     adds an edge between two existing nodes of an SDG

can be used.

To simplify the usage of SDGs, we also provide two functions that add gadgets
for certain variable structures to an SDG:

SCIPextendPermsymDetectionGraphLinear()     adds a gadget for a linear
expression $\sprod{a}{x} + b$
to an SDG;

SCIPaddSymgraphVarAggregation()     adds a gadget for aggregated variables
to an SDG.

The second function has been introduced, since we require that no
aggregated or fixed variables are present in an SDG.

§ DETAILED NUMERICAL RESULTS

In this appendix, we provide detailed numerical results for the tested
problem classes.
Tables <ref>–<ref> report on
the running times and primal-dual integrals for each instance of the 2- and
3-dimensional packing, kissing number, and energy problems that we
discussed in Section <ref>.
The number of items corresponds to the number of balls, spheres, and points
in these respective problems, whereas the settings refer to the settings
sym0–sym6 and the automatic setting as described in
Section <ref>.

Running times and primal-dual integrals for packing test set and dimension 2.

# items     sym0     sym1     sym2     sym3     sym4     sym5     sym6     auto.

9@lrunning time in seconds:

3        0.12        0.12        0.11        0.05        0.09        0.07        0.09        0.09

4        2.84        1.94        0.67        0.30        0.44        0.38        0.29        0.28

5        0.79        0.59        0.38        0.17        0.30        0.31        0.22        0.21

6       43.09       25.56        7.41        0.61        0.50        0.40        0.70        0.68

7     7200.00     7200.00     6606.63       22.81      201.83       16.16       11.01       16.23

8     7200.00     7200.00     4352.59       10.14       70.32       22.82        6.68       12.14

9     7200.00     7200.00     7200.00      644.78     7200.00      249.99      365.62      103.95

10     7200.00     7200.00     7200.00      267.77     7200.00      173.67      153.78       61.42

11     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00

12     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00

13     7200.00     7200.00     7200.00     7200.00     7200.00      358.37      351.27      301.84

14     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00

9@lprimal-dual integral:

3           5           5           6           2           3           2           3           4

4          35          28          16          11           9           9           8           8

5          29          23          19          10          14          14          12          11

6         802         527         151          22          12           9          26          24

7      120899      109585       31757         210         941         109         133         123

8      145839      118007       52138         285        1489         227          90         120

9      234515      269194      152641        6449       67037        1741        2826         667

10      266727      224203      229408        5454       73878        2567        1027         754

11      355432      344742      334154      154772      102571       66800       74434       50360

12      361376      353135      351864      199890      128913       20578       77572       68018

13      368337      354376      353221      190939       98344        7071        8769        6855

14      408953      408172      405794      277031      200833       46888      128374      105141

Running times and primal-dual integrals for packing test set and dimension 3.

# items     sym0     sym1     sym2     sym3     sym4     sym5     sym6     auto.

9@lrunning time in seconds:

3        2.09        0.93        0.35        0.29        0.22        0.25        0.27        0.41

4        1.49        0.85        0.29        0.29        0.31        0.31        0.29        0.28

5     7200.00     7200.00     5921.24      380.15      626.00      414.19       67.69       73.69

6     6663.82     1961.23      341.28       10.73       34.86       13.46        8.40        5.87

7     7200.00     7200.00     7200.00      631.20     2443.29      645.28      287.70      395.11

8     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00

9     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00

10     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00

11     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00

12     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00

13     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00

14     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00

9@lprimal-dual integral:

3          17          11           9           6           5           6           7           9

4          19          12           5          11          10          10           9           8

5       52376       22897        5366         437         566         421          86          88

6       48670       15778        2794         210         474         128         126          94

7      154538      164221      110088        6448       10830        3605        2901        2755

8      259241      231341      176139      133512      126275      120202      106350      106084

9      281270      271892      224430      154967       92668       66342       79984       69743

10      319452      304708      285611      206888      154403      158242      138025      127942

11      342112      334613      314555      251442      144333      128962      117638      115709

12      361785      345513      334208      267405      171725      163406      166023      145259

13      362833      355450      341441      277927      143289      173997      127984      115555

14      373245      388226      343286      299399      227500      207009      208462      153976

Running times and primal-dual integrals for kissing test set and dimension 2.

# items     sym0     sym1     sym2     sym3     sym4     sym5     sym6     auto.

9@lrunning time in seconds:

3        0.02        0.03        0.08        0.01        0.06        0.02        0.04        0.04

4        0.07        0.18        0.16        0.14        0.10        0.15        0.16        0.17

5        0.06        0.35        0.28        0.17        0.39        0.29        0.17        0.17

6        0.16        0.48        0.47        0.23        0.31        0.61        0.61        0.26

7     7200.00     7200.00     1136.23        2.36       22.42        3.62        1.75        2.72

8     7200.00     7200.00     7200.00        5.89      340.06       18.33        6.34        6.11

9     7200.00     7200.00     7200.00       11.85      451.55        9.87        7.22        4.50

10     7200.00     7200.00     7200.00       51.76     7200.00       86.73       54.52       17.89

11     7200.00     7200.00     7200.00      175.53     7200.00      355.42      126.20       29.58

12     7200.00     7200.00     7200.00      448.92     7200.00     1408.36     1578.97      380.60

13     7200.00     7200.00     7200.00     1928.51     7200.00     3137.21     2976.38      116.21

14     7200.00     7200.00     7200.00     5501.44     7200.00     7200.00     7200.00      849.15

9@lprimal-dual integral:

3           2           3           7           1           6           2           2           3

4           5          14          16           8           9          11           9          10

5           6          30          24           9          31          29          14          14

6           9          42          45          20          25          47          33          17

7       38404       21079        5254          99         209         159          54          74

8      245267      213439      101851         264        1873         651         149         201

9      313339      306493      286624         342        1978         193         159         205

10      392407      414595      404572        1598       94434        1503        1095         406

11      491427      491463      484365        4463      104650        5516        1994         682

12      527099      527103      524806        9572      239255       13014       18173        5704

13      555106      555107      555094       45741      152492       34012       29414        1369

14      577444      578765      577441      130808      177820      328974      121905       10812

Running times and primal-dual integrals for kissing test set and dimension 3.

# items     sym0     sym1     sym2     sym3     sym4     sym5     sym6     auto.

9@lrunning time in seconds:

3        0.02        0.02        0.03        0.02        0.05        0.02        0.01        0.02

4        0.02        0.03        0.04        0.03        0.05        0.03        0.02        0.02

5        0.04        0.04        0.04        0.01        0.07        0.02        0.02        0.02

6        0.04        0.03        0.03        0.03        0.07        0.02        0.03        0.02

7        0.05        0.05        0.06        0.04        0.08        0.04        0.03        0.04

8        0.04        0.04        0.07        0.03        0.06        0.03        0.05        0.08

9        0.07        0.08        0.07        0.04        0.08        1.37        0.07        0.03

10        0.09        0.09        0.08        0.05        0.13        0.07        0.73        0.05

11        0.10        0.13        0.54        0.07        0.09        1.97        0.12        0.92

12        0.10        1.09        0.14        0.06        0.15        2.08        0.20        0.07

13     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00

14     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00

9@lprimal-dual integral:

3           2           2           3           2           5           2           1           2

4           2           3           4           3           5           3           2           2

5           4           4           4           1           7           2           2           2

6           4           3           3           3           7           2           3           2

7           5           4           6           4           8           4           3           4

8           4           4           7           3           6           3           5           8

9           7           8           7           4           8         137           7           3

10           9           9           8           5          13           7          18           5

11          10          13          54           7           9         197           9          33

12          10          22          14           6          15         193          11           7

13       61672       61411       61412       61568       61612       61583       62150       62493

14       92105       92101       92102       93511       92106       92116       93969       92138

Running times and primal-dual integrals for energy test set and dimension 2.

# items     sym0     sym1     sym2     sym3     sym4     sym5     sym6     auto.

9@lrunning time in seconds:

3        2.17        0.96        0.37        0.14        0.31        0.30        0.20        0.14

4       28.92       14.75        4.69        0.90        3.64        1.87        1.36        1.35

5      637.64      330.00       80.48        3.96        6.50        3.74        2.06        2.00

6     7200.00     7200.00     7200.00       20.58      169.95       36.29       19.36       19.48

7     7200.00     7200.00     7200.00      131.63     1669.89      171.30       95.21       80.70

8     7200.00     7200.00     7200.00      733.38     7200.00     2358.56     7200.00     7200.00

9     7200.00     7200.00     7200.00     7200.00     7200.00     3555.87     3074.32     1258.01

10     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00

11     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00

12     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00

13     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00

14     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00

9@lprimal-dual integral:

3           6           4           3           3           2           3           5           3

4          59          30          22           5          14          12          11          11

5        1048         733         175          38          25          21          17          13

6       27577       16940       15275         105         339          95          67          60

7      103265       91918       71028         558        3115         460         300         221

8      165382      159741      138685        3004       61183        5533        8552        4083

9      209730      204553      183465       13291       78012        9411        7622        3704

10      248606      244166      225948       56374      138046       64303       60746       38110

11      268921      265619      246456      101186      161925       98961       98287       76904

12      284270      286673      276693      153360      201235      146922      145253      126222

13      303207      301486      290631      178830      210680      163230      162411      123375

14      313677      311820      308313      214843      243370      206209      200767      165437

Running times and primal-dual integrals for energy test set and dimension 3.

# items     sym0     sym1     sym2     sym3     sym4     sym5     sym6     auto.

9@lrunning time in seconds:

3     7200.00     1904.11      179.29       73.14      358.43      357.69       24.35      772.85

4     7200.00     7200.00     7200.00     1889.64     7200.00     7200.00     1448.27     1602.78

5     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00

6     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00

7     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00

8     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00

9     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00

10     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00

11     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00

12     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00

13     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00

14     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00     7200.00

9@lprimal-dual integral:

3        1528         447          44          23          80          80          11         186

4       14737        6781        2051         491        2658        1855         385         408

5       77151       59093       36746       12087       16408       12026        5531        5184

6      121289      104487       84450       41473       61835       46654       31565       31476

7      157158      143920      126967       86664      100511       88191       72522       71813

8      178762      168522      156057      120629      133847      120473      109227      108159

9      195888      186304      172755      146157      134636      128682      116954      115462

10      208562      199569      187568      164420      158820      151279      141834      140935

11      219253      213879      201490      185224      174839      168161      159821      159343

12      225733      223219      210340      192819      188728      182468      175968      174244

13      233884      229828      221233      205331      197518      190162      185491      184832

14      240182      238349      227048      216176      209512      202497      198844      198483