Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update OR-Tools to v9.11 #88

Draft
wants to merge 4 commits into
base: main
Choose a base branch
from
Draft

Update OR-Tools to v9.11 #88

wants to merge 4 commits into from

Conversation

hanno-becker
Copy link
Collaborator

No description provided.

@mkannwischer mkannwischer force-pushed the ortools_9_11 branch 4 times, most recently from 0660009 to 2c5f3be Compare November 28, 2024 12:21
@mkannwischer
Copy link
Collaborator

mkannwischer commented Dec 5, 2024

I've run some more experiments comapring v9.7 with v9.11 on my Apple M1 MacMini.
I followed the steps described here to extract models: https://github.com/slothy-optimizer/slothy/tree/main/paper/scripts/models. This was after #110 was merged.
I've done two exports one with v9.7 and one with v9.11.

Then I run each of the models with the respective version five times with the following code:

import os
from statistics import mean, median
import ortools
from ortools.sat.python import cp_model
from google.protobuf import text_format

TIMEOUT=300
ITERATIONS=5

def load_and_solve(file, i):
    model = cp_model.CpModel()
    with open(file, "r") as file:
        text_format.Parse(file.read(), model.Proto())
    print (f"[{i}]: Solve using OR-Tools v{ortools.__version__} ... ", end='', flush=True)
    solver = cp_model.CpSolver()
    solver.parameters.max_time_in_seconds = TIMEOUT
    # solver.parameters.search_branching = cp_model.FIXED_SEARCH
    solver.parameters.num_workers = 16

    status = solver.Solve(model)
    status_str = solver.StatusName(status)
    print(f"{status_str}, wall time: {solver.WallTime():.4f} s")
    return solver.WallTime()


def do_it(file):
    print(file)
    times = []
    for i in range(ITERATIONS):
        walltime = load_and_solve(file, i)
        times.append(walltime)
    return times


measurements = []

def printit(o):
    f = o["file"]
    m = o["measurements"]
    minimum = min(m)
    maximum = max(m)
    meanv    = mean(m)
    medianv  = median(m)
    m = ",".join([str(x) for x in m])
    print(f"| {f} | {minimum} | {maximum} | {meanv} | {medianv} | {m} |")

for f in os.listdir("./"):
    if not f.endswith(".txt"):
        continue
    if f == "results.txt" or f == "requirements.txt":
        continue
    times = do_it(f)
    measurements.append({"file": f, "measurements": times})
    printit(measurements[-1])


print("| model | minimum | maximum | mean | median | raw times |")
print("| ----- | ------- | ------- | ---- | ------ | --------- |")
for m in measurements:
    printit(m)

I attached the extracted models together with the results.
models-2024-11-29-v9.7.tar.gz
models-2024-11-29-v9.11.tar.gz

Here are the results:
v9.7

model minimum maximum mean median raw times
ntt_kyber_123_4567_a55_1732885055020.txt 3.8571500000000003 6.240216 4.9326628 4.759714000000001 3.8571500000000003,4.759714000000001,5.489064,4.31717,6.240216
ntt_kyber_123_4567_a55_1732885056836.txt 0.007852000000000001 0.009293000000000001 0.008247800000000001 0.007898 0.008328,0.007898,0.007852000000000001,0.007868,0.009293000000000001
ntt_kyber_123_4567_a55_1732885065031.txt 11.899611 28.321216000000003 18.096505800000003 12.1859 26.167989000000002,28.321216000000003,11.899611,11.907813,12.1859
ntt_kyber_123_4567_a55_1732885325573.txt 120.60413600000001 160.404051 135.9461598 126.95619900000001 120.60413600000001,160.404051,121.60764700000001,126.95619900000001,150.158766
ntt_kyber_123_4567_a55_1732885327861.txt 1.6431300000000002 1.8903800000000002 1.7666366000000002 1.761678 1.7218300000000002,1.816165,1.761678,1.6431300000000002,1.8903800000000002
ntt_kyber_123_4567_a55_1732885328779.txt 0.639811 0.7426550000000001 0.7062676000000001 0.7146480000000001 0.7270420000000001,0.7426550000000001,0.7071820000000001,0.639811,0.7146480000000001
ntt_dilithium_123_45678_a55_1732885332072.txt 2.92423 3.135123 3.0020146000000003 2.9849020000000004 2.92423,2.969087,2.996731,2.9849020000000004,3.135123
ntt_dilithium_123_45678_a55_1732885336878.txt 2.891496 3.0164060000000004 2.9689988 2.978653 2.958774,2.9996650000000002,2.978653,3.0164060000000004,2.891496
ntt_dilithium_123_45678_a55_1732885339706.txt 0.12004300000000001 0.12115200000000001 0.12040680000000001 0.120119 0.12115200000000001,0.120119,0.12008300000000001,0.12004300000000001,0.12063700000000001
ntt_dilithium_123_45678_a55_1732885340795.txt 0.053585 0.285972 0.10142520000000001 0.055871000000000004 0.055871000000000004,0.054004,0.053585,0.285972,0.057694
ntt_dilithium_123_45678_a55_1732885344008.txt 2.99427 4.457601 3.4125836 3.2024380000000003 2.99427,3.2024380000000003,3.21204,3.196569,4.457601
ntt_dilithium_123_45678_a55_1732885346741.txt 0.053627 0.054607 0.054209600000000004 0.054346000000000005 0.054346000000000005,0.054607,0.054576000000000006,0.053627,0.053892
ntt_dilithium_123_45678_a55_1732885347861.txt 0.053945 0.054598 0.054183800000000004 0.054092 0.054598,0.053945,0.053992000000000005,0.054292,0.054092
ntt_dilithium_123_45678_a55_1732885351142.txt 3.104048 3.2909270000000004 3.1898280000000003 3.148896 3.269061,3.1362080000000003,3.104048,3.148896,3.2909270000000004
ntt_dilithium_123_45678_a55_1732885352949.txt 0.004739 0.006444 0.005409000000000001 0.004819 0.006444,0.004772,0.0062710000000000005,0.004819,0.004739
ntt_dilithium_123_45678_a55_1732885353196.txt 0.005193000000000001 0.005999 0.0054044 0.005298000000000001 0.005318000000000001,0.005298000000000001,0.005214000000000001,0.005999,0.005193000000000001
ntt_dilithium_123_45678_a55_1732885353413.txt 0.029988 0.034355000000000004 0.031188000000000004 0.030510000000000002 0.034355000000000004,0.029988,0.030937000000000003,0.030150000000000003,0.030510000000000002
ntt_dilithium_123_45678_a55_1732885353644.txt 0.017288 0.019251 0.017748800000000002 0.017344000000000002 0.019251,0.017518000000000002,0.017288,0.017343,0.017344000000000002
ntt_dilithium_123_45678_a55_1732885353841.txt 0.00246 0.0028640000000000002 0.0027042000000000004 0.0027180000000000004 0.00246,0.0028640000000000002,0.0027180000000000004,0.002713,0.0027660000000000002
ntt_dilithium_123_45678_a55_1732885353978.txt 0.00318 0.003412 0.0032608000000000003 0.0032310000000000004 0.0032310000000000004,0.003203,0.00318,0.003412,0.003278
ntt_dilithium_123_45678_a55_1732885354115.txt 0.0025220000000000004 0.0029760000000000003 0.0026528000000000003 0.0025800000000000003 0.0026590000000000003,0.002527,0.0025220000000000004,0.0029760000000000003,0.0025800000000000003
ntt_dilithium_123_45678_a55_1732885354252.txt 0.0028190000000000003 0.0031590000000000003 0.0029656 0.00287 0.003125,0.0028550000000000003,0.00287,0.0031590000000000003,0.0028190000000000003
ntt_dilithium_123_45678_a55_1732885354898.txt 1.034337 1.0578800000000002 1.0406476 1.036824 1.035741,1.034337,1.038456,1.0578800000000002,1.036824
ntt_dilithium_123_45678_a55_1732885360687.txt 6.006772000000001 27.180334000000002 12.415998 7.209042 7.209042,15.635321000000001,27.180334000000002,6.048521,6.006772000000001
ntt_dilithium_123_45678_a55_1732885361270.txt 0.042037000000000005 0.043388 0.042491600000000004 0.042441 0.042145,0.043388,0.042447000000000006,0.042441,0.042037000000000005
ntt_dilithium_123_45678_a55_1732885365496.txt 4.434914 4.574657 4.480385 4.457141 4.574657,4.457141,4.4443150000000005,4.434914,4.4908980000000005
ntt_dilithium_123_45678_a55_1732885366538.txt 0.513374 0.523958 0.5160888 0.5140020000000001 0.523958,0.515277,0.513833,0.513374,0.5140020000000001
ntt_dilithium_123_45678_a55_1732885370383.txt 4.023289 7.554695000000001 4.8078034 4.204504 4.023289,4.204504,4.0339,4.222629,7.554695000000001
ntt_dilithium_123_45678_a55_1732885374280.txt 3.880288 4.665069 4.1241762 4.0349900000000005 4.665069,3.880288,4.05882,3.981714,4.0349900000000005
ntt_dilithium_123_45678_a55_1732885377574.txt 0.330726 0.36618700000000004 0.33908140000000003 0.332807 0.332201,0.332807,0.330726,0.333486,0.36618700000000004
ntt_dilithium_123_45678_a55_1732885380187.txt 0.139793 0.140431 0.1402016 0.140299 0.140431,0.14036200000000001,0.140123,0.140299,0.139793
ntt_dilithium_123_45678_a55_1732885390931.txt 12.203178000000001 15.105032000000001 13.325942200000002 12.541884000000001 15.105032000000001,14.548656000000001,12.203178000000001,12.230961,12.541884000000001
ntt_dilithium_123_45678_a55_1732885405454.txt 13.912942000000001 16.171456000000003 15.329707400000002 15.772991000000001 16.171456000000003,16.142747,15.772991000000001,13.912942000000001,14.648401000000002
ntt_dilithium_123_45678_a55_1732885411690.txt 0.142658 0.14402900000000002 0.14311800000000002 0.14305600000000002 0.14316500000000001,0.142682,0.14305600000000002,0.142658,0.14402900000000002
ntt_dilithium_123_45678_a55_1732885422870.txt 12.627319 12.766092 12.6720044 12.656943 12.629631000000002,12.627319,12.766092,12.680037,12.656943
ntt_dilithium_123_45678_a55_1732885437785.txt 15.382563000000001 34.308823000000004 24.9041786 27.493090000000002 34.308823000000004,30.015042,15.382563000000001,17.321375,27.493090000000002
ntt_dilithium_123_45678_a55_1732885635545.txt 23.303088000000002 31.392468 27.2441904 27.083133 25.464611,23.303088000000002,31.392468,28.977652000000003,27.083133
ntt_dilithium_123_45678_a55_1732885642491.txt 0.142655 0.144134 0.1431524 0.14305700000000002 0.143202,0.142714,0.14305700000000002,0.142655,0.144134
ntt_dilithium_123_45678_a55_1732885643420.txt 0.11729400000000001 0.11761800000000001 0.11739400000000001 0.11732400000000001 0.11742100000000001,0.117313,0.11761800000000001,0.11732400000000001,0.11729400000000001
ntt_dilithium_123_45678_a55_1732885646441.txt 2.915623 3.3671710000000004 3.248437 3.3237620000000003 3.3671710000000004,2.915623,3.308882,3.3237620000000003,3.326747
ntt_dilithium_123_45678_a55_1732885647750.txt 0.090241 0.477457 0.319485 0.465471 0.090241,0.47116600000000003,0.465471,0.477457,0.09309
ntt_dilithium_123_45678_a55_1732885650549.txt 3.932561 5.95711 4.7546610000000005 4.703219000000001 5.95711,4.756838,4.423577,4.703219000000001,3.932561
ntt_dilithium_123_45678_a55_1732885651869.txt 0.086136 0.086832 0.0863786 0.086245 0.086832,0.086136,0.08623,0.086245,0.08645
ntt_dilithium_123_45678_a55_1732885652670.txt 0.0864 0.08779400000000001 0.087121 0.087219 0.08779400000000001,0.086929,0.08726300000000001,0.0864,0.087219
ntt_dilithium_123_45678_a55_1732885653472.txt 0.084271 0.08481000000000001 0.0846298 0.084721 0.084721,0.084551,0.08479600000000001,0.08481000000000001,0.084271
ntt_dilithium_123_45678_a55_1732885654303.txt 0.08583800000000001 0.08894 0.086829 0.086735 0.08583800000000001,0.086735,0.08676500000000001,0.08894,0.085867
ntt_dilithium_123_45678_a55_1732885656149.txt 1.810532 5.226015 2.526564 1.8677780000000002 5.226015,1.810532,1.852896,1.875599,1.8677780000000002
ntt_dilithium_123_45678_a55_1732885656606.txt 0.022131 0.022279 0.0222176 0.022226000000000003 0.022279,0.022244,0.022208000000000002,0.022226000000000003,0.022131
ntt_dilithium_123_45678_a55_1732885656865.txt 0.022653000000000003 0.022999000000000002 0.022789200000000003 0.022764000000000003 0.022999000000000002,0.022764000000000003,0.022721,0.022653000000000003,0.022809000000000003
ntt_dilithium_123_45678_a55_1732885658515.txt 1.5161120000000001 1.633658 1.5747808 1.5766 1.633658,1.5162760000000002,1.5766,1.631258,1.5161120000000001
ntt_dilithium_123_45678_a55_1732885659854.txt 0.9639000000000001 1.090753 1.0202998 1.020364 1.0378610000000001,1.020364,1.090753,0.9639000000000001,0.9886210000000001
ntt_dilithium_123_45678_a55_1732885660278.txt 0.022796 0.022995 0.0228794 0.022819000000000002 0.022995,0.022979000000000003,0.022819000000000002,0.022808000000000002,0.022796
slothy_ci_fft_1732885734387.txt 0.012919000000000002 0.014506000000000002 0.0135846 0.013272 0.013272,0.014506000000000002,0.014296000000000001,0.012919000000000002,0.01293
slothy_ci_fft_1732885738142.txt 2.6173870000000004 2.8493470000000003 2.7385944 2.771567 2.6173870000000004,2.6475950000000004,2.771567,2.8493470000000003,2.8070760000000003
slothy_ci_fft_1732885740333.txt 2.471765 3.682631 3.0370368 3.1836320000000002 2.653855,3.1836320000000002,3.1933010000000004,3.682631,2.471765
slothy_ci_fft_1732885740719.txt 0.006296 0.007304000000000001 0.0067796 0.006645000000000001 0.006296,0.006645000000000001,0.007304000000000001,0.007298000000000001,0.006355
slothy_ci_fft_1732885741083.txt 0.300082 0.37296500000000005 0.33066120000000004 0.322761 0.37296500000000005,0.300082,0.35086500000000004,0.30663300000000004,0.322761
slothy_ci_fft_1732886041823.txt 300.07329100000004 300.08311100000003 300.0765498 300.075129 300.08311100000003,300.076653,300.07329100000004,300.074565,300.075129
slothy_ci_fft_1732886044279.txt 2.482616 3.099448 2.9266796000000004 2.9907410000000003 2.9907410000000003,2.9799870000000004,2.482616,3.099448,3.0806060000000004
slothy_ci_fft_1732886044669.txt 0.004667 0.0051920000000000004 0.004878400000000001 0.0048330000000000005 0.0048330000000000005,0.004667,0.0051920000000000004,0.004727,0.004973
slothy_ci_fft_1732886046623.txt 1.740823 2.4079330000000003 2.0627752 2.119635 2.119635,2.155132,1.740823,2.4079330000000003,1.8903530000000002
slothy_ci_fft_1732886047260.txt 0.011814000000000002 0.013729000000000002 0.012580200000000001 0.012534 0.01267,0.011814000000000002,0.012534,0.013729000000000002,0.012154000000000002
slothy_ci_fft_1732886050975.txt 2.86289 4.797361 3.8230956000000003 4.234861 4.797361,4.236548,4.234861,2.9838180000000003,2.86289
slothy_ci_fft_1732886055093.txt 6.795723000000001 36.122422 15.591973000000001 8.155078000000001 19.635652,36.122422,6.795723000000001,7.250990000000001,8.155078000000001
slothy_ci_fft_1732886055624.txt 0.013954000000000001 0.016560000000000002 0.015050400000000002 0.015174000000000002 0.015558,0.014006000000000001,0.016560000000000002,0.013954000000000001,0.015174000000000002
slothy_ci_fft_1732886055886.txt 0.206268 0.23117700000000002 0.21464080000000002 0.21036000000000002 0.21036000000000002,0.206268,0.209346,0.23117700000000002,0.21605300000000002
slothy_ci_fft_1732886059170.txt 2.614574 2.7675620000000003 2.6909122 2.6914360000000004 2.7675620000000003,2.6238010000000003,2.6914360000000004,2.757188,2.614574
slothy_ci_fft_1732886060990.txt 2.2570140000000003 3.3534080000000004 2.6628666 2.592667 2.333314,2.592667,2.2570140000000003,2.77793,3.3534080000000004
slothy_ci_fft_1732886061474.txt 0.0008190000000000001 0.0025510000000000003 0.0018138000000000002 0.002424 0.0025510000000000003,0.002424,0.0008190000000000001,0.0008470000000000001,0.002428
slothy_ci_fft_1732886061862.txt 0.352943 1.5356830000000001 0.628673 0.41231700000000004 1.5356830000000001,0.39460700000000004,0.447815,0.352943,0.41231700000000004
slothy_ci_sqmag_1732886169661.txt 0.004684000000000001 0.005378 0.0049286 0.0047220000000000005 0.004684000000000001,0.0047220000000000005,0.00472,0.005378,0.005139
slothy_ci_sqmag_1732886170206.txt 0.021544 0.023722 0.0229366 0.023112 0.023112,0.023517000000000003,0.023722,0.021544,0.022788000000000003
slothy_ci_sqmag_1732886170260.txt 0.031306 0.038464000000000005 0.035275400000000005 0.03554 0.038313,0.031306,0.038464000000000005,0.03554,0.032754000000000005
slothy_ci_sqmag_1732886170365.txt 0.000293 0.000364 0.0003128 0.000304 0.000364,0.000304,0.000297,0.000306,0.000293
slothy_ci_sqmag_1732886170376.txt 0.000858 0.000955 0.000885 0.0008690000000000001 0.000955,0.0008820000000000001,0.0008690000000000001,0.000861,0.000858
slothy_ci_sqmag_1732886170392.txt 0.005966 0.006614 0.0062182 0.00617 0.006614,0.005966,0.00617,0.006214000000000001,0.0061270000000000005
slothy_ci_sqmag_1732886170416.txt 0.0027990000000000003 0.0032150000000000004 0.002928 0.002869 0.0032150000000000004,0.002869,0.0027990000000000003,0.0028020000000000002,0.002955
slothy_ci_sqmag_1732886171089.txt 0.12621500000000002 0.13414500000000001 0.13111620000000002 0.13209700000000002 0.133161,0.129963,0.13414500000000001,0.13209700000000002,0.12621500000000002
slothy_ci_sqmag_1732886171273.txt 0.001399 0.0017690000000000002 0.00152 0.001494 0.0017690000000000002,0.0015110000000000002,0.001494,0.001427,0.001399
slothy_ci_sqmag_1732886171295.txt 0.006135000000000001 0.006658000000000001 0.0063376000000000005 0.006246000000000001 0.006658000000000001,0.006246000000000001,0.006411,0.006135000000000001,0.0062380000000000005
slothy_ci_sqmag_1732886171334.txt 0.004269 0.004385 0.0043282 0.004344000000000001 0.00436,0.004385,0.004344000000000001,0.004283,0.004269
slothy_ci_sqmag_1732886175260.txt 3.252313 14.663152 6.3624268 4.682596 5.275483,14.663152,3.93859,3.252313,4.682596
slothy_ci_sqmag_1732886175632.txt 0.00619 0.0062510000000000005 0.0062066000000000005 0.006195 0.0062510000000000005,0.006203,0.00619,0.006195,0.006194000000000001
slothy_ci_sqmag_1732886175721.txt 0.041046 0.057291 0.0477548 0.047290000000000006 0.041550000000000004,0.041046,0.051597000000000004,0.057291,0.047290000000000006
slothy_ci_sqmag_1732886175796.txt 0.012802000000000001 0.013705 0.013135800000000001 0.012910000000000001 0.012802000000000001,0.012844000000000001,0.013418000000000001,0.012910000000000001,0.013705
slothy_ci_sqmag_1732886176368.txt 0.009096 0.009213 0.009130000000000001 0.009113000000000001 0.009213,0.009112,0.009113000000000001,0.009096,0.009116
slothy_ci_sqmag_1732886176419.txt 0.025705000000000002 0.031898 0.0299862 0.030996000000000003 0.025705000000000002,0.029602000000000003,0.03173,0.030996000000000003,0.031898
slothy_ci_sqmag_1732886176494.txt 0.052306000000000005 0.061907000000000004 0.056356 0.05505 0.05505,0.052306000000000005,0.055005000000000005,0.057512,0.061907000000000004
slothy_ci_sqmag_1732886176599.txt 0.000821 0.002398 0.0017602000000000002 0.0022990000000000003 0.0009390000000000001,0.0022990000000000003,0.002398,0.002344,0.000821
slothy_ci_sqmag_1732886176618.txt 0.000642 0.0007300000000000001 0.0006714000000000001 0.0006580000000000001 0.0007300000000000001,0.0006770000000000001,0.0006580000000000001,0.0006500000000000001,0.000642
slothy_ci_sqmag_1732886176630.txt 0.0037530000000000003 0.004047 0.0039040000000000004 0.00391 0.004047,0.004015,0.00391,0.003795,0.0037530000000000003
slothy_ci_sqmag_1732886177335.txt 0.166549 0.19278 0.18300860000000002 0.18821000000000002 0.18821000000000002,0.19278,0.166549,0.179098,0.18840600000000002
slothy_ci_sqmag_1732886177520.txt 0.0007490000000000001 0.0007830000000000001 0.0007586 0.000754 0.0007830000000000001,0.000758,0.000754,0.0007490000000000001,0.0007490000000000001
slothy_ci_sqmag_1732886177538.txt 0.001374 0.004523 0.0020978 0.0014080000000000002 0.004523,0.001801,0.0014080000000000002,0.0013830000000000001,0.001374
slothy_ci_sqmag_1732886177556.txt 0.0015010000000000002 0.0016640000000000001 0.0015418 0.001521 0.0016640000000000001,0.001521,0.001521,0.001502,0.0015010000000000002
slothy_ci_sqmag_1732886177585.txt 0.012623 0.014497000000000001 0.013570200000000001 0.013614000000000001 0.013888000000000001,0.012623,0.013614000000000001,0.013229000000000001,0.014497000000000001
slothy_ci_sqmag_1732886177643.txt 0.009238000000000001 2.336047 0.47509500000000005 0.009964 0.009255000000000001,0.009964,0.009238000000000001,2.336047,0.010971
slothy_ci_sqmag_1732886179892.txt 2.333314 2.914677 2.7022772 2.8902680000000003 2.4625500000000002,2.910577,2.333314,2.8902680000000003,2.914677
slothy_ci_sqmag_1732886180266.txt 0.003105 0.003343 0.0032064000000000003 0.0031980000000000003 0.003343,0.0031980000000000003,0.0032280000000000004,0.003158,0.003105
slothy_ci_sqmag_1732886180320.txt 0.005409000000000001 0.005730000000000001 0.0054882 0.005438 0.005730000000000001,0.005409000000000001,0.005453,0.005411,0.005438
slothy_ci_sqmag_1732886180391.txt 0.045104000000000005 0.053138000000000005 0.05010540000000001 0.05133600000000001 0.045104000000000005,0.05133600000000001,0.051342000000000006,0.049607000000000005,0.053138000000000005
slothy_ci_sqmag_1732886180468.txt 0.015358 0.019602 0.016546 0.0161 0.015533000000000002,0.015358,0.0161,0.019602,0.016137000000000002

v9.11

model minimum maximum mean median raw times
ntt_kyber_123_4567_a55_1732871550401.txt 6.498353000000001 14.698543 9.220224 8.565066 6.498353000000001,8.565066,14.698543,7.446459000000001,8.892699
ntt_kyber_123_4567_a55_1732871552283.txt 0.018437000000000002 0.020701 0.019421200000000003 0.0187 0.020699000000000002,0.0187,0.018437000000000002,0.020701,0.018569000000000002
ntt_kyber_123_4567_a55_1732871559472.txt 11.095245 11.247692 11.1687808 11.172014 11.183826,11.247692,11.095245,11.145127,11.172014
ntt_kyber_123_4567_a55_1732871567261.txt 15.908032 140.037687 44.7931826 18.570816 31.052470000000003,18.570816,15.908032,18.396908,140.037687
ntt_kyber_123_4567_a55_1732871573800.txt 5.748259 6.841776 6.4292208 6.76288 6.814819000000001,5.97837,6.841776,5.748259,6.76288
ntt_kyber_123_4567_a55_1732871574381.txt 0.286238 0.33154500000000003 0.30832780000000004 0.304866 0.286238,0.304866,0.298454,0.33154500000000003,0.32053600000000004
ntt_dilithium_123_45678_a55_1732871577687.txt 3.368634 6.288773000000001 4.131989 3.4335590000000002 4.169288,6.288773000000001,3.368634,3.4335590000000002,3.3996910000000002
ntt_dilithium_123_45678_a55_1732871582700.txt 3.716978 4.100865000000001 3.8803742000000003 3.88898 3.88898,3.796593,3.716978,4.100865000000001,3.8984550000000002
ntt_dilithium_123_45678_a55_1732871585475.txt 0.11296400000000001 0.11478000000000001 0.11396840000000001 0.11446200000000001 0.11460200000000001,0.11478000000000001,0.11303400000000001,0.11446200000000001,0.11296400000000001
ntt_dilithium_123_45678_a55_1732871586516.txt 0.05469 0.055776000000000006 0.055090400000000005 0.054934000000000004 0.05469,0.055776000000000006,0.054762000000000005,0.055290000000000006,0.054934000000000004
ntt_dilithium_123_45678_a55_1732871589927.txt 3.8146970000000002 9.878825 5.3708762000000005 4.111192 3.8146970000000002,4.984507000000001,9.878825,4.0651600000000006,4.111192
ntt_dilithium_123_45678_a55_1732871592855.txt 0.05471500000000001 0.055985 0.055425600000000005 0.055482000000000004 0.055985,0.055755000000000006,0.05471500000000001,0.055482000000000004,0.055191000000000004
ntt_dilithium_123_45678_a55_1732871593899.txt 0.055068000000000006 0.05559 0.0552968 0.055294 0.05559,0.055068000000000006,0.055337000000000004,0.055195,0.055294
ntt_dilithium_123_45678_a55_1732871597518.txt 4.086258 4.39123 4.2429428 4.256788 4.160668,4.256788,4.39123,4.31977,4.086258
ntt_dilithium_123_45678_a55_1732871600604.txt 2.590552 2.7991460000000004 2.7062464000000004 2.737222 2.7991460000000004,2.737222,2.590552,2.6546320000000003,2.74968
ntt_dilithium_123_45678_a55_1732871609337.txt 9.025665 951.283582 197.541901 9.11326 9.10757,9.025665,9.11326,9.179428,951.283582
ntt_dilithium_123_45678_a55_1732871609913.txt 0.036363 0.037484 0.0371402 0.037307 0.036363,0.037484,0.037317,0.037307,0.03723
ntt_dilithium_123_45678_a55_1732871617015.txt 7.499091000000001 7.6277550000000005 7.5449998 7.540185 7.549301000000001,7.6277550000000005,7.540185,7.508667000000001,7.499091000000001
ntt_dilithium_123_45678_a55_1732871617578.txt 0.039536 0.040897 0.0398912 0.039687 0.040897,0.039572,0.039687,0.039536,0.039764
ntt_dilithium_123_45678_a55_1732871617943.txt 0.040211000000000004 0.041964 0.0406568 0.040365000000000005 0.041964,0.04025,0.040365000000000005,0.040494,0.040211000000000004
ntt_dilithium_123_45678_a55_1732871618289.txt 0.040604 0.040852 0.040713200000000005 0.040682 0.040682,0.040773000000000004,0.040852,0.040655000000000004,0.040604
ntt_dilithium_123_45678_a55_1732871618658.txt 0.041007 0.043503 0.041797 0.041093000000000005 0.043503,0.042350000000000006,0.041032000000000006,0.041007,0.041093000000000005
ntt_dilithium_123_45678_a55_1732871618731.txt 0.016914000000000002 0.026740000000000003 0.019812200000000002 0.018053 0.018053,0.016914000000000002,0.017597,0.019757,0.026740000000000003
ntt_dilithium_123_45678_a55_1732871618882.txt 0.020705 0.028434 0.023754 0.023849000000000002 0.028434,0.021367,0.023849000000000002,0.020705,0.024415000000000003
ntt_dilithium_123_45678_a55_1732871619064.txt 0.10001900000000001 0.123757 0.1080836 0.101498 0.101498,0.10081200000000001,0.10001900000000001,0.114332,0.123757
ntt_dilithium_123_45678_a55_1732871619212.txt 0.06957200000000001 0.07186000000000001 0.070757 0.07036500000000001 0.07036500000000001,0.071732,0.07186000000000001,0.06957200000000001,0.070256
ntt_dilithium_123_45678_a55_1732871619340.txt 0.0036530000000000004 0.004731 0.0038844 0.003673 0.004731,0.003699,0.003673,0.0036530000000000004,0.0036660000000000004
ntt_dilithium_123_45678_a55_1732871619419.txt 0.060058 0.067669 0.0638244 0.063154 0.066664,0.060058,0.067669,0.063154,0.06157700000000001
ntt_dilithium_123_45678_a55_1732871619534.txt 0.007721 0.009629 0.008204 0.007764 0.008156,0.009629,0.007764,0.007750000000000001,0.007721
ntt_dilithium_123_45678_a55_1732871619612.txt 0.05254 0.066288 0.057676000000000005 0.057524000000000006 0.053532,0.058496000000000006,0.066288,0.05254,0.057524000000000006
ntt_dilithium_123_45678_a55_1732871622559.txt 0.307792 0.311456 0.30890900000000004 0.30831600000000003 0.311456,0.30869,0.30829100000000004,0.307792,0.30831600000000003
ntt_dilithium_123_45678_a55_1732871625134.txt 0.126586 0.129186 0.12775 0.127831 0.127831,0.129186,0.126675,0.128472,0.126586
ntt_dilithium_123_45678_a55_1732871684991.txt 14.418077 16.449798 15.1966166 14.808354000000001 14.808354000000001,14.495005,16.449798,15.811849,14.418077
ntt_dilithium_123_45678_a55_1732871842631.txt 35.591963 101.776319 65.566213 65.488027 89.264656,35.591963,101.776319,35.710100000000004,65.488027
ntt_dilithium_123_45678_a55_1732871848968.txt 0.12423200000000001 0.126715 0.1257208 0.12660200000000002 0.12660200000000002,0.124419,0.126636,0.12423200000000001,0.126715
ntt_dilithium_123_45678_a55_1732871976221.txt 23.838494 51.196215 39.4520416 37.911116 23.838494,37.911116,51.196215,33.3941,50.920283000000005
ntt_dilithium_123_45678_a55_1732872018645.txt 19.401859 116.74107400000001 64.4812392 40.137691000000004 40.137691000000004,39.78504,19.401859,116.74107400000001,106.34053200000001
ntt_dilithium_123_45678_a55_1732872081773.txt 69.78313800000001 1005.7191710000001 267.0434816 94.523769 1005.7191710000001,94.691991,94.523769,69.78313800000001,70.499339
ntt_dilithium_123_45678_a55_1732872088088.txt 0.126392 0.127465 0.12665320000000002 0.126427 0.127465,0.126427,0.12640900000000002,0.12657300000000002,0.126392
ntt_dilithium_123_45678_a55_1732872089126.txt 0.142236 0.143287 0.142795 0.1429 0.143287,0.142405,0.143147,0.142236,0.1429
ntt_dilithium_123_45678_a55_1732872090056.txt 0.094889 0.09592300000000001 0.095375 0.09541300000000001 0.09541300000000001,0.095156,0.09592300000000001,0.094889,0.09549400000000001
ntt_dilithium_123_45678_a55_1732872093989.txt 4.524652000000001 4.801609 4.7053254 4.774102 4.524652000000001,4.774102,4.801609,4.780121,4.646143
ntt_dilithium_123_45678_a55_1732872098461.txt 4.356883 4.899605 4.707584600000001 4.8071850000000005 4.8560430000000006,4.618207,4.8071850000000005,4.899605,4.356883
ntt_dilithium_123_45678_a55_1732872103048.txt 4.892204 5.851543 5.189521 4.931142 4.892204,5.851543,4.896426,5.37629,4.931142
ntt_dilithium_123_45678_a55_1732872104601.txt 0.076965 0.078651 0.0775136 0.077308 0.078651,0.07711900000000001,0.07752500000000001,0.076965,0.077308
ntt_dilithium_123_45678_a55_1732872105497.txt 0.06970900000000001 0.071123 0.07045140000000001 0.070634 0.071123,0.06975400000000001,0.071037,0.06970900000000001,0.070634
ntt_dilithium_123_45678_a55_1732872109595.txt 5.320600000000001 6.322648 5.6188456 5.472672 6.322648,5.320600000000001,5.545637,5.472672,5.432671
ntt_dilithium_123_45678_a55_1732872111130.txt 0.069761 0.07129500000000001 0.0702908 0.070132 0.07129500000000001,0.070132,0.069761,0.07005,0.070216
ntt_dilithium_123_45678_a55_1732872111187.txt 0.0029400000000000003 0.003769 0.003183 0.003 0.003769,0.003223,0.002983,0.003,0.0029400000000000003
ntt_dilithium_123_45678_a55_1732872111224.txt 0.0028840000000000003 0.003537 0.0031052000000000002 0.0029530000000000003 0.0032040000000000003,0.002948,0.003537,0.0028840000000000003,0.0029530000000000003
ntt_dilithium_123_45678_a55_1732872111286.txt 0.004485 0.004732 0.0046154 0.004633 0.004727,0.004732,0.004485,0.004633,0.0045000000000000005
ntt_dilithium_123_45678_a55_1732872111322.txt 0.0028290000000000004 0.0030540000000000003 0.002934 0.002898 0.0030540000000000003,0.002896,0.002898,0.0028290000000000004,0.002993
ntt_dilithium_123_45678_a55_1732872111355.txt 0.000164 0.00017 0.0001676 0.00016900000000000002 0.00017,0.00016900000000000002,0.00016900000000000002,0.00016600000000000002,0.000164
ntt_dilithium_123_45678_a55_1732872111368.txt 0.00016900000000000002 0.00024400000000000002 0.0001878 0.000173 0.00024400000000000002,0.000182,0.000173,0.00016900000000000002,0.000171
ntt_dilithium_123_45678_a55_1732872111381.txt 0.000184 0.00022700000000000002 0.0001968 0.000191 0.00022700000000000002,0.00019600000000000002,0.000191,0.00018600000000000002,0.000184
slothy_ci_fft_1732872454264.txt 0.014616 0.01793 0.016534200000000002 0.016485 0.017320000000000002,0.01793,0.01632,0.016485,0.014616
slothy_ci_fft_1732872457745.txt 3.3231110000000004 5.019269 4.066391 4.0288520000000005 4.046725,3.3231110000000004,5.019269,4.0288520000000005,3.9139980000000003
slothy_ci_fft_1732872460115.txt 2.905796 4.783691 3.6741398000000003 2.961487 2.9542800000000002,2.961487,4.783691,4.765445000000001,2.905796
slothy_ci_fft_1732872460497.txt 0.006246000000000001 0.007214000000000001 0.006619600000000001 0.006259000000000001 0.007214000000000001,0.006259000000000001,0.00713,0.006249,0.006246000000000001
slothy_ci_fft_1732872460930.txt 0.48587 0.669656 0.5565898 0.5415800000000001 0.669656,0.548508,0.537335,0.48587,0.5415800000000001
slothy_ci_fft_1732872464714.txt 300.084463 300.204678 300.1453854 300.162228 300.162228,300.16783200000003,300.107726,300.084463,300.204678
slothy_ci_fft_1732872466679.txt 1.632489 1.6646 1.6521816 1.6594250000000001 1.639798,1.6594250000000001,1.6645960000000002,1.632489,1.6646
slothy_ci_fft_1732872467049.txt 0.002206 0.0029660000000000003 0.0025256000000000002 0.002526 0.0022800000000000003,0.00265,0.0029660000000000003,0.002526,0.002206
slothy_ci_fft_1732872467065.txt 0.002223 0.003082 0.0024514000000000003 0.002252 0.0024720000000000002,0.002228,0.002252,0.002223,0.003082
slothy_ci_fft_1732872467644.txt 0.014128000000000002 0.015282 0.0146776 0.014582000000000001 0.014582000000000001,0.014128000000000002,0.015086,0.015282,0.014310000000000002
slothy_ci_fft_1732872471249.txt 4.9394610000000005 11.256466000000001 8.5007454 8.390454 7.761466,8.390454,10.15588,11.256466000000001,4.9394610000000005
slothy_ci_fft_1732872473702.txt 2.66319 2.741223 2.7088934 2.707247 2.707247,2.741223,2.66319,2.701398,2.731409
slothy_ci_fft_1732872474172.txt 0.001438 0.0035710000000000004 0.0026992 0.0026290000000000003 0.001438,0.0035710000000000004,0.0026290000000000003,0.0032310000000000004,0.002627
slothy_ci_fft_1732872474591.txt 0.611055 0.670377 0.6376040000000001 0.6318640000000001 0.6318640000000001,0.616394,0.611055,0.670377,0.6583300000000001
slothy_ci_fft_1732872478674.txt 3.3267700000000002 3.920983 3.6765114000000003 3.8650460000000004 3.920983,3.3267700000000002,3.8837040000000003,3.386054,3.8650460000000004
slothy_ci_fft_1732872480043.txt 4.287905 5.243915 4.7220158 4.5511230000000005 5.243915,5.139880000000001,4.287905,4.5511230000000005,4.387256000000001
slothy_ci_fft_1732872480850.txt 0.710121 0.8274590000000001 0.7413374 0.716439 0.710121,0.8274590000000001,0.738568,0.7141000000000001,0.716439
slothy_ci_fft_1732872480975.txt 0.001554 0.002364 0.0021238000000000003 0.0022040000000000002 0.002364,0.002199,0.001554,0.0022040000000000002,0.002298
slothy_ci_sqmag_1732872606741.txt 0.005568 0.0061860000000000005 0.0059762 0.006095000000000001 0.005858,0.006095000000000001,0.0061860000000000005,0.005568,0.006174000000000001
slothy_ci_sqmag_1732872607307.txt 0.020548 0.024790000000000003 0.0230096 0.023061 0.022543,0.023061,0.020548,0.024790000000000003,0.024106000000000002
slothy_ci_sqmag_1732872607373.txt 0.036340000000000004 0.043975 0.039625600000000004 0.040041 0.043975,0.040041,0.041231000000000004,0.036340000000000004,0.036541000000000004
slothy_ci_sqmag_1732872607459.txt 0.00018500000000000002 0.00026700000000000004 0.0002066 0.000191 0.00026700000000000004,0.000202,0.000191,0.00018800000000000002,0.00018500000000000002
slothy_ci_sqmag_1732872607470.txt 0.002536 0.0029430000000000003 0.0027370000000000003 0.0028150000000000002 0.0029430000000000003,0.0028160000000000004,0.0028150000000000002,0.002575,0.002536
slothy_ci_sqmag_1732872607492.txt 0.003153 0.004002 0.0035292 0.003527 0.003412,0.004002,0.003552,0.003527,0.003153
slothy_ci_sqmag_1732872608184.txt 0.160061 0.17938500000000002 0.1666908 0.163054 0.17938500000000002,0.163054,0.16107100000000002,0.160061,0.169883
slothy_ci_sqmag_1732872608364.txt 0.000858 0.0010400000000000001 0.0009034000000000001 0.00087 0.0010400000000000001,0.000888,0.00087,0.000861,0.000858
slothy_ci_sqmag_1732872608379.txt 0.001636 0.004054 0.0030838000000000003 0.0032400000000000003 0.0032400000000000003,0.001636,0.004054,0.0034630000000000004,0.003026
slothy_ci_sqmag_1732872608416.txt 0.005986 0.006648 0.0063584 0.006436000000000001 0.006648,0.005986,0.006436000000000001,0.006256,0.006466
slothy_ci_sqmag_1732872612771.txt 5.870426 908.8210700000001 186.94872660000001 6.862021 6.166686,7.02343,6.862021,5.870426,908.8210700000001
slothy_ci_sqmag_1732872613139.txt 0.011565 0.012906 0.0119804 0.011605 0.012906,0.012233,0.011593000000000001,0.011605,0.011565
slothy_ci_sqmag_1732872613222.txt 0.060323 0.070721 0.06492780000000001 0.06647900000000001 0.070721,0.060386,0.06673,0.060323,0.06647900000000001
slothy_ci_sqmag_1732872613311.txt 0.016491000000000002 0.019287000000000002 0.017904 0.018139000000000002 0.018139000000000002,0.018164,0.016491000000000002,0.019287000000000002,0.017439
slothy_ci_sqmag_1732872613875.txt 0.0030840000000000004 0.0031680000000000002 0.0031136000000000002 0.0030940000000000004 0.003135,0.0031680000000000002,0.0030840000000000004,0.0030940000000000004,0.0030870000000000003
slothy_ci_sqmag_1732872613949.txt 0.028744000000000002 0.03832 0.0346396 0.036809 0.032505,0.036820000000000006,0.036809,0.028744000000000002,0.03832
slothy_ci_sqmag_1732872614019.txt 0.068321 0.08583700000000001 0.0764248 0.075667 0.068321,0.081374,0.08583700000000001,0.070925,0.075667
slothy_ci_sqmag_1732872614104.txt 0.000947 0.003578 0.0021894 0.0025900000000000003 0.002834,0.003578,0.0025900000000000003,0.000947,0.000998
slothy_ci_sqmag_1732872614123.txt 0.000181 0.00026900000000000003 0.0002044 0.000189 0.00026900000000000003,0.000199,0.000189,0.000181,0.000184
slothy_ci_sqmag_1732872614131.txt 0.000165 0.00022500000000000002 0.00018260000000000002 0.000172 0.00022500000000000002,0.000182,0.000172,0.00016900000000000002,0.000165
slothy_ci_sqmag_1732872614141.txt 0.0007530000000000001 0.0009050000000000001 0.0007970000000000001 0.000772 0.0009050000000000001,0.0007930000000000001,0.000772,0.0007620000000000001,0.0007530000000000001
slothy_ci_sqmag_1732872614156.txt 0.007523 0.008042 0.007799800000000001 0.007878000000000001 0.007878000000000001,0.007523,0.008032000000000001,0.007524,0.008042
slothy_ci_sqmag_1732872614904.txt 0.21260800000000002 0.266633 0.2281738 0.21555700000000003 0.21433600000000003,0.23173500000000002,0.266633,0.21555700000000003,0.21260800000000002
slothy_ci_sqmag_1732872615084.txt 0.000509 0.000586 0.0005298000000000001 0.0005160000000000001 0.000586,0.000528,0.0005160000000000001,0.00051,0.000509
slothy_ci_sqmag_1732872615101.txt 0.0014 0.001637 0.0014574000000000002 0.0014130000000000002 0.001637,0.0014310000000000002,0.0014130000000000002,0.0014060000000000001,0.0014
slothy_ci_sqmag_1732872615122.txt 0.005969 0.007353 0.006562200000000001 0.0064210000000000005 0.006155000000000001,0.005969,0.0064210000000000005,0.007353,0.006913000000000001
slothy_ci_sqmag_1732872615160.txt 0.004220000000000001 0.004874000000000001 0.0045332 0.0045460000000000006 0.004874000000000001,0.004446,0.0045460000000000006,0.004220000000000001,0.00458
slothy_ci_sqmag_1732872618787.txt 3.11606 4.740810000000001 3.9801702000000003 4.240719 4.240719,3.11606,3.3054180000000004,4.740810000000001,4.497844000000001
slothy_ci_sqmag_1732872619156.txt 0.00394 0.004481000000000001 0.0041272 0.004069000000000001 0.004481000000000001,0.004205,0.003941,0.00394,0.004069000000000001
slothy_ci_sqmag_1732872619191.txt 0.003954 0.004397 0.0042450000000000005 0.004285000000000001 0.003954,0.004308,0.004281,0.004397,0.004285000000000001
slothy_ci_sqmag_1732872619323.txt 0.11873 0.137213 0.125539 0.121506 0.137213,0.12004100000000001,0.121506,0.13020500000000002,0.11873
slothy_ci_sqmag_1732872619403.txt 0.00044600000000000005 0.0005510000000000001 0.00047940000000000005 0.000462 0.0005510000000000001,0.000478,0.000462,0.00046,0.00044600000000000005
slothy_ci_sqmag_1732872619415.txt 0.003261 0.004288 0.0036068000000000003 0.0034530000000000003 0.0034530000000000003,0.004288,0.0036100000000000004,0.003422,0.003261

@mkannwischer mkannwischer force-pushed the ortools_9_11 branch 5 times, most recently from afc3227 to d08fe2f Compare December 5, 2024 10:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants