-
Notifications
You must be signed in to change notification settings - Fork 2
/
README.html
1002 lines (889 loc) · 31.9 KB
/
README.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en">
<head>
<!-- 2020-04-21 tis 16:15 -->
<meta http-equiv="Content-Type" content="text/html;charset=utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>PAPER 99: Improving Program Locality in the GC using Hotness</title>
<meta name="generator" content="Org mode" />
<meta name="author" content="Tobias Wrigstad" />
<style type="text/css">
<!--/*--><![CDATA[/*><!--*/
.title { text-align: center;
margin-bottom: .2em; }
.subtitle { text-align: center;
font-size: medium;
font-weight: bold;
margin-top:0; }
.todo { font-family: monospace; color: red; }
.done { font-family: monospace; color: green; }
.priority { font-family: monospace; color: orange; }
.tag { background-color: #eee; font-family: monospace;
padding: 2px; font-size: 80%; font-weight: normal; }
.timestamp { color: #bebebe; }
.timestamp-kwd { color: #5f9ea0; }
.org-right { margin-left: auto; margin-right: 0px; text-align: right; }
.org-left { margin-left: 0px; margin-right: auto; text-align: left; }
.org-center { margin-left: auto; margin-right: auto; text-align: center; }
.underline { text-decoration: underline; }
#postamble p, #preamble p { font-size: 90%; margin: .2em; }
p.verse { margin-left: 3%; }
pre {
border: 1px solid #ccc;
box-shadow: 3px 3px 3px #eee;
padding: 8pt;
font-family: monospace;
overflow: auto;
margin: 1.2em;
}
pre.src {
position: relative;
overflow: visible;
padding-top: 1.2em;
}
pre.src:before {
display: none;
position: absolute;
background-color: white;
top: -10px;
right: 10px;
padding: 3px;
border: 1px solid black;
}
pre.src:hover:before { display: inline;}
/* Languages per Org manual */
pre.src-asymptote:before { content: 'Asymptote'; }
pre.src-awk:before { content: 'Awk'; }
pre.src-C:before { content: 'C'; }
/* pre.src-C++ doesn't work in CSS */
pre.src-clojure:before { content: 'Clojure'; }
pre.src-css:before { content: 'CSS'; }
pre.src-D:before { content: 'D'; }
pre.src-ditaa:before { content: 'ditaa'; }
pre.src-dot:before { content: 'Graphviz'; }
pre.src-calc:before { content: 'Emacs Calc'; }
pre.src-emacs-lisp:before { content: 'Emacs Lisp'; }
pre.src-fortran:before { content: 'Fortran'; }
pre.src-gnuplot:before { content: 'gnuplot'; }
pre.src-haskell:before { content: 'Haskell'; }
pre.src-hledger:before { content: 'hledger'; }
pre.src-java:before { content: 'Java'; }
pre.src-js:before { content: 'Javascript'; }
pre.src-latex:before { content: 'LaTeX'; }
pre.src-ledger:before { content: 'Ledger'; }
pre.src-lisp:before { content: 'Lisp'; }
pre.src-lilypond:before { content: 'Lilypond'; }
pre.src-lua:before { content: 'Lua'; }
pre.src-matlab:before { content: 'MATLAB'; }
pre.src-mscgen:before { content: 'Mscgen'; }
pre.src-ocaml:before { content: 'Objective Caml'; }
pre.src-octave:before { content: 'Octave'; }
pre.src-org:before { content: 'Org mode'; }
pre.src-oz:before { content: 'OZ'; }
pre.src-plantuml:before { content: 'Plantuml'; }
pre.src-processing:before { content: 'Processing.js'; }
pre.src-python:before { content: 'Python'; }
pre.src-R:before { content: 'R'; }
pre.src-ruby:before { content: 'Ruby'; }
pre.src-sass:before { content: 'Sass'; }
pre.src-scheme:before { content: 'Scheme'; }
pre.src-screen:before { content: 'Gnu Screen'; }
pre.src-sed:before { content: 'Sed'; }
pre.src-sh:before { content: 'shell'; }
pre.src-sql:before { content: 'SQL'; }
pre.src-sqlite:before { content: 'SQLite'; }
/* additional languages in org.el's org-babel-load-languages alist */
pre.src-forth:before { content: 'Forth'; }
pre.src-io:before { content: 'IO'; }
pre.src-J:before { content: 'J'; }
pre.src-makefile:before { content: 'Makefile'; }
pre.src-maxima:before { content: 'Maxima'; }
pre.src-perl:before { content: 'Perl'; }
pre.src-picolisp:before { content: 'Pico Lisp'; }
pre.src-scala:before { content: 'Scala'; }
pre.src-shell:before { content: 'Shell Script'; }
pre.src-ebnf2ps:before { content: 'ebfn2ps'; }
/* additional language identifiers per "defun org-babel-execute"
in ob-*.el */
pre.src-cpp:before { content: 'C++'; }
pre.src-abc:before { content: 'ABC'; }
pre.src-coq:before { content: 'Coq'; }
pre.src-groovy:before { content: 'Groovy'; }
/* additional language identifiers from org-babel-shell-names in
ob-shell.el: ob-shell is the only babel language using a lambda to put
the execution function name together. */
pre.src-bash:before { content: 'bash'; }
pre.src-csh:before { content: 'csh'; }
pre.src-ash:before { content: 'ash'; }
pre.src-dash:before { content: 'dash'; }
pre.src-ksh:before { content: 'ksh'; }
pre.src-mksh:before { content: 'mksh'; }
pre.src-posh:before { content: 'posh'; }
/* Additional Emacs modes also supported by the LaTeX listings package */
pre.src-ada:before { content: 'Ada'; }
pre.src-asm:before { content: 'Assembler'; }
pre.src-caml:before { content: 'Caml'; }
pre.src-delphi:before { content: 'Delphi'; }
pre.src-html:before { content: 'HTML'; }
pre.src-idl:before { content: 'IDL'; }
pre.src-mercury:before { content: 'Mercury'; }
pre.src-metapost:before { content: 'MetaPost'; }
pre.src-modula-2:before { content: 'Modula-2'; }
pre.src-pascal:before { content: 'Pascal'; }
pre.src-ps:before { content: 'PostScript'; }
pre.src-prolog:before { content: 'Prolog'; }
pre.src-simula:before { content: 'Simula'; }
pre.src-tcl:before { content: 'tcl'; }
pre.src-tex:before { content: 'TeX'; }
pre.src-plain-tex:before { content: 'Plain TeX'; }
pre.src-verilog:before { content: 'Verilog'; }
pre.src-vhdl:before { content: 'VHDL'; }
pre.src-xml:before { content: 'XML'; }
pre.src-nxml:before { content: 'XML'; }
/* add a generic configuration mode; LaTeX export needs an additional
(add-to-list 'org-latex-listings-langs '(conf " ")) in .emacs */
pre.src-conf:before { content: 'Configuration File'; }
table { border-collapse:collapse; }
caption.t-above { caption-side: top; }
caption.t-bottom { caption-side: bottom; }
td, th { vertical-align:top; }
th.org-right { text-align: center; }
th.org-left { text-align: center; }
th.org-center { text-align: center; }
td.org-right { text-align: right; }
td.org-left { text-align: left; }
td.org-center { text-align: center; }
dt { font-weight: bold; }
.footpara { display: inline; }
.footdef { margin-bottom: 1em; }
.figure { padding: 1em; }
.figure p { text-align: center; }
.equation-container {
display: table;
text-align: center;
width: 100%;
}
.equation {
vertical-align: middle;
}
.equation-label {
display: table-cell;
text-align: right;
vertical-align: middle;
}
.inlinetask {
padding: 10px;
border: 2px solid gray;
margin: 10px;
background: #ffffcc;
}
#org-div-home-and-up
{ text-align: right; font-size: 70%; white-space: nowrap; }
textarea { overflow-x: auto; }
.linenr { font-size: smaller }
.code-highlighted { background-color: #ffff00; }
.org-info-js_info-navigation { border-style: none; }
#org-info-js_console-label
{ font-size: 10px; font-weight: bold; white-space: nowrap; }
.org-info-js_search-highlight
{ background-color: #ffff00; color: #000000; font-weight: bold; }
.org-svg { width: 90%; }
/*]]>*/-->
</style>
<style> code { background: #ddd; padding: 2px; } </style>
<style> body { width: 800px; margin: auto; } </style>
<style> blockquote { background: #fcc; padding: 2px 14px 2px 14px; } </style>
<script type="text/javascript">
/*
@licstart The following is the entire license notice for the
JavaScript code in this tag.
Copyright (C) 2012-2019 Free Software Foundation, Inc.
The JavaScript code in this tag is free software: you can
redistribute it and/or modify it under the terms of the GNU
General Public License (GNU GPL) as published by the Free Software
Foundation, either version 3 of the License, or (at your option)
any later version. The code is distributed WITHOUT ANY WARRANTY;
without even the implied warranty of MERCHANTABILITY or FITNESS
FOR A PARTICULAR PURPOSE. See the GNU GPL for more details.
As additional permission under GNU GPL version 3 section 7, you
may distribute non-source (e.g., minimized or compacted) forms of
that code without the copy of the GNU GPL normally required by
section 4, provided you include this license notice and a URL
through which recipients can access the Corresponding Source.
@licend The above is the entire license notice
for the JavaScript code in this tag.
*/
<!--/*--><![CDATA[/*><!--*/
function CodeHighlightOn(elem, id)
{
var target = document.getElementById(id);
if(null != target) {
elem.cacheClassElem = elem.className;
elem.cacheClassTarget = target.className;
target.className = "code-highlighted";
elem.className = "code-highlighted";
}
}
function CodeHighlightOff(elem, id)
{
var target = document.getElementById(id);
if(elem.cacheClassElem)
elem.className = elem.cacheClassElem;
if(elem.cacheClassTarget)
target.className = elem.cacheClassTarget;
}
/*]]>*///-->
</script>
</head>
<body>
<div id="content">
<h1 class="title">PAPER 99: Improving Program Locality in the GC using Hotness</h1>
<div id="table-of-contents">
<h2>Table of Contents</h2>
<div id="text-table-of-contents">
<ul>
<li><a href="#org3a2bb4d">1. Overview</a></li>
<li><a href="#orga9de12c">2. Getting Started</a>
<ul>
<li><a href="#org1d495a5">2.1. OPTION 1: Pre-Packaged Virtual Machine </a></li>
<li><a href="#orgebb61e3">2.2. OPTION 2: Manual Build on Debian/Ubuntu </a></li>
</ul>
</li>
<li><a href="#orgf61d9d1">3. Step-by-Step</a>
<ul>
<li><a href="#org3ff15e6">3.1. Running the Simplified Benchmarks</a>
<ul>
<li><a href="#org9dc217d">3.1.1. Kick-the-Tires Benchmark Fast (~15 minutes)</a></li>
<li><a href="#org4a7f684">3.1.2. Kick-the-Tires Benchmark Slow (optional – note: ~1 day)</a></li>
<li><a href="#orgbf55303">3.1.3. Full Benchmark Slow (Weeks(!))</a></li>
<li><a href="#org4da65c3">3.1.4. Exporting the Results for Viewing Outside the VM</a></li>
</ul>
</li>
<li><a href="#orga89ebec">3.2. How to Evaluate this Artefact</a></li>
<li><a href="#org7a6f71d">3.3. Viewing the Results</a></li>
</ul>
</li>
</ul>
</div>
</div>
<div id="outline-container-org3a2bb4d" class="outline-2">
<h2 id="org3a2bb4d"><span class="section-number-2">1</span> Overview</h2>
<div class="outline-text-2" id="text-1">
<p>
Paper 99 presents HCSGC, built on-top of the ZGC collector in
OpenJDK. HCSGC use the mutators (the applications' threads) to
relocate objects on the heap to improve their placement. As a
consequence of this design, objects touched by a mutator during
the relocation phase of the GC cycle will be laid out in access
order which may improve cache performance on subsequent accesses.
Furthermore, HCSGC can track objects' "hotness". An object is
"hot" if it was accessed during the last GC cycle, otherwise
"cold". During GC, hot objects and cold objects are segregated to
increase the density of hot objects, and weights can be assigned
to cold objects so that they count less towards the live bytes of
a ZGC memory page. This can be used to improve the selection of
what memory pages to relocate during GC, as GC is no longer only
concerned with freeing memory, but with improving object locality
in the program.
</p>
<p>
We demonstrate the impact of our design choices, and the
applicability of our techniques, through a selection of
benchmarks.
</p>
<p>
As the number of tuning knobs are relatively large, the paper
deals with 19 different configurations for each benchmark with
several runs (the exact number depending on the benchmark)
to reach a steady state. To save artefact reviewers' time, the we
provide a simple test script that generates Figure 4 from
the paper with a single run. We also provide
a medium-weight testing option to generate the full Figure 4-6 with
31 runs (same as in the paper).
</p>
<p>
The full benchmarks<sup><a id="fnr.1" class="footref" href="#fn.1">1</a></sup> are also provided and can
take weeks to run, depending on the machine.
</p>
<p>
After the benchmarks are run, an HTML file is open that shows the
produced plots with links to the plots of the paper for
comparison.
</p>
<p>
This artefact consists of the following:
</p>
<ol class="org-ol">
<li>The sources for HCSGC in the form of a single patch file for
OpenJDK (<code>0001-hcsgc.patch</code>).</li>
<li><p>
The benchmark programs from our paper (§4):
</p>
<ul class="org-ul">
<li>Our "sanity check" micro benchmark from §4.4;</li>
<li>The JGraphT benchmark from §4.5; and</li>
<li>The DaCapo benchmark suite from which we run Tradebeans and h2 in §4.6.</li>
</ul>
<p>
NOTE: Due to copyright restrictions, we cannot include SPECjbb2015 in the artefact.
</p></li>
<li>A virtual machine containing all of the above, including
pre-built OpenJDK and HCSGC. As many virtual machine
environments do not support <code>perf</code> (which we rely on for cache
statistics), the full experience of this artefact is likely ony
possible with a native installation.</li>
<li>Scripts to automate running the the benchmarks and producing
the plots.</li>
</ol>
<p>
Beware that this artefact requires several gigabytes of disk space
for all dependencies and the build itself.
</p>
</div>
</div>
<div id="outline-container-orga9de12c" class="outline-2">
<h2 id="orga9de12c"><span class="section-number-2">2</span> Getting Started</h2>
<div class="outline-text-2" id="text-2">
<p>
This section explains how to install HCSGC alongside an OpenJDK
baseline. <a href="#orgf61d9d1">The next section</a> explains how to run the benchmarks.
</p>
<p>
Please note that OpenJDK is a large project with many
dependencies, and that that our HCSGC paper compares 19 different
configurations which naturally takes some time to run. We explain
simplifications made to keep time short to kick the tires, and how
to run the full benchmarks to reproduce the results from the paper
(possibly modulo <code>perf</code>, when running inside a virtual machine).
</p>
<p>
We provide two different ways to explore this artefact. To
kick-the-tires, we suggest Option 1 (prebuilt VM). For someone
working deep in this space or wanting to fully reproduce our
results, we suggestion Option 2 (manual native install).
</p>
<ul class="org-ul">
<li><a href="#org51b4a34">Option 1</a> is a full VM (~8GB) download. This requires no other setup or
installation other than the virtual machine infrastructure.</li>
<li><a href="#orgcb1b57b">Option 2</a> is a set of manual instructions for how to build and
install HCSGC on Debian/Ubuntu.</li>
</ul>
<p>
Notes: Option 1, due to running inside virtual machines, typically
<b>cannot</b> record perf data and may skew the results.
</p>
</div>
<div id="outline-container-org1d495a5" class="outline-3">
<h3 id="org1d495a5"><span class="section-number-3">2.1</span> OPTION 1: Pre-Packaged Virtual Machine <a id="org51b4a34"></a></h3>
<div class="outline-text-3" id="text-2-1">
<p>
You can use your favorite virtualisation software. We have tested
this VM using <a href="https://www.virtualbox.org/">Virtual Box</a>.
</p>
<p>
After uncompressing the artifact, one can import the VM (<code>pldi.ova</code>) directly;
start and login to the VM with the following credentials:
</p>
<pre class="example">
user: vagrant
password: vagrant
</pre>
<p>
See <a href="#orgf61d9d1">the next section</a> for instructions for how to run the
benchmarks.
</p>
<p>
All interaction with the VM will happen in the terminal. The only
need for a graphical user interface is to view graphical plots
from the benchmarking.
</p>
</div>
</div>
<div id="outline-container-orgebb61e3" class="outline-3">
<h3 id="orgebb61e3"><span class="section-number-3">2.2</span> OPTION 2: Manual Build on Debian/Ubuntu <a id="orgcb1b57b"></a></h3>
<div class="outline-text-3" id="text-2-2">
<p>
Because of the large number of dependencies for OpenJDK, we limit ourselves to
providing manual instructions for building on debian/ubuntu. At the time we
implemented HCSGC, ZGC, which we build upon, was only available for 64bit Linux.
</p>
<p>
Create a directory for the artefact, e.g. <code>mkdir ~/paper99; cd ~/paper99</code>. All
commands from now on are assumed to be issued from within that directory.
</p>
<p>
After uncompressing the artifact, you will see the following files and
directories (discarding the .ova VM file):
</p>
<pre class="example">
~paper99
├── 0001-hcsgc.patch <-- HCSGC sources patch
├── benchmarks <-- benchmark programs & scripts for running/plotting
├── jtreg <-- OpenJDK build dependency
├── LICENSE
├── README.html
├── README.org
└── README.txt
</pre>
<p>
Install the basic building blocks:
</p>
<pre class="example">
sudo apt-get install -y build-essential autoconf git git-lfs
</pre>
<p>
Utilities
</p>
<pre class="example">
sudo apt-get install -y curl perf-tools-unstable time util-linux
</pre>
<p>
(Note that sometimes, the <code>perf-tools-unstable</code> package naming may vary.)
</p>
<p>
Make sure you have the permissions to use <code>perf</code>:
</p>
<pre class="example">
sudo sh -c 'echo -1 > /proc/sys/kernel/perf_event_paranoid'
</pre>
<p>
Install OpenJDK dependencies
</p>
<pre class="example">
sudo apt-get install -y zip unzip libx11-dev libxext-dev libxrender-dev libxrandr-dev libxtst-dev libxt-dev libcups2-dev libfontconfig1-dev libasound2-dev
</pre>
<p>
Note that you need a graphical environment to build OpenJDK. If you don't have that, OpenJDK will complain and suggest packages that will remedy the situation.
</p>
<p>
You also need to have a Java installation. If you can install OpenJDK 13 or above, this will simplify things later:
</p>
<pre class="example">
sudo apt-get install -y openjdk-13-jdk-headless
</pre>
<p>
<b>Otherwise, you will still need one to build the benchmark.</b> Either of the following should work:
</p>
<pre class="example">
sudo apt-get install -y openjdk-11-jdk-headless
sudo apt-get install -y openjdk-8-jdk-headless
</pre>
<p>
If you could not install OpenJDK 13 above, you <b>need</b> to install modern boot JDK to build:
</p>
<pre class="example">
curl -O https://download.java.net/java/GA/jdk13.0.2/d4173c853231432d94f001e99d882ca7/8/GPL/openjdk-13.0.2_linux-x64_bin.tar.gz
tar zxf openjdk-13.0.2_linux-x64_bin.tar.gz
</pre>
<p>
(If you can install <code>openjdk-13-jdk-headless</code>, you can skip this step and omit the <code>--with-boot-jdk</code> flag when running <code>configure</code> later.)
</p>
<p>
Clone baseline JDK from GitHub mirror, and checkout the commit on-top of which HCSGC was authored.
</p>
<pre class="example">
git clone https://github.com/openjdk/jdk.git openjdk
cd openjdk
git checkout 67a89143dde6e545adbfc3c79bb89d954307f8bc
cd ..
</pre>
<p>
Create a copy of the baseline to build HCSGC.
</p>
<pre class="example">
cp -R openjdk hcsgc
</pre>
<p>
Configure and finally build OpenJDK.
</p>
<pre class="example">
cd openjdk
bash configure --with-target-bits=64 --with-native-debug-symbols=none --with-jtreg=../jtreg --with-boot-jdk=../jdk-13.0.2 --disable-warnings-as-errors --with-extra-cflags='-Wno-nonnull-compare -Wno-format -Wno-stringop-truncation ' --with-extra-cxxflags='-std=gnu++11'
make CONF=release
bash configure --enable-debug --with-target-bits=64 --with-native-debug-symbols=internal --with-jtreg=../jtreg --with-boot-jdk=../jdk-13.0.2 --disable-warnings-as-errors --with-extra-cflags='-Wno-nonnull-compare -Wno-format -Wno-stringop-truncation ' --with-extra-cxxflags='-std=gnu++11'
make CONF=debug
cd ..
</pre>
<p>
(If you did install <code>openjdk-13-jdk-headless</code> using apt, skip the <code>--with-boot-jdk</code> flag above.)
</p>
<p>
Patch OpenJDK with the HCSGC patch.
</p>
<pre class="example">
cd hcsgc
git am < ../0001-hcsgc.patch
</pre>
<p>
Configure and finally build OpenJDK/HCSGC.
</p>
<pre class="example">
bash configure --with-target-bits=64 --with-native-debug-symbols=none --with-jtreg=../jtreg --with-boot-jdk=../jdk-13.0.2 --disable-warnings-as-errors --with-extra-cflags='-Wno-nonnull-compare -Wno-format -Wno-stringop-truncation ' --with-extra-cxxflags='-std=gnu++11'
make CONF=release
bash configure --enable-debug --with-target-bits=64 --with-native-debug-symbols=internal --with-jtreg=../jtreg --with-boot-jdk=../jdk-13.0.2 --disable-warnings-as-errors --with-extra-cflags='-Wno-nonnull-compare -Wno-format -Wno-stringop-truncation ' --with-extra-cxxflags='-std=gnu++11'
make CONF=debug
cd ..
</pre>
<p>
(If you did install <code>openjdk-13-jdk-headless</code> using apt, skip the <code>--with-boot-jdk</code> flag above.)
</p>
<p>
Note that the <code>make</code> steps will take a long time. Using <code>make -j
<#jobs></code> is not needed, because the <code>bash configure ...</code> step has
already picked up the number of cores availalbe on the machine so
that <code>make</code> alone will do a parallel build. More documentation on
this can be viewed at the office OpenJDK
<a href="https://github.com/openjdk/jdk/blob/master/doc/building.md">repo</a>.
</p>
<p>
After building, the baseline <code>java</code> and binary is available in
<code>~/paper99/openjdk/build/linux-x86_64-server-release/jdk/bin/</code>,
and the HCSGC build in
<code>~/paper99/hcsgc/build/linux-x86_64-server-release/jdk/bin/</code>
(debug builds in <code>linux-x86_64-server-fastdebug</code>). However, you do
not need to use these directly, as scripts for running the
individual benchmarks are provided. This is detailed in the next
section.
</p>
<p>
Finally, to extract data from logs and plot the data, some additional tools are needed:
</p>
<pre class="example">
wget -qO- https://get.haskellstack.org/ | sh
stack install ghc
stack install turtle
sudo apt-get install -y ruby octave octave-statistics zsh
</pre>
<blockquote>
<p>
Finally, <b>unless</b> you placed the <code>paper99</code> directory directly
under <code>~</code>, you must edit certain files to reflect the location of
this project. In <code>benchmark/Makefile</code>, change the first line to
indicate project root. In <code>benchmark/programs/hcsgc_engine.hs</code> and
<code>benchmark/programs/hcsgc_engine_single_core.hs</code>, change the paths
to the newly built OpenJDK and HCSGC binaries (lines 43, 44, 46,
and 47 in both files). <b>If you skip this step, the benchmark
scripts will not run!</b>
</p>
</blockquote>
</div>
</div>
</div>
<div id="outline-container-orgf61d9d1" class="outline-2">
<h2 id="orgf61d9d1"><span class="section-number-2">3</span> Step-by-Step</h2>
<div class="outline-text-2" id="text-3">
<p>
This section contains instructions for running the benchmarks from
the paper (again, excluding SPECjbb2015 for copyright reasons).
</p>
<p>
A text describing how to evaluate the results is found in the file
<code>result.html</code> in the <code>benchmarks</code> directory, and which is opened
automatically when the benchmarks finish (inside the VM) and
contains all the plots with accompanying text (mostly adapted from
the paper).
</p>
</div>
<div id="outline-container-org3ff15e6" class="outline-3">
<h3 id="org3ff15e6"><span class="section-number-3">3.1</span> Running the Simplified Benchmarks</h3>
<div class="outline-text-3" id="text-3-1">
<p>
Due to the large number of configurations, and the large number of
runs needed for statistical significance, running the
benchmarks is <b>very time consuming</b>, and is expected to take at
least 24 hours on a modern laptop.
</p>
<p>
To this end, we provide a simplified benchmark setting where we
only run the synthetic benchmark for a single run (for each
19 configurations) that is intended to prove that HCSGC is indeed
built correctly, and that its tuning knobs do work (because
different configurations do see different results).
</p>
<p>
Based on our experience, we estimate the following run-times for
the benchmarks:
</p>
<table border="2" cellspacing="0" cellpadding="6" rules="groups" frame="hsides">
<colgroup>
<col class="org-left" />
<col class="org-left" />
</colgroup>
<thead>
<tr>
<th scope="col" class="org-left">Benchmark</th>
<th scope="col" class="org-left">Run-time</th>
</tr>
</thead>
<tbody>
<tr>
<td class="org-left">synthetic</td>
<td class="org-left">5h</td>
</tr>
<tr>
<td class="org-left">synthetic phases</td>
<td class="org-left">15h</td>
</tr>
<tr>
<td class="org-left">synthetic cold</td>
<td class="org-left">7h</td>
</tr>
<tr>
<td class="org-left">cc-uk</td>
<td class="org-left">7h</td>
</tr>
<tr>
<td class="org-left">cc-enwiki</td>
<td class="org-left">7h</td>
</tr>
<tr>
<td class="org-left">mc-uk</td>
<td class="org-left">40h</td>
</tr>
<tr>
<td class="org-left">mc-enwiki</td>
<td class="org-left">32h</td>
</tr>
<tr>
<td class="org-left">tradebeans</td>
<td class="org-left">98h</td>
</tr>
<tr>
<td class="org-left">h2</td>
<td class="org-left">158h</td>
</tr>
</tbody>
</table>
</div>
<div id="outline-container-org9dc217d" class="outline-4">
<h4 id="org9dc217d"><span class="section-number-4">3.1.1</span> Kick-the-Tires Benchmark Fast (~15 minutes)</h4>
<div class="outline-text-4" id="text-3-1-1">
<table border="2" cellspacing="0" cellpadding="6" rules="groups" frame="hsides">
<colgroup>
<col class="org-left" />
<col class="org-left" />
<col class="org-left" />
<col class="org-right" />
</colgroup>
<thead>
<tr>
<th scope="col" class="org-left">Benchmark</th>
<th scope="col" class="org-left">Figure in Paper</th>
<th scope="col" class="org-left">Runs</th>
<th scope="col" class="org-right">Configurations</th>
</tr>
</thead>
<tbody>
<tr>
<td class="org-left">synthetic</td>
<td class="org-left">Fig. 4</td>
<td class="org-left">1 (see note)</td>
<td class="org-right">19</td>
</tr>
</tbody>
</table>
<p>
<b>Note:</b> To avoid changing the scripts, we copy the resulting logs
31 times to fake the 31 runs that we ran in the paper. Naturally
this setting cannot be used to obtain any results with any
statistical significance, and any jitter on your machine could
skew the results.
</p>
<p>
To run this benchmark, which should take 15 minutes (in <code>~/benchmarks</code> on VM or <code>~/paper99/benchmarks</code> in the manual install):
</p>
<pre class="example">
zsh
make test
</pre>
<p>
This compiles the synthetic benchmark, runs the parts of it
relevant for Figure 4, produces the corresponding plots and opens
a browser to show them together with accompanying text.
</p>
</div>
</div>
<div id="outline-container-org4a7f684" class="outline-4">
<h4 id="org4a7f684"><span class="section-number-4">3.1.2</span> Kick-the-Tires Benchmark Slow (optional – note: ~1 day)</h4>
<div class="outline-text-4" id="text-3-1-2">
<table border="2" cellspacing="0" cellpadding="6" rules="groups" frame="hsides">
<colgroup>
<col class="org-left" />
<col class="org-left" />
<col class="org-right" />
<col class="org-right" />
</colgroup>
<thead>
<tr>
<th scope="col" class="org-left">Benchmark</th>
<th scope="col" class="org-left">Figure in Paper</th>
<th scope="col" class="org-right">Runs</th>
<th scope="col" class="org-right">Configurations</th>
</tr>
</thead>
<tbody>
<tr>
<td class="org-left">synthetic</td>
<td class="org-left">Fig. 4</td>
<td class="org-right">31</td>
<td class="org-right">19</td>
</tr>
<tr>
<td class="org-left">synthetic phases</td>
<td class="org-left">Fig. 5</td>
<td class="org-right">31</td>
<td class="org-right">19</td>
</tr>
<tr>
<td class="org-left">synthetic cold</td>
<td class="org-left">Fig. 6</td>
<td class="org-right">31</td>
<td class="org-right">19</td>
</tr>
</tbody>
</table>
<p>
To run the synthetic benchmarks for Figure 4 with the full 31
runs from the paper (in <code>~/benchmarks</code> on VM or <code>~/paper99/benchmarks</code> in the manual install):
</p>
<pre class="example">
zsh
make test_full
</pre>
</div>
</div>
<div id="outline-container-orgbf55303" class="outline-4">
<h4 id="orgbf55303"><span class="section-number-4">3.1.3</span> Full Benchmark Slow (Weeks(!))</h4>
<div class="outline-text-4" id="text-3-1-3">
<table border="2" cellspacing="0" cellpadding="6" rules="groups" frame="hsides">
<colgroup>
<col class="org-left" />
<col class="org-left" />
<col class="org-right" />
<col class="org-right" />
</colgroup>
<thead>
<tr>
<th scope="col" class="org-left">Benchmark</th>
<th scope="col" class="org-left">Figure in Paper</th>
<th scope="col" class="org-right">Runs</th>
<th scope="col" class="org-right">Configurations</th>
</tr>
</thead>
<tbody>
<tr>
<td class="org-left">synthetic</td>
<td class="org-left">Fig. 4</td>
<td class="org-right">31</td>
<td class="org-right">19</td>
</tr>
<tr>
<td class="org-left">synthetic phases</td>
<td class="org-left">Fig. 5</td>
<td class="org-right">31</td>
<td class="org-right">19</td>
</tr>
<tr>
<td class="org-left">synthetic cold</td>
<td class="org-left">Fig. 6</td>
<td class="org-right">31</td>
<td class="org-right">19</td>
</tr>
</tbody>
<tbody>
<tr>
<td class="org-left">jgrapht suite</td>
<td class="org-left">Fig. 7-10</td>
<td class="org-right">31</td>
<td class="org-right">19</td>
</tr>
</tbody>
<tbody>
<tr>
<td class="org-left">dacapo h2/tradebeans</td>
<td class="org-left">Fig. 11-12</td>
<td class="org-right">5 (25 iters per run)</td>
<td class="org-right">19</td>
</tr>
</tbody>
</table>
<p>
Finally, to run the <b>full</b> benchmarks from the paper (in <code>~/benchmarks</code> on VM or <code>~/paper99/benchmarks</code> in the manual install):
</p>
<pre class="example">
zsh
make bench
</pre>
<p>
This compiles all benchmarks (DaCapo is provided as a jar file),
runs all the benchmarks, produces the plots and finally opens a
browser to show the results together with accompanying text.
</p>
<p>
The plots are are placed in
<code>data/images/evaluation/</code> under the <code>benchmark</code> directory. The file
<code>results.html</code> in the <code>benchmark</code> directory shows all the plots in a single HTML
file together with pointers to the paper and a discussion of their
interpretation. The directory
<code>submitted_evaluation</code> contains the
plots from the paper for comparison and all these figures are shown in <code>results.html</code>.
</p>
</div>
</div>
<div id="outline-container-org4da65c3" class="outline-4">
<h4 id="org4da65c3"><span class="section-number-4">3.1.4</span> Exporting the Results for Viewing Outside the VM</h4>
<div class="outline-text-4" id="text-3-1-4">
<p>
You can create a zip file, <code>results.zip</code>, with all the plots and
<code>results.html</code> like so:
</p>
<pre class="example">
make zip
</pre>
<p>
And use whatever means to export the zip file. The simplest way is
probably by using <code>scp</code> to copy <code>results.zip</code> to some other machine.
</p>
</div>
</div>
</div>
<div id="outline-container-orga89ebec" class="outline-3">
<h3 id="orga89ebec"><span class="section-number-3">3.2</span> How to Evaluate this Artefact</h3>
<div class="outline-text-3" id="text-3-2">
<p>
This is described in <a href="benchmarks/results.html">results.html</a> along with the figures produced
by running the benchmark scripts using the makefile as detailed
above.
</p>
</div>
</div>
<div id="outline-container-org7a6f71d" class="outline-3">
<h3 id="org7a6f71d"><span class="section-number-3">3.3</span> Viewing the Results</h3>
<div class="outline-text-3" id="text-3-3">
<p>
The file <a href="benchmarks/results.html">results.html</a> is opened by the main makefile after
benchmarking. Should you wish to open it again without rerunning
the benchmarks:
</p>
<pre class="example">
make view
</pre>
</div>
</div>
</div>
<div id="footnotes">
<h2 class="footnotes">Footnotes: </h2>
<div id="text-footnotes">
<div class="footdef"><sup><a id="fn.1" class="footnum" href="#fnr.1">1</a></sup> <div class="footpara"><p class="footpara">Except for SPECJbb2015 which cannot be
included due to copyright restrictions.</p></div></div>
</div>
</div></div>
<div id="postamble" class="status">
<p class="author">Author: Tobias Wrigstad</p>
<p class="date">Created: 2020-04-21 tis 16:15</p>
<p class="validation"><a href="http://validator.w3.org/check?uri=referer">Validate</a></p>
</div>