TTC 2018: Case 1 "Quality-based Software-Selection and Hardware-Mapping as Model Transformation Problem"
In order to get the case working, perform the following steps:
- Install GLPK to get the reference implementation working (otherwise the test, and therefore build, will fail)
- Go to https://www.gnu.org/software/glpk/ (Linux) or http://winglpk.sourceforge.net/ (Windows) and follow the instructions
- Tested with latest version
4.64
of GLPK
- Clone the repository:
git clone https://git-st.inf.tu-dresden.de/stgroup/ttc18.git && cd ttc18
- Verify, that the path to GLPK is correct in
gradle.properties
(and change it, if necessary) - Build it:
./gradlew build
(orgradlew.bat build
on Windows) - Run the benchmark:
./gradlew benchmarkFull
- As this might take long, running a set of scenarios is possible with
./gradlew benchmarkFull -Pscenario=0,small
(comma separated list of ids and/or names) - Alternatively, the timeout can be set to a smaller value in
jastadd-mquat-benchmark/src/main/resources/scenarios.json
- Please do not alter the definition of the scenarios, instead use the Custom Benchmark
- As this might take long, running a set of scenarios is possible with
All modules are prefixed with jastadd-mquat
, as this is an implementation of MQuAT (Multi-Quality AutoTuning) based on JastAdd. There are 5 modules:
base
: Contains the specifications for grammar and attributes, (de-)serializers and the model generatorbenchmark
: Benchmark infrastructure and settingssolver
: Interfaces for solvers, and a small testsuitesolver-ilp
: Reference implementation using ILPsolver-simple
: Naïve, brute-force solver written in Java
A new solution should be created using a new module (or multiple, if necessary). You can use the simple-solver module as an example. The following steps need to be completed:
- Create an implementation of
de.tudresden.inf.st.mquat.solving.BenchmarkableSolver
(which extends theSolver
interface). The main method here ispublic Solution solve(Root model) throws SolvingException
, which takes a model as input an returns a solution - Add an include of your project to
settings.gradle
- Optional step: Create a test case by extending the
HandwrittenTestSuite
- Add a compile dependency to your project in
build.gradle
of the projectjastadd-mquat-benchmark
- Update [
de.tudresden.inf.st.mquat.benchmark.SolverFactory.createAvailableSolversIfNeeded
] (https://git-st.inf.tu-dresden.de/stgroup/ttc18/blob/master/jastadd-mquat-benchmark/src/main/java/de/tudresden/inf/st/mquat/benchmark/SolverFactory.java#L22) to create a new instance of your solver - Add the name of your solver to the benchmark settings
- Use
jastadd-mquat-benchmark/src/main/resources/scenarios.json
for the Gradle taskbenchmarkFull
- Use
jastadd-mquat-benchmark/src/main/resources/local-benchmark-settings.json
for the Gralde taskbenchmarkCustom
(see Custom Benchmark for details)
- Use
- Run the benchmark, either
./gradlew benchmarkFull
or./gradlew benchmarkCustom
To test your solution, the Gradle task benchmarkCustom
can be used. This task generates a custom set of models and runs a benchmark for them.
All default parameters are specified in the file benchmark-settings.json
within the directory jastadd-mquat-benchmark/src/main/resources
.
To change them, create a new file in this directory named local-benchmark-settings.json
.
In this local version, all parameter values override the default settings, but are ignored when committing.
To test your solver with the name fancy-solver
along with the reference implementation using a model with 10
and 15
requests and a timeout of 50 seconds, the file local-benchmark-settings.json
would be as follows.
{
"solvers": [
"ilp-direct",
"fancy-solver"
],
"basic": {
"verbose": true,
"minRequests": 10,
"maxRequests": 15,
"stepRequests": 5,
"timeoutValue": 50,
"timeoutUnit": "SECONDS",
"total": 2
}
}
The value total
is used to constrain the total number of models to be generated. Set this to null
(the default) to generate all value for the defined parameter ranges.
Refer to de.tudresden.inf.st.mquat.generator.ScenarioDescription
for a description of the possible parameters.
- Please use the gradle wrapper script, as different version of Gradle might not work with the setup
- The wrapper script uses version
3.3
- The wrapper script uses version
- If anything is not working as expected, feel free to contact on of the authors of the TTC case