Scala version of Flash-Fill for Excel 2013 by Gulwani et Al. See http://rise4fun.com/QuickCode/dates
It is the tool used in the paper StriSynth: Synthesis for Live Programming
- Key features
- Usage
- StringSolver command line Quickstart
- Automated Bash commands
- API
- Test, export and import
Build using sbt 0.13 and scala 2.10.3.
- Supports counters in files.
- "..." continues an expression if there is a loop.
- Semi-automated file renaming commands, file processing, filter and partition
- Command-line to create programs by example and export them to Powershell Scripts.
Add the following to your build.sbt file:
libraryDependencies += "ch.epfl.lara" %% "stringsolver" % "1.2"
- Install SBT
git clone https://github.com/MikaelMayer/StringSolver.gitcd StringSolversbt publish-local
You can use the StringSolver command line to
- create transformations functions from examples
- create reduction functions from examples
- create partition functions from example
- Create filter functions from examples
- Compose these functions with any other scala function
- Export transformations to powershell scripts.
If you use Ammnonite, just enter the following one-liner to get started:
import $ivy.`ch.epfl.lara::stringsolver:1.2`, ch.epfl.lara.synthesis.stringsolver._, ImperativeProgram._ , import CurrentInstance._, import ProgramTypes._ ; HELP; NEW
If you use standard consoles, download StringSolver, enter its folder in command line and write:
sbt console
Note that before the scala console pops out, the command line displays the main commands that you can use. Let's create a script which rename all javascript files to typescript files. Enter the lines one by one to understand the interaction:
val rename = ("myfile.js" ==> "mv myfile.js myfile.ts")
val filter = {YES ==> ("index.js", "aux.js"; NO ==> "index.html" }
val filter = YES ==> "reporting.js"
val t = filter andThen rename
t in PowerShell to "script.ps1"
Now move the newly created script to a folder where you have some files like main.js, a.js, x.doc, and run it in PowerShell:
PS> .\script.ps1
mv main.js main.ts
mv a.js a.ts
Go back to the StringSolver console, and try out the following (it might take a little longer)
val combine = ("report1.pdf", "report2.pdf") ==> "convert report1.pdf report2.pdf... report.pdf"
val filter = { YES ==> ("report.pdf", "index.pdf"); NO ==> "report.js"}
filter andThen combine in PowerShell to "script.ps1"
Executing the script in a folder containing among others a1.pdf, a2.pdf and a3.pdf would give:
PS> .\script.ps1
convert a1.pdf a2.pdf a3.pdf a.pdf
StringSolver includes a nice automatic renaming tool and an automated command generalizer.
Installation:
- Build the project using
sbt one-jaror download it from sonatype (see URL above) - Use the following alias to rename file using the tool (e.g. in your
.bashrcfile:)
export STRINGSOLVERPATH = [/path/to/StringSolver/target/scala/]
alias mv='java -jar "$STRINGSOLVERPATH/stringsolver_2.10-1.1-one-jar.jar" move'
alias auto='java -jar "$STRINGSOLVERPATH/stringsolver_2.10-1.1-one-jar.jar" auto'
alias partition='java -jar "$STRINGSOLVERPATH/stringsolver_2.10-1.1-one-jar.jar" partition'
alias filter='java -jar "$STRINGSOLVERPATH/stringsolver_2.10-1.1-one-jar.jar" filter'
This windows shell extension recursively monitors folders for renaming and suggests renamings.
-
Install python https://www.python.org/downloads/
-
Make sure that python can be invoked from the command line.
-
Edit the first line of
Monitor.ps1to indicate which root folder to monitor (it could be c:) -
Run PowerShell and navigate to the repository
-
Inside Powershell, run
PowerShell -Stato start single threaded mode. -
Start:
.\Monitor.ps1
-
Now navigate to the
StringSolverrepository -
Type
sbt -
Type
run server
Now the service is launched.
Now, within the explorer, rename two files or two folders in the same folder. A balloon should appear with the suggestion. Click it and the renaming is automatically done.
To stop the service, either close the command line utility, or on another shell at the StringSolver repository, write:
- echo "stop" | Send-TcpRequest localhost 12345;
The semi-automatic rename tool overrides the actual one:
mv file1 file2
However, when a mapping is detected, the algorithm displays it and you can then use mv to trigger it for all other files.
mv
This is equivalent to perform the global transformation in a single line using -a or --auto modifier, when you trust enough the system:
mv -a file1 file2
There are other commands after a renaming is done:
mv -eor--explainprovides a high-level english explanation of the transformationmv -tor--testprovides a visualization of the transformation if applied.
Adding options -e or -t along with filenames helps the system to refine the request without modifying any files.
-p or --propertiesprovides owner name and last modification date.
If you experience trouble with the mv command, you can always use the -c or --clear option to clear the history stored in a temporary file.
To produce an equivalent bash script which would produce the result, add the option -b or --bash
The standard way to run bash commands is the following:
auto filename "my unix command depending on filename"
For example, to convert a like Algebra2Week5.txt to Week5/Algebra2.pdf and remove the original file, you can do the following:
auto Algebra2Week5.txt "convert Algebra2Week5.txt Week5/Algebra2.pdf;rm Algebra2Week5.txt"
To perform this transformation for all files, just type again
auto
The last two commands could be combined in a single command if you trust the system enough by using the -a or --auto command.
auto -a Algebra2Week5.txt "convert Algebra2Week5.txt Week5/Algebra2.pdf;rm Algebra2Week5.txt"
To produce an equivalent bash script which would produce the result, add the option -b or --bash (to come soon)
The previous command can also be abbreviated by letting the program infer what is the file the command depends on. So writing this would be equivalent to the previous command:
auto -a "convert Algebra2Week5.txt Week5/Algebra2.pdf;rm Algebra2Week5.txt"
To directs the system to use the file content line by line instead of just the filename, use the flag -l or --lines.
If you do not trust the system, you can run auto -e before your command, or after any command having used auto to check what the global transformation would be. In the other hand, auto -t visualizes the transformation.
For example, running the following command will not trigger it, but will display a list of command that would be executed if the user runs auto
auto -t "convert Algebra2Week5.txt Week5/Algebra2.pdf;rm Algebra2Week5.txt"
If you experience trouble with the auto command, you can always run auto --clear to clear the history stored in a temporary file.
If you have a set of files, by providing at least two files of each partition for at least two partitions, you can split the files into as many folders as there are partitions. For example:
partition --test myphoto1.jpg images otherpicture2.jpg images mytextfile.txt text otherdoc.txt text
partition
It will move all files ending with .jpg to images, all files ending with .txt to text, and if there are files ending .pdf, they will be moved in a folder named .pdf, etc.
Provided partition names can be unrelated constants (red, blue, etc.), numbers (1, 2, 3...), strings transformation from the common substring of each partition (set-TXT,set-JPG,....) or any combination of these three.
StringSolver provides a way to change the content of an entire file based on examples. Given an even number of inputs, map will start to compute the transformation from inputs to outputs.
If it is given one input, it will apply the current transformation to the lines of the file.
map ' case France => "0033"' ' case France => "+33"'
map CountryMapping.scala
The --filter flag can be used to apply the transformation on a subset of lines. See below.
filter is slightly different from the previous commands. It is similar to partition, in the sense that it can move files to different folder. The difference is that it separates files given a property, and is lazy to move files, so it can be used for the other commands.
It has the --test option by default and can effectively move files only when using the filter alone or if the modifier --auto is set.
For example:
filter myphoto1.jpg images otherpicture2.jpg images mytextfile.txt . otherdoc.pdf .
filter
The first line considers that the images is the tag for accepted files. It will find out that accepted files end with .jpg.
The second line asks to move accepted files to a folder images and keep the others in the current directory .
mv, auto and partition also accept the --filter modifier. If set, it will perform the last filter --test and apply their transformation only on filtered files and not the others.
To use the filter command with the content of a file and the map command, you have to provide the --lines flag to filter and then the --filter flag to map. For example:
filter --lines "element {01}" ok "element {02}" ok "Title" notok
map "element {01}" "element {1,el}" "content {10}" "content {10,co}"
map --filter mycontent.json
import ch.epfl.lara.synthesis.stringsolver._
object Test {
val c = StringSolver()
c.add("file499.pdf -> 01file.pdf")
c.add("report_761.pdf -> 02report.pdf")
c.add("credits##.pdf -> 03credits.pdf")
/* Prints:
* a 2-digit counter incrementing starting at 1
* + the first input until the end of the first lowercase word
* + the first input starting at the first '.'
*/
println(Printer(c.solve().get))
/* Prints:
* 04input.pdf
**/
println(c.solve("input%^$.pdf"))
}
object Test2 {
val c = StringSolver()
c.add(List("Alg1.txt"), "convert Alg1.txt Alg001.pdf")
/* Prints:
* the constant string 'convert ' + the first input
+ the constant string ' ' + the first input until the end of the first word
+ a 3-digit number from the first number in previous output
+ the constant string '.pdf'
*/
println(Printer(c.solve().get))
/* Prints:
* "convert Math2.txt Math002.pdf"
*/
println(c.solve("Math2.txt"))
val c2 = StringSolver()
c2.add(List("Alg001.pdf", "Alg002.pdf", "Alg003.pdf"), "convert Alg001.pdf Alg002.pdf... AlgBook.pdf")
/* Prints:
* the constant string 'convert ' + concatenates all inputs separated by ' '
* + the constant string ' ' + the first input until the end of the first word
* + the constant string 'Book' + the first input starting at the last non-number
*/
println(Printer(c2.solve().get))
/* Prints:
* convert Math1.pdf Math2.pdf Math3.pdf Math4.pdf MathBook.pdf
*/
println(c2.solve("Math1.pdf | Math2.pdf | Math3.pdf | Math4.pdf"))
// prints "
}The other ways to add input/output examples in StringSolver are the following, given that c is a StringSolver instance.
// Exactly one input and one output
c.add("input1 -> output1")
// c.add returns a set of solution programs. You can .takeBest on it
// Three inputs and two outputs
c.add("input1 | input2 | input3 | output1 | output2", 3)
// Three inputs and two outputs
c.add(List("input1", "input2", "input3"), List("output1", "output2"))
// Three inputs and one output
c.add(List("input1", "input2", "input3"), "output1)
// Three inputs and two outputs two times
c.add("""input1 | input2 | input3 | output1 | output2
input4 | input5 | input6 | output3 | output4""", 2)To solve and print an existing StringSolver instance, do the following:
val c = StringSolver()
c.add(List("a","b"),List("ab","ba"))
// If one output was provided (equivalent to c.solve(0))
c.solve() match { // The first input + the second input
case Some(prog) => println(Printer(prog))
case None =>
}
// Retrieve the second output program (0-index based)
c.solve(1) match { // The second input + the first input
case Some(prog) => println(Printer(prog))
case None =>
}
// Returns "cd | dc"
c.solve("c | d")
// Returns List("cd", "dc")
c.solve(List("c", "d"))To solve a problem from scratch, you can also do the following:
val c = StringSolver()
c.add(List("a","b"),List("ab","ba"))
// Return: "a | b | ab | ba
// c | d | cd | dc"
c.solve("""
a | b | ab | ba
c | d""", 2)Most of the StringSolver usage is fully automated, and does not require to change the following options. For some cases, they can be useful to trigger on/off.
val c = StringSolver()
/**
* Use numbering from previous input option
*/
c.setUseNumbers(b: Boolean) = {ff.numbering = b; this}
/**
* Loop level. 0 will not look for loops
*/
c.setLoopLevel(i: Int) = {ff.DEFAULT_REC_LOOP_LEVEL = i; this}
/**
* Timeout in seconds to add a new input/output example.
* This is approximate. Default is 15s
*/
c.setTimeout(seconds: Int) = {ff.TIMEOUT_SECONDS = seconds; this}
/**
* If looking for loops, what could be the maximum separator length
*/
c.setMaxSeparatorLength(length: Int) = {ff.MAX_SEPARATOR_LENGTH = length; this}
/**
* If only interesting positions (aka word, special chars and digit separators)
* are considered when looking for loops
*/
c.setOnlyInterestingPositions(b: Boolean) = {ff.onlyInterestingPositions = b; this}
/**
* Outputs programs steps. Useful for debugging an other.
*/
c.setVerbose(b: Boolean) = {ff.verbose = b; this}
c.isVerbose = ff.verbose
/**
* Allows to iterate over inputs.
*/
c.setIterateInput(b: Boolean) = ff.iterateInput = b
/**
* Allows to use the example index for positions
*/
c.setUseIndexForPosition(b: Boolean) = ff.useIndexForPosition = b
/**
* Retrieves statistics
*/
c.getStatistics(): String = ff.statistics()
/**
* Advanced stats.
*/
c.setAdvancedStats(b: Boolean) = ff.advanced_stats = b
/**
* Extra time to merge as a proportion of timeout
*/
c.setExtraTimeToMerge(f: Float) = extra_time_to_merge = fTo test StringSolver, run `sbt test' but this might take a while.
To create a usable jar file containing everything, including scala, run:
sbt one-jar
The jar file will be in the target/ repository
When you want to include StringSolver in your own scala or java project, follow these steps:
- Add the following lines separated by blank lines in your
build.sbtat the root of your project
name := "MyProject"
version := "1.0"
organization := "com.example"
scalaVersion := "2.10.3"
mainClass in (Compile, run) := Some("com.example.Custom")
resolvers += "Sonatype.org" at "https://oss.sonatype.org/service/local/repositories/releases/content"
libraryDependencies += "ch.epfl.lara" %% "stringsolver" % "1.1"- import the various declarations in your source file.
If for example your project contains a file named src/main/scala/com/example/Custom.scala:
package com.example
import ch.epfl.lara.synthesis.stringsolver._
object Custom {
val c = StringSolver()
def main(a: Array[String]): Unit = {
c.add(a(0).split("\\|").toList, a(1))
c.solve() match {
case Some(a) => println(Printer(a))
case None => println("No program found")
}
}
}- Then you can run in command line
sbt run "tEsT|inPuT1" TESTinput001
and it should output:
the first input uppercase + the lowercase second input until the end of the first word
+ a 3-digit number from the second input starting at the first number
Not sure how it works, but a colleague of mine used the following in the maven build file:
<dependency>
<groupId>ch.epfl.lara</groupId>
<artifactId>stringsolver_2.10</artifactId>
<version>1.1</version>
</dependency>
The remaining steps are similar to the previous paragraph.