diff --git a/dev/HerbConstraints/index.html b/dev/HerbConstraints/index.html index 3c9ccf3..553175e 100644 --- a/dev/HerbConstraints/index.html +++ b/dev/HerbConstraints/index.html @@ -1,5 +1,5 @@ -HerbConstraints.jl · Herb.jl
Edit on GitHub

HerbConstraints.jl Documentation

HerbConstraints.AbstractMatchNode — Type
abstract type AbstractMatchNode

Tree structure to which rulenode trees can be matched. Consists of MatchNodes, which can match a specific RuleNode, and MatchVars, which is a variable that can be filled in with any RuleNode.

HerbConstraints.ComesAfter — Type
ComesAfter <: PropagatorConstraint

A ComesAfter constraint is a PropagatorConstraint containing the following:

  • rule::Int: A reference to a rule in the grammar
  • predecessors: A list of rules in the grammar

This Constraint enforces that the rule can only be applied if every rule in predecessors is used in the path from the root of the tree to the current hole in the order that they are given. Even though the rules must be in order, there might be other rules inbetween.

For example, consider the tree 1(a, 2(b, 3(c, d)))):

  • ComesAfter(4, [2, 3]) would enforce that rule 4 can only be used if 2 and 3 are used in the path from the root in that order. Therefore, only hole c and d can be filled with 4.
  • ComesAfter(4, [1, 3]) also allows c and d to be filled, since 1 and 3 are still used in the correct order. It does not matter that 2 is also used in the path to the root.
  • ComesAfter(4, [3, 2]) does not allow any hole to be filled with 4, since either the predecessors are either not in the path or in the wrong order for each of the holes.
HerbConstraints.Forbidden — Type
Forbidden <: PropagatorConstraint

This [PropagatorConstraint] forbids any subtree that matches the pattern given by tree to be generated. A pattern is a tree of AbstractMatchNodes. Such a node can either be a MatchNode, which contains a rule index corresponding to the rule index in the Grammar and the appropriate number of children, similar to RuleNodes. It can also contain a MatchVar, which contains a single identifier symbol. A MatchVar can match any subtree, but if there are multiple instances of the same variable in the pattern, the matched subtrees must be identical. Any rule in the domain that makes the match attempt successful is removed.

For example, consider the tree 1(a, 2(b, 3(c, 4)))):

  • Forbidden(MatchNode(3, [MatchNode(5), MatchNode(4)])) forbids c to be filled with 5.
  • Forbidden(MatchNode(3, [MatchVar(:v), MatchNode(4)])) forbids c to be filled, since a [MatchVar] can match any rule, thus making the match attempt successful for the entire domain of c. Therefore, this tree invalid.
  • Forbidden(MatchNode(3, [MatchVar(:v), MatchVar(:v)])) forbids c to be filled with 4, since that would make both assignments to v equal, which causes a successful match.
Warning

The Forbidden constraint makes use of LocalConstraints to make sure that constraints are also enforced in the future when the context of a Hole changes. Therefore, Forbidden can only be used in implementations that keep track of the LocalConstraints and propagate them at the right moments.

HerbConstraints.ForbiddenPath — Type
ForbiddenPath <: PropagatorConstraint

A [PropagatorConstraint] that forbids a certain derivation sequence. sequence defines the forbidden sequence. Each rule that would complete the sequence when expanding a Hole in an AbstractRuleNode tree is removed from the domain. The derivation sequence is the path from the root to the hole.

For example, consider the tree 1(a, 2(b, 3(c, d)))):

  • ForbiddenPath([1, 2, 4]) enforces that rule 4 cannot be applied at b, since it completes the sequence. However, it can be applied at a, c and d.
  • ForbiddenPath([3, 1]) enforces that rule 1 cannot be applied at c or d.
HerbConstraints.GrammarContext — Type
mutable struct GrammarContext

Structure used to track the context. Contains: - the expression being modified - the path to the hole that is being expanded, represented as a sequence of child indices. e.g., [2, 1] would point to the first child of the second child of the root. - a vector with local constraints that should be propagated upon expansion.

HerbConstraints.LocalConstraint — Type
abstract type LocalConstraint <: Constraint

Abstract type representing all local constraints. Local constraints correspond to a specific (partial) AbstractRuleNode tree. Each local constraint contains a path to a specific location in the tree. Each local constraint has an implementation of a propagate-function that takes

The propagate-function returns a tuple containing

Warning

By default, LocalConstraints are only propagated once. Constraints that have to be propagated more frequently should return themselves in the list of new local constraints.

HerbConstraints.LocalForbidden — Type
LocalForbidden

Forbids the a subtree that matches the MatchNode tree to be generated at the location provided by the path. Use a Forbidden constraint for enforcing this throughout the entire search space.

HerbConstraints.LocalOrdered — Type

Enforces an order over two or more subtrees that fill the variables specified in order when the pattern is applied at the location given by path. Use an Ordered constraint for enforcing this throughout the entire search space.

HerbConstraints.MatchFail — Type
@enum MatchFail hardfail softfail

This enum is used for distinguishing between two types of failures when trying to match a RuleNode either with another RuleNode or with an AbstractMatchNode

  • Hardfail means that there is no match, and there is no way to fill in the holes to get a match.
  • Softfail means that there is no match, but there might be a way to fill the holes that results in a match.
HerbConstraints.MatchNode — Type
struct MatchNode <: AbstractMatchNode

Match a specific rulenode, where the grammar rule index is rule_ind and children matches the children of the RuleNode. Example usage:

MatchNode(3, [MatchNode(1), MatchNode(2)])

This matches RuleNode(3, [RuleNode(1), RuleNode(2)])

HerbConstraints.MatchVar — Type
struct MatchVar <: AbstractMatchNode

Matches anything and assigns it to a variable. The ForbiddenTree constraint will not match if identical variable symbols match to different trees. Example usage:

MatchNode(3, [MatchVar(:x), MatchVar(:x)])

This matches RuleNode(3, [RuleNode(1), RuleNode(1)]), RuleNode(3, [RuleNode(2), RuleNode(2)]), etc.

HerbConstraints.Ordered — Type
Ordered <: PropagatorConstraint

A PropagatorConstraint that enforces a specific order in MatchVar assignments in the pattern defined by tree. A pattern is a tree of AbstractMatchNodes. Such a node can either be a MatchNode, which contains a rule index corresponding to the rule index in the Grammar and the appropriate number of children, similar to RuleNodes. It can also contain a MatchVar, which contains a single identifier symbol. A MatchVar can match any subtree, but if there are multiple instances of the same variable in the pattern, the matched subtrees must be identical.

The order defines an order between the variable assignments. For example, if the order is [x, y], the constraint will require the assignment to x to be less than or equal to the assignment to y. The order is recursively defined by RuleNode indices. For more information, see Base.isless(rn₁::AbstractRuleNode, rn₂::AbstractRuleNode).

For example, consider the tree 1(a, 2(b, 3(c, 4)))):

  • Ordered(MatchNode(3, [MatchVar(:v), MatchVar(:w)]), [:v, :w]) removes every rule with an index of 5 or greater from the domain of c, since that would make the index of the assignment to v greater than the index of the assignment to w, violating the order.
  • Ordered(MatchNode(3, [MatchVar(:v), MatchVar(:w)]), [:w, :v]) removes every rule with an index of 4 or less from the domain of c, since that would make the index of the assignment to v less than the index of the assignment to w, violating the order.
Warning

The Ordered constraint makes use of LocalConstraints to make sure that constraints are also enforced in the future when the context of a Hole changes. Therefore, Ordered can only be used in implementations that keep track of the LocalConstraints and propagate them at the right moments.

HerbConstraints.PropagatorConstraint — Type
PropagatorConstraint <: Constraint

Abstract type representing all propagator constraints. Each propagator constraint has an implementation of a propagate-function that takes

The propagate-function returns a tuple containing

HerbConstraints.RequireOnLeft — Type

Rules have to be used in the specified order. That is, rule at index K can only be used if rules at indices [1...K-1] are used in the left subtree of the current expression

HerbConstraints.@csgrammar_annotated — Macro

@csgrammar_annotated Define an annotated grammar and return it as a ContextSensitiveGrammar. Allows for adding optional annotations per rule. As well as that, allows for adding optional labels per rule, which can be referenced in annotations. Syntax is backwards-compatible with @csgrammar. Examples:

g₁ = @csgrammar_annotated begin
+HerbConstraints.jl · Herb.jl
Edit on GitHub
HerbConstraints.jl Documentation
HerbConstraints.AbstractMatchNode — Type
abstract type AbstractMatchNode
Tree structure to which rulenode trees can be matched. Consists of MatchNodes, which can match a specific RuleNode, and MatchVars, which is a variable that can be filled in with any RuleNode.
source
HerbConstraints.ComesAfter — Type
ComesAfter <: PropagatorConstraint
A ComesAfter constraint is a PropagatorConstraint containing the following:
  • rule::Int: A reference to a rule in the grammar
  • predecessors: A list of rules in the grammar
This Constraint enforces that the rule can only be applied if every rule in  predecessors is used in the path from the root of the tree  to the current hole in the order  that they are given. Even though the rules must be in order, there might be other rules inbetween.
For example, consider the tree 1(a, 2(b, 3(c, d)))):
  • ComesAfter(4, [2, 3]) would enforce that rule 4 can only be used if 2 and 3  are used in the path from the root in that order. Therefore, only hole c and d can be filled with 4.
  • ComesAfter(4, [1, 3]) also allows c and d to be filled, since 1 and 3 are still used in the  correct order. It does not matter that 2 is also used in the path to the root.
  • ComesAfter(4, [3, 2]) does not allow any hole to be filled with 4, since either the predecessors are  either not in the path or in the wrong order for each of the holes. 
source
HerbConstraints.Forbidden — Type
Forbidden <: PropagatorConstraint
This [PropagatorConstraint] forbids any subtree that matches the pattern given by tree to be generated. A pattern is a tree of AbstractMatchNodes.  Such a node can either be a MatchNode, which contains a rule index corresponding to the  rule index in the Grammar and the appropriate number of children, similar to RuleNodes. It can also contain a MatchVar, which contains a single identifier symbol. A MatchVar can match any subtree, but if there are multiple instances of the same variable in the pattern, the matched subtrees must be identical. Any rule in the domain that makes the match attempt successful is removed.
For example, consider the tree 1(a, 2(b, 3(c, 4)))):
  • Forbidden(MatchNode(3, [MatchNode(5), MatchNode(4)])) forbids c to be filled with 5.
  • Forbidden(MatchNode(3, [MatchVar(:v), MatchNode(4)])) forbids c to be filled, since a [MatchVar] can  match any rule, thus making the match attempt successful for the entire domain of c.  Therefore, this tree invalid.
  • Forbidden(MatchNode(3, [MatchVar(:v), MatchVar(:v)])) forbids c to be filled with 4, since that would  make both assignments to v equal, which causes a successful match.
Warning
The Forbidden constraint makes use of LocalConstraints to make sure that constraints  are also enforced in the future when the context of a Hole changes.  Therefore, Forbidden can only be used in implementations that keep track of the  LocalConstraints and propagate them at the right moments.
source
HerbConstraints.ForbiddenPath — Type
ForbiddenPath <: PropagatorConstraint
A [PropagatorConstraint] that forbids a certain derivation sequence. sequence defines the forbidden sequence.  Each rule that would complete the sequence when expanding a Hole in an  AbstractRuleNode tree is removed from the domain. The derivation sequence is the path from the root to the hole.
For example, consider the tree 1(a, 2(b, 3(c, d)))):
  • ForbiddenPath([1, 2, 4]) enforces that rule 4 cannot be applied at b,  since it completes the sequence. However, it can be applied at a, c and d.
  • ForbiddenPath([3, 1]) enforces that rule 1 cannot be applied at c or d.
source
HerbConstraints.GrammarContext — Type
mutable struct GrammarContext
Structure used to track the context. Contains: 	- the expression being modified  	- the path to the hole that is being expanded, represented as a sequence of child indices. 	  e.g., [2, 1] would point to the first child of the second child of the root. 	- a vector with local constraints that should be propagated upon expansion.
source
HerbConstraints.LocalConstraint — Type
abstract type LocalConstraint <: Constraint
Abstract type representing all local constraints. Local constraints correspond to a specific (partial) AbstractRuleNode tree. Each local constraint contains a path to a specific location in the tree.   Each local constraint has an implementation of a propagate-function that takes
The propagate-function returns a tuple containing
Warning
By default, LocalConstraints are only propagated once. Constraints that have to be propagated more frequently should return  themselves in the list of new local constraints.
source
HerbConstraints.LocalForbidden — Type
LocalForbidden
Forbids the a subtree that matches the MatchNode tree to be generated at the location  provided by the path.  Use a Forbidden constraint for enforcing this throughout the entire search space.
source
HerbConstraints.LocalOrdered — Type
Enforces an order over two or more subtrees that fill the variables  specified in order when the pattern is applied at the location given by path. Use an Ordered constraint for enforcing this throughout the entire search space.
source
HerbConstraints.MatchFail — Type
@enum MatchFail hardfail softfail
This enum is used for distinguishing between two types of failures when trying to  match a RuleNode either with another RuleNode or with an AbstractMatchNode
  • Hardfail means that there is no match, and there is no way to fill in the holes to get a match.
  • Softfail means that there is no match, but there might be a way to fill the holes that results in a match.
source
HerbConstraints.MatchNode — Type
struct MatchNode <: AbstractMatchNode
Match a specific rulenode, where the grammar rule index is rule_ind  and children matches the children of the RuleNode. Example usage:
MatchNode(3, [MatchNode(1), MatchNode(2)])
This matches RuleNode(3, [RuleNode(1), RuleNode(2)])
source
HerbConstraints.MatchVar — Type
struct MatchVar <: AbstractMatchNode
Matches anything and assigns it to a variable.  The ForbiddenTree constraint will not match if identical variable symbols match to different trees. Example usage:
MatchNode(3, [MatchVar(:x), MatchVar(:x)])
This matches RuleNode(3, [RuleNode(1), RuleNode(1)]), RuleNode(3, [RuleNode(2), RuleNode(2)]), etc.
source
HerbConstraints.Ordered — Type
Ordered <: PropagatorConstraint
A PropagatorConstraint that enforces a specific order in MatchVar  assignments in the pattern defined by tree. A pattern is a tree of AbstractMatchNodes.  Such a node can either be a MatchNode, which contains a rule index corresponding to the  rule index in the Grammar and the appropriate number of children, similar to RuleNodes. It can also contain a MatchVar, which contains a single identifier symbol. A MatchVar can match any subtree, but if there are multiple instances of the same variable in the pattern, the matched subtrees must be identical.
The order defines an order between the variable assignments.  For example, if the order is [x, y], the constraint will require  the assignment to x to be less than or equal to the assignment to y. The order is recursively defined by RuleNode indices.  For more information, see Base.isless(rn₁::AbstractRuleNode, rn₂::AbstractRuleNode).
For example, consider the tree 1(a, 2(b, 3(c, 4)))):
  • Ordered(MatchNode(3, [MatchVar(:v), MatchVar(:w)]), [:v, :w]) removes every rule  with an index of 5 or greater from the domain of c, since that would make the index of the  assignment to v greater than the index of the assignment to w, violating the order.
  • Ordered(MatchNode(3, [MatchVar(:v), MatchVar(:w)]), [:w, :v]) removes every rule  with an index of 4 or less from the domain of c, since that would make the index of the  assignment to v less than the index of the assignment to w, violating the order.
Warning
The Ordered constraint makes use of LocalConstraints to make sure that constraints  are also enforced in the future when the context of a Hole changes.  Therefore, Ordered can only be used in implementations that keep track of the  LocalConstraints and propagate them at the right moments.
source
HerbConstraints.PropagatorConstraint — Type
PropagatorConstraint <: Constraint
Abstract type representing all propagator constraints. Each propagator constraint has an implementation of a propagate-function that takes
The propagate-function returns a tuple containing
source
HerbConstraints.RequireOnLeft — Type
Rules have to be used in the specified order. That is, rule at index K can only be used if rules at indices [1...K-1] are used in the left subtree of the current expression
source
HerbConstraints.@csgrammar_annotated — Macro
@csgrammar_annotated Define an annotated grammar and return it as a ContextSensitiveGrammar. Allows for adding optional annotations per rule. As well as that, allows for adding optional labels per rule, which can be referenced in annotations.  Syntax is backwards-compatible with @csgrammar. Examples:
g₁ = @csgrammar_annotated begin
     Element = 1
     Element = x
     Element = Element + Element := commutative
@@ -18,4 +18,4 @@
                                                        forbidden_path([:addition, :one]) || forbidden_path([:one, :variable])
                                                     )
     multiplication:: Element = Element * Element := (commutative, transitive)
-end
Base.show — Method
Base.show(io::IO, node::MatchNode; separator=",", last_child::Bool=true)
Prints a found MatchNode given an and the respective children to IO. 
Base.show — Method
Base.show(io::IO, node::MatchVar; separator=",", last_child::Bool=true)
Prints a matching variable assignment described by MatchVar to IO.
HerbConstraints._matchnode2expr — Function
_matchnode2expr(expr::Expr, pattern::MatchVar, grammar::Grammar, j=0)
Internal function for matchnode2expr, recursively iterating over a matched variable and converting it to an expression. This is primarily useful for pretty-printing a pattern. Returns the corresponding expression and the current child index.
HerbConstraints._matchnode2expr — Function
_matchnode2expr(typ::Symbol, pattern::MatchNode, grammar::Grammar, j=0)
Internal function for matchnode2expr, returning the matched translated symbol. This is primarily useful for pretty-printing a pattern. Returns the corresponding expression, i.e. the variable name and the current child index.
HerbConstraints._matchnode2expr — Function
_matchnode2expr(typ::Symbol, pattern::MatchVar, grammar::Grammar, j=0)
Internal function for matchnode2expr. This is primarily useful for pretty-printing a pattern. Returns the corresponding expression, i.e. the variable name and the current child index.
HerbConstraints._matchnode2expr — Function
_matchnode2expr(expr::Expr, pattern::MatchNode, grammar::Grammar, j=0)
Internal function for matchnode2expr, recursively iterating over a matched pattern and converting it to an expression. This is primarily useful for pretty-printing a pattern. Returns the corresponding expression and the current child index.
HerbConstraints._pattern_match — Method
_pattern_match(rn::RuleNode, mn::MatchNode, vars::Dict{Symbol, AbstractRuleNode})::Union{Nothing, MatchFail}
Tries to match RuleNode rn with MatchNode mn. Modifies the variable assignment dictionary vars. Returns nothing if the match is successful. If the match is unsuccessful, it returns whether it is a softfail or hardfail (see MatchFail docstring)
HerbConstraints._pattern_match — Method
_pattern_match(rn::RuleNode, mv::MatchVar, vars::Dict{Symbol, AbstractRuleNode})::Union{Nothing, MatchFail}
Matching RuleNode rn with MatchVar mv. If the variable is already assigned, the rulenode is matched with the specific variable value. Returns nothing if the match is succesful.  If the match is unsuccessful, it returns whether it is a softfail or hardfail (see MatchFail docstring)
HerbConstraints._pattern_match_with_hole — Method
_pattern_match_with_hole(::Hole, mn::MatchNode, hole_location::Vector{Int}, ::Dict{Symbol, AbstractRuleNode})::Union{Int, Symbol, MatchFail, Tuple{Symbol, Vector{Int}}}
Matches the Hole with the given MatchNode. 
TODO check this behaviour?
HerbConstraints._pattern_match_with_hole — Method
_pattern_match_with_hole(rn::RuleNode, mn::MatchNode, hole_location::Vector{Int}, vars::Dict{Symbol, AbstractRuleNode})::Union{Int, Symbol, MatchFail, Tuple{Symbol, Vector{Int}}}
Tries to match RuleNode rn with MatchNode mn and fill in the domain of the hole at hole_location.  Returns if match is successful either:
  • The id for the node which fills the hole
  • A symbol for the variable that fills the hole
  • A tuple containing:
    • The variable that matched (the subtree containing) the hole
    • The location of the hole in this subtree
If the match is unsuccessful, it returns:
  • hardfail if there are no holes that can be filled in such a way that the match will become succesful
  • softfail if the match could become successful if the holes are filled in a certain way
HerbConstraints._pattern_match_with_hole — Method
_pattern_match_with_hole(rn::RuleNode, mv::MatchVar, hole_location::Vector{Int}, vars::Dict{Symbol, AbstractRuleNode})::Union{Int, Symbol, MatchFail, Tuple{Symbol, Vector{Int}}}
Tries to match RuleNode rn with MatchVar mv and fill in the domain of the hole at hole_location. If the variable name is already assigned in vars, the rulenode is matched with the hole. Otherwise the variable and the hole location are returned.
HerbConstraints._rulenode_match_with_hole — Method
_rulenode_match_with_hole(rn₁::RuleNode, rn₂::RuleNode, hole_location::Vector{Int})::Union{Int, MatchFail}
Matches two rulenodes.  Returns how to fill in the hole in rn₁ to make it match rn₂ if:
  • rn₁ has a single hole at the provided location
  • rn₂ doesn't have any holes
  • rn₁ matches rn₂ apart from the single hole location.
If the match fails, it returns whether it is a softfail or a hardfail (see MatchFail docstring)
HerbConstraints.addparent! — Method
addparent!(context::GrammarContext, parent::Int)
Adds a parent to the context. The parent is defined by the grammar rule id.
HerbConstraints.annotation2constraint — Method
Converts an annotation to a constraint. commutative: creates an Ordered constraint transitive: creates an (incorrect) Forbidden constraint forbidden_path(path::Vector{Union{Symbol, Int}}): creates a ForbiddenPath constraint with the original rule included ... || ...: creates a OneOf constraint (also works with ... || ... || ... et cetera, though not very performant)
HerbConstraints.copy_and_insert — Method
copy_and_insert(old_context::GrammarContext, parent::Int)
Copies the given context and insert the parent in the node location.
HerbConstraints.make_smaller_or_equal — Method
Filters the domain of the hole at hole_location in rn₁ to make rn₁ be ordered before rn₂.   Returns the filtered domain, and a boolean indicating if this constraint can be deleted.
HerbConstraints.matchnode2expr — Method
matchnode2expr(pattern::MatchNode, grammar::Grammar)
Converts a MatchNode tree into a Julia expression.  This is primarily useful for pretty-printing a pattern. Returns the corresponding expression.
HerbConstraints.matchnode2expr — Method
matchnode2expr(pattern::MatchVar, grammar::Grammar)
Converts a MatchVar into an expression by returning the variable directly. This is primarily useful for pretty-printing a pattern.
HerbConstraints.propagate — Method
propagate(c::Forbidden, g::Grammar, context::GrammarContext, domain::Vector{Int})::Tuple{Vector{Int}, Vector{LocalConstraint}}
Propagates the Forbidden constraint. It removes the rules from the domain that would complete the forbidden tree.
Warning
The Forbidden constraint makes use of LocalConstraints to make sure that constraints  are also enforced in the future when the context of a Hole changes.  Therefore, Forbidden can only be used in implementations that keep track of the  LocalConstraints and propagate them at the right moments.
HerbConstraints.propagate — Method
Propagates the LocalOneOf constraint. It enforces that at least one of its given constraints hold.
HerbConstraints.propagate — Method
Propagates the LocalForbidden constraint. It removes rules from the domain that would make the RuleNode at the given path match the pattern defined by the MatchNode.
HerbConstraints.propagate — Method
Propagates the LocalOrdered constraint. It removes rules from the domain that would violate the order of variables as defined in the  constraint.
HerbConstraints.propagate — Method
Propagates the OneOf constraint. It enforces that at least one of its given constraints hold.
HerbConstraints.propagate — Method
propagate(c::Ordered, g::Grammar, context::GrammarContext, domain::Vector{Int})::Tuple{Vector{Int}, Vector{LocalConstraint}}
Propagates the Ordered constraint. Any rule that violates the order as defined by the contraint is removed from the domain.
Warning
The Ordered constraint makes use of LocalConstraints to make sure that constraints  are also enforced in the future when the context of a Hole changes.  Therefore, Ordered can only be used in implementations that keep track of the  LocalConstraints and propagate them at the right moments.
HerbConstraints.propagate — Method
Propagates the RequireOnLeft constraint. It removes every element from the domain that does not have a necessary  predecessor in the left subtree.
Index

+end
source
Base.show — Method
Base.show(io::IO, node::MatchNode; separator=",", last_child::Bool=true)

Prints a found MatchNode given an and the respective children to IO.

source
Base.show — Method
Base.show(io::IO, node::MatchVar; separator=",", last_child::Bool=true)

Prints a matching variable assignment described by MatchVar to IO.

source
HerbConstraints._matchnode2expr — Function
_matchnode2expr(expr::Expr, pattern::MatchNode, grammar::Grammar, j=0)

Internal function for matchnode2expr, recursively iterating over a matched pattern and converting it to an expression. This is primarily useful for pretty-printing a pattern. Returns the corresponding expression and the current child index.

source
HerbConstraints._matchnode2expr — Function
_matchnode2expr(typ::Symbol, pattern::MatchNode, grammar::Grammar, j=0)

Internal function for matchnode2expr, returning the matched translated symbol. This is primarily useful for pretty-printing a pattern. Returns the corresponding expression, i.e. the variable name and the current child index.

source
HerbConstraints._matchnode2expr — Function
_matchnode2expr(expr::Expr, pattern::MatchVar, grammar::Grammar, j=0)

Internal function for matchnode2expr, recursively iterating over a matched variable and converting it to an expression. This is primarily useful for pretty-printing a pattern. Returns the corresponding expression and the current child index.

source
HerbConstraints._matchnode2expr — Function
_matchnode2expr(typ::Symbol, pattern::MatchVar, grammar::Grammar, j=0)

Internal function for matchnode2expr. This is primarily useful for pretty-printing a pattern. Returns the corresponding expression, i.e. the variable name and the current child index.

source
HerbConstraints._pattern_match — Method
_pattern_match(rn::RuleNode, mn::MatchNode, vars::Dict{Symbol, AbstractRuleNode})::Union{Nothing, MatchFail}

Tries to match RuleNode rn with MatchNode mn. Modifies the variable assignment dictionary vars. Returns nothing if the match is successful. If the match is unsuccessful, it returns whether it is a softfail or hardfail (see MatchFail docstring)

source
HerbConstraints._pattern_match — Method
_pattern_match(rn::RuleNode, mv::MatchVar, vars::Dict{Symbol, AbstractRuleNode})::Union{Nothing, MatchFail}

Matching RuleNode rn with MatchVar mv. If the variable is already assigned, the rulenode is matched with the specific variable value. Returns nothing if the match is succesful. If the match is unsuccessful, it returns whether it is a softfail or hardfail (see MatchFail docstring)

source
HerbConstraints._pattern_match_with_hole — Method
_pattern_match_with_hole(rn::RuleNode, mn::MatchNode, hole_location::Vector{Int}, vars::Dict{Symbol, AbstractRuleNode})::Union{Int, Symbol, MatchFail, Tuple{Symbol, Vector{Int}}}

Tries to match RuleNode rn with MatchNode mn and fill in the domain of the hole at hole_location. Returns if match is successful either:

  • The id for the node which fills the hole
  • A symbol for the variable that fills the hole
  • A tuple containing:
    • The variable that matched (the subtree containing) the hole
    • The location of the hole in this subtree

If the match is unsuccessful, it returns:

  • hardfail if there are no holes that can be filled in such a way that the match will become succesful
  • softfail if the match could become successful if the holes are filled in a certain way
source
HerbConstraints._pattern_match_with_hole — Method
_pattern_match_with_hole(rn::RuleNode, mv::MatchVar, hole_location::Vector{Int}, vars::Dict{Symbol, AbstractRuleNode})::Union{Int, Symbol, MatchFail, Tuple{Symbol, Vector{Int}}}

Tries to match RuleNode rn with MatchVar mv and fill in the domain of the hole at hole_location. If the variable name is already assigned in vars, the rulenode is matched with the hole. Otherwise the variable and the hole location are returned.

source
HerbConstraints._rulenode_match_with_hole — Method
_rulenode_match_with_hole(rn₁::RuleNode, rn₂::RuleNode, hole_location::Vector{Int})::Union{Int, MatchFail}

Matches two rulenodes. Returns how to fill in the hole in rn₁ to make it match rn₂ if:

  • rn₁ has a single hole at the provided location
  • rn₂ doesn't have any holes
  • rn₁ matches rn₂ apart from the single hole location.

If the match fails, it returns whether it is a softfail or a hardfail (see MatchFail docstring)

source
HerbConstraints.addparent! — Method
addparent!(context::GrammarContext, parent::Int)

Adds a parent to the context. The parent is defined by the grammar rule id.

source
HerbConstraints.annotation2constraint — Method

Converts an annotation to a constraint. commutative: creates an Ordered constraint transitive: creates an (incorrect) Forbidden constraint forbidden_path(path::Vector{Union{Symbol, Int}}): creates a ForbiddenPath constraint with the original rule included ... || ...: creates a OneOf constraint (also works with ... || ... || ... et cetera, though not very performant)

source
HerbConstraints.make_smaller_or_equal — Method

Filters the domain of the hole at hole_location in rn₁ to make rn₁ be ordered before rn₂. Returns the filtered domain, and a boolean indicating if this constraint can be deleted.

source
HerbConstraints.matchnode2expr — Method
matchnode2expr(pattern::MatchNode, grammar::Grammar)

Converts a MatchNode tree into a Julia expression. This is primarily useful for pretty-printing a pattern. Returns the corresponding expression.

source
HerbConstraints.matchnode2expr — Method
matchnode2expr(pattern::MatchVar, grammar::Grammar)

Converts a MatchVar into an expression by returning the variable directly. This is primarily useful for pretty-printing a pattern.

source
HerbConstraints.propagate — Method
propagate(c::Forbidden, g::Grammar, context::GrammarContext, domain::Vector{Int})::Tuple{Vector{Int}, Vector{LocalConstraint}}

Propagates the Forbidden constraint. It removes the rules from the domain that would complete the forbidden tree.

Warning

The Forbidden constraint makes use of LocalConstraints to make sure that constraints are also enforced in the future when the context of a Hole changes. Therefore, Forbidden can only be used in implementations that keep track of the LocalConstraints and propagate them at the right moments.

source
HerbConstraints.propagate — Method

Propagates the LocalForbidden constraint. It removes rules from the domain that would make the RuleNode at the given path match the pattern defined by the MatchNode.

source
HerbConstraints.propagate — Method

Propagates the LocalOrdered constraint. It removes rules from the domain that would violate the order of variables as defined in the constraint.

source
HerbConstraints.propagate — Method
propagate(c::Ordered, g::Grammar, context::GrammarContext, domain::Vector{Int})::Tuple{Vector{Int}, Vector{LocalConstraint}}

Propagates the Ordered constraint. Any rule that violates the order as defined by the contraint is removed from the domain.

Warning

The Ordered constraint makes use of LocalConstraints to make sure that constraints are also enforced in the future when the context of a Hole changes. Therefore, Ordered can only be used in implementations that keep track of the LocalConstraints and propagate them at the right moments.

source
HerbConstraints.propagate — Method

Propagates the RequireOnLeft constraint. It removes every element from the domain that does not have a necessary predecessor in the left subtree.

source

Index

diff --git a/dev/HerbCore/index.html b/dev/HerbCore/index.html index 2d8071d..ac22df8 100644 --- a/dev/HerbCore/index.html +++ b/dev/HerbCore/index.html @@ -1,2 +1,2 @@ -HerbCore.jl Documentation · Herb.jl
Edit on GitHub

HerbCore.jl Documentation

HerbCore.AbstractRuleNode — Type
AbstractRuleNode

Abstract type for representing expression trees. Expression trees consist of RuleNodes and Holes.

  • A RuleNode represents a certain production rule in the Grammar.
  • A Hole is a placeholder where certain rules in the grammar still can be applied.
HerbCore.Grammar — Type
Grammar

Abstract type representing all grammars. It is assumed that all grammar structs have at least the following attributes:

  • rules::Vector{Any}: A list of RHS of rules (subexpressions).
  • types::Vector{Symbol}: A list of LHS of rules (types, all symbols).
  • isterminal::BitVector: A bitvector where bit i represents whether rule i is terminal.
  • iseval::BitVector: A bitvector where bit i represents whether rule i is an eval rule.
  • bytype::Dict{Symbol,Vector{Int}}: A dictionary that maps a type to all rules of said type.
  • domains::Dict{Symbol, BitVector}: A dictionary that maps a type to a domain bitvector. The domain bitvector has bit i set to true iff the ith rule is of this type.
  • childtypes::Vector{Vector{Symbol}}: A list of types of the children for each rule.

If a rule is terminal, the corresponding list is empty.

  • log_probabilities::Union{Vector{Real}, Nothing}: A list of probabilities for each rule.

If the grammar is non-probabilistic, the list can be nothing.

For concrete types, see ContextFreeGrammar and ContextSensitiveGrammar.

HerbCore.Hole — Type
Hole <: AbstractRuleNode

A Hole is a placeholder where certain rules from the grammar can still be applied. The domain of a Hole defines which rules can be applied. The domain is a bitvector, where the ith bit is set to true if the ith rule in the grammar can be applied.

HerbCore.HoleReference — Type
HoleReference

Contains a hole and the path to the hole from the root of the tree.

HerbCore.RuleNode — Type
RuleNode <: AbstractRuleNode

A RuleNode represents a node in an expression tree. Each node corresponds to a certain rule in the Grammar. A RuleNode consists of:

  • ind: The index of the rule in the Grammar which this node is representing.
  • _val: Field for storing immediately evaluated values
  • children: The children of this node in the expression tree
Compat

Evaluate immediately functionality is not yet supported by most of Herb.jl.

HerbCore.RuleNode — Method
RuleNode(ind::Int, _val::Any)

Create a RuleNode for the Grammar rule with index ind, _val as immediately evaluated value and no children

Warning

Only use this constructor if you are absolutely certain that a rule is terminal and cannot have children. Use [RuleNode(ind::Int, grammar::Grammar)] for rules that might have children. In general, Holes should be used as a placeholder when the children of a node are not yet known.

Compat

Evaluate immediately functionality is not yet supported by most of Herb.jl.

HerbCore.RuleNode — Method
RuleNode(ind::Int, children::Vector{AbstractRuleNode})

Create a RuleNode for the Grammar rule with index ind and children as subtrees.

Base.isless — Method
Base.isless(rn₁::AbstractRuleNode, rn₂::AbstractRuleNode)::Bool

Compares two RuleNodes. Returns true if the left RuleNode is less than the right RuleNode. Order is determined from the index of the RuleNodes. If both RuleNodes have the same index, a depth-first search is performed in both RuleNodes until nodes with a different index are found.

Base.length — Method
Base.length(root::RuleNode)

Return the number of nodes in the tree rooted at root. Holes don't count.

Base.length — Method
Base.length(root::RuleNode)

Return the number of nodes in the tree rooted at root. Holes don't count.

HerbCore.get_rulesequence — Method
get_rulesequence(node::RuleNode, path::Vector{Int})

Extract the derivation sequence from a path (sequence of child indices) and an AbstractRuleNode. If the path is deeper than the deepest node, it returns what it has.

HerbCore.node_depth — Method
node_depth(root::AbstractRuleNode, node::AbstractRuleNode)::Int

Return the depth of node for an AbstractRuleNode tree rooted at root. Depth is 1 when root == node.

Warning

node must be a subtree of root in order for this function to work.

HerbCore.rulesonleft — Method
rulesonleft(expr::RuleNode, path::Vector{Int})::Set{Int}

Finds all rules that are used in the left subtree defined by the path.

HerbCore.swap_node — Method
swap_node(expr::AbstractRuleNode, new_expr::AbstractRuleNode, path::Vector{Int})

Replace a node in expr, specified by path, with new_expr. Path is a sequence of child indices, starting from the root node.

HerbCore.swap_node — Method
swap_node(expr::RuleNode, node::RuleNode, child_index::Int, new_expr::RuleNode)

Replace child i of a node, a part of larger expr, with new_expr.

Index

+HerbCore.jl Documentation · Herb.jl
Edit on GitHub

HerbCore.jl Documentation

HerbCore.Grammar — Type
Grammar

Abstract type representing all grammars. It is assumed that all grammar structs have at least the following attributes:

  • rules::Vector{Any}: A list of RHS of rules (subexpressions).
  • types::Vector{Symbol}: A list of LHS of rules (types, all symbols).
  • isterminal::BitVector: A bitvector where bit i represents whether rule i is terminal.
  • iseval::BitVector: A bitvector where bit i represents whether rule i is an eval rule.
  • bytype::Dict{Symbol,Vector{Int}}: A dictionary that maps a type to all rules of said type.
  • domains::Dict{Symbol, BitVector}: A dictionary that maps a type to a domain bitvector. The domain bitvector has bit i set to true iff the ith rule is of this type.
  • childtypes::Vector{Vector{Symbol}}: A list of types of the children for each rule.

If a rule is terminal, the corresponding list is empty.

  • log_probabilities::Union{Vector{Real}, Nothing}: A list of probabilities for each rule.

If the grammar is non-probabilistic, the list can be nothing.

For concrete types, see ContextFreeGrammar and ContextSensitiveGrammar.

source
HerbCore.Hole — Type
Hole <: AbstractRuleNode

A Hole is a placeholder where certain rules from the grammar can still be applied. The domain of a Hole defines which rules can be applied. The domain is a bitvector, where the ith bit is set to true if the ith rule in the grammar can be applied.

source
HerbCore.RuleNode — Type
RuleNode <: AbstractRuleNode

A RuleNode represents a node in an expression tree. Each node corresponds to a certain rule in the Grammar. A RuleNode consists of:

  • ind: The index of the rule in the Grammar which this node is representing.
  • _val: Field for storing immediately evaluated values
  • children: The children of this node in the expression tree
Compat

Evaluate immediately functionality is not yet supported by most of Herb.jl.

source
HerbCore.RuleNode — Method
RuleNode(ind::Int, _val::Any)

Create a RuleNode for the Grammar rule with index ind, _val as immediately evaluated value and no children

Warning

Only use this constructor if you are absolutely certain that a rule is terminal and cannot have children. Use [RuleNode(ind::Int, grammar::Grammar)] for rules that might have children. In general, Holes should be used as a placeholder when the children of a node are not yet known.

Compat

Evaluate immediately functionality is not yet supported by most of Herb.jl.

source
Base.isless — Method
Base.isless(rn₁::AbstractRuleNode, rn₂::AbstractRuleNode)::Bool

Compares two RuleNodes. Returns true if the left RuleNode is less than the right RuleNode. Order is determined from the index of the RuleNodes. If both RuleNodes have the same index, a depth-first search is performed in both RuleNodes until nodes with a different index are found.

source
Base.length — Method
Base.length(root::RuleNode)

Return the number of nodes in the tree rooted at root. Holes don't count.

source
Base.length — Method
Base.length(root::RuleNode)

Return the number of nodes in the tree rooted at root. Holes don't count.

source
HerbCore.get_rulesequence — Method
get_rulesequence(node::RuleNode, path::Vector{Int})

Extract the derivation sequence from a path (sequence of child indices) and an AbstractRuleNode. If the path is deeper than the deepest node, it returns what it has.

source
HerbCore.node_depth — Method
node_depth(root::AbstractRuleNode, node::AbstractRuleNode)::Int

Return the depth of node for an AbstractRuleNode tree rooted at root. Depth is 1 when root == node.

Warning

node must be a subtree of root in order for this function to work.

source
HerbCore.rulesonleft — Method
rulesonleft(expr::RuleNode, path::Vector{Int})::Set{Int}

Finds all rules that are used in the left subtree defined by the path.

source
HerbCore.swap_node — Method
swap_node(expr::AbstractRuleNode, new_expr::AbstractRuleNode, path::Vector{Int})

Replace a node in expr, specified by path, with new_expr. Path is a sequence of child indices, starting from the root node.

source
HerbCore.swap_node — Method
swap_node(expr::RuleNode, node::RuleNode, child_index::Int, new_expr::RuleNode)

Replace child i of a node, a part of larger expr, with new_expr.

source

Index

diff --git a/dev/HerbData/index.html b/dev/HerbData/index.html index c052441..7717c22 100644 --- a/dev/HerbData/index.html +++ b/dev/HerbData/index.html @@ -1,2 +1,2 @@ -HerbCore.jl · Herb.jl
Edit on GitHub

HerbData.jl Documentation

HerbData.IOExample — Type
struct IOExample <: Example

An input-output example. input is a Dict of {Symbol,Any} where the symbol represents a variable in a program. output can be anything.

HerbData.IOPExample — Type
struct IOPExample <: Example

An input-output example with an associated program. ex is an IOExample. program is a program of arbitrary form. Please note that this is a pure container, and thus does not guarantee any checks on the validity of the program.

HerbData.read_IOPexamples — Method
read_IOPexamples(filepath::AbstractString)::Vector{Tuple{IOPExample}

Reads serialized IO + program examples from disk after type checking.

HerbData.read_IOexamples — Method
read_IOexamples(filepath::AbstractString)::Vector{IOExample}

Reads serialized IO examples from disk after type checking.

HerbData.readdata — Method
readdata(directory::AbstractString, lineparser::Function)::Vector{Problem}

Reads all files in the given directory and parses them line by line into an ExampleProblem using the given lineparser.

TODO: Turn this into an iterator that doesn't load all data into memory at initialization.

HerbData.readfile — Method
readfile(filepath::AbstractString, lineparser::Function)::Problem

Reads a file and parses every non-empty line using the line parser.

HerbData.write_IOPexamples — Method
write_IOPexamples(filepath::AbstractString, examples::Vector{Tuple{IOExample, Any}})

Writes IO examples and the corresponding programs to disk by serializing them into a file using HDF5 checking for and appending the .xiop.

HerbData.write_IOexamples — Method
write_IOexamples(filepath::AbstractString, examples::Vector{IOExample})

Writes IO examples to disk by serializing them into a file using HDF5 checking for and appending the .xio file ending.

Index

+HerbCore.jl · Herb.jl
Edit on GitHub

HerbData.jl Documentation

HerbData.IOExample — Type
struct IOExample <: Example

An input-output example. input is a Dict of {Symbol,Any} where the symbol represents a variable in a program. output can be anything.

source
HerbData.IOPExample — Type
struct IOPExample <: Example

An input-output example with an associated program. ex is an IOExample. program is a program of arbitrary form. Please note that this is a pure container, and thus does not guarantee any checks on the validity of the program.

source
HerbData.read_IOPexamples — Method
read_IOPexamples(filepath::AbstractString)::Vector{Tuple{IOPExample}

Reads serialized IO + program examples from disk after type checking.

source
HerbData.read_IOexamples — Method
read_IOexamples(filepath::AbstractString)::Vector{IOExample}

Reads serialized IO examples from disk after type checking.

source
HerbData.readdata — Method
readdata(directory::AbstractString, lineparser::Function)::Vector{Problem}

Reads all files in the given directory and parses them line by line into an ExampleProblem using the given lineparser.

TODO: Turn this into an iterator that doesn't load all data into memory at initialization.

source
HerbData.readfile — Method
readfile(filepath::AbstractString, lineparser::Function)::Problem

Reads a file and parses every non-empty line using the line parser.

source
HerbData.write_IOPexamples — Method
write_IOPexamples(filepath::AbstractString, examples::Vector{Tuple{IOExample, Any}})

Writes IO examples and the corresponding programs to disk by serializing them into a file using HDF5 checking for and appending the .xiop.

source
HerbData.write_IOexamples — Method
write_IOexamples(filepath::AbstractString, examples::Vector{IOExample})

Writes IO examples to disk by serializing them into a file using HDF5 checking for and appending the .xio file ending.

source

Index

diff --git a/dev/HerbGrammar/index.html b/dev/HerbGrammar/index.html index aabb013..688f873 100644 --- a/dev/HerbGrammar/index.html +++ b/dev/HerbGrammar/index.html @@ -1,42 +1,42 @@ -HerbGrammar.jl · Herb.jl
Edit on GitHub

HerbGrammar.jl Documentation

HerbGrammar.ContextFreeGrammar — Type
ContextFreeGrammar <: Grammar

Represents a context-free grammar and its production rules. Consists of:

  • rules::Vector{Any}: A list of RHS of rules (subexpressions).
  • types::Vector{Symbol}: A list of LHS of rules (types, all symbols).
  • isterminal::BitVector: A bitvector where bit i represents whether rule i is terminal.
  • iseval::BitVector: A bitvector where bit i represents whether rule i is an eval rule.
  • bytype::Dict{Symbol,Vector{Int}}: A dictionary that maps a type to all rules of said type.
  • domains::Dict{Symbol, BitVector}: A dictionary that maps a type to a domain bitvector. The domain bitvector has bit i set to true iff the ith rule is of this type.
  • childtypes::Vector{Vector{Symbol}}: A list of types of the children for each rule. If a rule is terminal, the corresponding list is empty.
  • log_probabilities::Union{Vector{Real}, Nothing}: A list of probabilities for each rule. If the grammar is non-probabilistic, the list can be nothing.

Use the @cfgrammar macro to create a ContextFreeGrammar object. Use the @pcfgrammar macro to create a ContextFreeGrammar object with probabilities. For context-sensitive grammars, see ContextSensitiveGrammar.

HerbGrammar.ContextSensitiveGrammar — Type
ContextSensitiveGrammar <: Grammar

Represents a context-sensitive grammar. Extends Grammar with constraints.

Consists of:

  • rules::Vector{Any}: A list of RHS of rules (subexpressions).
  • types::Vector{Symbol}: A list of LHS of rules (types, all symbols).
  • isterminal::BitVector: A bitvector where bit i represents whether rule i is terminal.
  • iseval::BitVector: A bitvector where bit i represents whether rule i is an eval rule.
  • bytype::Dict{Symbol,Vector{Int}}: A dictionary that maps a type to all rules of said type.
  • domains::Dict{Symbol, BitVector}: A dictionary that maps a type to a domain bitvector. The domain bitvector has bit i set to true iff the ith rule is of this type.
  • childtypes::Vector{Vector{Symbol}}: A list of types of the children for each rule. If a rule is terminal, the corresponding list is empty.
  • log_probabilities::Union{Vector{Real}, Nothing}: A list of probabilities for each rule. If the grammar is non-probabilistic, the list can be nothing.
  • constraints::Vector{Constraint}: A list of constraints that programs in this grammar have to abide.

Use the @csgrammar macro to create a ContextSensitiveGrammar object. Use the @pcsgrammar macro to create a ContextSensitiveGrammar object with probabilities. For context-free grammars, see ContextFreeGrammar.

HerbGrammar.NodeLoc — Type

NodeLoc A helper struct that points to a node in the tree via its parent such that the child can be easily swapped out. If i is 0 the node pointed to is the root node and parent is the node itself.

HerbGrammar.@cfgrammar — Macro
@cfgrammar

A macro for defining a ContextFreeGrammar.

Example usage:

grammar = @cfgrammar begin
+HerbGrammar.jl · Herb.jl
Edit on GitHub
HerbGrammar.jl Documentation
HerbGrammar.ContextFreeGrammar — Type
ContextFreeGrammar <: Grammar
Represents a context-free grammar and its production rules. Consists of:
  • rules::Vector{Any}: A list of RHS of rules (subexpressions).
  • types::Vector{Symbol}: A list of LHS of rules (types, all symbols).
  • isterminal::BitVector: A bitvector where bit i represents whether rule i is terminal.
  • iseval::BitVector: A bitvector where bit i represents whether rule i is an eval rule.
  • bytype::Dict{Symbol,Vector{Int}}: A dictionary that maps a type to all rules of said type.
  • domains::Dict{Symbol, BitVector}: A dictionary that maps a type to a domain bitvector.  The domain bitvector has bit i set to true iff the ith rule is of this type.
  • childtypes::Vector{Vector{Symbol}}: A list of types of the children for each rule.  If a rule is terminal, the corresponding list is empty.
  • log_probabilities::Union{Vector{Real}, Nothing}: A list of probabilities for each rule.  If the grammar is non-probabilistic, the list can be nothing.
Use the @cfgrammar macro to create a ContextFreeGrammar object. Use the @pcfgrammar macro to create a ContextFreeGrammar object with probabilities. For context-sensitive grammars, see ContextSensitiveGrammar.
source
HerbGrammar.ContextSensitiveGrammar — Type
ContextSensitiveGrammar <: Grammar
Represents a context-sensitive grammar. Extends Grammar with constraints.
Consists of:
  • rules::Vector{Any}: A list of RHS of rules (subexpressions).
  • types::Vector{Symbol}: A list of LHS of rules (types, all symbols).
  • isterminal::BitVector: A bitvector where bit i represents whether rule i is terminal.
  • iseval::BitVector: A bitvector where bit i represents whether rule i is an eval rule.
  • bytype::Dict{Symbol,Vector{Int}}: A dictionary that maps a type to all rules of said type.
  • domains::Dict{Symbol, BitVector}: A dictionary that maps a type to a domain bitvector.  The domain bitvector has bit i set to true iff the ith rule is of this type.
  • childtypes::Vector{Vector{Symbol}}: A list of types of the children for each rule.  If a rule is terminal, the corresponding list is empty.
  • log_probabilities::Union{Vector{Real}, Nothing}: A list of probabilities for each rule.  If the grammar is non-probabilistic, the list can be nothing.
  • constraints::Vector{Constraint}: A list of constraints that programs in this grammar have to abide.
Use the @csgrammar macro to create a ContextSensitiveGrammar object. Use the @pcsgrammar macro to create a ContextSensitiveGrammar object with probabilities. For context-free grammars, see ContextFreeGrammar.
source
HerbGrammar.NodeLoc — Type
NodeLoc A helper struct that points to a node in the tree via its parent such that the child can be easily swapped out. If i is 0 the node pointed to is the root node and parent is the node itself.
source
HerbGrammar.@cfgrammar — Macro
@cfgrammar
A macro for defining a ContextFreeGrammar. 
Example usage:
grammar = @cfgrammar begin
 	R = x
 	R = 1 | 2
 	R = R + R
-end
Syntax:
  • Literals: Symbols that are already defined in Julia are considered literals, such as 1, 2, or π. For example: R = 1.
  • Variables: A variable is a symbol that is not a nonterminal symbol and not already defined in Julia. For example: R = x.
  • Functions: Functions and infix operators that are defined in Julia or the Main module can be used  with the default evaluator. For example: R = R + R, R = f(a, b).
  • Combinations: Multiple rules can be defined on a single line in the grammar definition using the | symbol. For example: R = 1 | 2 | 3.
  • Iterators: Another way to define multiple rules is by providing a Julia iterator after a | symbol. For example: R = |(1:9).
Related:
HerbGrammar.@csgrammar — Macro
@csgrammar
A macro for defining a ContextSensitiveGrammar.  Constraints can be added afterwards using the addconstraint! function.
Example usage:
grammar = @csgrammar begin
+end
Syntax:
  • Literals: Symbols that are already defined in Julia are considered literals, such as 1, 2, or π. For example: R = 1.
  • Variables: A variable is a symbol that is not a nonterminal symbol and not already defined in Julia. For example: R = x.
  • Functions: Functions and infix operators that are defined in Julia or the Main module can be used  with the default evaluator. For example: R = R + R, R = f(a, b).
  • Combinations: Multiple rules can be defined on a single line in the grammar definition using the | symbol. For example: R = 1 | 2 | 3.
  • Iterators: Another way to define multiple rules is by providing a Julia iterator after a | symbol. For example: R = |(1:9).
Related:
source
HerbGrammar.@csgrammar — Macro
@csgrammar
A macro for defining a ContextSensitiveGrammar.  Constraints can be added afterwards using the addconstraint! function.
Example usage:
grammar = @csgrammar begin
 	R = x
 	R = 1 | 2
 	R = R + R
-end
Syntax:
  • Literals: Symbols that are already defined in Julia are considered literals, such as 1, 2, or π. For example: R = 1.
  • Variables: A variable is a symbol that is not a nonterminal symbol and not already defined in Julia. For example: R = x.
  • Functions: Functions and infix operators that are defined in Julia or the Main module can be used  with the default evaluator. For example: R = R + R, R = f(a, b).
  • Combinations: Multiple rules can be defined on a single line in the grammar definition using the | symbol. For example: R = 1 | 2 | 3.
  • Iterators: Another way to define multiple rules is by providing a Julia iterator after a | symbol. For example: R = |(1:9).
Related:
HerbGrammar.@pcfgrammar — Macro
@pcfgrammar
A macro for defining a probabilistic ContextFreeGrammar. 
Example usage:
grammar = @pcfgrammar begin
+end
Syntax:
  • Literals: Symbols that are already defined in Julia are considered literals, such as 1, 2, or π. For example: R = 1.
  • Variables: A variable is a symbol that is not a nonterminal symbol and not already defined in Julia. For example: R = x.
  • Functions: Functions and infix operators that are defined in Julia or the Main module can be used  with the default evaluator. For example: R = R + R, R = f(a, b).
  • Combinations: Multiple rules can be defined on a single line in the grammar definition using the | symbol. For example: R = 1 | 2 | 3.
  • Iterators: Another way to define multiple rules is by providing a Julia iterator after a | symbol. For example: R = |(1:9).
Related:
source
HerbGrammar.@pcfgrammar — Macro
@pcfgrammar
A macro for defining a probabilistic ContextFreeGrammar. 
Example usage:
grammar = @pcfgrammar begin
 	0.5 : R = x
 	0.3 : R = 1 | 2
 	0.2 : R = R + R
-end
Syntax:
The syntax of rules is identical to the syntax used by @cfgrammar:
  • Literals: Symbols that are already defined in Julia are considered literals, such as 1, 2, or π. For example: R = 1.
  • Variables: A variable is a symbol that is not a nonterminal symbol and not already defined in Julia. For example: R = x.
  • Functions: Functions and infix operators that are defined in Julia or the Main module can be used  with the default evaluator. For example: R = R + R, R = f(a, b).
  • Combinations: Multiple rules can be defined on a single line in the grammar definition using the | symbol. For example: R = 1 | 2 | 3.
  • Iterators: Another way to define multiple rules is by providing a Julia iterator after a | symbol. For example: R = |(1:9).
Every rule is also prefixed with a probability. Rules and probabilities are separated using the : symbol. If multiple rules are defined on a single line, the probability is equally divided between the rules. The sum of probabilities for all rules of a certain non-terminal symbol should be equal to 1.  The probabilities are automatically scaled if this isn't the case.
Related:
HerbGrammar.@pcsgrammar — Macro
@pcsgrammar
A macro for defining a probabilistic ContextSensitiveGrammar. 
Example usage:
grammar = @pcsgrammar begin
+end
Syntax:
The syntax of rules is identical to the syntax used by @cfgrammar:
  • Literals: Symbols that are already defined in Julia are considered literals, such as 1, 2, or π. For example: R = 1.
  • Variables: A variable is a symbol that is not a nonterminal symbol and not already defined in Julia. For example: R = x.
  • Functions: Functions and infix operators that are defined in Julia or the Main module can be used  with the default evaluator. For example: R = R + R, R = f(a, b).
  • Combinations: Multiple rules can be defined on a single line in the grammar definition using the | symbol. For example: R = 1 | 2 | 3.
  • Iterators: Another way to define multiple rules is by providing a Julia iterator after a | symbol. For example: R = |(1:9).
Every rule is also prefixed with a probability. Rules and probabilities are separated using the : symbol. If multiple rules are defined on a single line, the probability is equally divided between the rules. The sum of probabilities for all rules of a certain non-terminal symbol should be equal to 1.  The probabilities are automatically scaled if this isn't the case.
Related:
source
HerbGrammar.@pcsgrammar — Macro
@pcsgrammar
A macro for defining a probabilistic ContextSensitiveGrammar. 
Example usage:
grammar = @pcsgrammar begin
 	0.5 : R = x
 	0.3 : R = 1 | 2
 	0.2 : R = R + R
-end
Syntax:
The syntax of rules is identical to the syntax used by @csgrammar:
  • Literals: Symbols that are already defined in Julia are considered literals, such as 1, 2, or π. For example: R = 1.
  • Variables: A variable is a symbol that is not a nonterminal symbol and not already defined in Julia. For example: R = x.
  • Functions: Functions and infix operators that are defined in Julia or the Main module can be used  with the default evaluator. For example: R = R + R, R = f(a, b).
  • Combinations: Multiple rules can be defined on a single line in the grammar definition using the | symbol. For example: R = 1 | 2 | 3.
  • Iterators: Another way to define multiple rules is by providing a Julia iterator after a | symbol. For example: R = |(1:9).
Every rule is also prefixed with a probability. Rules and probabilities are separated using the : symbol. If multiple rules are defined on a single line, the probability is equally divided between the rules. The sum of probabilities for all rules of a certain non-terminal symbol should be equal to 1.  The probabilities are automatically scaled if this isn't the case.
Related:
Base.get — Method
get(root::RuleNode, loc::NodeLoc) Obtain the node pointed to by loc.
Base.insert! — Method
insert!(loc::NodeLoc, rulenode::RuleNode) Replaces the subtree pointed to by loc with the given rulenode.
HerbGrammar.add_rule! — Method
add_rule!(g::Grammar, e::Expr)
Adds a rule to the grammar. 
Usage:
	add_rule!(grammar, :("Real = Real + Real"))
The syntax is identical to the syntax of @csgrammar and @cfgrammar, but only single rules are supported.
Warning
Calls to this function are ignored if a rule is already in the grammar.
HerbGrammar.child_types — Method
child_types(grammar::Grammar, rule_index::Int)
Returns the types of the children (nonterminals) of the production rule at rule_index.
HerbGrammar.child_types — Method
child_types(grammar::Grammar, node::RuleNode)
Returns the list of child types (nonterminal symbols) in the production rule used by node.
HerbGrammar.cleanup_removed_rules! — Method
cleanup_removed_rules!(g::Grammar)
Removes any placeholders for previously deleted rules.  This means that indices get shifted.
Warning
When indices are shifted, this grammar can no longer be used to interpret  AbstractRuleNode trees created before the call to this function. These trees become meaningless. 
HerbGrammar.containedin — Method
containedin(vec1::Vector, vec2::Vector)
Checks if elements of vec1 are contained in vec2 in the same order (possibly with elements in between)
HerbGrammar.contains_returntype — Function
contains_returntype(node::RuleNode, grammar::Grammar, sym::Symbol, maxdepth::Int=typemax(Int))
Returns true if the tree rooted at node contains at least one node at depth less than maxdepth with the given return type or nonterminal symbol.
HerbGrammar.expr2cfgrammar — Method
expr2cfgrammar(ex::Expr)::ContextFreeGrammar
A function for converting an Expr to a ContextFreeGrammar. If the expression is hardcoded, you should use the @cfgrammar macro. Only expressions in the correct format (see @cfgrammar) can be converted.
Example usage:
grammar = expr2cfgrammar(
+end
Syntax:
The syntax of rules is identical to the syntax used by @csgrammar:
  • Literals: Symbols that are already defined in Julia are considered literals, such as 1, 2, or π. For example: R = 1.
  • Variables: A variable is a symbol that is not a nonterminal symbol and not already defined in Julia. For example: R = x.
  • Functions: Functions and infix operators that are defined in Julia or the Main module can be used  with the default evaluator. For example: R = R + R, R = f(a, b).
  • Combinations: Multiple rules can be defined on a single line in the grammar definition using the | symbol. For example: R = 1 | 2 | 3.
  • Iterators: Another way to define multiple rules is by providing a Julia iterator after a | symbol. For example: R = |(1:9).
Every rule is also prefixed with a probability. Rules and probabilities are separated using the : symbol. If multiple rules are defined on a single line, the probability is equally divided between the rules. The sum of probabilities for all rules of a certain non-terminal symbol should be equal to 1.  The probabilities are automatically scaled if this isn't the case.
Related:
source
Base.get — Method
get(root::RuleNode, loc::NodeLoc) Obtain the node pointed to by loc.
source
Base.insert! — Method
insert!(loc::NodeLoc, rulenode::RuleNode) Replaces the subtree pointed to by loc with the given rulenode.
source
HerbGrammar.add_rule! — Method
add_rule!(g::Grammar, e::Expr)
Adds a rule to the grammar. 
Usage:
	add_rule!(grammar, :("Real = Real + Real"))
The syntax is identical to the syntax of @csgrammar and @cfgrammar, but only single rules are supported.
Warning
Calls to this function are ignored if a rule is already in the grammar.
source
HerbGrammar.child_types — Method
child_types(grammar::Grammar, rule_index::Int)
Returns the types of the children (nonterminals) of the production rule at rule_index.
source
HerbGrammar.child_types — Method
child_types(grammar::Grammar, node::RuleNode)
Returns the list of child types (nonterminal symbols) in the production rule used by node.
source
HerbGrammar.cleanup_removed_rules! — Method
cleanup_removed_rules!(g::Grammar)
Removes any placeholders for previously deleted rules.  This means that indices get shifted.
Warning
When indices are shifted, this grammar can no longer be used to interpret  AbstractRuleNode trees created before the call to this function. These trees become meaningless. 
source
HerbGrammar.containedin — Method
containedin(vec1::Vector, vec2::Vector)
Checks if elements of vec1 are contained in vec2 in the same order (possibly with elements in between)
source
HerbGrammar.contains_returntype — Function
contains_returntype(node::RuleNode, grammar::Grammar, sym::Symbol, maxdepth::Int=typemax(Int))
Returns true if the tree rooted at node contains at least one node at depth less than maxdepth with the given return type or nonterminal symbol.
source
HerbGrammar.expr2cfgrammar — Method
expr2cfgrammar(ex::Expr)::ContextFreeGrammar
A function for converting an Expr to a ContextFreeGrammar. If the expression is hardcoded, you should use the @cfgrammar macro. Only expressions in the correct format (see @cfgrammar) can be converted.
Example usage:
grammar = expr2cfgrammar(
 	begin
 		R = x
 		R = 1 | 2
 		R = R + R
 	end
-)
HerbGrammar.expr2csgrammar — Method
expr2csgrammar(ex::Expr)::ContextSensitiveGrammar
A function for converting an Expr to a ContextSensitiveGrammar. If the expression is hardcoded, you should use the @csgrammar macro. Only expressions in the correct format (see @csgrammar) can be converted.
Example usage:
grammar = expr2csgrammar(
 	begin
 		R = x
 		R = 1 | 2
 		R = R + R
 	end
-)
HerbGrammar.expr2pcfgrammar — Method
Function for converting an Expr to a ContextFreeGrammar with probabilities. If the expression is hardcoded, you should use the @pcfgrammar macro. Only expressions in the correct format (see @pcfgrammar) can be converted.
Example usage:
grammar = expr2pcsgrammar(
 	begin
 		0.5 : R = x
 		0.3 : R = 1 | 2
 		0.2 : R = R + R
 	end
-)
HerbGrammar.expr2pcsgrammar — Method
Function for converting an Expr to a ContextSensitiveGrammar with probabilities. If the expression is hardcoded, you should use the @pcsgrammar macro. Only expressions in the correct format (see @pcsgrammar) can be converted.
Example usage:
grammar = expr2pcsgrammar(
 	begin
 		0.5 : R = x
 		0.3 : R = 1 | 2
 		0.2 : R = R + R
 	end
-)
HerbGrammar.get_childtypes — Method
get_childtypes(rule::Any, types::AbstractVector{Symbol})
Returns the child types/nonterminals of a production rule.
HerbGrammar.get_domain — Method
get_domain(g::Grammar, type::Symbol)::BitVector
Returns the domain for the hole of a certain type as a BitVector of the same length as the number of  rules in the grammar. Bit i is set to true iff rule i is of type type.
Info
Since this function can be intensively used when exploring a program space defined by a grammar, the outcomes of this function are precomputed and stored in the domains field in a Grammar.
HerbGrammar.get_domain — Method
get_domain(g::Grammar, rules::Vector{Int})::BitVector
Takes a domain rules defined as a vector of ints and converts it to a domain defined as a BitVector.
HerbGrammar.get_rulesequence — Method
get_rulesequence(node::RuleNode, path::Vector{Int})
Extract the derivation sequence from a path (sequence of child indices) and an AbstractRuleNode. If the path is deeper than the deepest node, it returns what it has.
HerbGrammar.iscomplete — Method
iscomplete(grammar::Grammar, node::RuleNode)
Returns true if the expression represented by the RuleNode is a complete expression,  meaning that it is fully defined and doesn't have any Holes.
HerbGrammar.iseval — Method
iseval(rule)
Returns true if the rule is the special evaluate immediately function, i.e., _()
Compat
evaluate immediately functionality is not yet supported by most of Herb.jl
HerbGrammar.iseval — Method
iseval(grammar::Grammar, index::Int)::Bool
Returns true if the production rule at rule_index contains the special _() eval function.
Compat
evaluate immediately functionality is not yet supported by most of Herb.jl
HerbGrammar.iseval — Method
iseval(grammar::Grammar)::Bool
Returns true if any production rules in grammar contain the special _() eval function.
Compat
evaluate immediately functionality is not yet supported by most of Herb.jl
HerbGrammar.isterminal — Method
isterminal(rule::Any, types::AbstractVector{Symbol})
Returns true if the rule is terminal, i.e., it does not contain any of the types in the provided vector. For example, :(x) is terminal, and :(1+1) is terminal, but :(Real + Real) is typically not.
HerbGrammar.isterminal — Method
isterminal(grammar::Grammar, rule_index::Int)::Bool
Returns true if the production rule at rule_index is terminal, i.e., does not contain any nonterminal symbols.
HerbGrammar.isterminal — Method
isterminal(grammar::Grammar, node::RuleNode)::Bool
Returns true if the production rule used by node is terminal, i.e., does not contain any nonterminal symbols.
HerbGrammar.isvariable — Method
isvariable(grammar::Grammar, node::RuleNode)::Bool
Returns true if the rule used by node represents a variable.
HerbGrammar.log_probability — Method
log_probability(grammar::Grammar, index::Int)::Real
Returns the log probability for the rule at index in the grammar.
Warning
If the grammar is not probabilistic, a warning is displayed, and a uniform probability is assumed.
HerbGrammar.max_arity — Method
max_arity(grammar::Grammar)::Int
Returns the maximum arity (number of children) over all production rules in the Grammar.
HerbGrammar.mindepth — Method
mindepth(grammar::Grammar, typ::Symbol, dmap::AbstractVector{Int})
Returns the minimum depth achievable for a given nonterminal symbol. The minimum depth is the depth of the lowest tree that can be made using typ  as a start symbol. dmap can be obtained from mindepth_map.
HerbGrammar.mindepth_map — Method
mindepth_map(grammar::Grammar)
Returns the minimum depth achievable for each production rule in the Grammar. In other words, this function finds the depths of the lowest trees that can be made  using each of the available production rules as a root.
HerbGrammar.nchildren — Method
nchildren(grammar::Grammar, rule_index::Int)::Int
Returns the number of children (nonterminals) of the production rule at rule_index.
HerbGrammar.nchildren — Method
nchildren(grammar::Grammar, node::RuleNode)::Int
Returns the number of children in the production rule used by node.
HerbGrammar.probability — Method
probability(grammar::Grammar, index::Int)::Real
Return the probability for a rule in the grammar. Use log_probability whenever possible.
Warning
If the grammar is not probabilistic, a warning is displayed, and a uniform probability is assumed.
HerbGrammar.read_cfg — Method
read_cfg(filepath::AbstractString)::ContextFreeGrammar
Reads a ContextFreeGrammar from the file provided in filepath.
Danger
Only open trusted grammars.  Parts of the grammar can be passed to Julia's eval function.  
HerbGrammar.read_csg — Method
read_csg(grammarpath::AbstractString, constraintspath::AbstractString)::ContextSensitiveGrammar
Reads a ContextSensitiveGrammar from the files at grammarpath and constraintspath. The grammar path may also point to a ContextFreeGrammar.
Danger
Only open trusted grammars.  Parts of the grammar can be passed to Julia's eval function.  
HerbGrammar.read_pcfg — Method
read_pcfg(filepath::AbstractString)::ContextFreeGrammar
Reads a probabilistic ContextFreeGrammar from a file provided in filepath.
Danger
Only open trusted grammars.  Parts of the grammar can be passed to Julia's eval function.  
HerbGrammar.read_pcsg — Method
read_pcsg(grammarpath::AbstractString, constraintspath::AbstractString)::ContextSensitiveGrammar
Reads a probabilistic ContextSensitiveGrammar from the files at grammarpath and constraintspath. The grammar path may also point to a ContextFreeGrammar.
Danger
Only open trusted grammars.  Parts of the grammar can be passed to Julia's eval function.  
HerbGrammar.remove_rule! — Method
remove_rule!(g::Grammar, idx::Int)
Removes the rule corresponding to idx from the grammar.  In order to avoid shifting indices, the rule is replaced with nothing, and all other data structures are updated accordingly.
HerbGrammar.return_type — Method
return_type(grammar::Grammar, rule_index::Int)::Symbol
Returns the type of the production rule at rule_index.
HerbGrammar.return_type — Method
return_type(grammar::Grammar, node::RuleNode)
Gives the return type or nonterminal symbol in the production rule used by node.
HerbGrammar.rulenode2expr — Method
rulenode2expr(rulenode::RuleNode, grammar::Grammar)
Converts a RuleNode into a Julia expression corresponding to the rule definitions in the grammar. The returned expression can be evaluated with Julia semantics using eval().
HerbGrammar.rulesoftype — Method
rulesoftype(node::RuleNode, grammar::Grammar, ruletype::Symbol, ignoreNode::RuleNode)
Returns every rule of nonterminal symbol ruletype that is also used in the AbstractRuleNode tree, but not in the ignoreNode subtree.
Warning
The ignoreNode must be a subtree of node for it to have an effect.
HerbGrammar.rulesoftype — Method
rulesoftype(node::RuleNode, ruleset::Set{Int}, ignoreNode::RuleNode)
Returns every rule in the ruleset that is also used in the AbstractRuleNode tree, but not in the ignoreNode subtree.
Warning
The ignoreNode must be a subtree of node for it to have an effect.
HerbGrammar.rulesonleft — Method
rulesonleft(expr::RuleNode, path::Vector{Int})::Set{Int}
Finds all rules that are used in the left subtree defined by the path.
HerbGrammar.subsequenceof — Method
subsequenceof(vec1::Vector{Int}, vec2::Vector{Int})
Checks if vec1 is a subsequence of vec2.
HerbGrammar.swap_node — Method
swap_node(expr::AbstractRuleNode, new_expr::AbstractRuleNode, path::Vector{Int})
Replace a node in expr, specified by path, with new_expr. Path is a sequence of child indices, starting from the root node.
HerbGrammar.swap_node — Method
swap_node(expr::RuleNode, node::RuleNode, child_index::Int, new_expr::RuleNode)
Replace child i of a node, a part of larger expr, with new_expr.
Index

+)
source
HerbGrammar.get_domain — Method
get_domain(g::Grammar, type::Symbol)::BitVector

Returns the domain for the hole of a certain type as a BitVector of the same length as the number of rules in the grammar. Bit i is set to true iff rule i is of type type.

Info

Since this function can be intensively used when exploring a program space defined by a grammar, the outcomes of this function are precomputed and stored in the domains field in a Grammar.

source
HerbGrammar.get_domain — Method
get_domain(g::Grammar, rules::Vector{Int})::BitVector

Takes a domain rules defined as a vector of ints and converts it to a domain defined as a BitVector.

source
HerbGrammar.get_rulesequence — Method
get_rulesequence(node::RuleNode, path::Vector{Int})

Extract the derivation sequence from a path (sequence of child indices) and an AbstractRuleNode. If the path is deeper than the deepest node, it returns what it has.

source
HerbGrammar.iscomplete — Method
iscomplete(grammar::Grammar, node::RuleNode)

Returns true if the expression represented by the RuleNode is a complete expression, meaning that it is fully defined and doesn't have any Holes.

source
HerbGrammar.iseval — Method
iseval(rule)

Returns true if the rule is the special evaluate immediately function, i.e., _()

Compat

evaluate immediately functionality is not yet supported by most of Herb.jl

source
HerbGrammar.iseval — Method
iseval(grammar::Grammar, index::Int)::Bool

Returns true if the production rule at rule_index contains the special _() eval function.

Compat

evaluate immediately functionality is not yet supported by most of Herb.jl

source
HerbGrammar.iseval — Method
iseval(grammar::Grammar)::Bool

Returns true if any production rules in grammar contain the special _() eval function.

Compat

evaluate immediately functionality is not yet supported by most of Herb.jl

source
HerbGrammar.isterminal — Method
isterminal(rule::Any, types::AbstractVector{Symbol})

Returns true if the rule is terminal, i.e., it does not contain any of the types in the provided vector. For example, :(x) is terminal, and :(1+1) is terminal, but :(Real + Real) is typically not.

source
HerbGrammar.isterminal — Method
isterminal(grammar::Grammar, rule_index::Int)::Bool

Returns true if the production rule at rule_index is terminal, i.e., does not contain any nonterminal symbols.

source
HerbGrammar.isterminal — Method
isterminal(grammar::Grammar, node::RuleNode)::Bool

Returns true if the production rule used by node is terminal, i.e., does not contain any nonterminal symbols.

source
HerbGrammar.isvariable — Method
isvariable(grammar::Grammar, node::RuleNode)::Bool

Returns true if the rule used by node represents a variable.

source
HerbGrammar.log_probability — Method
log_probability(grammar::Grammar, index::Int)::Real

Returns the log probability for the rule at index in the grammar.

Warning

If the grammar is not probabilistic, a warning is displayed, and a uniform probability is assumed.

source
HerbGrammar.mindepth — Method
mindepth(grammar::Grammar, typ::Symbol, dmap::AbstractVector{Int})

Returns the minimum depth achievable for a given nonterminal symbol. The minimum depth is the depth of the lowest tree that can be made using typ as a start symbol. dmap can be obtained from mindepth_map.

source
HerbGrammar.mindepth_map — Method
mindepth_map(grammar::Grammar)

Returns the minimum depth achievable for each production rule in the Grammar. In other words, this function finds the depths of the lowest trees that can be made using each of the available production rules as a root.

source
HerbGrammar.nchildren — Method
nchildren(grammar::Grammar, rule_index::Int)::Int

Returns the number of children (nonterminals) of the production rule at rule_index.

source
HerbGrammar.nchildren — Method
nchildren(grammar::Grammar, node::RuleNode)::Int

Returns the number of children in the production rule used by node.

source
HerbGrammar.probability — Method
probability(grammar::Grammar, index::Int)::Real

Return the probability for a rule in the grammar. Use log_probability whenever possible.

Warning

If the grammar is not probabilistic, a warning is displayed, and a uniform probability is assumed.

source
HerbGrammar.read_cfg — Method
read_cfg(filepath::AbstractString)::ContextFreeGrammar

Reads a ContextFreeGrammar from the file provided in filepath.

Danger

Only open trusted grammars. Parts of the grammar can be passed to Julia's eval function.

source
HerbGrammar.read_csg — Method
read_csg(grammarpath::AbstractString, constraintspath::AbstractString)::ContextSensitiveGrammar

Reads a ContextSensitiveGrammar from the files at grammarpath and constraintspath. The grammar path may also point to a ContextFreeGrammar.

Danger

Only open trusted grammars. Parts of the grammar can be passed to Julia's eval function.

source
HerbGrammar.read_pcfg — Method
read_pcfg(filepath::AbstractString)::ContextFreeGrammar

Reads a probabilistic ContextFreeGrammar from a file provided in filepath.

Danger

Only open trusted grammars. Parts of the grammar can be passed to Julia's eval function.

source
HerbGrammar.read_pcsg — Method
read_pcsg(grammarpath::AbstractString, constraintspath::AbstractString)::ContextSensitiveGrammar

Reads a probabilistic ContextSensitiveGrammar from the files at grammarpath and constraintspath. The grammar path may also point to a ContextFreeGrammar.

Danger

Only open trusted grammars. Parts of the grammar can be passed to Julia's eval function.

source
HerbGrammar.remove_rule! — Method
remove_rule!(g::Grammar, idx::Int)

Removes the rule corresponding to idx from the grammar. In order to avoid shifting indices, the rule is replaced with nothing, and all other data structures are updated accordingly.

source
HerbGrammar.return_type — Method
return_type(grammar::Grammar, rule_index::Int)::Symbol

Returns the type of the production rule at rule_index.

source
HerbGrammar.return_type — Method
return_type(grammar::Grammar, node::RuleNode)

Gives the return type or nonterminal symbol in the production rule used by node.

source
HerbGrammar.rulenode2expr — Method
rulenode2expr(rulenode::RuleNode, grammar::Grammar)

Converts a RuleNode into a Julia expression corresponding to the rule definitions in the grammar. The returned expression can be evaluated with Julia semantics using eval().

source
HerbGrammar.rulesoftype — Method
rulesoftype(node::RuleNode, grammar::Grammar, ruletype::Symbol, ignoreNode::RuleNode)

Returns every rule of nonterminal symbol ruletype that is also used in the AbstractRuleNode tree, but not in the ignoreNode subtree.

Warning

The ignoreNode must be a subtree of node for it to have an effect.

source
HerbGrammar.rulesoftype — Method
rulesoftype(node::RuleNode, ruleset::Set{Int}, ignoreNode::RuleNode)

Returns every rule in the ruleset that is also used in the AbstractRuleNode tree, but not in the ignoreNode subtree.

Warning

The ignoreNode must be a subtree of node for it to have an effect.

source
HerbGrammar.rulesonleft — Method
rulesonleft(expr::RuleNode, path::Vector{Int})::Set{Int}

Finds all rules that are used in the left subtree defined by the path.

source
HerbGrammar.swap_node — Method
swap_node(expr::AbstractRuleNode, new_expr::AbstractRuleNode, path::Vector{Int})

Replace a node in expr, specified by path, with new_expr. Path is a sequence of child indices, starting from the root node.

source
HerbGrammar.swap_node — Method
swap_node(expr::RuleNode, node::RuleNode, child_index::Int, new_expr::RuleNode)

Replace child i of a node, a part of larger expr, with new_expr.

source

Index

diff --git a/dev/HerbInterpret/index.html b/dev/HerbInterpret/index.html index 0bde7ab..55f8661 100644 --- a/dev/HerbInterpret/index.html +++ b/dev/HerbInterpret/index.html @@ -1,4 +1,4 @@ -HerbInterpret.jl · Herb.jl
Edit on GitHub

HerbInterpret.jl Documentation

HerbInterpret.evaluate_program — Method
evaluate_program(program::RuleNode, examples::Vector{<:Example}, grammar::Grammar, evaluation_function::Function)

Runs a program on the examples and returns tuples of actual desired output and the program's output

HerbInterpret.execute_on_examples — Method
execute_on_examples(tab::SymbolTable, expr::Any, example_inputs::Vector{Dict{Symbol, Any}})::Vector{Any}

Executes a given expression on a set of inputs and returns the respective outputs. WARNING: This function throws exceptions that are caused in the given expression. These exceptions have to be handled by the caller of this function.

HerbInterpret.interpret — Method
interpret(tab::SymbolTable, ex::Expr)

Evaluates an expression without compiling it. Uses AST and symbol lookups. Only supports :call and :(=) expressions at the moment.

Example usage:

tab = SymbolTable(:f => f, :x => x)
+HerbInterpret.jl · Herb.jl
Edit on GitHub
HerbInterpret.jl Documentation
HerbInterpret.evaluate_program — Method
evaluate_program(program::RuleNode, examples::Vector{<:Example}, grammar::Grammar, evaluation_function::Function)
Runs a program on the examples and returns tuples of actual desired output and the program's output
source
HerbInterpret.execute_on_examples — Method
execute_on_examples(tab::SymbolTable, expr::Any, example_inputs::Vector{Dict{Symbol, Any}})::Vector{Any}
Executes a given expression on a set of inputs and returns the respective outputs. WARNING: This function throws exceptions that are caused in the given expression. These exceptions have to be handled by the caller of this function.
source
HerbInterpret.interpret — Method
interpret(tab::SymbolTable, ex::Expr)
Evaluates an expression without compiling it. Uses AST and symbol lookups. Only supports :call and :(=) expressions at the moment.
Example usage:
tab = SymbolTable(:f => f, :x => x)
 ex = :(f(x))
-interpret(tab, ex)
WARNING: This function throws exceptions that are caused in the given expression. These exceptions have to be handled by the caller of this function.
HerbInterpret.test_all_examples — Method
test_all_examples(tab::SymbolTable, expr::Any, examples::Vector{Example})::Vector{Bool}
Runs the interpreter on all examples with the given input table and expression.  The symbol table defines everything (functions, symbols) that are not input variables to the program to be synthesised. Returns a list of true/false values indicating if the expression satisfies the corresponding example. WARNING: This function throws exceptions that are caused in the given expression. These exceptions have to be handled by the caller of this function.
HerbInterpret.test_examples — Method
test_examples(tab::SymbolTable, expr::Any, examples::Vector{Example})::Bool
Evaluates all examples and returns true iff all examples pass. Shortcircuits as soon as an example is found for which the program doesn't work.  Returns false if one of the examples produces an error.
HerbInterpret.test_with_input — Method
test_with_input(tab::SymbolTable, expr::Any, input::Dict)
Interprets an expression or symbol with the given symboltable and the input. WARNING: This function throws exceptions that are caused in the given expression. These exceptions have to be handled by the caller of this function.
Index

+interpret(tab, ex)

WARNING: This function throws exceptions that are caused in the given expression. These exceptions have to be handled by the caller of this function.

source
HerbInterpret.test_all_examples — Method
test_all_examples(tab::SymbolTable, expr::Any, examples::Vector{Example})::Vector{Bool}

Runs the interpreter on all examples with the given input table and expression. The symbol table defines everything (functions, symbols) that are not input variables to the program to be synthesised. Returns a list of true/false values indicating if the expression satisfies the corresponding example. WARNING: This function throws exceptions that are caused in the given expression. These exceptions have to be handled by the caller of this function.

source
HerbInterpret.test_examples — Method
test_examples(tab::SymbolTable, expr::Any, examples::Vector{Example})::Bool

Evaluates all examples and returns true iff all examples pass. Shortcircuits as soon as an example is found for which the program doesn't work. Returns false if one of the examples produces an error.

source
HerbInterpret.test_with_input — Method
test_with_input(tab::SymbolTable, expr::Any, input::Dict)

Interprets an expression or symbol with the given symboltable and the input. WARNING: This function throws exceptions that are caused in the given expression. These exceptions have to be handled by the caller of this function.

source

Index

diff --git a/dev/HerbSearch/index.html b/dev/HerbSearch/index.html index 889993b..96a90b2 100644 --- a/dev/HerbSearch/index.html +++ b/dev/HerbSearch/index.html @@ -1,9 +1,9 @@ -HerbSearch.jl · Herb.jl
Edit on GitHub

HerbSearch.jl Documentation

HerbSearch.ContextSensitivePriorityEnumerator — Type
mutable struct ContextSensitivePriorityEnumerator <: ExpressionIterator

Enumerates a context-free grammar starting at Symbol sym with respect to the grammar up to a given depth and a given size. The exploration is done using the given priority function for derivations, and the expand function for discovered nodes.

HerbSearch.ExpandFailureReason — Type
@enum ExpandFailureReason limit_reached=1 already_complete=2

Representation of the different reasons why expanding a partial tree failed. Currently, there are two possible causes of the expansion failing:

  • limit_reached: The depth limit or the size limit of the partial tree would be violated by the expansion
  • already_complete: There is no hole left in the tree, so nothing can be expanded.
HerbSearch.GeneticSearchIterator — Type
GeneticSearchIterator{FitnessFunction,CrossOverFunction,MutationFunction,SelectParentsFunction,EvaluationFunction} <: ExpressionIterator

Defines an ExpressionIterator using genetic search.

Consists of:

  • grammar::ContextSensitiveGrammar: the grammar to search over

  • examples::Vector{<:Example}: a collection of examples defining the specification

  • fitness::FitnessFunction: assigns a numerical value (fitness score) to each individual based on how closely it meets the desired objective

  • cross_over::CrossOverFunction: combines the program from two parent individuals to create one or more offspring individuals

  • mutation!::MutationFunction: mutates the program of an invididual

  • select_parents::SelectParentsFunction: selects two parents for the crossover

  • evaluation_function::EvaluationFunction: interpreter to evaluate the individual programs

  • start_symbol::Symbol: defines the start symbol from which the search should be started

  • population_size::Int64: number of inviduals in the population

  • mutation_probability::Float64: probability of mutation for each individual

  • maximum_initial_population_depth::Int64: maximum depth of trees when population is initialized

end

HerbSearch.PriorityQueueItem — Type
struct PriorityQueueItem

Represents an item in the priority enumerator priority queue. An item contains of:

  • tree: A partial AST
  • size: The size of the tree. This is a cached value which prevents having to traverse the entire tree each time the size is needed.
  • constraints: The local constraints that apply to this tree. These constraints are enforced each time the tree is modified.
HerbSearch.PropagateResult — Type
@enum PropagateResult tree_complete=1 tree_incomplete=2 tree_infeasible=3

Representation of the possible results of a constraint propagation. At the moment there are three possible outcomes:

  • tree_complete: The propagation was applied successfully and the tree does not contain any holes anymore. Thus no constraints can be applied anymore.
  • tree_incomplete: The propagation was applied successfully and the tree does contain more holes. Thus more constraints may be applied to further prune the respective domains.
  • tree_infeasible: The propagation was succesful, but there are holes with empty domains. Hence, the tree is now infeasible.
HerbSearch.StochasticSearchEnumerator — Type
Base.@kwdef struct StochasticSearchEnumerator <: ExpressionIterator

A unified struct for the algorithms Metropolis Hastings, Very Large Scale Neighbourhood and Simulated Annealing. Each algorithm implements neighbourhood propose accept and temperature functions. Below the signiture of all this function is shown

Signatures

Returns a node location from the program that is the neighbourhood. It can also return other information using dict

neighbourhood(program::RuleNode, grammar::Grammar) -> (loc::NodeLocation, dict::Dict)

Proposes a list of programs using the location provided by neighbourhood and the dict.

propose(current_program, loc::NodeLocation, grammar::Grammar, max_depth::Int64, dict::Dict) -> Iter[RuleNode]

Based on the current program and possible cost and temperature it accepts the program or not. Usually we would always want to accept better programs but we might get stuck if we do so. That is why some implementations of the accept function accept with a probability costs that are worse. cost means how different are the outcomes of the program compared to the correct outcomes. The lower the cost the better the program performs on the examples. The cost is provided by the cost_function

accept(current_cost::Real, possible_cost::Real, temperature::Real) -> Bool

Returns the new temperature based on the previous temperature. Higher the temperature means that the algorithm will explore more.

temperature(previous_temperature::Real) -> Real

Returns the cost of the current program. It receives a list of tuples (expected, found) and gives back a cost.

cost_function(outcomes::Tuple{<:Number,<:Number}[]) -> Real

Fields

  • grammar::ContextSensitiveGrammar grammar that the algorithm uses
  • max_depth::Int64 = 5 maximum depth of the program to generate
  • examples::Vector{Example} example used to check the program
  • neighbourhood::Function
  • propose::Function
  • accept::Function
  • temperature::Function
  • cost_function::Function
  • start_symbol::Symbol the start symbol of the algorithm :Real or :Int
  • initial_temperature::Real = 1
  • evaluation_function::Function that evaluates the julia expressions

An iterator over all possible expressions of a grammar up to max_depth with start symbol sym.

Base.iterate — Method
Base.iterate(iter::ContextSensitivePriorityEnumerator, pq::DataStructures.PriorityQueue)

Describes the iteration for a given ContextSensitivePriorityEnumerator and a PriorityQueue over the grammar without enqueueing new items to the priority queue. Recursively returns the result for the priority queue.

Base.iterate — Method
Base.iterate(iter::ContextSensitivePriorityEnumerator)

Describes the iteration for a given ContextSensitivePriorityEnumerator over the grammar. The iteration constructs a PriorityQueue first and then prunes it propagating the active constraints. Recursively returns the result for the priority queue.

Base.iterate — Method
Base.iterate(iter::GeneticSearchIterator, current_state::GeneticIteratorState)

Iterates the search space using a genetic algorithm. Takes the iterator and the current state to mutate and crossover random inviduals. Returns the best program-so-far and the state of the iterator.

Base.iterate — Method
Base.iterate(iter::GeneticSearchIterator)

Iterates the search space using a genetic algorithm. First generates a population sampling random programs. Returns the best program-so-far, and the state of the iterator.

Base.iterate — Method
Base.iterate(iter::StochasticSearchEnumerator, current_state::IteratorState)

The algorithm that constructs the iterator of StochasticSearchEnumerator. It has the following structure:

  1. get a random node location -> location,dict = neighbourhood(current_program)
  2. call propose on the current program getting a list of possbile replacements in the node location
  3. iterate through all the possible replacements and perform the replacement in the current program
    1. accept the new program by modifying the next_program or reject the new program
  4. return the new next_program
Base.rand — Function
rand(::Type{RuleNode}, grammar::Grammar, typ::Symbol, max_depth::Int=10)

Generates a random RuleNode of return type typ and maximum depth max_depth.

Base.rand — Function
rand(::Type{RuleNode}, grammar::Grammar, typ::Symbol, dmap::AbstractVector{Int}, max_depth::Int=10)

Generates a random RuleNode, i.e. an expression tree, of root type typ and maximum depth max_depth guided by a depth map dmap if possible.

Base.rand — Function
rand(::Type{RuleNode}, grammar::Grammar, max_depth::Int=10)

Generates a random RuleNode of arbitrary type and maximum depth max_depth.

HerbSearch._expand — Method
_expand(node::Hole, grammar::ContextSensitiveGrammar, ::Int, max_holes::Int, context::GrammarContext, hole_heuristic::Function, derivation_heuristic::Function)::Union{ExpandFailureReason, Vector{TreeConstraints}}

Expands a given hole that was found in _expand using the given derivation heuristic. Returns the list of discovered nodes in that order and with their respective constraints.

HerbSearch._expand — Method
_expand(root::RuleNode, grammar::ContextSensitiveGrammar, max_depth::Int, max_holes::Int, context::GrammarContext, hole_heuristic::Function, derivation_heuristic::Function)::Union{ExpandFailureReason, Vector{TreeConstraints}}

Recursive expand function used in multiple enumeration techniques. Expands one hole/undefined leaf of the given RuleNode tree found using the given hole heuristic. If the expansion was successful, returns a list of new trees and a list of lists of hole locations, corresponding to the holes of each newly expanded tree. Returns nothing if tree is already complete (i.e. contains no holes). Returns an empty list if the tree is partial (i.e. contains holes), but they could not be expanded because of the depth limit.

HerbSearch._find_next_complete_tree — Method
_find_next_complete_tree(grammar::ContextSensitiveGrammar, max_depth::Int, max_size::Int, priority_function::Function, expand_function::Function, pq::PriorityQueue)::Union{Tuple{RuleNode, PriorityQueue}, Nothing}

Takes a priority queue and returns the smallest AST from the grammar it can obtain from the queue or by (repeatedly) expanding trees that are in the queue. Returns nothing if there are no trees left within the depth limit.

HerbSearch.best_accept — Method
best_accept(current_cost::Real, next_cost::Real, temperature)

Returns true if the cost of the proposed program is smaller than the cost of the current program. Otherwise, returns false.

Arguments

  • current_cost::Real: the cost of the current program.
  • next_cost::Real: the cost of the proposed program.
  • temperature::Real: the temperature; not used.
HerbSearch.calculate_cost — Method
calculate_cost(program::RuleNode, cost_function::Function, examples::AbstractVector{Example}, grammar::Grammar, evaluation_function::Function)

Returns the cost of the program using the examples and the cost_function. It first convert the program to an expression and evaluates it on all the examples using HerbInterpret.evaluate_program.

HerbSearch.const_temperature — Method
const_temperature(current_temperature)

Returns the temperature unchanged. This function is used by Metropolis Hastings and Very Large Neighbourhood Search algorithms.

Arguments

  • current_temperature::Real: the current temperature of the search.
HerbSearch.constructNeighbourhood — Method
constructNeighbourhood(current_program::RuleNode, grammar::Grammar)

The neighbourhood node location is chosen at random. The dictionary is nothing.

Arguments

  • current_program::RuleNode: the current program.
  • grammar::Grammar: the grammar.
HerbSearch.constructNeighbourhoodRuleSubset — Method
constructNeighbourhoodRuleSubset(current_program::RuleNode, grammar::Grammar)

The neighbourhood node location is chosen at random. The dictionary is contains one entry with key "rule_subset" and value of type Vector{Any} being a random subset of grammar rules.

Arguments

  • current_program::RuleNode: the current program.
  • grammar::Grammar: the grammar.
HerbSearch.count_expressions — Method
count_expressions(iter::ExpressionIterator)

Counts and returns the number of possible expressions in the expression iterator. The Iterator is not modified.

HerbSearch.count_expressions — Method
count_expressions(grammar::Grammar, max_depth::Int, max_size::Int, sym::Symbol)

Counts and returns the number of possible expressions of a grammar up to max_depth with start symbol sym.

HerbSearch.crossover_swap_children_1 — Method
crossover_swap_children_1(parent1::RuleNode, parent2::RuleNode)

Performs a random crossover of two parents of type RuleNode. The subprograms are swapped and only one altered parent program is returned.

HerbSearch.crossover_swap_children_2 — Method
crossover_swap_children_2(parent1::RuleNode, parent2::RuleNode)

Performs a random crossover of two parents of type RuleNode. The subprograms are swapped and both altered parent programs are returned.

HerbSearch.decreasing_temperature — Method
decreasing_temperature(percentage::Real)

Returns a function that produces a temperature decreased by percentage%. This function is used by the Simmulated Annealing algorithm.

Arguments

  • percentage::Real: the percentage to decrease the temperature by.
HerbSearch.default_error_function — Method
default_error_function(old_error, output, expected_output)

Default error function for search_best.

- old_error         - The existing total error
+HerbSearch.jl · Herb.jl
Edit on GitHub
HerbSearch.jl Documentation
HerbSearch.ContextSensitivePriorityEnumerator — Type
mutable struct ContextSensitivePriorityEnumerator <: ExpressionIterator
Enumerates a context-free grammar starting at Symbol sym with respect to the grammar up to a given depth and a given size. The exploration is done using the given priority function for derivations, and the expand function for discovered nodes.
source
HerbSearch.ExpandFailureReason — Type
@enum ExpandFailureReason limit_reached=1 already_complete=2
Representation of the different reasons why expanding a partial tree failed.  Currently, there are two possible causes of the expansion failing:
  • limit_reached: The depth limit or the size limit of the partial tree would   be violated by the expansion
  • already_complete: There is no hole left in the tree, so nothing can be   expanded.
source
HerbSearch.GeneticSearchIterator — Type
GeneticSearchIterator{FitnessFunction,CrossOverFunction,MutationFunction,SelectParentsFunction,EvaluationFunction} <: ExpressionIterator
Defines an ExpressionIterator using genetic search. 
Consists of:
  • grammar::ContextSensitiveGrammar: the grammar to search over
  • examples::Vector{<:Example}: a collection of examples defining the specification 
  • fitness::FitnessFunction: assigns a numerical value (fitness score) to each individual based on how closely it meets the desired objective
  • cross_over::CrossOverFunction: combines the program from two parent individuals to create one or more offspring individuals
  • mutation!::MutationFunction: mutates the program of an invididual
  • select_parents::SelectParentsFunction: selects two parents for the crossover
  • evaluation_function::EvaluationFunction: interpreter to evaluate the individual programs
  • start_symbol::Symbol: defines the start symbol from which the search should be started
  • population_size::Int64: number of inviduals in the population
  • mutation_probability::Float64: probability of mutation for each individual
  • maximum_initial_population_depth::Int64: maximum depth of trees when population is initialized 
end
source
HerbSearch.PriorityQueueItem — Type
struct PriorityQueueItem
Represents an item in the priority enumerator priority queue. An item contains of:
  • tree: A partial AST
  • size: The size of the tree. This is a cached value which prevents  having to traverse the entire tree each time the size is needed.
  • constraints: The local constraints that apply to this tree.   These constraints are enforced each time the tree is modified.
source
HerbSearch.PropagateResult — Type
@enum PropagateResult tree_complete=1 tree_incomplete=2 tree_infeasible=3
Representation of the possible results of a constraint propagation.  At the moment there are three possible outcomes:
  • tree_complete: The propagation was applied successfully and the tree does not contain any holes anymore. Thus no constraints can be applied anymore.
  • tree_incomplete: The propagation was applied successfully and the tree does contain more holes. Thus more constraints may be applied to further prune the respective domains.
  • tree_infeasible: The propagation was succesful, but there are holes with empty domains. Hence, the tree is now infeasible.
source
HerbSearch.StochasticSearchEnumerator — Type
Base.@kwdef struct StochasticSearchEnumerator <: ExpressionIterator
A unified struct for the algorithms Metropolis Hastings, Very Large Scale Neighbourhood and Simulated Annealing. Each algorithm implements neighbourhood propose accept and temperature functions. Below the signiture of all this function is shown
Signatures
Returns a node location from the program that is the neighbourhood. It can also return other information using  dict
neighbourhood(program::RuleNode, grammar::Grammar) -> (loc::NodeLocation, dict::Dict)
Proposes a list of programs using the location provided by neighbourhood and the dict.
propose(current_program, loc::NodeLocation, grammar::Grammar, max_depth::Int64, dict::Dict) -> Iter[RuleNode]
Based on the current program and possible cost and temperature it accepts the program or not. Usually we would always want to accept better programs but we might get stuck if we do so. That is why some implementations of the accept function accept with a probability  costs that are worse.  cost means how different are the outcomes of the program compared to the correct outcomes. The lower the cost the better the program performs on the examples. The cost is provided by the cost_function
accept(current_cost::Real, possible_cost::Real, temperature::Real) -> Bool
Returns the new temperature based on the previous temperature. Higher the temperature means that the algorithm will explore more.
temperature(previous_temperature::Real) -> Real
Returns the cost of the current program. It receives a list of tuples (expected, found) and gives back a cost.
cost_function(outcomes::Tuple{<:Number,<:Number}[]) -> Real
Fields
  • grammar::ContextSensitiveGrammar grammar that the algorithm uses
  • max_depth::Int64 = 5  maximum depth of the program to generate
  • examples::Vector{Example} example used to check the program
  • neighbourhood::Function 
  • propose::Function
  • accept::Function
  • temperature::Function
  • cost_function::Function
  • start_symbol::Symbol the start symbol of the algorithm :Real or :Int
  • initial_temperature::Real = 1 
  • evaluation_function::Function that evaluates the julia expressions
An iterator over all possible expressions of a grammar up to max_depth with start symbol sym.
source
Base.iterate — Method
Base.iterate(iter::ContextSensitivePriorityEnumerator, pq::DataStructures.PriorityQueue)
Describes the iteration for a given ContextSensitivePriorityEnumerator and a PriorityQueue over the grammar without enqueueing new items to the priority queue. Recursively returns the result for the priority queue.
source
Base.iterate — Method
Base.iterate(iter::ContextSensitivePriorityEnumerator)
Describes the iteration for a given ContextSensitivePriorityEnumerator over the grammar. The iteration constructs a PriorityQueue first and then prunes it propagating the active constraints. Recursively returns the result for the priority queue.
source
Base.iterate — Method
Base.iterate(iter::GeneticSearchIterator, current_state::GeneticIteratorState)
Iterates the search space using a genetic algorithm. Takes the iterator and the current state to mutate and crossover random inviduals. Returns the best program-so-far and the state of the iterator.
source
Base.iterate — Method
Base.iterate(iter::GeneticSearchIterator)
Iterates the search space using a genetic algorithm. First generates a population sampling random programs. Returns the best program-so-far, and the state of the iterator.
source
Base.iterate — Method
Base.iterate(iter::StochasticSearchEnumerator, current_state::IteratorState)
The algorithm that constructs the iterator of StochasticSearchEnumerator. It has the following structure:
  1. get a random node location -> location,dict = neighbourhood(current_program)
  2. call propose on the current program getting a list of possbile replacements in the node location 
  3. iterate through all the possible replacements and perform the replacement in the current program 
    1. accept the new program by modifying the next_program or reject the new program
  4. return the new next_program
source
Base.rand — Function
rand(::Type{RuleNode}, grammar::Grammar, max_depth::Int=10)
Generates a random RuleNode of arbitrary type and maximum depth max_depth.
source
Base.rand — Function
rand(::Type{RuleNode}, grammar::Grammar, typ::Symbol, dmap::AbstractVector{Int}, max_depth::Int=10)
Generates a random RuleNode, i.e. an expression tree, of root type typ and maximum depth max_depth guided by a depth map dmap if possible.
source
Base.rand — Function
rand(::Type{RuleNode}, grammar::Grammar, typ::Symbol, max_depth::Int=10)
Generates a random RuleNode of return type typ and maximum depth max_depth.
source
HerbSearch._expand — Method
_expand(node::Hole, grammar::ContextSensitiveGrammar, ::Int, max_holes::Int, context::GrammarContext, hole_heuristic::Function, derivation_heuristic::Function)::Union{ExpandFailureReason, Vector{TreeConstraints}}
Expands a given hole that was found in _expand using the given derivation heuristic. Returns the list of discovered nodes in that order and with their respective constraints.
source
HerbSearch._expand — Method
_expand(root::RuleNode, grammar::ContextSensitiveGrammar, max_depth::Int, max_holes::Int, context::GrammarContext, hole_heuristic::Function, derivation_heuristic::Function)::Union{ExpandFailureReason, Vector{TreeConstraints}}
Recursive expand function used in multiple enumeration techniques. Expands one hole/undefined leaf of the given RuleNode tree found using the given hole heuristic. If the expansion was successful, returns a list of new trees and a list of lists of hole locations, corresponding to the holes of each newly expanded tree.  Returns nothing if tree is already complete (i.e. contains no holes). Returns an empty list if the tree is partial (i.e. contains holes), but they could not be expanded because of the depth limit.
source
HerbSearch._find_next_complete_tree — Method
_find_next_complete_tree(grammar::ContextSensitiveGrammar, max_depth::Int, max_size::Int, priority_function::Function, expand_function::Function, pq::PriorityQueue)::Union{Tuple{RuleNode, PriorityQueue}, Nothing}
Takes a priority queue and returns the smallest AST from the grammar it can obtain from the queue or by (repeatedly) expanding trees that are in the queue. Returns nothing if there are no trees left within the depth limit.
source
HerbSearch.best_accept — Method
best_accept(current_cost::Real, next_cost::Real, temperature)
Returns true if the cost of the proposed program is smaller than the cost of the current program. Otherwise, returns false.
Arguments
  • current_cost::Real: the cost of the current program.
  • next_cost::Real: the cost of the proposed program.
  • temperature::Real: the temperature; not used.
source
HerbSearch.calculate_cost — Method
calculate_cost(program::RuleNode, cost_function::Function, examples::AbstractVector{Example}, grammar::Grammar, evaluation_function::Function)
Returns the cost of the program using the examples and the cost_function. It first convert the program to an expression and evaluates it on all the examples using HerbInterpret.evaluate_program.
source
HerbSearch.const_temperature — Method
const_temperature(current_temperature)
Returns the temperature unchanged. This function is used by Metropolis Hastings and Very Large Neighbourhood Search algorithms.
Arguments
  • current_temperature::Real: the current temperature of the search.
source
HerbSearch.constructNeighbourhood — Method
constructNeighbourhood(current_program::RuleNode, grammar::Grammar)
The neighbourhood node location is chosen at random. The dictionary is nothing.
Arguments
  • current_program::RuleNode: the current program.
  • grammar::Grammar: the grammar.
source
HerbSearch.constructNeighbourhoodRuleSubset — Method
constructNeighbourhoodRuleSubset(current_program::RuleNode, grammar::Grammar)
The neighbourhood node location is chosen at random. The dictionary is contains one entry with key "rule_subset" and value of type Vector{Any} being a random subset of grammar rules.
Arguments
  • current_program::RuleNode: the current program.
  • grammar::Grammar: the grammar.
source
HerbSearch.count_expressions — Method
count_expressions(iter::ExpressionIterator)
Counts and returns the number of possible expressions in the expression iterator. The Iterator is not modified.
source
HerbSearch.count_expressions — Method
count_expressions(grammar::Grammar, max_depth::Int, max_size::Int, sym::Symbol)
Counts and returns the number of possible expressions of a grammar up to max_depth with start symbol sym.
source
HerbSearch.crossover_swap_children_1 — Method
crossover_swap_children_1(parent1::RuleNode, parent2::RuleNode)
Performs a random crossover of two parents of type RuleNode. The subprograms are swapped and only one altered parent program is returned.
source
HerbSearch.crossover_swap_children_2 — Method
crossover_swap_children_2(parent1::RuleNode, parent2::RuleNode)
Performs a random crossover of two parents of type RuleNode. The subprograms are swapped and both altered parent programs are returned.
source
HerbSearch.decreasing_temperature — Method
decreasing_temperature(percentage::Real)
Returns a function that produces a temperature decreased by percentage%. This function is used by the Simmulated Annealing algorithm.
Arguments
  • percentage::Real: the percentage to decrease the temperature by.
source
HerbSearch.default_error_function — Method
default_error_function(old_error, output, expected_output)
Default error function for search_best.
- old_error         - The existing total error
 - output            - The actual output of the evaluator
-- expected_output   - The expected output for the example
The default function returns 0 if the outputs match and 1 otherwise.
HerbSearch.default_fitness — Method
default_fitness(program, results)
Defines the default fitness function taking the program and its results. Results are a vector of tuples, where each tuple is in the form Tuple{expected_output, actual_output}. As we are looking for individuals with the highest fitness function, the error is inverted. 
HerbSearch.enumerate_neighbours_propose — Method
enumerate_neighbours_propose(enumeration_depth::Int64)
The return function is a function that produces a list with all the subprograms with depth at most enumeration_depth.
Arguments
  • enumeration_depth::Int64: the maximum enumeration depth.
HerbSearch.get_best_program — Method
get_best_program(population::Array{RuleNode}, iter:: GeneticSearchIterator)::RuleNode
Returns the best program within the population with respect to the fitness function.
HerbSearch.get_bfs_enumerator — Function
get_bfs_enumerator(grammar::ContextFreeGrammar, max_depth::Int, max_size::Int, sym::Symbol, hole_heuristic::Function=heuristic_leftmost, derivation_heuristic::Function=(a,_) -> a)::ContextSensitivePriorityEnumerator
Returns a breadth-first search enumerator given a ContextFreeGrammar. Returns trees in the grammar in increasing order of size. 
HerbSearch.get_bfs_enumerator — Function
get_bfs_enumerator(grammar::ContextSensitiveGrammar, max_depth::Int, max_size::Int, sym::Symbol, hole_heuristic::Function=heuristic_leftmost, derivation_heuristic::Function=(a,_) -> a)::ContextSensitivePriorityEnumerator
Returns a breadth-first search enumerator given a ContextSensitiveGrammar. Returns trees in the grammar in increasing order of size. 
HerbSearch.get_dfs_enumerator — Function
get_dfs_enumerator(grammar::ContextFreeGrammar, max_depth::Int, max_size::Int, sym::Symbol, hole_heuristic::Function=heuristic_leftmost, derivation_heuristic::Function=(a,_) -> a)::ContextSensitivePriorityEnumerator
Returns a depth-first search enumerator given a ContextFreeGrammar. Returns trees in the grammar in decreasing order of size.
HerbSearch.get_dfs_enumerator — Function
get_dfs_enumerator(grammar::ContextSensitiveGrammar, max_depth::Int, max_size::Int, sym::Symbol, hole_heuristic::Function=heuristic_leftmost, derivation_heuristic::Function=(a,_) -> a)::ContextSensitivePriorityEnumerator
Returns a depth-first search enumerator given a ContextSensitiveGrammar. Returns trees in the grammar in decreasing order of size.
HerbSearch.get_genetic_enumerator — Method
get_genetic_enumerator(examples; fitness_function = HerbSearch.default_fitness, initial_population_size = 10, maximum_initial_population_depth = 3, mutation_probability = 0.1, cross_over = HerbSearch.crossover_swap_children_2, select_parents = HerbSearch.select_fitness_proportional_parents, evaluation_function::Function=HerbInterpret.test_with_input)
Returns a GeneticSearchIterator given a grammar. The iterator is fitted against the examples provided evaluated using the fitness function. All other arguments are hyperparameters for the genetic search procedure.
HerbSearch.get_mh_enumerator — Function
get_mh_enumerator(examples::AbstractArray{<:Example}, cost_function::Function, evaluation_function::Function=HerbInterpret.test_with_input)
Returns an enumerator that runs according to the Metropolis Hastings algorithm.
  • examples : array of examples
  • cost_function : cost function to evaluate the programs proposed
  • evaluation_function : evaluation function that evaluates the program generated and produces an output 
The propose function is randomfillpropose and the accept function is probabilistic. The temperature value of the algorithm remains constant over time. 
HerbSearch.get_most_likely_first_enumerator — Function
get_most_likely_first_enumerator(grammar::ContextFreeGrammar, max_depth::Int, max_size::Int, sym::Symbol, hole_heuristic::Function=heuristic_leftmost, derivation_heuristic::Function=(a,_) -> a)::ContextSensitivePriorityEnumerator
Returns an enumerator that enumerates expressions in the grammar in decreasing order of probability. Only use this function with probabilistic grammars.
HerbSearch.get_most_likely_first_enumerator — Function
get_most_likely_first_enumerator(grammar::ContextSensitiveGrammar, max_depth::Int, max_size::Int, sym::Symbol, hole_heuristic::Function=heuristic_leftmost, derivation_heuristic::Function=(a,_) -> a)::ContextSensitivePriorityEnumerator
Returns an enumerator that enumerates expressions in the grammar in decreasing order of probability. Only use this function with probabilistic grammars.
HerbSearch.get_sa_enumerator — Function
get_sa_enumerator(examples, cost_function, initial_temperature=1, temperature_decreasing_factor = 0.99, evaluation_function::Function=HerbInterpret.test_with_input)
Returns an enumerator that runs according to the Very Large Scale Neighbourhood Search algorithm.
  • examples : array of examples
  • cost_function : cost function to evaluate the programs proposed
  • initial_temperature : the starting temperature of the algorithm
  • temperature_decreasing_factor : the decreasing factor of the temperature of the time
  • evaluation_function : evaluation function that evaluates the program generated and produces an output 
The propose function is random_fill_propose (the same as for Metropolis Hastings). The accept function is probabilistic but takes into account the tempeerature too.
HerbSearch.get_vlsn_enumerator — Function
get_vlsn_enumerator(examples, cost_function, enumeration_depth = 2, evaluation_function::Function=HerbInterpret.test_with_input)
Returns an enumerator that runs according to the Very Large Scale Neighbourhood Search algorithm.
  • examples : array of examples
  • cost_function : cost function to evaluate the programs proposed
  • enumeration_depth : the enumeration depth to search for a best program at a time
  • evaluation_function : evaluation function that evaluates the program generated and produces an output 
The propose function consists of all possible programs of the given enumeration_depth. The accept function accepts the program with the lowest cost according to the cost_function. The temperature value of the algorithm remains constant over time. 
HerbSearch.heuristic_leftmost — Method
heuristic_leftmost(node::AbstractRuleNode, max_depth::Int)::Union{ExpandFailureReason, HoleReference}
Defines a heuristic over holes, where the left-most hole always gets considered first. Returns a HoleReference once a hole is found. This is the default option for enumerators.
HerbSearch.heuristic_random — Method
heuristic_random(node::AbstractRuleNode, max_depth::Int)::Union{ExpandFailureReason, HoleReference}
Defines a heuristic over holes, where random holes get chosen randomly using random exploration. Returns a HoleReference once a hole is found.
HerbSearch.heuristic_rightmost — Method
heuristic_rightmost(node::AbstractRuleNode, max_depth::Int)::Union{ExpandFailureReason, HoleReference}
Defines a heuristic over holes, where the right-most hole always gets considered first. Returns a HoleReference once a hole is found. 
HerbSearch.heuristic_smallest_domain — Method
heuristic_smallest_domain(node::AbstractRuleNode, max_depth::Int)::Union{ExpandFailureReason, HoleReference}
Defines a heuristic over all available holes in the unfinished AST, by considering the size of their respective domains. A domain here describes the number of possible derivations with respect to the constraints. Returns a HoleReference once a hole is found. 
HerbSearch.mean_squared_error — Method
mean_squared_error(results::AbstractVector{Tuple{<:Number,<:Number}})
Returns the mean squared error of results.
Arguments
  • results<:AbstractVector{<:Tuple{Number,Number}}: the vector of tuples, where each tuple is in the form Tuple{expected_output, actual_output}.
HerbSearch.misclassification — Method
misclassification(results::AbstractVector{Tuple{<:Number,<:Number}})
Returns the amount of misclassified examples, i.e. how many tuples with non-matching entries are there in results.
Arguments
  • results<:AbstractVector{<:Tuple{Number,Number}}: the vector of tuples, where each tuple is in the form Tuple{expected_output, actual_output}.
HerbSearch.most_likely_priority_function — Method
most_likely_priority_function(g::ContextSensitiveGrammar, tree::AbstractRuleNode, ::Union{Real, Tuple{Vararg{Real}}})
Calculates logit for all possible derivations for a node in a tree and returns them.
HerbSearch.mse_error_function — Method
mse_error_function(old_error, output, expected_output)
Mean squared error function for search_best.
- old_error         - The existing total error
+- expected_output   - The expected output for the example
The default function returns 0 if the outputs match and 1 otherwise.
source
HerbSearch.default_fitness — Method
default_fitness(program, results)
Defines the default fitness function taking the program and its results. Results are a vector of tuples, where each tuple is in the form Tuple{expected_output, actual_output}. As we are looking for individuals with the highest fitness function, the error is inverted. 
source
HerbSearch.enumerate_neighbours_propose — Method
enumerate_neighbours_propose(enumeration_depth::Int64)
The return function is a function that produces a list with all the subprograms with depth at most enumeration_depth.
Arguments
  • enumeration_depth::Int64: the maximum enumeration depth.
source
HerbSearch.get_best_program — Method
get_best_program(population::Array{RuleNode}, iter:: GeneticSearchIterator)::RuleNode
Returns the best program within the population with respect to the fitness function.
source
HerbSearch.get_bfs_enumerator — Function
get_bfs_enumerator(grammar::ContextFreeGrammar, max_depth::Int, max_size::Int, sym::Symbol, hole_heuristic::Function=heuristic_leftmost, derivation_heuristic::Function=(a,_) -> a)::ContextSensitivePriorityEnumerator
Returns a breadth-first search enumerator given a ContextFreeGrammar. Returns trees in the grammar in increasing order of size. 
source
HerbSearch.get_bfs_enumerator — Function
get_bfs_enumerator(grammar::ContextSensitiveGrammar, max_depth::Int, max_size::Int, sym::Symbol, hole_heuristic::Function=heuristic_leftmost, derivation_heuristic::Function=(a,_) -> a)::ContextSensitivePriorityEnumerator
Returns a breadth-first search enumerator given a ContextSensitiveGrammar. Returns trees in the grammar in increasing order of size. 
source
HerbSearch.get_dfs_enumerator — Function
get_dfs_enumerator(grammar::ContextFreeGrammar, max_depth::Int, max_size::Int, sym::Symbol, hole_heuristic::Function=heuristic_leftmost, derivation_heuristic::Function=(a,_) -> a)::ContextSensitivePriorityEnumerator
Returns a depth-first search enumerator given a ContextFreeGrammar. Returns trees in the grammar in decreasing order of size.
source
HerbSearch.get_dfs_enumerator — Function
get_dfs_enumerator(grammar::ContextSensitiveGrammar, max_depth::Int, max_size::Int, sym::Symbol, hole_heuristic::Function=heuristic_leftmost, derivation_heuristic::Function=(a,_) -> a)::ContextSensitivePriorityEnumerator
Returns a depth-first search enumerator given a ContextSensitiveGrammar. Returns trees in the grammar in decreasing order of size.
source
HerbSearch.get_genetic_enumerator — Method
get_genetic_enumerator(examples; fitness_function = HerbSearch.default_fitness, initial_population_size = 10, maximum_initial_population_depth = 3, mutation_probability = 0.1, cross_over = HerbSearch.crossover_swap_children_2, select_parents = HerbSearch.select_fitness_proportional_parents, evaluation_function::Function=HerbInterpret.test_with_input)
Returns a GeneticSearchIterator given a grammar. The iterator is fitted against the examples provided evaluated using the fitness function. All other arguments are hyperparameters for the genetic search procedure.
source
HerbSearch.get_mh_enumerator — Function
get_mh_enumerator(examples::AbstractArray{<:Example}, cost_function::Function, evaluation_function::Function=HerbInterpret.test_with_input)
Returns an enumerator that runs according to the Metropolis Hastings algorithm.
  • examples : array of examples
  • cost_function : cost function to evaluate the programs proposed
  • evaluation_function : evaluation function that evaluates the program generated and produces an output 
The propose function is randomfillpropose and the accept function is probabilistic. The temperature value of the algorithm remains constant over time. 
source
HerbSearch.get_most_likely_first_enumerator — Function
get_most_likely_first_enumerator(grammar::ContextFreeGrammar, max_depth::Int, max_size::Int, sym::Symbol, hole_heuristic::Function=heuristic_leftmost, derivation_heuristic::Function=(a,_) -> a)::ContextSensitivePriorityEnumerator
Returns an enumerator that enumerates expressions in the grammar in decreasing order of probability. Only use this function with probabilistic grammars.
source
HerbSearch.get_most_likely_first_enumerator — Function
get_most_likely_first_enumerator(grammar::ContextSensitiveGrammar, max_depth::Int, max_size::Int, sym::Symbol, hole_heuristic::Function=heuristic_leftmost, derivation_heuristic::Function=(a,_) -> a)::ContextSensitivePriorityEnumerator
Returns an enumerator that enumerates expressions in the grammar in decreasing order of probability. Only use this function with probabilistic grammars.
source
HerbSearch.get_sa_enumerator — Function
get_sa_enumerator(examples, cost_function, initial_temperature=1, temperature_decreasing_factor = 0.99, evaluation_function::Function=HerbInterpret.test_with_input)
Returns an enumerator that runs according to the Very Large Scale Neighbourhood Search algorithm.
  • examples : array of examples
  • cost_function : cost function to evaluate the programs proposed
  • initial_temperature : the starting temperature of the algorithm
  • temperature_decreasing_factor : the decreasing factor of the temperature of the time
  • evaluation_function : evaluation function that evaluates the program generated and produces an output 
The propose function is random_fill_propose (the same as for Metropolis Hastings). The accept function is probabilistic but takes into account the tempeerature too.
source
HerbSearch.get_vlsn_enumerator — Function
get_vlsn_enumerator(examples, cost_function, enumeration_depth = 2, evaluation_function::Function=HerbInterpret.test_with_input)
Returns an enumerator that runs according to the Very Large Scale Neighbourhood Search algorithm.
  • examples : array of examples
  • cost_function : cost function to evaluate the programs proposed
  • enumeration_depth : the enumeration depth to search for a best program at a time
  • evaluation_function : evaluation function that evaluates the program generated and produces an output 
The propose function consists of all possible programs of the given enumeration_depth. The accept function accepts the program with the lowest cost according to the cost_function. The temperature value of the algorithm remains constant over time. 
source
HerbSearch.heuristic_leftmost — Method
heuristic_leftmost(node::AbstractRuleNode, max_depth::Int)::Union{ExpandFailureReason, HoleReference}
Defines a heuristic over holes, where the left-most hole always gets considered first. Returns a HoleReference once a hole is found. This is the default option for enumerators.
source
HerbSearch.heuristic_random — Method
heuristic_random(node::AbstractRuleNode, max_depth::Int)::Union{ExpandFailureReason, HoleReference}
Defines a heuristic over holes, where random holes get chosen randomly using random exploration. Returns a HoleReference once a hole is found.
source
HerbSearch.heuristic_rightmost — Method
heuristic_rightmost(node::AbstractRuleNode, max_depth::Int)::Union{ExpandFailureReason, HoleReference}
Defines a heuristic over holes, where the right-most hole always gets considered first. Returns a HoleReference once a hole is found. 
source
HerbSearch.heuristic_smallest_domain — Method
heuristic_smallest_domain(node::AbstractRuleNode, max_depth::Int)::Union{ExpandFailureReason, HoleReference}
Defines a heuristic over all available holes in the unfinished AST, by considering the size of their respective domains. A domain here describes the number of possible derivations with respect to the constraints. Returns a HoleReference once a hole is found. 
source
HerbSearch.mean_squared_error — Method
mean_squared_error(results::AbstractVector{Tuple{<:Number,<:Number}})
Returns the mean squared error of results.
Arguments
  • results<:AbstractVector{<:Tuple{Number,Number}}: the vector of tuples, where each tuple is in the form Tuple{expected_output, actual_output}.
source
HerbSearch.misclassification — Method
misclassification(results::AbstractVector{Tuple{<:Number,<:Number}})
Returns the amount of misclassified examples, i.e. how many tuples with non-matching entries are there in results.
Arguments
  • results<:AbstractVector{<:Tuple{Number,Number}}: the vector of tuples, where each tuple is in the form Tuple{expected_output, actual_output}.
source
HerbSearch.most_likely_priority_function — Method
most_likely_priority_function(g::ContextSensitiveGrammar, tree::AbstractRuleNode, ::Union{Real, Tuple{Vararg{Real}}})
Calculates logit for all possible derivations for a node in a tree and returns them.
source
HerbSearch.mse_error_function — Method
mse_error_function(old_error, output, expected_output)
Mean squared error function for search_best.
- old_error         - The existing total error
 - output            - The actual output of the evaluator
-- expected_output   - The expected output for the example
The function build the mean squared error from output and expected_output`.
HerbSearch.mutate_random! — Function
mutate_random!(program::RuleNode, grammar::Grammar, max_depth::Int64 = 2)
Mutates the given program by inserting a randomly generated sub-program at a random location.
HerbSearch.probabilistic_accept — Method
probabilistic_accept(current_cost::Real, next_cost::Real, temperature::Real)
Probabilistically decides whether to accept the new program (next) based on the ratio of costs (smaller is better) between the previous and new program. Returns True if the new program is accepted, False otherwise.
Arguments
  • current_cost::Real: the cost of the current program.
  • next_cost::Real: the cost of the proposed program.
  • temperature::Real: the temperature; not used.
HerbSearch.probabilistic_accept_with_temperature — Method
probabilistic_accept_with_temperature(current_cost::Real, next_cost::Real, temperature::Real)
Returns true if the cost of the proposed program is smaller than the cost of the current program. Otherwise, returns true with the probability equal to: 
\[1 / (1 + exp(delta / temperature))\]
In any other case, returns false.
Arguments
  • current_cost::Real: the cost of the current program.
  • next_cost::Real: the cost of the proposed program.
  • temperature::Real: the temperature of the search.
HerbSearch.probabilistic_accept_with_temperature_fraction — Method
probabilistic_accept_with_temperature_fraction(current_cost::Real, program_to_consider_cost::Real, temperature::Real)
Probabilistically decides whether to accept the new program (next) based on the ratio of costs (smaller is better) between the previous and new program multiplied by the temperature. Returns True if the new program is accepted, False otherwise.
Arguments
  • current_cost::Real: the cost of the current program.
  • next_cost::Real: the cost of the proposed program.
  • temperature::Real: the current temperature 
HerbSearch.propagate_constraints — Function
function propagate_constraints(root::AbstractRuleNode, grammar::ContextSensitiveGrammar, local_constraints::Set{LocalConstraint}, max_holes::Int, filled_hole::Union{HoleReference, Nothing}=nothing)::Tuple{PropagateResult, Set{LocalConstraint}}
Propagates a set of local constraints recursively to all children of a given root node. As propagate_constraints gets often called when a hole was just filled, filled_hole helps keeping track to propagate the constraints to relevant nodes, e.g. children of filled_hole. max_holes makes sure that max_size of Base.iterate is not violated.  The function returns the PropagateResult and the set of relevant LocalConstraints.
HerbSearch.random_fill_propose — Method
random_fill_propose(current_program, neighbourhood_node_loc, grammar, max_depth, dict)
Returns a list with only one proposed, completely random, subprogram.
Arguments
  • current_program::RuleNode: the current program.
  • neighbourhood_node_loc::NodeLoc: the location of the program to replace.
  • grammar::Grammar: the grammar used to create programs.
  • max_depth::Int: the maximum depth of the resulting programs.
  • dmap::AbstractVector{Int} : the minimum possible depth to reach for each rule
  • dict::Dict{String, Any}: the dictionary with additional arguments; not used.
HerbSearch.search — Method
search(g::Grammar, problem::Problem, start::Symbol; evaluator::Function=test_with_input, enumerator::Function=get_bfs_enumerator, max_depth::Union{Int, Nothing}=nothing, max_size::Union{Int, Nothing}=nothing, max_time::Union{Int, Nothing}=nothing, max_enumerations::Union{Int, Nothing}=nothing, allow_evaluation_errors::Bool=false)::Union{Any, Nothing}
Searches for a program by calling search_rulenode starting from Symbol start guided by enumerator and Grammar trying to satisfy  the higher-order constraints in form of input/output examples defined in the Problem.  This is the heart of the Herb's search for satisfying programs. Returns the found program when the evaluation calculated using evaluator is successful.
HerbSearch.search_best — Method
search_best(g::Grammar, problem::Problem, start::Symbol; evaluator::Function=test_with_input, enumerator::Function=get_bfs_enumerator, error_function::Function=default_error_function, max_depth::Union{Int, Nothing}=nothing, max_size::Union{Int, Nothing}=nothing, max_time::Union{Int, Nothing}=nothing, max_enumerations::Union{Int, Nothing}=nothing, allow_evaluation_errors::Bool=false)::Tuple{Any, Real}
Searches the grammar for the program that satisfies the maximum number of examples in the problem. The evaluator should be a function that takes a SymbolTable, expression and a dictionary with      input variable assignments and returns the output of the expression.
- g                 - The grammar that defines the search space
+- expected_output   - The expected output for the example
The function build the mean squared error from output and expected_output`.
source
HerbSearch.mutate_random! — Function
mutate_random!(program::RuleNode, grammar::Grammar, max_depth::Int64 = 2)
Mutates the given program by inserting a randomly generated sub-program at a random location.
source
HerbSearch.probabilistic_accept — Method
probabilistic_accept(current_cost::Real, next_cost::Real, temperature::Real)
Probabilistically decides whether to accept the new program (next) based on the ratio of costs (smaller is better) between the previous and new program. Returns True if the new program is accepted, False otherwise.
Arguments
  • current_cost::Real: the cost of the current program.
  • next_cost::Real: the cost of the proposed program.
  • temperature::Real: the temperature; not used.
source
HerbSearch.probabilistic_accept_with_temperature — Method
probabilistic_accept_with_temperature(current_cost::Real, next_cost::Real, temperature::Real)
Returns true if the cost of the proposed program is smaller than the cost of the current program. Otherwise, returns true with the probability equal to: 
\[1 / (1 + exp(delta / temperature))\]
In any other case, returns false.
Arguments
  • current_cost::Real: the cost of the current program.
  • next_cost::Real: the cost of the proposed program.
  • temperature::Real: the temperature of the search.
source
HerbSearch.probabilistic_accept_with_temperature_fraction — Method
probabilistic_accept_with_temperature_fraction(current_cost::Real, program_to_consider_cost::Real, temperature::Real)
Probabilistically decides whether to accept the new program (next) based on the ratio of costs (smaller is better) between the previous and new program multiplied by the temperature. Returns True if the new program is accepted, False otherwise.
Arguments
  • current_cost::Real: the cost of the current program.
  • next_cost::Real: the cost of the proposed program.
  • temperature::Real: the current temperature 
source
HerbSearch.propagate_constraints — Function
function propagate_constraints(root::AbstractRuleNode, grammar::ContextSensitiveGrammar, local_constraints::Set{LocalConstraint}, max_holes::Int, filled_hole::Union{HoleReference, Nothing}=nothing)::Tuple{PropagateResult, Set{LocalConstraint}}
Propagates a set of local constraints recursively to all children of a given root node. As propagate_constraints gets often called when a hole was just filled, filled_hole helps keeping track to propagate the constraints to relevant nodes, e.g. children of filled_hole. max_holes makes sure that max_size of Base.iterate is not violated.  The function returns the PropagateResult and the set of relevant LocalConstraints.
source
HerbSearch.random_fill_propose — Method
random_fill_propose(current_program, neighbourhood_node_loc, grammar, max_depth, dict)
Returns a list with only one proposed, completely random, subprogram.
Arguments
  • current_program::RuleNode: the current program.
  • neighbourhood_node_loc::NodeLoc: the location of the program to replace.
  • grammar::Grammar: the grammar used to create programs.
  • max_depth::Int: the maximum depth of the resulting programs.
  • dmap::AbstractVector{Int} : the minimum possible depth to reach for each rule
  • dict::Dict{String, Any}: the dictionary with additional arguments; not used.
source
HerbSearch.search — Method
search(g::Grammar, problem::Problem, start::Symbol; evaluator::Function=test_with_input, enumerator::Function=get_bfs_enumerator, max_depth::Union{Int, Nothing}=nothing, max_size::Union{Int, Nothing}=nothing, max_time::Union{Int, Nothing}=nothing, max_enumerations::Union{Int, Nothing}=nothing, allow_evaluation_errors::Bool=false, mod::Module=Main)::Union{Any, Nothing}
Searches for a program by calling search_rulenode starting from Symbol start guided by enumerator and Grammar trying to satisfy  the higher-order constraints in form of input/output examples defined in the Problem.  This is the heart of the Herb's search for satisfying programs. Returns the found program when the evaluation calculated using evaluator is successful.
source
HerbSearch.search_best — Method
search_best(g::Grammar, problem::Problem, start::Symbol; evaluator::Function=test_with_input, enumerator::Function=get_bfs_enumerator, error_function::Function=default_error_function, max_depth::Union{Int, Nothing}=nothing, max_size::Union{Int, Nothing}=nothing, max_time::Union{Int, Nothing}=nothing, max_enumerations::Union{Int, Nothing}=nothing, allow_evaluation_errors::Bool=false, mod::Module=Main)::Tuple{Any, Real}
Searches the grammar for the program that satisfies the maximum number of examples in the problem. The evaluator should be a function that takes a SymbolTable, expression and a dictionary with      input variable assignments and returns the output of the expression.
- g                 - The grammar that defines the search space
 - problem           - The problem definition with IO examples
 - start             - The start symbol in the grammar
 - evaluator         - The evaluation function. Takes a SymbolTable, expression and a dictionary with 
@@ -14,7 +14,8 @@
 - max_depth         - The maximum depth of the search
 - max_time          - The maximum time allowed for the search in seconds
 - max_enumerations  - The maximum number of programs to enumerate and test
-- allow_evaluation_errors - Whether the search should crash if an exception is thrown in the evaluation
Returns a tuple with the best found program so far and the error.  Can be considerably slower than search due to having to evaluate each expression on each example.
HerbSearch.search_rulenode — Method
search_rulenode(g::Grammar, problem::Problem, start::Symbol; evaluator::Function=test_with_input, enumerator::Function=get_bfs_enumerator, max_depth::Union{Int, Nothing}=nothing, max_size::Union{Int, Nothing}=nothing, max_time::Union{Int, Nothing}=nothing, max_enumerations::Union{Int, Nothing}=nothing, allow_evaluation_errors::Bool=false)::Union{Tuple{RuleNode, Any}, Nothing}
Searches the grammar for the program that satisfies the maximum number of examples in the problem.
    - g                 - The grammar that defines the search space
+- allow_evaluation_errors - Whether the search should crash if an exception is thrown in the evaluation
+- mod               - A module containing definitions for the functions in the grammar that do not exist in Main
Returns a tuple with the best found program so far and the error.  Can be considerably slower than search due to having to evaluate each expression on each example.
source
HerbSearch.search_rulenode — Method
search_rulenode(g::Grammar, problem::Problem, start::Symbol; evaluator::Function=test_with_input, enumerator::Function=get_bfs_enumerator, max_depth::Union{Int, Nothing}=nothing, max_size::Union{Int, Nothing}=nothing, max_time::Union{Int, Nothing}=nothing, max_enumerations::Union{Int, Nothing}=nothing, allow_evaluation_errors::Bool=false, mod::Module=Main)::Union{Tuple{RuleNode, Any}, Nothing}
Searches the grammar for the program that satisfies the maximum number of examples in the problem.
    - g                 - The grammar that defines the search space
     - problem           - The problem definition with IO examples
     - start             - The start symbol in the grammar
     - evaluator         - The evaluation function. Takes a SymbolTable, expression and a dictionary with 
@@ -25,6 +26,7 @@
     - max_time          - The maximum time allowed for the search in seconds
     - max_enumerations  - The maximum number of programs to enumerate and test'
     - allow_evaluation_errors - Whether the search should crash if an exception is thrown in the evaluation
+    - mod               - A module containing definitions for the functions in the grammar that do not exist in Main
 Returns a tuple of the rulenode and the expression of the solution program once it has been found, 
-or nothing otherwise.
HerbSearch.select_chromosome — Method
select_chromosome(population::Array{RuleNode}, fitness_array::Array{<:Real})::RuleNode
Selects a chromosome (individual) from the population based on a fitness array. The function uses a fitness-proportionate selection strategy, often referred to as "roulette wheel" selection. Assumes fitness_array to be normalized already.
HerbSearch.select_fitness_proportional_parents — Method
select_fitness_proportional_parents(population::Array{RuleNode}, fitness_array::Array{<:Real})::Tuple{RuleNode,RuleNode}
Selects two parent chromosomes (individuals) from a population based on fitness-proportionate selection. The selected parents can be used for genetic crossover in the next steps of the algorithm.
StatsBase.sample — Function
sample(::Type{NodeLoc}, root::RuleNode, typ::Symbol, grammar::Grammar)
Uniformly selects a random node in the tree of a given type, specified using its parent such that the subtree can be replaced. Returns a NodeLoc.
StatsBase.sample — Function
sample(::Type{NodeLoc}, root::RuleNode, maxdepth::Int=typemax(Int))
Uniformly selects a random node in the tree no deeper than maxdepth using reservoir sampling. Returns a NodeLoc that specifies the location using its parent so that the subtree can be replaced.
StatsBase.sample — Function
sample(root::RuleNode, typ::Symbol, grammar::Grammar, maxdepth::Int=typemax(Int))
Uniformly samples a random node from the tree limited to maxdepth.
StatsBase.sample — Function
sample(root::RuleNode, typ::Symbol, grammar::Grammar,
-                      maxdepth::Int=typemax(Int))
Uniformly selects a random node of the given return type typ limited by maxdepth.
The HerbSearch package takes care of all operations related to searching for the desired program. This includes
  • the functionality to sample a certain program given a grammar,
  • the implementation of several heuristic functions,
  • searching for a program that satisfies the specification, and
  • implementations of several search algorithms in terms of how they enumerate the search space
    • Breadth-First Search 
    • Depth-First Search 
    • Metropolis Hastings 
    • Very Large Scale Neighbourhood Search 
    • Simulated Annealing
    • Genetic Search
Index

+or nothing otherwise.
source
HerbSearch.select_chromosome — Method
select_chromosome(population::Array{RuleNode}, fitness_array::Array{<:Real})::RuleNode

Selects a chromosome (individual) from the population based on a fitness array. The function uses a fitness-proportionate selection strategy, often referred to as "roulette wheel" selection. Assumes fitness_array to be normalized already.

source
HerbSearch.select_fitness_proportional_parents — Method
select_fitness_proportional_parents(population::Array{RuleNode}, fitness_array::Array{<:Real})::Tuple{RuleNode,RuleNode}

Selects two parent chromosomes (individuals) from a population based on fitness-proportionate selection. The selected parents can be used for genetic crossover in the next steps of the algorithm.

source
StatsBase.sample — Function
sample(root::RuleNode, typ::Symbol, grammar::Grammar, maxdepth::Int=typemax(Int))

Uniformly samples a random node from the tree limited to maxdepth.

source
StatsBase.sample — Function
sample(::Type{NodeLoc}, root::RuleNode, typ::Symbol, grammar::Grammar)

Uniformly selects a random node in the tree of a given type, specified using its parent such that the subtree can be replaced. Returns a NodeLoc.

source
StatsBase.sample — Function
sample(::Type{NodeLoc}, root::RuleNode, maxdepth::Int=typemax(Int))

Uniformly selects a random node in the tree no deeper than maxdepth using reservoir sampling. Returns a NodeLoc that specifies the location using its parent so that the subtree can be replaced.

source
StatsBase.sample — Function
sample(root::RuleNode, typ::Symbol, grammar::Grammar,
+                      maxdepth::Int=typemax(Int))

Uniformly selects a random node of the given return type typ limited by maxdepth.

source

The HerbSearch package takes care of all operations related to searching for the desired program. This includes

  • the functionality to sample a certain program given a grammar,
  • the implementation of several heuristic functions,
  • searching for a program that satisfies the specification, and
  • implementations of several search algorithms in terms of how they enumerate the search space
    • Breadth-First Search
    • Depth-First Search
    • Metropolis Hastings
    • Very Large Scale Neighbourhood Search
    • Simulated Annealing
    • Genetic Search

Index

diff --git a/dev/concepts/index.html b/dev/concepts/index.html index 6eca0d5..f72c292 100644 --- a/dev/concepts/index.html +++ b/dev/concepts/index.html @@ -1,2 +1,2 @@ -Architecture and core concepts · Herb.jl
Edit on GitHub
+Architecture and core concepts · Herb.jl
Edit on GitHub
diff --git a/dev/get_started/index.html b/dev/get_started/index.html index e3542e9..57fba56 100644 --- a/dev/get_started/index.html +++ b/dev/get_started/index.html @@ -20,4 +20,4 @@ problem = Problem([IOExample(Dict(:x => x), 2x+1) for x ∈ 1:5]) solution = search(g, problem, :Number, max_depth=3) -test_with_input(SymbolTable(g), solution, Dict(:x => 6)) +test_with_input(SymbolTable(g), solution, Dict(:x => 6)) diff --git a/dev/index.html b/dev/index.html index 18b2e57..9fdce96 100644 --- a/dev/index.html +++ b/dev/index.html @@ -1,2 +1,2 @@ -Herb.jl · Herb.jl
Edit on GitHub

Herb.jl

A library for defining and efficiently solving program synthesis tasks in Julia.

Why Herb.jl?

When writing research software we almost always investigate highly specific properties or algorithms of our domain, leading to us building the tools from scratch over and over again. The very same holds for the field of program synthesis: Tools are hard to run, benchmarks are hard to get and prepare, and its hard to adapt our existing code to a novel idea.

Herb.jl will take care of this for you and helps you defining, solving and extending your program synthesis problems.

Herb.jl provides...

  • a unified and universal framework for program synthesis
  • Herb.jl allows you to describe all sorts of program synthesis problems using context-free grammars
  • a number of state-of-the-art benchmarks and solvers already implemented and usable out-of-the-box

Herb.jl's sub-packages provide fast and easily extendable implementations of

  • various static and dynamic search strategies,
  • learning search strategies, sampling techniques and more,
  • constraint formulation and propagation,
  • easy grammar formulation and usage,
  • wide-range of usable program interpreters and languages + the possibility to use your own, and
  • efficient data formulation.

Why Julia?

Julia is a perfect fit for program synthesis due to numerous reasons. Starting from scientific reasons like speed of execution and composability over to practical reasons like speed of writing Julia code. For a full ode on why to use Julia, please see the WhyJulia manifesto.

Sub-Modules

Herb's functionality is distributed among several sub-packages:

Basics

Advanced content

+Herb.jl · Herb.jl
Edit on GitHub

Herb.jl

A library for defining and efficiently solving program synthesis tasks in Julia.

Why Herb.jl?

When writing research software we almost always investigate highly specific properties or algorithms of our domain, leading to us building the tools from scratch over and over again. The very same holds for the field of program synthesis: Tools are hard to run, benchmarks are hard to get and prepare, and its hard to adapt our existing code to a novel idea.

Herb.jl will take care of this for you and helps you defining, solving and extending your program synthesis problems.

Herb.jl provides...

  • a unified and universal framework for program synthesis
  • Herb.jl allows you to describe all sorts of program synthesis problems using context-free grammars
  • a number of state-of-the-art benchmarks and solvers already implemented and usable out-of-the-box

Herb.jl's sub-packages provide fast and easily extendable implementations of

  • various static and dynamic search strategies,
  • learning search strategies, sampling techniques and more,
  • constraint formulation and propagation,
  • easy grammar formulation and usage,
  • wide-range of usable program interpreters and languages + the possibility to use your own, and
  • efficient data formulation.

Why Julia?

Julia is a perfect fit for program synthesis due to numerous reasons. Starting from scientific reasons like speed of execution and composability over to practical reasons like speed of writing Julia code. For a full ode on why to use Julia, please see the WhyJulia manifesto.

Sub-Modules

Herb's functionality is distributed among several sub-packages:

Basics

Advanced content

diff --git a/dev/install/index.html b/dev/install/index.html index baed360..f0c7f15 100644 --- a/dev/install/index.html +++ b/dev/install/index.html @@ -1,4 +1,4 @@ Installation Guide · Herb.jl
Edit on GitHub

Installation Guide

Before installing Herb.jl, ensure that you have a running Julia distribution installed (Julia version 1.7 and above were tested).

Thanks to Julia's package management, installing Herb.jl is very straighforward. Activate the default Julia REPL using

julia

or from within one of your projects using

julia --project=.

From the Julia REPL run 

]
 add Herb

or instead running

import Pkg
-Pkg.add("Herb")

which will both install all dependencies automatically.

For later convenience we can also add the respective dependencies to our project, so that we do not have to write Herb.HerbGrammar every time.

] add HerbConstraints HerbCore HerbData HerbInterpret HerbGrammar HerbSearch

And just like this you are done! Welcome to Herb.jl!

+Pkg.add("Herb")

which will both install all dependencies automatically.

For later convenience we can also add the respective dependencies to our project, so that we do not have to write Herb.HerbGrammar every time.

] add HerbConstraints HerbCore HerbData HerbInterpret HerbGrammar HerbSearch

And just like this you are done! Welcome to Herb.jl!

diff --git a/dev/search/index.html b/dev/search/index.html index 616726d..c1081f3 100644 --- a/dev/search/index.html +++ b/dev/search/index.html @@ -1,2 +1,2 @@ -Search · Herb.jl

Loading search...

    +Search · Herb.jl

    Loading search...

      diff --git a/dev/search_index.js b/dev/search_index.js index 0c4d981..e850075 100644 --- a/dev/search_index.js +++ b/dev/search_index.js @@ -1,3 +1,3 @@ var documenterSearchIndex = {"docs": -[{"location":"install/#Installation-Guide","page":"Installation Guide","title":"Installation Guide","text":"","category":"section"},{"location":"install/","page":"Installation Guide","title":"Installation Guide","text":"Before installing Herb.jl, ensure that you have a running Julia distribution installed (Julia version 1.7 and above were tested). ","category":"page"},{"location":"install/","page":"Installation Guide","title":"Installation Guide","text":"Thanks to Julia's package management, installing Herb.jl is very straighforward. Activate the default Julia REPL using","category":"page"},{"location":"install/","page":"Installation Guide","title":"Installation Guide","text":"julia","category":"page"},{"location":"install/","page":"Installation Guide","title":"Installation Guide","text":"or from within one of your projects using","category":"page"},{"location":"install/","page":"Installation Guide","title":"Installation Guide","text":"julia --project=.","category":"page"},{"location":"install/","page":"Installation Guide","title":"Installation Guide","text":"From the Julia REPL run ","category":"page"},{"location":"install/","page":"Installation Guide","title":"Installation Guide","text":"]\nadd Herb","category":"page"},{"location":"install/","page":"Installation Guide","title":"Installation Guide","text":"or instead running","category":"page"},{"location":"install/","page":"Installation Guide","title":"Installation Guide","text":"import Pkg\nPkg.add(\"Herb\")","category":"page"},{"location":"install/","page":"Installation Guide","title":"Installation Guide","text":"which will both install all dependencies automatically.","category":"page"},{"location":"install/","page":"Installation Guide","title":"Installation Guide","text":"For later convenience we can also add the respective dependencies to our project, so that we do not have to write Herb.HerbGrammar every time.","category":"page"},{"location":"install/","page":"Installation Guide","title":"Installation Guide","text":"] add HerbConstraints HerbCore HerbData HerbInterpret HerbGrammar HerbSearch","category":"page"},{"location":"install/","page":"Installation Guide","title":"Installation Guide","text":"And just like this you are done! Welcome to Herb.jl!","category":"page"},{"location":"concepts/#Architecture-and-core-concepts","page":"Architecture and core concepts","title":"Architecture and core concepts","text":"","category":"section"},{"location":"HerbSearch/#HerbSearch_docs","page":"HerbSearch.jl","title":"HerbSearch.jl Documentation","text":"","category":"section"},{"location":"HerbSearch/","page":"HerbSearch.jl","title":"HerbSearch.jl","text":"CurrentModule=HerbSearch","category":"page"},{"location":"HerbSearch/","page":"HerbSearch.jl","title":"HerbSearch.jl","text":"Modules = [HerbSearch]\nOrder = [:type, :const, :macro, :function]","category":"page"},{"location":"HerbSearch/#HerbSearch.ContextSensitivePriorityEnumerator","page":"HerbSearch.jl","title":"HerbSearch.ContextSensitivePriorityEnumerator","text":"mutable struct ContextSensitivePriorityEnumerator <: ExpressionIterator\n\nEnumerates a context-free grammar starting at Symbol sym with respect to the grammar up to a given depth and a given size. The exploration is done using the given priority function for derivations, and the expand function for discovered nodes.\n\n\n\n\n\n","category":"type"},{"location":"HerbSearch/#HerbSearch.ExpandFailureReason","page":"HerbSearch.jl","title":"HerbSearch.ExpandFailureReason","text":"@enum ExpandFailureReason limit_reached=1 already_complete=2\n\nRepresentation of the different reasons why expanding a partial tree failed. Currently, there are two possible causes of the expansion failing:\n\nlimit_reached: The depth limit or the size limit of the partial tree would be violated by the expansion\nalready_complete: There is no hole left in the tree, so nothing can be expanded.\n\n\n\n\n\n","category":"type"},{"location":"HerbSearch/#HerbSearch.ExpressionIterator","page":"HerbSearch.jl","title":"HerbSearch.ExpressionIterator","text":"abstract type ExpressionIterator\n\nAbstract super-type for all possible enumerators.\n\n\n\n\n\n","category":"type"},{"location":"HerbSearch/#HerbSearch.GeneticSearchIterator","page":"HerbSearch.jl","title":"HerbSearch.GeneticSearchIterator","text":"GeneticSearchIterator{FitnessFunction,CrossOverFunction,MutationFunction,SelectParentsFunction,EvaluationFunction} <: ExpressionIterator\n\nDefines an ExpressionIterator using genetic search. \n\nConsists of:\n\ngrammar::ContextSensitiveGrammar: the grammar to search over\nexamples::Vector{<:Example}: a collection of examples defining the specification \nfitness::FitnessFunction: assigns a numerical value (fitness score) to each individual based on how closely it meets the desired objective\ncross_over::CrossOverFunction: combines the program from two parent individuals to create one or more offspring individuals\nmutation!::MutationFunction: mutates the program of an invididual\nselect_parents::SelectParentsFunction: selects two parents for the crossover\nevaluation_function::EvaluationFunction: interpreter to evaluate the individual programs\nstart_symbol::Symbol: defines the start symbol from which the search should be started\npopulation_size::Int64: number of inviduals in the population\nmutation_probability::Float64: probability of mutation for each individual\nmaximum_initial_population_depth::Int64: maximum depth of trees when population is initialized \n\nend\n\n\n\n\n\n","category":"type"},{"location":"HerbSearch/#HerbSearch.PriorityQueueItem","page":"HerbSearch.jl","title":"HerbSearch.PriorityQueueItem","text":"struct PriorityQueueItem\n\nRepresents an item in the priority enumerator priority queue. An item contains of:\n\ntree: A partial AST\nsize: The size of the tree. This is a cached value which prevents having to traverse the entire tree each time the size is needed.\nconstraints: The local constraints that apply to this tree. These constraints are enforced each time the tree is modified.\n\n\n\n\n\n","category":"type"},{"location":"HerbSearch/#HerbSearch.PriorityQueueItem-Tuple{AbstractRuleNode, Int64}","page":"HerbSearch.jl","title":"HerbSearch.PriorityQueueItem","text":"PriorityQueueItem(tree::AbstractRuleNode, size::Int)\n\nConstructs PriorityQueueItem given only a tree and the size, but no constraints.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.PropagateResult","page":"HerbSearch.jl","title":"HerbSearch.PropagateResult","text":"@enum PropagateResult tree_complete=1 tree_incomplete=2 tree_infeasible=3\n\nRepresentation of the possible results of a constraint propagation. At the moment there are three possible outcomes:\n\ntree_complete: The propagation was applied successfully and the tree does not contain any holes anymore. Thus no constraints can be applied anymore.\ntree_incomplete: The propagation was applied successfully and the tree does contain more holes. Thus more constraints may be applied to further prune the respective domains.\ntree_infeasible: The propagation was succesful, but there are holes with empty domains. Hence, the tree is now infeasible.\n\n\n\n\n\n","category":"type"},{"location":"HerbSearch/#HerbSearch.StochasticSearchEnumerator","page":"HerbSearch.jl","title":"HerbSearch.StochasticSearchEnumerator","text":"Base.@kwdef struct StochasticSearchEnumerator <: ExpressionIterator\n\nA unified struct for the algorithms Metropolis Hastings, Very Large Scale Neighbourhood and Simulated Annealing. Each algorithm implements neighbourhood propose accept and temperature functions. Below the signiture of all this function is shown\n\nSignatures\n\n\n\nReturns a node location from the program that is the neighbourhood. It can also return other information using dict\n\nneighbourhood(program::RuleNode, grammar::Grammar) -> (loc::NodeLocation, dict::Dict)\n\n\n\nProposes a list of programs using the location provided by neighbourhood and the dict.\n\npropose(current_program, loc::NodeLocation, grammar::Grammar, max_depth::Int64, dict::Dict) -> Iter[RuleNode]\n\n\n\nBased on the current program and possible cost and temperature it accepts the program or not. Usually we would always want to accept better programs but we might get stuck if we do so. That is why some implementations of the accept function accept with a probability costs that are worse. cost means how different are the outcomes of the program compared to the correct outcomes. The lower the cost the better the program performs on the examples. The cost is provided by the cost_function\n\naccept(current_cost::Real, possible_cost::Real, temperature::Real) -> Bool\n\n\n\nReturns the new temperature based on the previous temperature. Higher the temperature means that the algorithm will explore more.\n\ntemperature(previous_temperature::Real) -> Real\n\n\n\nReturns the cost of the current program. It receives a list of tuples (expected, found) and gives back a cost.\n\ncost_function(outcomes::Tuple{<:Number,<:Number}[]) -> Real\n\n\n\nFields\n\ngrammar::ContextSensitiveGrammar grammar that the algorithm uses\nmax_depth::Int64 = 5 maximum depth of the program to generate\nexamples::Vector{Example} example used to check the program\nneighbourhood::Function \npropose::Function\naccept::Function\ntemperature::Function\ncost_function::Function\nstart_symbol::Symbol the start symbol of the algorithm :Real or :Int\ninitial_temperature::Real = 1 \nevaluation_function::Function that evaluates the julia expressions\n\nAn iterator over all possible expressions of a grammar up to max_depth with start symbol sym.\n\n\n\n\n\n","category":"type"},{"location":"HerbSearch/#Base.iterate-Tuple{ContextSensitivePriorityEnumerator, DataStructures.PriorityQueue}","page":"HerbSearch.jl","title":"Base.iterate","text":"Base.iterate(iter::ContextSensitivePriorityEnumerator, pq::DataStructures.PriorityQueue)\n\nDescribes the iteration for a given ContextSensitivePriorityEnumerator and a PriorityQueue over the grammar without enqueueing new items to the priority queue. Recursively returns the result for the priority queue.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#Base.iterate-Tuple{ContextSensitivePriorityEnumerator}","page":"HerbSearch.jl","title":"Base.iterate","text":"Base.iterate(iter::ContextSensitivePriorityEnumerator)\n\nDescribes the iteration for a given ContextSensitivePriorityEnumerator over the grammar. The iteration constructs a PriorityQueue first and then prunes it propagating the active constraints. Recursively returns the result for the priority queue.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#Base.iterate-Tuple{HerbSearch.GeneticSearchIterator, HerbSearch.GeneticIteratorState}","page":"HerbSearch.jl","title":"Base.iterate","text":"Base.iterate(iter::GeneticSearchIterator, current_state::GeneticIteratorState)\n\nIterates the search space using a genetic algorithm. Takes the iterator and the current state to mutate and crossover random inviduals. Returns the best program-so-far and the state of the iterator.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#Base.iterate-Tuple{HerbSearch.GeneticSearchIterator}","page":"HerbSearch.jl","title":"Base.iterate","text":"Base.iterate(iter::GeneticSearchIterator)\n\nIterates the search space using a genetic algorithm. First generates a population sampling random programs. Returns the best program-so-far, and the state of the iterator.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#Base.iterate-Tuple{HerbSearch.StochasticSearchEnumerator, HerbSearch.IteratorState}","page":"HerbSearch.jl","title":"Base.iterate","text":"Base.iterate(iter::StochasticSearchEnumerator, current_state::IteratorState)\n\nThe algorithm that constructs the iterator of StochasticSearchEnumerator. It has the following structure:\n\nget a random node location -> location,dict = neighbourhood(current_program)\ncall propose on the current program getting a list of possbile replacements in the node location \niterate through all the possible replacements and perform the replacement in the current program \naccept the new program by modifying the next_program or reject the new program\nreturn the new next_program\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#Base.rand","page":"HerbSearch.jl","title":"Base.rand","text":"rand(::Type{RuleNode}, grammar::Grammar, typ::Symbol, max_depth::Int=10)\n\nGenerates a random RuleNode of return type typ and maximum depth max_depth.\n\n\n\n\n\n","category":"function"},{"location":"HerbSearch/#Base.rand-2","page":"HerbSearch.jl","title":"Base.rand","text":"rand(::Type{RuleNode}, grammar::Grammar, typ::Symbol, dmap::AbstractVector{Int}, max_depth::Int=10)\n\nGenerates a random RuleNode, i.e. an expression tree, of root type typ and maximum depth max_depth guided by a depth map dmap if possible.\n\n\n\n\n\n","category":"function"},{"location":"HerbSearch/#Base.rand-3","page":"HerbSearch.jl","title":"Base.rand","text":"rand(::Type{RuleNode}, grammar::Grammar, max_depth::Int=10)\n\nGenerates a random RuleNode of arbitrary type and maximum depth max_depth.\n\n\n\n\n\n","category":"function"},{"location":"HerbSearch/#HerbSearch._expand-Tuple{Hole, ContextSensitiveGrammar, Int64, Int64, GrammarContext, Function, Function}","page":"HerbSearch.jl","title":"HerbSearch._expand","text":"_expand(node::Hole, grammar::ContextSensitiveGrammar, ::Int, max_holes::Int, context::GrammarContext, hole_heuristic::Function, derivation_heuristic::Function)::Union{ExpandFailureReason, Vector{TreeConstraints}}\n\nExpands a given hole that was found in _expand using the given derivation heuristic. Returns the list of discovered nodes in that order and with their respective constraints.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch._expand-Tuple{RuleNode, ContextSensitiveGrammar, Int64, Int64, GrammarContext, Function, Function}","page":"HerbSearch.jl","title":"HerbSearch._expand","text":"_expand(root::RuleNode, grammar::ContextSensitiveGrammar, max_depth::Int, max_holes::Int, context::GrammarContext, hole_heuristic::Function, derivation_heuristic::Function)::Union{ExpandFailureReason, Vector{TreeConstraints}}\n\nRecursive expand function used in multiple enumeration techniques. Expands one hole/undefined leaf of the given RuleNode tree found using the given hole heuristic. If the expansion was successful, returns a list of new trees and a list of lists of hole locations, corresponding to the holes of each newly expanded tree. Returns nothing if tree is already complete (i.e. contains no holes). Returns an empty list if the tree is partial (i.e. contains holes), but they could not be expanded because of the depth limit.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch._find_next_complete_tree-Tuple{ContextSensitiveGrammar, Int64, Int64, Function, Function, DataStructures.PriorityQueue}","page":"HerbSearch.jl","title":"HerbSearch._find_next_complete_tree","text":"_find_next_complete_tree(grammar::ContextSensitiveGrammar, max_depth::Int, max_size::Int, priority_function::Function, expand_function::Function, pq::PriorityQueue)::Union{Tuple{RuleNode, PriorityQueue}, Nothing}\n\nTakes a priority queue and returns the smallest AST from the grammar it can obtain from the queue or by (repeatedly) expanding trees that are in the queue. Returns nothing if there are no trees left within the depth limit.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.best_accept-Tuple{Real, Real, Any}","page":"HerbSearch.jl","title":"HerbSearch.best_accept","text":"best_accept(current_cost::Real, next_cost::Real, temperature)\n\nReturns true if the cost of the proposed program is smaller than the cost of the current program. Otherwise, returns false.\n\nArguments\n\ncurrent_cost::Real: the cost of the current program.\nnext_cost::Real: the cost of the proposed program.\ntemperature::Real: the temperature; not used.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.calculate_cost-Tuple{RuleNode, Function, AbstractVector{Example}, Grammar, Function}","page":"HerbSearch.jl","title":"HerbSearch.calculate_cost","text":"calculate_cost(program::RuleNode, cost_function::Function, examples::AbstractVector{Example}, grammar::Grammar, evaluation_function::Function)\n\nReturns the cost of the program using the examples and the cost_function. It first convert the program to an expression and evaluates it on all the examples using HerbInterpret.evaluate_program.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.const_temperature-Tuple{Any}","page":"HerbSearch.jl","title":"HerbSearch.const_temperature","text":"const_temperature(current_temperature)\n\nReturns the temperature unchanged. This function is used by Metropolis Hastings and Very Large Neighbourhood Search algorithms.\n\nArguments\n\ncurrent_temperature::Real: the current temperature of the search.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.constructNeighbourhood-Tuple{RuleNode, Grammar}","page":"HerbSearch.jl","title":"HerbSearch.constructNeighbourhood","text":"constructNeighbourhood(current_program::RuleNode, grammar::Grammar)\n\nThe neighbourhood node location is chosen at random. The dictionary is nothing.\n\nArguments\n\ncurrent_program::RuleNode: the current program.\ngrammar::Grammar: the grammar.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.constructNeighbourhoodRuleSubset-Tuple{RuleNode, Grammar}","page":"HerbSearch.jl","title":"HerbSearch.constructNeighbourhoodRuleSubset","text":"constructNeighbourhoodRuleSubset(current_program::RuleNode, grammar::Grammar)\n\nThe neighbourhood node location is chosen at random. The dictionary is contains one entry with key \"rule_subset\" and value of type Vector{Any} being a random subset of grammar rules.\n\nArguments\n\ncurrent_program::RuleNode: the current program.\ngrammar::Grammar: the grammar.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.count_expressions-Tuple{ExpressionIterator}","page":"HerbSearch.jl","title":"HerbSearch.count_expressions","text":"count_expressions(iter::ExpressionIterator)\n\nCounts and returns the number of possible expressions in the expression iterator. The Iterator is not modified.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.count_expressions-Tuple{Grammar, Int64, Int64, Symbol}","page":"HerbSearch.jl","title":"HerbSearch.count_expressions","text":"count_expressions(grammar::Grammar, max_depth::Int, max_size::Int, sym::Symbol)\n\nCounts and returns the number of possible expressions of a grammar up to max_depth with start symbol sym.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.crossover_swap_children_1-Tuple{RuleNode, RuleNode}","page":"HerbSearch.jl","title":"HerbSearch.crossover_swap_children_1","text":"crossover_swap_children_1(parent1::RuleNode, parent2::RuleNode)\n\nPerforms a random crossover of two parents of type RuleNode. The subprograms are swapped and only one altered parent program is returned.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.crossover_swap_children_2-Tuple{RuleNode, RuleNode}","page":"HerbSearch.jl","title":"HerbSearch.crossover_swap_children_2","text":"crossover_swap_children_2(parent1::RuleNode, parent2::RuleNode)\n\nPerforms a random crossover of two parents of type RuleNode. The subprograms are swapped and both altered parent programs are returned.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.decreasing_temperature-Tuple{Real}","page":"HerbSearch.jl","title":"HerbSearch.decreasing_temperature","text":"decreasing_temperature(percentage::Real)\n\nReturns a function that produces a temperature decreased by percentage%. This function is used by the Simmulated Annealing algorithm.\n\nArguments\n\npercentage::Real: the percentage to decrease the temperature by.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.default_error_function-Tuple{Any, Any, Any}","page":"HerbSearch.jl","title":"HerbSearch.default_error_function","text":"default_error_function(old_error, output, expected_output)\n\nDefault error function for search_best.\n\n- old_error - The existing total error\n- output - The actual output of the evaluator\n- expected_output - The expected output for the example\n\nThe default function returns 0 if the outputs match and 1 otherwise.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.default_fitness-Tuple{Any, Any}","page":"HerbSearch.jl","title":"HerbSearch.default_fitness","text":"default_fitness(program, results)\n\nDefines the default fitness function taking the program and its results. Results are a vector of tuples, where each tuple is in the form Tuple{expected_output, actual_output}. As we are looking for individuals with the highest fitness function, the error is inverted. \n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.enumerate_neighbours_propose-Tuple{Int64}","page":"HerbSearch.jl","title":"HerbSearch.enumerate_neighbours_propose","text":"enumerate_neighbours_propose(enumeration_depth::Int64)\n\nThe return function is a function that produces a list with all the subprograms with depth at most enumeration_depth.\n\nArguments\n\nenumeration_depth::Int64: the maximum enumeration depth.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.get_best_program-Tuple{Array{RuleNode}, HerbSearch.GeneticSearchIterator}","page":"HerbSearch.jl","title":"HerbSearch.get_best_program","text":"get_best_program(population::Array{RuleNode}, iter:: GeneticSearchIterator)::RuleNode\n\nReturns the best program within the population with respect to the fitness function.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.get_bfs_enumerator","page":"HerbSearch.jl","title":"HerbSearch.get_bfs_enumerator","text":"get_bfs_enumerator(grammar::ContextFreeGrammar, max_depth::Int, max_size::Int, sym::Symbol, hole_heuristic::Function=heuristic_leftmost, derivation_heuristic::Function=(a,_) -> a)::ContextSensitivePriorityEnumerator\n\nReturns a breadth-first search enumerator given a ContextFreeGrammar. Returns trees in the grammar in increasing order of size. \n\n\n\n\n\n","category":"function"},{"location":"HerbSearch/#HerbSearch.get_bfs_enumerator-2","page":"HerbSearch.jl","title":"HerbSearch.get_bfs_enumerator","text":"get_bfs_enumerator(grammar::ContextSensitiveGrammar, max_depth::Int, max_size::Int, sym::Symbol, hole_heuristic::Function=heuristic_leftmost, derivation_heuristic::Function=(a,_) -> a)::ContextSensitivePriorityEnumerator\n\nReturns a breadth-first search enumerator given a ContextSensitiveGrammar. Returns trees in the grammar in increasing order of size. \n\n\n\n\n\n","category":"function"},{"location":"HerbSearch/#HerbSearch.get_dfs_enumerator","page":"HerbSearch.jl","title":"HerbSearch.get_dfs_enumerator","text":"get_dfs_enumerator(grammar::ContextFreeGrammar, max_depth::Int, max_size::Int, sym::Symbol, hole_heuristic::Function=heuristic_leftmost, derivation_heuristic::Function=(a,_) -> a)::ContextSensitivePriorityEnumerator\n\nReturns a depth-first search enumerator given a ContextFreeGrammar. Returns trees in the grammar in decreasing order of size.\n\n\n\n\n\n","category":"function"},{"location":"HerbSearch/#HerbSearch.get_dfs_enumerator-2","page":"HerbSearch.jl","title":"HerbSearch.get_dfs_enumerator","text":"get_dfs_enumerator(grammar::ContextSensitiveGrammar, max_depth::Int, max_size::Int, sym::Symbol, hole_heuristic::Function=heuristic_leftmost, derivation_heuristic::Function=(a,_) -> a)::ContextSensitivePriorityEnumerator\n\nReturns a depth-first search enumerator given a ContextSensitiveGrammar. Returns trees in the grammar in decreasing order of size.\n\n\n\n\n\n","category":"function"},{"location":"HerbSearch/#HerbSearch.get_genetic_enumerator-Tuple{Any}","page":"HerbSearch.jl","title":"HerbSearch.get_genetic_enumerator","text":"get_genetic_enumerator(examples; fitness_function = HerbSearch.default_fitness, initial_population_size = 10, maximum_initial_population_depth = 3, mutation_probability = 0.1, cross_over = HerbSearch.crossover_swap_children_2, select_parents = HerbSearch.select_fitness_proportional_parents, evaluation_function::Function=HerbInterpret.test_with_input)\n\nReturns a GeneticSearchIterator given a grammar. The iterator is fitted against the examples provided evaluated using the fitness function. All other arguments are hyperparameters for the genetic search procedure.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.get_mh_enumerator","page":"HerbSearch.jl","title":"HerbSearch.get_mh_enumerator","text":"get_mh_enumerator(examples::AbstractArray{<:Example}, cost_function::Function, evaluation_function::Function=HerbInterpret.test_with_input)\n\nReturns an enumerator that runs according to the Metropolis Hastings algorithm.\n\nexamples : array of examples\ncost_function : cost function to evaluate the programs proposed\nevaluation_function : evaluation function that evaluates the program generated and produces an output \n\nThe propose function is randomfillpropose and the accept function is probabilistic. The temperature value of the algorithm remains constant over time. \n\n\n\n\n\n","category":"function"},{"location":"HerbSearch/#HerbSearch.get_most_likely_first_enumerator","page":"HerbSearch.jl","title":"HerbSearch.get_most_likely_first_enumerator","text":"get_most_likely_first_enumerator(grammar::ContextFreeGrammar, max_depth::Int, max_size::Int, sym::Symbol, hole_heuristic::Function=heuristic_leftmost, derivation_heuristic::Function=(a,_) -> a)::ContextSensitivePriorityEnumerator\n\nReturns an enumerator that enumerates expressions in the grammar in decreasing order of probability. Only use this function with probabilistic grammars.\n\n\n\n\n\n","category":"function"},{"location":"HerbSearch/#HerbSearch.get_most_likely_first_enumerator-2","page":"HerbSearch.jl","title":"HerbSearch.get_most_likely_first_enumerator","text":"get_most_likely_first_enumerator(grammar::ContextSensitiveGrammar, max_depth::Int, max_size::Int, sym::Symbol, hole_heuristic::Function=heuristic_leftmost, derivation_heuristic::Function=(a,_) -> a)::ContextSensitivePriorityEnumerator\n\nReturns an enumerator that enumerates expressions in the grammar in decreasing order of probability. Only use this function with probabilistic grammars.\n\n\n\n\n\n","category":"function"},{"location":"HerbSearch/#HerbSearch.get_sa_enumerator","page":"HerbSearch.jl","title":"HerbSearch.get_sa_enumerator","text":"get_sa_enumerator(examples, cost_function, initial_temperature=1, temperature_decreasing_factor = 0.99, evaluation_function::Function=HerbInterpret.test_with_input)\n\nReturns an enumerator that runs according to the Very Large Scale Neighbourhood Search algorithm.\n\nexamples : array of examples\ncost_function : cost function to evaluate the programs proposed\ninitial_temperature : the starting temperature of the algorithm\ntemperature_decreasing_factor : the decreasing factor of the temperature of the time\nevaluation_function : evaluation function that evaluates the program generated and produces an output \n\nThe propose function is random_fill_propose (the same as for Metropolis Hastings). The accept function is probabilistic but takes into account the tempeerature too.\n\n\n\n\n\n","category":"function"},{"location":"HerbSearch/#HerbSearch.get_vlsn_enumerator","page":"HerbSearch.jl","title":"HerbSearch.get_vlsn_enumerator","text":"get_vlsn_enumerator(examples, cost_function, enumeration_depth = 2, evaluation_function::Function=HerbInterpret.test_with_input)\n\nReturns an enumerator that runs according to the Very Large Scale Neighbourhood Search algorithm.\n\nexamples : array of examples\ncost_function : cost function to evaluate the programs proposed\nenumeration_depth : the enumeration depth to search for a best program at a time\nevaluation_function : evaluation function that evaluates the program generated and produces an output \n\nThe propose function consists of all possible programs of the given enumeration_depth. The accept function accepts the program with the lowest cost according to the cost_function. The temperature value of the algorithm remains constant over time. \n\n\n\n\n\n","category":"function"},{"location":"HerbSearch/#HerbSearch.heuristic_leftmost-Tuple{AbstractRuleNode, Int64}","page":"HerbSearch.jl","title":"HerbSearch.heuristic_leftmost","text":"heuristic_leftmost(node::AbstractRuleNode, max_depth::Int)::Union{ExpandFailureReason, HoleReference}\n\nDefines a heuristic over holes, where the left-most hole always gets considered first. Returns a HoleReference once a hole is found. This is the default option for enumerators.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.heuristic_random-Tuple{AbstractRuleNode, Int64}","page":"HerbSearch.jl","title":"HerbSearch.heuristic_random","text":"heuristic_random(node::AbstractRuleNode, max_depth::Int)::Union{ExpandFailureReason, HoleReference}\n\nDefines a heuristic over holes, where random holes get chosen randomly using random exploration. Returns a HoleReference once a hole is found.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.heuristic_rightmost-Tuple{AbstractRuleNode, Int64}","page":"HerbSearch.jl","title":"HerbSearch.heuristic_rightmost","text":"heuristic_rightmost(node::AbstractRuleNode, max_depth::Int)::Union{ExpandFailureReason, HoleReference}\n\nDefines a heuristic over holes, where the right-most hole always gets considered first. Returns a HoleReference once a hole is found. \n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.heuristic_smallest_domain-Tuple{AbstractRuleNode, Int64}","page":"HerbSearch.jl","title":"HerbSearch.heuristic_smallest_domain","text":"heuristic_smallest_domain(node::AbstractRuleNode, max_depth::Int)::Union{ExpandFailureReason, HoleReference}\n\nDefines a heuristic over all available holes in the unfinished AST, by considering the size of their respective domains. A domain here describes the number of possible derivations with respect to the constraints. Returns a HoleReference once a hole is found. \n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.mean_squared_error-Tuple{T} where T<:(AbstractVector{<:Tuple{Number, Number}})","page":"HerbSearch.jl","title":"HerbSearch.mean_squared_error","text":"mean_squared_error(results::AbstractVector{Tuple{<:Number,<:Number}})\n\nReturns the mean squared error of results.\n\nArguments\n\nresults<:AbstractVector{<:Tuple{Number,Number}}: the vector of tuples, where each tuple is in the form Tuple{expected_output, actual_output}.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.misclassification-Tuple{T} where T<:(AbstractVector{<:Tuple{Number, Number}})","page":"HerbSearch.jl","title":"HerbSearch.misclassification","text":"misclassification(results::AbstractVector{Tuple{<:Number,<:Number}})\n\nReturns the amount of misclassified examples, i.e. how many tuples with non-matching entries are there in results.\n\nArguments\n\nresults<:AbstractVector{<:Tuple{Number,Number}}: the vector of tuples, where each tuple is in the form Tuple{expected_output, actual_output}.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.most_likely_priority_function-Tuple{ContextSensitiveGrammar, AbstractRuleNode, Union{Real, Tuple{Vararg{Real}}}}","page":"HerbSearch.jl","title":"HerbSearch.most_likely_priority_function","text":"most_likely_priority_function(g::ContextSensitiveGrammar, tree::AbstractRuleNode, ::Union{Real, Tuple{Vararg{Real}}})\n\nCalculates logit for all possible derivations for a node in a tree and returns them.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.mse_error_function-Tuple{Any, Any, Any}","page":"HerbSearch.jl","title":"HerbSearch.mse_error_function","text":"mse_error_function(old_error, output, expected_output)\n\nMean squared error function for search_best.\n\n- old_error - The existing total error\n- output - The actual output of the evaluator\n- expected_output - The expected output for the example\n\nThe function build the mean squared error from output and expected_output`.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.mutate_random!","page":"HerbSearch.jl","title":"HerbSearch.mutate_random!","text":"mutate_random!(program::RuleNode, grammar::Grammar, max_depth::Int64 = 2)\n\nMutates the given program by inserting a randomly generated sub-program at a random location.\n\n\n\n\n\n","category":"function"},{"location":"HerbSearch/#HerbSearch.probabilistic_accept-Tuple{Real, Real, Real}","page":"HerbSearch.jl","title":"HerbSearch.probabilistic_accept","text":"probabilistic_accept(current_cost::Real, next_cost::Real, temperature::Real)\n\nProbabilistically decides whether to accept the new program (next) based on the ratio of costs (smaller is better) between the previous and new program. Returns True if the new program is accepted, False otherwise.\n\nArguments\n\ncurrent_cost::Real: the cost of the current program.\nnext_cost::Real: the cost of the proposed program.\ntemperature::Real: the temperature; not used.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.probabilistic_accept_with_temperature-Tuple{Real, Real, Real}","page":"HerbSearch.jl","title":"HerbSearch.probabilistic_accept_with_temperature","text":"probabilistic_accept_with_temperature(current_cost::Real, next_cost::Real, temperature::Real)\n\nReturns true if the cost of the proposed program is smaller than the cost of the current program. Otherwise, returns true with the probability equal to: \n\n1 (1 + exp(delta temperature))\n\nIn any other case, returns false.\n\nArguments\n\ncurrent_cost::Real: the cost of the current program.\nnext_cost::Real: the cost of the proposed program.\ntemperature::Real: the temperature of the search.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.probabilistic_accept_with_temperature_fraction-Tuple{Real, Real, Real}","page":"HerbSearch.jl","title":"HerbSearch.probabilistic_accept_with_temperature_fraction","text":"probabilistic_accept_with_temperature_fraction(current_cost::Real, program_to_consider_cost::Real, temperature::Real)\n\nProbabilistically decides whether to accept the new program (next) based on the ratio of costs (smaller is better) between the previous and new program multiplied by the temperature. Returns True if the new program is accepted, False otherwise.\n\nArguments\n\ncurrent_cost::Real: the cost of the current program.\nnext_cost::Real: the cost of the proposed program.\ntemperature::Real: the current temperature \n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.propagate_constraints","page":"HerbSearch.jl","title":"HerbSearch.propagate_constraints","text":"function propagate_constraints(root::AbstractRuleNode, grammar::ContextSensitiveGrammar, local_constraints::Set{LocalConstraint}, max_holes::Int, filled_hole::Union{HoleReference, Nothing}=nothing)::Tuple{PropagateResult, Set{LocalConstraint}}\n\nPropagates a set of local constraints recursively to all children of a given root node. As propagate_constraints gets often called when a hole was just filled, filled_hole helps keeping track to propagate the constraints to relevant nodes, e.g. children of filled_hole. max_holes makes sure that max_size of Base.iterate is not violated. The function returns the PropagateResult and the set of relevant LocalConstraints.\n\n\n\n\n\n","category":"function"},{"location":"HerbSearch/#HerbSearch.random_fill_propose-Tuple{RuleNode, NodeLoc, Grammar, Int64, AbstractVector{Int64}, Union{Nothing, Dict{String, Any}}}","page":"HerbSearch.jl","title":"HerbSearch.random_fill_propose","text":"random_fill_propose(current_program, neighbourhood_node_loc, grammar, max_depth, dict)\n\nReturns a list with only one proposed, completely random, subprogram.\n\nArguments\n\ncurrent_program::RuleNode: the current program.\nneighbourhood_node_loc::NodeLoc: the location of the program to replace.\ngrammar::Grammar: the grammar used to create programs.\nmax_depth::Int: the maximum depth of the resulting programs.\ndmap::AbstractVector{Int} : the minimum possible depth to reach for each rule\ndict::Dict{String, Any}: the dictionary with additional arguments; not used.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.search-Tuple{Grammar, Problem, Symbol}","page":"HerbSearch.jl","title":"HerbSearch.search","text":"search(g::Grammar, problem::Problem, start::Symbol; evaluator::Function=test_with_input, enumerator::Function=get_bfs_enumerator, max_depth::Union{Int, Nothing}=nothing, max_size::Union{Int, Nothing}=nothing, max_time::Union{Int, Nothing}=nothing, max_enumerations::Union{Int, Nothing}=nothing, allow_evaluation_errors::Bool=false)::Union{Any, Nothing}\n\nSearches for a program by calling search_rulenode starting from Symbol start guided by enumerator and Grammar trying to satisfy the higher-order constraints in form of input/output examples defined in the Problem. This is the heart of the Herb's search for satisfying programs. Returns the found program when the evaluation calculated using evaluator is successful.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.search_best-Tuple{Grammar, Problem, Symbol}","page":"HerbSearch.jl","title":"HerbSearch.search_best","text":"search_best(g::Grammar, problem::Problem, start::Symbol; evaluator::Function=test_with_input, enumerator::Function=get_bfs_enumerator, error_function::Function=default_error_function, max_depth::Union{Int, Nothing}=nothing, max_size::Union{Int, Nothing}=nothing, max_time::Union{Int, Nothing}=nothing, max_enumerations::Union{Int, Nothing}=nothing, allow_evaluation_errors::Bool=false)::Tuple{Any, Real}\n\nSearches the grammar for the program that satisfies the maximum number of examples in the problem. The evaluator should be a function that takes a SymbolTable, expression and a dictionary with input variable assignments and returns the output of the expression.\n\n- g - The grammar that defines the search space\n- problem - The problem definition with IO examples\n- start - The start symbol in the grammar\n- evaluator - The evaluation function. Takes a SymbolTable, expression and a dictionary with \n input variable assignments and returns the output of the expression.\n- enumerator - A constructor for the enumerator that should be used in the search\n- error_function - The error function. Takes the existing total error, the actual output of the evaluator \n and the expected value for the example.\n- max_depth - The maximum depth of the search\n- max_time - The maximum time allowed for the search in seconds\n- max_enumerations - The maximum number of programs to enumerate and test\n- allow_evaluation_errors - Whether the search should crash if an exception is thrown in the evaluation\n\nReturns a tuple with the best found program so far and the error. Can be considerably slower than search due to having to evaluate each expression on each example.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.search_rulenode-Tuple{Grammar, Problem, Symbol}","page":"HerbSearch.jl","title":"HerbSearch.search_rulenode","text":"search_rulenode(g::Grammar, problem::Problem, start::Symbol; evaluator::Function=test_with_input, enumerator::Function=get_bfs_enumerator, max_depth::Union{Int, Nothing}=nothing, max_size::Union{Int, Nothing}=nothing, max_time::Union{Int, Nothing}=nothing, max_enumerations::Union{Int, Nothing}=nothing, allow_evaluation_errors::Bool=false)::Union{Tuple{RuleNode, Any}, Nothing}\n\nSearches the grammar for the program that satisfies the maximum number of examples in the problem.\n\n - g - The grammar that defines the search space\n - problem - The problem definition with IO examples\n - start - The start symbol in the grammar\n - evaluator - The evaluation function. Takes a SymbolTable, expression and a dictionary with \n input variable assignments and returns the output of the expression.\n - enumerator - A constructor for the enumerator that should be used in the search\n - max_depth - The maximum depth of the search\n - max_size - The maximum number of nodes for ASTs in the search\n - max_time - The maximum time allowed for the search in seconds\n - max_enumerations - The maximum number of programs to enumerate and test'\n - allow_evaluation_errors - Whether the search should crash if an exception is thrown in the evaluation\nReturns a tuple of the rulenode and the expression of the solution program once it has been found, \nor nothing otherwise.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.select_chromosome-Tuple{Array{RuleNode}, Array{<:Real}}","page":"HerbSearch.jl","title":"HerbSearch.select_chromosome","text":"select_chromosome(population::Array{RuleNode}, fitness_array::Array{<:Real})::RuleNode\n\nSelects a chromosome (individual) from the population based on a fitness array. The function uses a fitness-proportionate selection strategy, often referred to as \"roulette wheel\" selection. Assumes fitness_array to be normalized already.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.select_fitness_proportional_parents-Tuple{Array{RuleNode}, Array{<:Real}}","page":"HerbSearch.jl","title":"HerbSearch.select_fitness_proportional_parents","text":"select_fitness_proportional_parents(population::Array{RuleNode}, fitness_array::Array{<:Real})::Tuple{RuleNode,RuleNode}\n\nSelects two parent chromosomes (individuals) from a population based on fitness-proportionate selection. The selected parents can be used for genetic crossover in the next steps of the algorithm.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.validate_iterator-Tuple{Any}","page":"HerbSearch.jl","title":"HerbSearch.validate_iterator","text":"validate_iterator(iter)\n\nValidates the parameters of the iterator\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#StatsBase.sample","page":"HerbSearch.jl","title":"StatsBase.sample","text":"sample(::Type{NodeLoc}, root::RuleNode, typ::Symbol, grammar::Grammar)\n\nUniformly selects a random node in the tree of a given type, specified using its parent such that the subtree can be replaced. Returns a NodeLoc.\n\n\n\n\n\n","category":"function"},{"location":"HerbSearch/#StatsBase.sample-2","page":"HerbSearch.jl","title":"StatsBase.sample","text":"sample(::Type{NodeLoc}, root::RuleNode, maxdepth::Int=typemax(Int))\n\nUniformly selects a random node in the tree no deeper than maxdepth using reservoir sampling. Returns a NodeLoc that specifies the location using its parent so that the subtree can be replaced.\n\n\n\n\n\n","category":"function"},{"location":"HerbSearch/#StatsBase.sample-3","page":"HerbSearch.jl","title":"StatsBase.sample","text":"sample(root::RuleNode, typ::Symbol, grammar::Grammar, maxdepth::Int=typemax(Int))\n\nUniformly samples a random node from the tree limited to maxdepth.\n\n\n\n\n\n","category":"function"},{"location":"HerbSearch/#StatsBase.sample-4","page":"HerbSearch.jl","title":"StatsBase.sample","text":"sample(root::RuleNode, typ::Symbol, grammar::Grammar,\n maxdepth::Int=typemax(Int))\n\nUniformly selects a random node of the given return type typ limited by maxdepth.\n\n\n\n\n\n","category":"function"},{"location":"HerbSearch/","page":"HerbSearch.jl","title":"HerbSearch.jl","text":"The HerbSearch package takes care of all operations related to searching for the desired program. This includes","category":"page"},{"location":"HerbSearch/","page":"HerbSearch.jl","title":"HerbSearch.jl","text":"the functionality to sample a certain program given a grammar,\nthe implementation of several heuristic functions,\nsearching for a program that satisfies the specification, and\nimplementations of several search algorithms in terms of how they enumerate the search space\nBreadth-First Search \nDepth-First Search \nMetropolis Hastings \nVery Large Scale Neighbourhood Search \nSimulated Annealing\nGenetic Search","category":"page"},{"location":"HerbSearch/#Index","page":"HerbSearch.jl","title":"Index","text":"","category":"section"},{"location":"HerbSearch/","page":"HerbSearch.jl","title":"HerbSearch.jl","text":"","category":"page"},{"location":"HerbCore/#HerbCore_docs","page":"HerbCore.jl Documentation","title":"HerbCore.jl Documentation","text":"","category":"section"},{"location":"HerbCore/","page":"HerbCore.jl Documentation","title":"HerbCore.jl Documentation","text":"CurrentModule=HerbCore","category":"page"},{"location":"HerbCore/","page":"HerbCore.jl Documentation","title":"HerbCore.jl Documentation","text":"Modules = [HerbCore]\nOrder = [:type, :const, :macro, :function]","category":"page"},{"location":"HerbCore/#HerbCore.AbstractRuleNode","page":"HerbCore.jl Documentation","title":"HerbCore.AbstractRuleNode","text":"AbstractRuleNode\n\nAbstract type for representing expression trees. Expression trees consist of RuleNodes and Holes.\n\nA RuleNode represents a certain production rule in the Grammar.\nA Hole is a placeholder where certain rules in the grammar still can be applied. \n\n\n\n\n\n","category":"type"},{"location":"HerbCore/#HerbCore.Constraint","page":"HerbCore.jl Documentation","title":"HerbCore.Constraint","text":"Represents a constraint for a ContextSensitiveGrammar. Concrete implementations can be found in HerbConstraints.jl.\n\n\n\n\n\n","category":"type"},{"location":"HerbCore/#HerbCore.Grammar","page":"HerbCore.jl Documentation","title":"HerbCore.Grammar","text":"Grammar\n\nAbstract type representing all grammars. It is assumed that all grammar structs have at least the following attributes:\n\nrules::Vector{Any}: A list of RHS of rules (subexpressions).\ntypes::Vector{Symbol}: A list of LHS of rules (types, all symbols).\nisterminal::BitVector: A bitvector where bit i represents whether rule i is terminal.\niseval::BitVector: A bitvector where bit i represents whether rule i is an eval rule.\nbytype::Dict{Symbol,Vector{Int}}: A dictionary that maps a type to all rules of said type.\ndomains::Dict{Symbol, BitVector}: A dictionary that maps a type to a domain bitvector. The domain bitvector has bit i set to true iff the ith rule is of this type.\nchildtypes::Vector{Vector{Symbol}}: A list of types of the children for each rule. \n\nIf a rule is terminal, the corresponding list is empty.\n\nlog_probabilities::Union{Vector{Real}, Nothing}: A list of probabilities for each rule. \n\nIf the grammar is non-probabilistic, the list can be nothing.\n\nFor concrete types, see ContextFreeGrammar and ContextSensitiveGrammar.\n\n\n\n\n\n","category":"type"},{"location":"HerbCore/#HerbCore.Hole","page":"HerbCore.jl Documentation","title":"HerbCore.Hole","text":"Hole <: AbstractRuleNode\n\nA Hole is a placeholder where certain rules from the grammar can still be applied. The domain of a Hole defines which rules can be applied. The domain is a bitvector, where the ith bit is set to true if the ith rule in the grammar can be applied. \n\n\n\n\n\n","category":"type"},{"location":"HerbCore/#HerbCore.HoleReference","page":"HerbCore.jl Documentation","title":"HerbCore.HoleReference","text":"HoleReference\n\nContains a hole and the path to the hole from the root of the tree.\n\n\n\n\n\n","category":"type"},{"location":"HerbCore/#HerbCore.RuleNode","page":"HerbCore.jl Documentation","title":"HerbCore.RuleNode","text":"RuleNode <: AbstractRuleNode\n\nA RuleNode represents a node in an expression tree. Each node corresponds to a certain rule in the Grammar. A RuleNode consists of:\n\nind: The index of the rule in the Grammar which this node is representing.\n_val: Field for storing immediately evaluated values\nchildren: The children of this node in the expression tree\n\ncompat: Compat\nEvaluate immediately functionality is not yet supported by most of Herb.jl.\n\n\n\n\n\n","category":"type"},{"location":"HerbCore/#HerbCore.RuleNode-Tuple{Int64, Any}","page":"HerbCore.jl Documentation","title":"HerbCore.RuleNode","text":"RuleNode(ind::Int, _val::Any)\n\nCreate a RuleNode for the Grammar rule with index ind, _val as immediately evaluated value and no children\n\nwarning: Warning\nOnly use this constructor if you are absolutely certain that a rule is terminal and cannot have children. Use [RuleNode(ind::Int, grammar::Grammar)] for rules that might have children. In general, Holes should be used as a placeholder when the children of a node are not yet known. \n\ncompat: Compat\nEvaluate immediately functionality is not yet supported by most of Herb.jl.\n\n\n\n\n\n","category":"method"},{"location":"HerbCore/#HerbCore.RuleNode-Tuple{Int64, Vector{AbstractRuleNode}}","page":"HerbCore.jl Documentation","title":"HerbCore.RuleNode","text":"RuleNode(ind::Int, children::Vector{AbstractRuleNode})\n\nCreate a RuleNode for the Grammar rule with index ind and children as subtrees.\n\n\n\n\n\n","category":"method"},{"location":"HerbCore/#Base.isless-Tuple{AbstractRuleNode, AbstractRuleNode}","page":"HerbCore.jl Documentation","title":"Base.isless","text":"Base.isless(rn₁::AbstractRuleNode, rn₂::AbstractRuleNode)::Bool\n\nCompares two RuleNodes. Returns true if the left RuleNode is less than the right RuleNode. Order is determined from the index of the RuleNodes. If both RuleNodes have the same index, a depth-first search is performed in both RuleNodes until nodes with a different index are found.\n\n\n\n\n\n","category":"method"},{"location":"HerbCore/#Base.length-Tuple{Hole}","page":"HerbCore.jl Documentation","title":"Base.length","text":"Base.length(root::RuleNode)\n\nReturn the number of nodes in the tree rooted at root. Holes don't count.\n\n\n\n\n\n","category":"method"},{"location":"HerbCore/#Base.length-Tuple{RuleNode}","page":"HerbCore.jl Documentation","title":"Base.length","text":"Base.length(root::RuleNode)\n\nReturn the number of nodes in the tree rooted at root. Holes don't count.\n\n\n\n\n\n","category":"method"},{"location":"HerbCore/#HerbCore.contains_hole-Tuple{RuleNode}","page":"HerbCore.jl Documentation","title":"HerbCore.contains_hole","text":"contains_hole(rn::RuleNode) = any(contains_hole(c) for c ∈ rn.children)\n\nChecks if an AbstractRuleNode tree contains a Hole.\n\n\n\n\n\n","category":"method"},{"location":"HerbCore/#HerbCore.depth-Tuple{RuleNode}","page":"HerbCore.jl Documentation","title":"HerbCore.depth","text":"depth(root::RuleNode)::Int\n\nReturn the depth of the AbstractRuleNode tree rooted at root. Holes don't count towards the depth.\n\n\n\n\n\n","category":"method"},{"location":"HerbCore/#HerbCore.get_node_at_location-Tuple{RuleNode, Vector{Int64}}","page":"HerbCore.jl Documentation","title":"HerbCore.get_node_at_location","text":"get_node_at_location(root::RuleNode, location::Vector{Int})\n\nRetrieves a RuleNode at the given location by reference. \n\n\n\n\n\n","category":"method"},{"location":"HerbCore/#HerbCore.get_rulesequence-Tuple{RuleNode, Vector{Int64}}","page":"HerbCore.jl Documentation","title":"HerbCore.get_rulesequence","text":"get_rulesequence(node::RuleNode, path::Vector{Int})\n\nExtract the derivation sequence from a path (sequence of child indices) and an AbstractRuleNode. If the path is deeper than the deepest node, it returns what it has.\n\n\n\n\n\n","category":"method"},{"location":"HerbCore/#HerbCore.node_depth-Tuple{AbstractRuleNode, AbstractRuleNode}","page":"HerbCore.jl Documentation","title":"HerbCore.node_depth","text":"node_depth(root::AbstractRuleNode, node::AbstractRuleNode)::Int\n\nReturn the depth of node for an AbstractRuleNode tree rooted at root. Depth is 1 when root == node.\n\nwarning: Warning\nnode must be a subtree of root in order for this function to work.\n\n\n\n\n\n","category":"method"},{"location":"HerbCore/#HerbCore.rulesoftype-Tuple{RuleNode, Set{Int64}}","page":"HerbCore.jl Documentation","title":"HerbCore.rulesoftype","text":"rulesoftype(node::RuleNode, ruleset::Set{Int})\n\nReturns every rule in the ruleset that is also used in the AbstractRuleNode tree.\n\n\n\n\n\n","category":"method"},{"location":"HerbCore/#HerbCore.rulesonleft-Tuple{RuleNode, Vector{Int64}}","page":"HerbCore.jl Documentation","title":"HerbCore.rulesonleft","text":"rulesonleft(expr::RuleNode, path::Vector{Int})::Set{Int}\n\nFinds all rules that are used in the left subtree defined by the path.\n\n\n\n\n\n","category":"method"},{"location":"HerbCore/#HerbCore.swap_node-Tuple{AbstractRuleNode, AbstractRuleNode, Vector{Int64}}","page":"HerbCore.jl Documentation","title":"HerbCore.swap_node","text":"swap_node(expr::AbstractRuleNode, new_expr::AbstractRuleNode, path::Vector{Int})\n\nReplace a node in expr, specified by path, with new_expr. Path is a sequence of child indices, starting from the root node.\n\n\n\n\n\n","category":"method"},{"location":"HerbCore/#HerbCore.swap_node-Tuple{RuleNode, RuleNode, Int64, RuleNode}","page":"HerbCore.jl Documentation","title":"HerbCore.swap_node","text":"swap_node(expr::RuleNode, node::RuleNode, child_index::Int, new_expr::RuleNode)\n\nReplace child i of a node, a part of larger expr, with new_expr.\n\n\n\n\n\n","category":"method"},{"location":"HerbCore/#Index","page":"HerbCore.jl Documentation","title":"Index","text":"","category":"section"},{"location":"HerbCore/","page":"HerbCore.jl Documentation","title":"HerbCore.jl Documentation","text":"","category":"page"},{"location":"HerbGrammar/#HerbGrammar_docs","page":"HerbGrammar.jl","title":"HerbGrammar.jl Documentation","text":"","category":"section"},{"location":"HerbGrammar/","page":"HerbGrammar.jl","title":"HerbGrammar.jl","text":"CurrentModule=HerbGrammar","category":"page"},{"location":"HerbGrammar/","page":"HerbGrammar.jl","title":"HerbGrammar.jl","text":"Modules = [HerbGrammar]\nOrder = [:type, :const, :macro, :function]","category":"page"},{"location":"HerbGrammar/#HerbGrammar.ContextFreeGrammar","page":"HerbGrammar.jl","title":"HerbGrammar.ContextFreeGrammar","text":"ContextFreeGrammar <: Grammar\n\nRepresents a context-free grammar and its production rules. Consists of:\n\nrules::Vector{Any}: A list of RHS of rules (subexpressions).\ntypes::Vector{Symbol}: A list of LHS of rules (types, all symbols).\nisterminal::BitVector: A bitvector where bit i represents whether rule i is terminal.\niseval::BitVector: A bitvector where bit i represents whether rule i is an eval rule.\nbytype::Dict{Symbol,Vector{Int}}: A dictionary that maps a type to all rules of said type.\ndomains::Dict{Symbol, BitVector}: A dictionary that maps a type to a domain bitvector. The domain bitvector has bit i set to true iff the ith rule is of this type.\nchildtypes::Vector{Vector{Symbol}}: A list of types of the children for each rule. If a rule is terminal, the corresponding list is empty.\nlog_probabilities::Union{Vector{Real}, Nothing}: A list of probabilities for each rule. If the grammar is non-probabilistic, the list can be nothing.\n\nUse the @cfgrammar macro to create a ContextFreeGrammar object. Use the @pcfgrammar macro to create a ContextFreeGrammar object with probabilities. For context-sensitive grammars, see ContextSensitiveGrammar.\n\n\n\n\n\n","category":"type"},{"location":"HerbGrammar/#HerbGrammar.ContextSensitiveGrammar","page":"HerbGrammar.jl","title":"HerbGrammar.ContextSensitiveGrammar","text":"ContextSensitiveGrammar <: Grammar\n\nRepresents a context-sensitive grammar. Extends Grammar with constraints.\n\nConsists of:\n\nrules::Vector{Any}: A list of RHS of rules (subexpressions).\ntypes::Vector{Symbol}: A list of LHS of rules (types, all symbols).\nisterminal::BitVector: A bitvector where bit i represents whether rule i is terminal.\niseval::BitVector: A bitvector where bit i represents whether rule i is an eval rule.\nbytype::Dict{Symbol,Vector{Int}}: A dictionary that maps a type to all rules of said type.\ndomains::Dict{Symbol, BitVector}: A dictionary that maps a type to a domain bitvector. The domain bitvector has bit i set to true iff the ith rule is of this type.\nchildtypes::Vector{Vector{Symbol}}: A list of types of the children for each rule. If a rule is terminal, the corresponding list is empty.\nlog_probabilities::Union{Vector{Real}, Nothing}: A list of probabilities for each rule. If the grammar is non-probabilistic, the list can be nothing.\nconstraints::Vector{Constraint}: A list of constraints that programs in this grammar have to abide.\n\nUse the @csgrammar macro to create a ContextSensitiveGrammar object. Use the @pcsgrammar macro to create a ContextSensitiveGrammar object with probabilities. For context-free grammars, see ContextFreeGrammar.\n\n\n\n\n\n","category":"type"},{"location":"HerbGrammar/#HerbGrammar.NodeLoc","page":"HerbGrammar.jl","title":"HerbGrammar.NodeLoc","text":"NodeLoc A helper struct that points to a node in the tree via its parent such that the child can be easily swapped out. If i is 0 the node pointed to is the root node and parent is the node itself.\n\n\n\n\n\n","category":"type"},{"location":"HerbGrammar/#HerbGrammar.SymbolTable","page":"HerbGrammar.jl","title":"HerbGrammar.SymbolTable","text":"SymbolTable\n\nData structure for mapping terminal symbols in the Grammar to their Julia interpretation.\n\n\n\n\n\n","category":"type"},{"location":"HerbGrammar/#HerbGrammar.SymbolTable-2","page":"HerbGrammar.jl","title":"HerbGrammar.SymbolTable","text":"SymbolTable(grammar::Grammar, mod::Module=Main)\n\nReturns a SymbolTable populated with a mapping from symbols in the Grammar to symbols in module mod or Main, if defined.\n\n\n\n\n\n","category":"type"},{"location":"HerbGrammar/#HerbGrammar.@cfgrammar-Tuple{Any}","page":"HerbGrammar.jl","title":"HerbGrammar.@cfgrammar","text":"@cfgrammar\n\nA macro for defining a ContextFreeGrammar. \n\nExample usage:\n\ngrammar = @cfgrammar begin\n\tR = x\n\tR = 1 | 2\n\tR = R + R\nend\n\nSyntax:\n\nLiterals: Symbols that are already defined in Julia are considered literals, such as 1, 2, or π. For example: R = 1.\nVariables: A variable is a symbol that is not a nonterminal symbol and not already defined in Julia. For example: R = x.\nFunctions: Functions and infix operators that are defined in Julia or the Main module can be used with the default evaluator. For example: R = R + R, R = f(a, b).\nCombinations: Multiple rules can be defined on a single line in the grammar definition using the | symbol. For example: R = 1 | 2 | 3.\nIterators: Another way to define multiple rules is by providing a Julia iterator after a | symbol. For example: R = |(1:9).\n\nRelated:\n\n@csgrammar uses the same syntax to create ContextSensitiveGrammars.\n@pcfgrammar uses a similar syntax to create probabilistic ContextFreeGrammars.\n\n\n\n\n\n","category":"macro"},{"location":"HerbGrammar/#HerbGrammar.@csgrammar-Tuple{Any}","page":"HerbGrammar.jl","title":"HerbGrammar.@csgrammar","text":"@csgrammar\n\nA macro for defining a ContextSensitiveGrammar. Constraints can be added afterwards using the addconstraint! function.\n\nExample usage:\n\ngrammar = @csgrammar begin\n\tR = x\n\tR = 1 | 2\n\tR = R + R\nend\n\nSyntax:\n\nLiterals: Symbols that are already defined in Julia are considered literals, such as 1, 2, or π. For example: R = 1.\nVariables: A variable is a symbol that is not a nonterminal symbol and not already defined in Julia. For example: R = x.\nFunctions: Functions and infix operators that are defined in Julia or the Main module can be used with the default evaluator. For example: R = R + R, R = f(a, b).\nCombinations: Multiple rules can be defined on a single line in the grammar definition using the | symbol. For example: R = 1 | 2 | 3.\nIterators: Another way to define multiple rules is by providing a Julia iterator after a | symbol. For example: R = |(1:9).\n\nRelated:\n\n@cfgrammar uses the same syntax to create ContextFreeGrammars.\n@pcsgrammar uses a similar syntax to create probabilistic ContextSensitiveGrammars.\n\n\n\n\n\n","category":"macro"},{"location":"HerbGrammar/#HerbGrammar.@pcfgrammar-Tuple{Any}","page":"HerbGrammar.jl","title":"HerbGrammar.@pcfgrammar","text":"@pcfgrammar\n\nA macro for defining a probabilistic ContextFreeGrammar. \n\nExample usage:\n\ngrammar = @pcfgrammar begin\n\t0.5 : R = x\n\t0.3 : R = 1 | 2\n\t0.2 : R = R + R\nend\n\nSyntax:\n\nThe syntax of rules is identical to the syntax used by @cfgrammar:\n\nLiterals: Symbols that are already defined in Julia are considered literals, such as 1, 2, or π. For example: R = 1.\nVariables: A variable is a symbol that is not a nonterminal symbol and not already defined in Julia. For example: R = x.\nFunctions: Functions and infix operators that are defined in Julia or the Main module can be used with the default evaluator. For example: R = R + R, R = f(a, b).\nCombinations: Multiple rules can be defined on a single line in the grammar definition using the | symbol. For example: R = 1 | 2 | 3.\nIterators: Another way to define multiple rules is by providing a Julia iterator after a | symbol. For example: R = |(1:9).\n\nEvery rule is also prefixed with a probability. Rules and probabilities are separated using the : symbol. If multiple rules are defined on a single line, the probability is equally divided between the rules. The sum of probabilities for all rules of a certain non-terminal symbol should be equal to 1. The probabilities are automatically scaled if this isn't the case.\n\nRelated:\n\n@pcsgrammar uses the same syntax to create probabilistic ContextSensitiveGrammars.\n@cfgrammar uses a similar syntax to create non-probabilistic ContextFreeGrammars.\n\n\n\n\n\n","category":"macro"},{"location":"HerbGrammar/#HerbGrammar.@pcsgrammar-Tuple{Any}","page":"HerbGrammar.jl","title":"HerbGrammar.@pcsgrammar","text":"@pcsgrammar\n\nA macro for defining a probabilistic ContextSensitiveGrammar. \n\nExample usage:\n\ngrammar = @pcsgrammar begin\n\t0.5 : R = x\n\t0.3 : R = 1 | 2\n\t0.2 : R = R + R\nend\n\nSyntax:\n\nThe syntax of rules is identical to the syntax used by @csgrammar:\n\nLiterals: Symbols that are already defined in Julia are considered literals, such as 1, 2, or π. For example: R = 1.\nVariables: A variable is a symbol that is not a nonterminal symbol and not already defined in Julia. For example: R = x.\nFunctions: Functions and infix operators that are defined in Julia or the Main module can be used with the default evaluator. For example: R = R + R, R = f(a, b).\nCombinations: Multiple rules can be defined on a single line in the grammar definition using the | symbol. For example: R = 1 | 2 | 3.\nIterators: Another way to define multiple rules is by providing a Julia iterator after a | symbol. For example: R = |(1:9).\n\nEvery rule is also prefixed with a probability. Rules and probabilities are separated using the : symbol. If multiple rules are defined on a single line, the probability is equally divided between the rules. The sum of probabilities for all rules of a certain non-terminal symbol should be equal to 1. The probabilities are automatically scaled if this isn't the case.\n\nRelated:\n\n@pcfgrammar uses the same syntax to create probabilistic ContextFreeGrammars.\n@csgrammar uses a similar syntax to create non-probabilistic ContextSensitiveGrammars.\n\n\n\n\n\n","category":"macro"},{"location":"HerbGrammar/#Base.get-Tuple{RuleNode, NodeLoc}","page":"HerbGrammar.jl","title":"Base.get","text":"get(root::RuleNode, loc::NodeLoc) Obtain the node pointed to by loc.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#Base.insert!-Tuple{RuleNode, NodeLoc, RuleNode}","page":"HerbGrammar.jl","title":"Base.insert!","text":"insert!(loc::NodeLoc, rulenode::RuleNode) Replaces the subtree pointed to by loc with the given rulenode.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.add_rule!-Tuple{Grammar, Expr}","page":"HerbGrammar.jl","title":"HerbGrammar.add_rule!","text":"add_rule!(g::Grammar, e::Expr)\n\nAdds a rule to the grammar. \n\nUsage:\n\n\tadd_rule!(grammar, :(\"Real = Real + Real\"))\n\nThe syntax is identical to the syntax of @csgrammar and @cfgrammar, but only single rules are supported.\n\nwarning: Warning\nCalls to this function are ignored if a rule is already in the grammar.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.addconstraint!-Tuple{ContextSensitiveGrammar, Constraint}","page":"HerbGrammar.jl","title":"HerbGrammar.addconstraint!","text":"addconstraint!(grammar::ContextSensitiveGrammar, c::Constraint)\n\nAdds a Constraint to a ContextSensitiveGrammar.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.cfg2csg-Tuple{ContextFreeGrammar}","page":"HerbGrammar.jl","title":"HerbGrammar.cfg2csg","text":"cfg2csg(g::ContextFreeGrammar)::ContextSensitiveGrammar\n\nConverts a ContextFreeGrammar to a ContextSensitiveGrammar without any Constraints.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.child_types-Tuple{Grammar, Int64}","page":"HerbGrammar.jl","title":"HerbGrammar.child_types","text":"child_types(grammar::Grammar, rule_index::Int)\n\nReturns the types of the children (nonterminals) of the production rule at rule_index.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.child_types-Tuple{Grammar, RuleNode}","page":"HerbGrammar.jl","title":"HerbGrammar.child_types","text":"child_types(grammar::Grammar, node::RuleNode)\n\nReturns the list of child types (nonterminal symbols) in the production rule used by node.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.cleanup_removed_rules!-Tuple{Grammar}","page":"HerbGrammar.jl","title":"HerbGrammar.cleanup_removed_rules!","text":"cleanup_removed_rules!(g::Grammar)\n\nRemoves any placeholders for previously deleted rules. This means that indices get shifted.\n\nwarning: Warning\nWhen indices are shifted, this grammar can no longer be used to interpret AbstractRuleNode trees created before the call to this function. These trees become meaningless. \n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.clearconstraints!-Tuple{ContextSensitiveGrammar}","page":"HerbGrammar.jl","title":"HerbGrammar.clearconstraints!","text":"Clear all constraints from the grammar\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.containedin-Tuple{Vector, Vector}","page":"HerbGrammar.jl","title":"HerbGrammar.containedin","text":"containedin(vec1::Vector, vec2::Vector)\n\nChecks if elements of vec1 are contained in vec2 in the same order (possibly with elements in between)\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.contains_returntype","page":"HerbGrammar.jl","title":"HerbGrammar.contains_returntype","text":"contains_returntype(node::RuleNode, grammar::Grammar, sym::Symbol, maxdepth::Int=typemax(Int))\n\nReturns true if the tree rooted at node contains at least one node at depth less than maxdepth with the given return type or nonterminal symbol.\n\n\n\n\n\n","category":"function"},{"location":"HerbGrammar/#HerbGrammar.expr2cfgrammar-Tuple{Expr}","page":"HerbGrammar.jl","title":"HerbGrammar.expr2cfgrammar","text":"expr2cfgrammar(ex::Expr)::ContextFreeGrammar\n\nA function for converting an Expr to a ContextFreeGrammar. If the expression is hardcoded, you should use the @cfgrammar macro. Only expressions in the correct format (see @cfgrammar) can be converted.\n\nExample usage:\n\ngrammar = expr2cfgrammar(\n\tbegin\n\t\tR = x\n\t\tR = 1 | 2\n\t\tR = R + R\n\tend\n)\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.expr2csgrammar-Tuple{Expr}","page":"HerbGrammar.jl","title":"HerbGrammar.expr2csgrammar","text":"expr2csgrammar(ex::Expr)::ContextSensitiveGrammar\n\nA function for converting an Expr to a ContextSensitiveGrammar. If the expression is hardcoded, you should use the @csgrammar macro. Only expressions in the correct format (see @csgrammar) can be converted.\n\nExample usage:\n\ngrammar = expr2csgrammar(\n\tbegin\n\t\tR = x\n\t\tR = 1 | 2\n\t\tR = R + R\n\tend\n)\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.expr2pcfgrammar-Tuple{Expr}","page":"HerbGrammar.jl","title":"HerbGrammar.expr2pcfgrammar","text":"Function for converting an Expr to a ContextFreeGrammar with probabilities. If the expression is hardcoded, you should use the @pcfgrammar macro. Only expressions in the correct format (see @pcfgrammar) can be converted.\n\nExample usage:\n\ngrammar = expr2pcsgrammar(\n\tbegin\n\t\t0.5 : R = x\n\t\t0.3 : R = 1 | 2\n\t\t0.2 : R = R + R\n\tend\n)\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.expr2pcsgrammar-Tuple{Expr}","page":"HerbGrammar.jl","title":"HerbGrammar.expr2pcsgrammar","text":"Function for converting an Expr to a ContextSensitiveGrammar with probabilities. If the expression is hardcoded, you should use the @pcsgrammar macro. Only expressions in the correct format (see @pcsgrammar) can be converted.\n\nExample usage:\n\ngrammar = expr2pcsgrammar(\n\tbegin\n\t\t0.5 : R = x\n\t\t0.3 : R = 1 | 2\n\t\t0.2 : R = R + R\n\tend\n)\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.get_childtypes-Tuple{Any, AbstractVector{Symbol}}","page":"HerbGrammar.jl","title":"HerbGrammar.get_childtypes","text":"get_childtypes(rule::Any, types::AbstractVector{Symbol})\n\nReturns the child types/nonterminals of a production rule.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.get_domain-Tuple{Grammar, Symbol}","page":"HerbGrammar.jl","title":"HerbGrammar.get_domain","text":"get_domain(g::Grammar, type::Symbol)::BitVector\n\nReturns the domain for the hole of a certain type as a BitVector of the same length as the number of rules in the grammar. Bit i is set to true iff rule i is of type type.\n\ninfo: Info\nSince this function can be intensively used when exploring a program space defined by a grammar, the outcomes of this function are precomputed and stored in the domains field in a Grammar.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.get_domain-Tuple{Grammar, Vector{Int64}}","page":"HerbGrammar.jl","title":"HerbGrammar.get_domain","text":"get_domain(g::Grammar, rules::Vector{Int})::BitVector\n\nTakes a domain rules defined as a vector of ints and converts it to a domain defined as a BitVector.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.get_node_at_location-Tuple{RuleNode, Vector{Int64}}","page":"HerbGrammar.jl","title":"HerbGrammar.get_node_at_location","text":"get_node_at_location(root::RuleNode, location::Vector{Int})\n\nRetrieves a RuleNode at the given location by reference. \n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.get_rulesequence-Tuple{RuleNode, Vector{Int64}}","page":"HerbGrammar.jl","title":"HerbGrammar.get_rulesequence","text":"get_rulesequence(node::RuleNode, path::Vector{Int})\n\nExtract the derivation sequence from a path (sequence of child indices) and an AbstractRuleNode. If the path is deeper than the deepest node, it returns what it has.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.has_children-Tuple{RuleNode}","page":"HerbGrammar.jl","title":"HerbGrammar.has_children","text":"has_children(node::RuleNode)\n\nReturns true if node has children\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.iscomplete-Tuple{Grammar, RuleNode}","page":"HerbGrammar.jl","title":"HerbGrammar.iscomplete","text":"iscomplete(grammar::Grammar, node::RuleNode)\n\nReturns true if the expression represented by the RuleNode is a complete expression, meaning that it is fully defined and doesn't have any Holes.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.iseval-Tuple{Any}","page":"HerbGrammar.jl","title":"HerbGrammar.iseval","text":"iseval(rule)\n\nReturns true if the rule is the special evaluate immediately function, i.e., _()\n\ncompat: Compat\nevaluate immediately functionality is not yet supported by most of Herb.jl\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.iseval-Tuple{Grammar, Int64}","page":"HerbGrammar.jl","title":"HerbGrammar.iseval","text":"iseval(grammar::Grammar, index::Int)::Bool\n\nReturns true if the production rule at rule_index contains the special _() eval function.\n\ncompat: Compat\nevaluate immediately functionality is not yet supported by most of Herb.jl\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.iseval-Tuple{Grammar}","page":"HerbGrammar.jl","title":"HerbGrammar.iseval","text":"iseval(grammar::Grammar)::Bool\n\nReturns true if any production rules in grammar contain the special _() eval function.\n\ncompat: Compat\nevaluate immediately functionality is not yet supported by most of Herb.jl\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.isprobabilistic-Tuple{Grammar}","page":"HerbGrammar.jl","title":"HerbGrammar.isprobabilistic","text":"isprobabilistic(grammar::Grammar)::Bool\n\nFunction returns whether a Grammar is probabilistic.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.isterminal-Tuple{Any, AbstractVector{Symbol}}","page":"HerbGrammar.jl","title":"HerbGrammar.isterminal","text":"isterminal(rule::Any, types::AbstractVector{Symbol})\n\nReturns true if the rule is terminal, i.e., it does not contain any of the types in the provided vector. For example, :(x) is terminal, and :(1+1) is terminal, but :(Real + Real) is typically not.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.isterminal-Tuple{Grammar, Int64}","page":"HerbGrammar.jl","title":"HerbGrammar.isterminal","text":"isterminal(grammar::Grammar, rule_index::Int)::Bool\n\nReturns true if the production rule at rule_index is terminal, i.e., does not contain any nonterminal symbols.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.isterminal-Tuple{Grammar, RuleNode}","page":"HerbGrammar.jl","title":"HerbGrammar.isterminal","text":"isterminal(grammar::Grammar, node::RuleNode)::Bool\n\nReturns true if the production rule used by node is terminal, i.e., does not contain any nonterminal symbols.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.isvariable-Tuple{Grammar, RuleNode}","page":"HerbGrammar.jl","title":"HerbGrammar.isvariable","text":"isvariable(grammar::Grammar, node::RuleNode)::Bool\n\nReturns true if the rule used by node represents a variable.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.log_probability-Tuple{Grammar, Int64}","page":"HerbGrammar.jl","title":"HerbGrammar.log_probability","text":"log_probability(grammar::Grammar, index::Int)::Real\n\nReturns the log probability for the rule at index in the grammar.\n\nwarning: Warning\nIf the grammar is not probabilistic, a warning is displayed, and a uniform probability is assumed.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.max_arity-Tuple{Grammar}","page":"HerbGrammar.jl","title":"HerbGrammar.max_arity","text":"max_arity(grammar::Grammar)::Int\n\nReturns the maximum arity (number of children) over all production rules in the Grammar.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.mindepth-Tuple{Grammar, Symbol, AbstractVector{Int64}}","page":"HerbGrammar.jl","title":"HerbGrammar.mindepth","text":"mindepth(grammar::Grammar, typ::Symbol, dmap::AbstractVector{Int})\n\nReturns the minimum depth achievable for a given nonterminal symbol. The minimum depth is the depth of the lowest tree that can be made using typ as a start symbol. dmap can be obtained from mindepth_map.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.mindepth_map-Tuple{Grammar}","page":"HerbGrammar.jl","title":"HerbGrammar.mindepth_map","text":"mindepth_map(grammar::Grammar)\n\nReturns the minimum depth achievable for each production rule in the Grammar. In other words, this function finds the depths of the lowest trees that can be made using each of the available production rules as a root.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.nchildren-Tuple{Grammar, Int64}","page":"HerbGrammar.jl","title":"HerbGrammar.nchildren","text":"nchildren(grammar::Grammar, rule_index::Int)::Int\n\nReturns the number of children (nonterminals) of the production rule at rule_index.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.nchildren-Tuple{Grammar, RuleNode}","page":"HerbGrammar.jl","title":"HerbGrammar.nchildren","text":"nchildren(grammar::Grammar, node::RuleNode)::Int\n\nReturns the number of children in the production rule used by node.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.nonterminals-Tuple{Grammar}","page":"HerbGrammar.jl","title":"HerbGrammar.nonterminals","text":"nonterminals(grammar::Grammar)::Vector{Symbol}\n\nReturns a list of the nonterminals or types in the Grammar.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.probability-Tuple{Grammar, Int64}","page":"HerbGrammar.jl","title":"HerbGrammar.probability","text":"probability(grammar::Grammar, index::Int)::Real\n\nReturn the probability for a rule in the grammar. Use log_probability whenever possible.\n\nwarning: Warning\nIf the grammar is not probabilistic, a warning is displayed, and a uniform probability is assumed.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.read_cfg-Tuple{AbstractString}","page":"HerbGrammar.jl","title":"HerbGrammar.read_cfg","text":"read_cfg(filepath::AbstractString)::ContextFreeGrammar\n\nReads a ContextFreeGrammar from the file provided in filepath.\n\ndanger: Danger\nOnly open trusted grammars. Parts of the grammar can be passed to Julia's eval function. \n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.read_csg-Tuple{AbstractString, AbstractString}","page":"HerbGrammar.jl","title":"HerbGrammar.read_csg","text":"read_csg(grammarpath::AbstractString, constraintspath::AbstractString)::ContextSensitiveGrammar\n\nReads a ContextSensitiveGrammar from the files at grammarpath and constraintspath. The grammar path may also point to a ContextFreeGrammar.\n\ndanger: Danger\nOnly open trusted grammars. Parts of the grammar can be passed to Julia's eval function. \n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.read_pcfg-Tuple{AbstractString}","page":"HerbGrammar.jl","title":"HerbGrammar.read_pcfg","text":"read_pcfg(filepath::AbstractString)::ContextFreeGrammar\n\nReads a probabilistic ContextFreeGrammar from a file provided in filepath.\n\ndanger: Danger\nOnly open trusted grammars. Parts of the grammar can be passed to Julia's eval function. \n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.read_pcsg-Tuple{AbstractString, AbstractString}","page":"HerbGrammar.jl","title":"HerbGrammar.read_pcsg","text":"read_pcsg(grammarpath::AbstractString, constraintspath::AbstractString)::ContextSensitiveGrammar\n\nReads a probabilistic ContextSensitiveGrammar from the files at grammarpath and constraintspath. The grammar path may also point to a ContextFreeGrammar.\n\ndanger: Danger\nOnly open trusted grammars. Parts of the grammar can be passed to Julia's eval function. \n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.remove_rule!-Tuple{Grammar, Int64}","page":"HerbGrammar.jl","title":"HerbGrammar.remove_rule!","text":"remove_rule!(g::Grammar, idx::Int)\n\nRemoves the rule corresponding to idx from the grammar. In order to avoid shifting indices, the rule is replaced with nothing, and all other data structures are updated accordingly.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.return_type-Tuple{Grammar, Int64}","page":"HerbGrammar.jl","title":"HerbGrammar.return_type","text":"return_type(grammar::Grammar, rule_index::Int)::Symbol\n\nReturns the type of the production rule at rule_index.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.return_type-Tuple{Grammar, RuleNode}","page":"HerbGrammar.jl","title":"HerbGrammar.return_type","text":"return_type(grammar::Grammar, node::RuleNode)\n\nGives the return type or nonterminal symbol in the production rule used by node.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.root_node_loc-Tuple{RuleNode}","page":"HerbGrammar.jl","title":"HerbGrammar.root_node_loc","text":"rootnodeloc(root::RuleNode) Returns a NodeLoc pointing to the root node.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.rulenode2expr-Tuple{RuleNode, Grammar}","page":"HerbGrammar.jl","title":"HerbGrammar.rulenode2expr","text":"rulenode2expr(rulenode::RuleNode, grammar::Grammar)\n\nConverts a RuleNode into a Julia expression corresponding to the rule definitions in the grammar. The returned expression can be evaluated with Julia semantics using eval().\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.rulenode_log_probability-Tuple{RuleNode, Grammar}","page":"HerbGrammar.jl","title":"HerbGrammar.rulenode_log_probability","text":"Calculates the log probability associated with a rulenode in a probabilistic grammar.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.rulesoftype-Tuple{RuleNode, Grammar, Symbol, RuleNode}","page":"HerbGrammar.jl","title":"HerbGrammar.rulesoftype","text":"rulesoftype(node::RuleNode, grammar::Grammar, ruletype::Symbol, ignoreNode::RuleNode)\n\nReturns every rule of nonterminal symbol ruletype that is also used in the AbstractRuleNode tree, but not in the ignoreNode subtree.\n\nwarning: Warning\nThe ignoreNode must be a subtree of node for it to have an effect.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.rulesoftype-Tuple{RuleNode, Grammar, Symbol}","page":"HerbGrammar.jl","title":"HerbGrammar.rulesoftype","text":"rulesoftype(node::RuleNode, grammar::Grammar, ruletype::Symbol)\n\nReturns every rule of nonterminal symbol ruletype that is also used in the AbstractRuleNode tree.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.rulesoftype-Tuple{RuleNode, Set{Int64}, RuleNode}","page":"HerbGrammar.jl","title":"HerbGrammar.rulesoftype","text":"rulesoftype(node::RuleNode, ruleset::Set{Int}, ignoreNode::RuleNode)\n\nReturns every rule in the ruleset that is also used in the AbstractRuleNode tree, but not in the ignoreNode subtree.\n\nwarning: Warning\nThe ignoreNode must be a subtree of node for it to have an effect.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.rulesonleft-Tuple{RuleNode, Vector{Int64}}","page":"HerbGrammar.jl","title":"HerbGrammar.rulesonleft","text":"rulesonleft(expr::RuleNode, path::Vector{Int})::Set{Int}\n\nFinds all rules that are used in the left subtree defined by the path.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.store_cfg-Tuple{AbstractString, ContextFreeGrammar}","page":"HerbGrammar.jl","title":"HerbGrammar.store_cfg","text":"store_cfg(filepath::AbstractString, grammar::ContextFreeGrammar)\n\nWrites a ContextFreeGrammar to the file provided by filepath.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.store_csg-Tuple{AbstractString, AbstractString, ContextSensitiveGrammar}","page":"HerbGrammar.jl","title":"HerbGrammar.store_csg","text":"store_csg(grammarpath::AbstractString, constraintspath::AbstractString, g::ContextSensitiveGrammar)\n\nWrites a ContextSensitiveGrammar to the files at grammarpath and constraintspath. The grammarpath file will contain a ContextSensitiveGrammar definition, and the constraintspath file will contain the Constraints of the ContextSensitiveGrammar.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.subsequenceof-Tuple{Vector{Int64}, Vector{Int64}}","page":"HerbGrammar.jl","title":"HerbGrammar.subsequenceof","text":"subsequenceof(vec1::Vector{Int}, vec2::Vector{Int})\n\nChecks if vec1 is a subsequence of vec2.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.swap_node-Tuple{AbstractRuleNode, AbstractRuleNode, Vector{Int64}}","page":"HerbGrammar.jl","title":"HerbGrammar.swap_node","text":"swap_node(expr::AbstractRuleNode, new_expr::AbstractRuleNode, path::Vector{Int})\n\nReplace a node in expr, specified by path, with new_expr. Path is a sequence of child indices, starting from the root node.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.swap_node-Tuple{RuleNode, RuleNode, Int64, RuleNode}","page":"HerbGrammar.jl","title":"HerbGrammar.swap_node","text":"swap_node(expr::RuleNode, node::RuleNode, child_index::Int, new_expr::RuleNode)\n\nReplace child i of a node, a part of larger expr, with new_expr.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#Index","page":"HerbGrammar.jl","title":"Index","text":"","category":"section"},{"location":"HerbGrammar/","page":"HerbGrammar.jl","title":"HerbGrammar.jl","text":"","category":"page"},{"location":"HerbConstraints/#HerbConstraints_docs","page":"HerbConstraints.jl","title":"HerbConstraints.jl Documentation","text":"","category":"section"},{"location":"HerbConstraints/","page":"HerbConstraints.jl","title":"HerbConstraints.jl","text":"CurrentModule=HerbConstraints","category":"page"},{"location":"HerbConstraints/","page":"HerbConstraints.jl","title":"HerbConstraints.jl","text":"Modules = [HerbConstraints]\nOrder = [:type, :const, :macro, :function]","category":"page"},{"location":"HerbConstraints/#HerbConstraints.AbstractMatchNode","page":"HerbConstraints.jl","title":"HerbConstraints.AbstractMatchNode","text":"abstract type AbstractMatchNode\n\nTree structure to which rulenode trees can be matched. Consists of MatchNodes, which can match a specific RuleNode, and MatchVars, which is a variable that can be filled in with any RuleNode.\n\n\n\n\n\n","category":"type"},{"location":"HerbConstraints/#HerbConstraints.ComesAfter","page":"HerbConstraints.jl","title":"HerbConstraints.ComesAfter","text":"ComesAfter <: PropagatorConstraint\n\nA ComesAfter constraint is a PropagatorConstraint containing the following:\n\nrule::Int: A reference to a rule in the grammar\npredecessors: A list of rules in the grammar\n\nThis Constraint enforces that the rule can only be applied if every rule in predecessors is used in the path from the root of the tree to the current hole in the order that they are given. Even though the rules must be in order, there might be other rules inbetween.\n\nFor example, consider the tree 1(a, 2(b, 3(c, d)))):\n\nComesAfter(4, [2, 3]) would enforce that rule 4 can only be used if 2 and 3 are used in the path from the root in that order. Therefore, only hole c and d can be filled with 4.\nComesAfter(4, [1, 3]) also allows c and d to be filled, since 1 and 3 are still used in the correct order. It does not matter that 2 is also used in the path to the root.\nComesAfter(4, [3, 2]) does not allow any hole to be filled with 4, since either the predecessors are either not in the path or in the wrong order for each of the holes. \n\n\n\n\n\n","category":"type"},{"location":"HerbConstraints/#HerbConstraints.ComesAfter-Tuple{Int64, Int64}","page":"HerbConstraints.jl","title":"HerbConstraints.ComesAfter","text":"ComesAfter(rule::Int, predecessor::Int)\n\nCreates a ComesAfter constraint with only a single predecessor. \n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints.Forbidden","page":"HerbConstraints.jl","title":"HerbConstraints.Forbidden","text":"Forbidden <: PropagatorConstraint\n\nThis [PropagatorConstraint] forbids any subtree that matches the pattern given by tree to be generated. A pattern is a tree of AbstractMatchNodes. Such a node can either be a MatchNode, which contains a rule index corresponding to the rule index in the Grammar and the appropriate number of children, similar to RuleNodes. It can also contain a MatchVar, which contains a single identifier symbol. A MatchVar can match any subtree, but if there are multiple instances of the same variable in the pattern, the matched subtrees must be identical. Any rule in the domain that makes the match attempt successful is removed.\n\nFor example, consider the tree 1(a, 2(b, 3(c, 4)))):\n\nForbidden(MatchNode(3, [MatchNode(5), MatchNode(4)])) forbids c to be filled with 5.\nForbidden(MatchNode(3, [MatchVar(:v), MatchNode(4)])) forbids c to be filled, since a [MatchVar] can match any rule, thus making the match attempt successful for the entire domain of c. Therefore, this tree invalid.\nForbidden(MatchNode(3, [MatchVar(:v), MatchVar(:v)])) forbids c to be filled with 4, since that would make both assignments to v equal, which causes a successful match.\n\nwarning: Warning\nThe Forbidden constraint makes use of LocalConstraints to make sure that constraints are also enforced in the future when the context of a Hole changes. Therefore, Forbidden can only be used in implementations that keep track of the LocalConstraints and propagate them at the right moments.\n\n\n\n\n\n","category":"type"},{"location":"HerbConstraints/#HerbConstraints.ForbiddenPath","page":"HerbConstraints.jl","title":"HerbConstraints.ForbiddenPath","text":"ForbiddenPath <: PropagatorConstraint\n\nA [PropagatorConstraint] that forbids a certain derivation sequence. sequence defines the forbidden sequence. Each rule that would complete the sequence when expanding a Hole in an AbstractRuleNode tree is removed from the domain. The derivation sequence is the path from the root to the hole.\n\nFor example, consider the tree 1(a, 2(b, 3(c, d)))):\n\nForbiddenPath([1, 2, 4]) enforces that rule 4 cannot be applied at b, since it completes the sequence. However, it can be applied at a, c and d.\nForbiddenPath([3, 1]) enforces that rule 1 cannot be applied at c or d.\n\n\n\n\n\n","category":"type"},{"location":"HerbConstraints/#HerbConstraints.GrammarContext","page":"HerbConstraints.jl","title":"HerbConstraints.GrammarContext","text":"mutable struct GrammarContext\n\nStructure used to track the context. Contains: \t- the expression being modified \t- the path to the hole that is being expanded, represented as a sequence of child indices. \t e.g., [2, 1] would point to the first child of the second child of the root. \t- a vector with local constraints that should be propagated upon expansion.\n\n\n\n\n\n","category":"type"},{"location":"HerbConstraints/#HerbConstraints.LocalConstraint","page":"HerbConstraints.jl","title":"HerbConstraints.LocalConstraint","text":"abstract type LocalConstraint <: Constraint\n\nAbstract type representing all local constraints. Local constraints correspond to a specific (partial) AbstractRuleNode tree. Each local constraint contains a path to a specific location in the tree. Each local constraint has an implementation of a propagate-function that takes\n\nthe LocalConstraint\na Grammar\na GrammarContext, which most importantly contains the tree and the location in the tree where propagation should take place.\nThe domain which the propagate-function prunes. \n\nThe propagate-function returns a tuple containing\n\nThe pruned domain\nA list of new LocalConstraints\n\nwarning: Warning\nBy default, LocalConstraints are only propagated once. Constraints that have to be propagated more frequently should return themselves in the list of new local constraints.\n\n\n\n\n\n","category":"type"},{"location":"HerbConstraints/#HerbConstraints.LocalForbidden","page":"HerbConstraints.jl","title":"HerbConstraints.LocalForbidden","text":"LocalForbidden\n\nForbids the a subtree that matches the MatchNode tree to be generated at the location provided by the path. Use a Forbidden constraint for enforcing this throughout the entire search space.\n\n\n\n\n\n","category":"type"},{"location":"HerbConstraints/#HerbConstraints.LocalOneOf","page":"HerbConstraints.jl","title":"HerbConstraints.LocalOneOf","text":"Meta-constraint that enforces the disjunction of its given constraints.\n\n\n\n\n\n","category":"type"},{"location":"HerbConstraints/#HerbConstraints.LocalOrdered","page":"HerbConstraints.jl","title":"HerbConstraints.LocalOrdered","text":"Enforces an order over two or more subtrees that fill the variables specified in order when the pattern is applied at the location given by path. Use an Ordered constraint for enforcing this throughout the entire search space.\n\n\n\n\n\n","category":"type"},{"location":"HerbConstraints/#HerbConstraints.MatchFail","page":"HerbConstraints.jl","title":"HerbConstraints.MatchFail","text":"@enum MatchFail hardfail softfail\n\nThis enum is used for distinguishing between two types of failures when trying to match a RuleNode either with another RuleNode or with an AbstractMatchNode\n\nHardfail means that there is no match, and there is no way to fill in the holes to get a match.\nSoftfail means that there is no match, but there might be a way to fill the holes that results in a match.\n\n\n\n\n\n","category":"type"},{"location":"HerbConstraints/#HerbConstraints.MatchNode","page":"HerbConstraints.jl","title":"HerbConstraints.MatchNode","text":"struct MatchNode <: AbstractMatchNode\n\nMatch a specific rulenode, where the grammar rule index is rule_ind and children matches the children of the RuleNode. Example usage:\n\nMatchNode(3, [MatchNode(1), MatchNode(2)])\n\nThis matches RuleNode(3, [RuleNode(1), RuleNode(2)])\n\n\n\n\n\n","category":"type"},{"location":"HerbConstraints/#HerbConstraints.MatchVar","page":"HerbConstraints.jl","title":"HerbConstraints.MatchVar","text":"struct MatchVar <: AbstractMatchNode\n\nMatches anything and assigns it to a variable. The ForbiddenTree constraint will not match if identical variable symbols match to different trees. Example usage:\n\nMatchNode(3, [MatchVar(:x), MatchVar(:x)])\n\nThis matches RuleNode(3, [RuleNode(1), RuleNode(1)]), RuleNode(3, [RuleNode(2), RuleNode(2)]), etc.\n\n\n\n\n\n","category":"type"},{"location":"HerbConstraints/#HerbConstraints.OneOf","page":"HerbConstraints.jl","title":"HerbConstraints.OneOf","text":"Meta-constraint that enforces the disjunction of its given constraints.\n\n\n\n\n\n","category":"type"},{"location":"HerbConstraints/#HerbConstraints.Ordered","page":"HerbConstraints.jl","title":"HerbConstraints.Ordered","text":"Ordered <: PropagatorConstraint\n\nA PropagatorConstraint that enforces a specific order in MatchVar assignments in the pattern defined by tree. A pattern is a tree of AbstractMatchNodes. Such a node can either be a MatchNode, which contains a rule index corresponding to the rule index in the Grammar and the appropriate number of children, similar to RuleNodes. It can also contain a MatchVar, which contains a single identifier symbol. A MatchVar can match any subtree, but if there are multiple instances of the same variable in the pattern, the matched subtrees must be identical.\n\nThe order defines an order between the variable assignments. For example, if the order is [x, y], the constraint will require the assignment to x to be less than or equal to the assignment to y. The order is recursively defined by RuleNode indices. For more information, see Base.isless(rn₁::AbstractRuleNode, rn₂::AbstractRuleNode).\n\nFor example, consider the tree 1(a, 2(b, 3(c, 4)))):\n\nOrdered(MatchNode(3, [MatchVar(:v), MatchVar(:w)]), [:v, :w]) removes every rule with an index of 5 or greater from the domain of c, since that would make the index of the assignment to v greater than the index of the assignment to w, violating the order.\nOrdered(MatchNode(3, [MatchVar(:v), MatchVar(:w)]), [:w, :v]) removes every rule with an index of 4 or less from the domain of c, since that would make the index of the assignment to v less than the index of the assignment to w, violating the order.\n\nwarning: Warning\nThe Ordered constraint makes use of LocalConstraints to make sure that constraints are also enforced in the future when the context of a Hole changes. Therefore, Ordered can only be used in implementations that keep track of the LocalConstraints and propagate them at the right moments.\n\n\n\n\n\n","category":"type"},{"location":"HerbConstraints/#HerbConstraints.PropagatorConstraint","page":"HerbConstraints.jl","title":"HerbConstraints.PropagatorConstraint","text":"PropagatorConstraint <: Constraint\n\nAbstract type representing all propagator constraints. Each propagator constraint has an implementation of a propagate-function that takes\n\nthe PropagatorConstraint\na Grammar\na GrammarContext, which most importantly contains the tree and the location in the tree where propagation should take place.\nThe domain which the propagate-function prunes. \n\nThe propagate-function returns a tuple containing\n\nThe pruned domain\nA list of new LocalConstraints\n\n\n\n\n\n","category":"type"},{"location":"HerbConstraints/#HerbConstraints.RequireOnLeft","page":"HerbConstraints.jl","title":"HerbConstraints.RequireOnLeft","text":"Rules have to be used in the specified order. That is, rule at index K can only be used if rules at indices [1...K-1] are used in the left subtree of the current expression\n\n\n\n\n\n","category":"type"},{"location":"HerbConstraints/#HerbConstraints.@csgrammar_annotated-Tuple{Any}","page":"HerbConstraints.jl","title":"HerbConstraints.@csgrammar_annotated","text":"@csgrammar_annotated Define an annotated grammar and return it as a ContextSensitiveGrammar. Allows for adding optional annotations per rule. As well as that, allows for adding optional labels per rule, which can be referenced in annotations. Syntax is backwards-compatible with @csgrammar. Examples:\n\ng₁ = @csgrammar_annotated begin\n Element = 1\n Element = x\n Element = Element + Element := commutative\n Element = Element * Element := (commutative, transitive)\nend\n\ng₁ = @csgrammar_annotated begin\n Element = 1\n Element = x\n Element = Element + Element := forbidden_path([3, 1])\n Element = Element * Element := (commutative, transitive)\nend\n\ng₁ = @csgrammar_annotated begin\n one:: Element = 1\n variable:: Element = x\n addition:: Element = Element + Element := (\n commutative,\n transitive,\n forbidden_path([:addition, :one]) || forbidden_path([:one, :variable])\n )\n multiplication:: Element = Element * Element := (commutative, transitive)\nend\n\n\n\n\n\n","category":"macro"},{"location":"HerbConstraints/#Base.show-Tuple{IO, MatchNode}","page":"HerbConstraints.jl","title":"Base.show","text":"Base.show(io::IO, node::MatchNode; separator=\",\", last_child::Bool=true)\n\nPrints a found MatchNode given an and the respective children to IO. \n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#Base.show-Tuple{IO, MatchVar}","page":"HerbConstraints.jl","title":"Base.show","text":"Base.show(io::IO, node::MatchVar; separator=\",\", last_child::Bool=true)\n\nPrints a matching variable assignment described by MatchVar to IO.\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints._matchnode2expr","page":"HerbConstraints.jl","title":"HerbConstraints._matchnode2expr","text":"_matchnode2expr(expr::Expr, pattern::MatchVar, grammar::Grammar, j=0)\n\nInternal function for matchnode2expr, recursively iterating over a matched variable and converting it to an expression. This is primarily useful for pretty-printing a pattern. Returns the corresponding expression and the current child index.\n\n\n\n\n\n","category":"function"},{"location":"HerbConstraints/#HerbConstraints._matchnode2expr-2","page":"HerbConstraints.jl","title":"HerbConstraints._matchnode2expr","text":"_matchnode2expr(typ::Symbol, pattern::MatchNode, grammar::Grammar, j=0)\n\nInternal function for matchnode2expr, returning the matched translated symbol. This is primarily useful for pretty-printing a pattern. Returns the corresponding expression, i.e. the variable name and the current child index.\n\n\n\n\n\n","category":"function"},{"location":"HerbConstraints/#HerbConstraints._matchnode2expr-3","page":"HerbConstraints.jl","title":"HerbConstraints._matchnode2expr","text":"_matchnode2expr(typ::Symbol, pattern::MatchVar, grammar::Grammar, j=0)\n\nInternal function for matchnode2expr. This is primarily useful for pretty-printing a pattern. Returns the corresponding expression, i.e. the variable name and the current child index.\n\n\n\n\n\n","category":"function"},{"location":"HerbConstraints/#HerbConstraints._matchnode2expr-4","page":"HerbConstraints.jl","title":"HerbConstraints._matchnode2expr","text":"_matchnode2expr(expr::Expr, pattern::MatchNode, grammar::Grammar, j=0)\n\nInternal function for matchnode2expr, recursively iterating over a matched pattern and converting it to an expression. This is primarily useful for pretty-printing a pattern. Returns the corresponding expression and the current child index.\n\n\n\n\n\n","category":"function"},{"location":"HerbConstraints/#HerbConstraints._pattern_match-Tuple{RuleNode, MatchNode, Dict{Symbol, AbstractRuleNode}}","page":"HerbConstraints.jl","title":"HerbConstraints._pattern_match","text":"_pattern_match(rn::RuleNode, mn::MatchNode, vars::Dict{Symbol, AbstractRuleNode})::Union{Nothing, MatchFail}\n\nTries to match RuleNode rn with MatchNode mn. Modifies the variable assignment dictionary vars. Returns nothing if the match is successful. If the match is unsuccessful, it returns whether it is a softfail or hardfail (see MatchFail docstring)\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints._pattern_match-Tuple{RuleNode, MatchVar, Dict{Symbol, AbstractRuleNode}}","page":"HerbConstraints.jl","title":"HerbConstraints._pattern_match","text":"_pattern_match(rn::RuleNode, mv::MatchVar, vars::Dict{Symbol, AbstractRuleNode})::Union{Nothing, MatchFail}\n\nMatching RuleNode rn with MatchVar mv. If the variable is already assigned, the rulenode is matched with the specific variable value. Returns nothing if the match is succesful. If the match is unsuccessful, it returns whether it is a softfail or hardfail (see MatchFail docstring)\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints._pattern_match_with_hole-Tuple{Hole, MatchNode, Vector{Int64}, Dict{Symbol, AbstractRuleNode}}","page":"HerbConstraints.jl","title":"HerbConstraints._pattern_match_with_hole","text":"_pattern_match_with_hole(::Hole, mn::MatchNode, hole_location::Vector{Int}, ::Dict{Symbol, AbstractRuleNode})::Union{Int, Symbol, MatchFail, Tuple{Symbol, Vector{Int}}}\n\nMatches the Hole with the given MatchNode. \n\nTODO check this behaviour?\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints._pattern_match_with_hole-Tuple{Hole, MatchVar, Vector{Int64}, Dict{Symbol, AbstractRuleNode}}","page":"HerbConstraints.jl","title":"HerbConstraints._pattern_match_with_hole","text":"_pattern_match_with_hole(::Hole, mn::MatchNode, hole_location::Vector{Int}, ::Dict{Symbol, AbstractRuleNode})::Union{Int, Symbol, MatchFail, Tuple{Symbol, Vector{Int}}}\n\nMatches the Hole with the given MatchVar, similar to _pattern_match_with_hole.\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints._pattern_match_with_hole-Tuple{RuleNode, MatchNode, Vector{Int64}, Dict{Symbol, AbstractRuleNode}}","page":"HerbConstraints.jl","title":"HerbConstraints._pattern_match_with_hole","text":"_pattern_match_with_hole(rn::RuleNode, mn::MatchNode, hole_location::Vector{Int}, vars::Dict{Symbol, AbstractRuleNode})::Union{Int, Symbol, MatchFail, Tuple{Symbol, Vector{Int}}}\n\nTries to match RuleNode rn with MatchNode mn and fill in the domain of the hole at hole_location. Returns if match is successful either:\n\nThe id for the node which fills the hole\nA symbol for the variable that fills the hole\nA tuple containing:\nThe variable that matched (the subtree containing) the hole\nThe location of the hole in this subtree\n\nIf the match is unsuccessful, it returns:\n\nhardfail if there are no holes that can be filled in such a way that the match will become succesful\nsoftfail if the match could become successful if the holes are filled in a certain way\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints._pattern_match_with_hole-Tuple{RuleNode, MatchVar, Vector{Int64}, Dict{Symbol, AbstractRuleNode}}","page":"HerbConstraints.jl","title":"HerbConstraints._pattern_match_with_hole","text":"_pattern_match_with_hole(rn::RuleNode, mv::MatchVar, hole_location::Vector{Int}, vars::Dict{Symbol, AbstractRuleNode})::Union{Int, Symbol, MatchFail, Tuple{Symbol, Vector{Int}}}\n\nTries to match RuleNode rn with MatchVar mv and fill in the domain of the hole at hole_location. If the variable name is already assigned in vars, the rulenode is matched with the hole. Otherwise the variable and the hole location are returned.\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints._rulenode_compare-Tuple{RuleNode, RuleNode}","page":"HerbConstraints.jl","title":"HerbConstraints._rulenode_compare","text":"Returns -1 if rn₁ < rn₂\nReturns 0 if rn₁ == rn₂ \nReturns 1 if rn₁ > rn₂\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints._rulenode_match_with_hole-Tuple{RuleNode, RuleNode, Vector{Int64}}","page":"HerbConstraints.jl","title":"HerbConstraints._rulenode_match_with_hole","text":"_rulenode_match_with_hole(rn₁::RuleNode, rn₂::RuleNode, hole_location::Vector{Int})::Union{Int, MatchFail}\n\nMatches two rulenodes. Returns how to fill in the hole in rn₁ to make it match rn₂ if:\n\nrn₁ has a single hole at the provided location\nrn₂ doesn't have any holes\nrn₁ matches rn₂ apart from the single hole location.\n\nIf the match fails, it returns whether it is a softfail or a hardfail (see MatchFail docstring)\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints.addparent!-Tuple{GrammarContext, Int64}","page":"HerbConstraints.jl","title":"HerbConstraints.addparent!","text":"addparent!(context::GrammarContext, parent::Int)\n\nAdds a parent to the context. The parent is defined by the grammar rule id.\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints.annotation2constraint-Tuple{Any, Int64, Vector{String}}","page":"HerbConstraints.jl","title":"HerbConstraints.annotation2constraint","text":"Converts an annotation to a constraint. commutative: creates an Ordered constraint transitive: creates an (incorrect) Forbidden constraint forbidden_path(path::Vector{Union{Symbol, Int}}): creates a ForbiddenPath constraint with the original rule included ... || ...: creates a OneOf constraint (also works with ... || ... || ... et cetera, though not very performant)\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints.check_tree-Tuple{ComesAfter, Grammar, AbstractRuleNode}","page":"HerbConstraints.jl","title":"HerbConstraints.check_tree","text":"Checks if the given tree abides the constraint.\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints.check_tree-Tuple{Forbidden, Grammar, RuleNode}","page":"HerbConstraints.jl","title":"HerbConstraints.check_tree","text":"check_tree(c::Forbidden, g::Grammar, tree::RuleNode)::Bool\n\nChecks if the given AbstractRuleNode tree abides the Forbidden constraint.\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints.check_tree-Tuple{ForbiddenPath, Grammar, AbstractRuleNode}","page":"HerbConstraints.jl","title":"HerbConstraints.check_tree","text":"Checks if the given tree abides the constraint.\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints.check_tree-Tuple{HerbConstraints.Condition, Grammar, RuleNode}","page":"HerbConstraints.jl","title":"HerbConstraints.check_tree","text":"Checks if the given tree abides the constraint.\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints.check_tree-Tuple{OneOf, Grammar, AbstractRuleNode}","page":"HerbConstraints.jl","title":"HerbConstraints.check_tree","text":"Checks if the given tree abides the constraint.\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints.check_tree-Tuple{Ordered, Grammar, RuleNode}","page":"HerbConstraints.jl","title":"HerbConstraints.check_tree","text":"check_tree(c::Ordered, g::Grammar, tree::RuleNode)::Bool\n\nChecks if the given AbstractRuleNode tree abides the Ordered constraint.\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints.check_tree-Tuple{RequireOnLeft, Grammar, AbstractRuleNode}","page":"HerbConstraints.jl","title":"HerbConstraints.check_tree","text":"Checks if the given tree abides the constraint.\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints.copy_and_insert-Tuple{GrammarContext, Int64}","page":"HerbConstraints.jl","title":"HerbConstraints.copy_and_insert","text":"copy_and_insert(old_context::GrammarContext, parent::Int)\n\nCopies the given context and insert the parent in the node location.\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints.make_smaller_or_equal-Tuple{RuleNode, RuleNode, Vector{Int64}, Vector{Int64}}","page":"HerbConstraints.jl","title":"HerbConstraints.make_smaller_or_equal","text":"Filters the domain of the hole at hole_location in rn₁ to make rn₁ be ordered before rn₂. Returns the filtered domain, and a boolean indicating if this constraint can be deleted.\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints.matchnode2expr-Tuple{MatchNode, Grammar}","page":"HerbConstraints.jl","title":"HerbConstraints.matchnode2expr","text":"matchnode2expr(pattern::MatchNode, grammar::Grammar)\n\nConverts a MatchNode tree into a Julia expression. This is primarily useful for pretty-printing a pattern. Returns the corresponding expression.\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints.matchnode2expr-Tuple{MatchVar, Grammar}","page":"HerbConstraints.jl","title":"HerbConstraints.matchnode2expr","text":"matchnode2expr(pattern::MatchVar, grammar::Grammar)\n\nConverts a MatchVar into an expression by returning the variable directly. This is primarily useful for pretty-printing a pattern.\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints.propagate-Tuple{Forbidden, Grammar, GrammarContext, Vector{Int64}, Union{Nothing, HoleReference}}","page":"HerbConstraints.jl","title":"HerbConstraints.propagate","text":"propagate(c::Forbidden, g::Grammar, context::GrammarContext, domain::Vector{Int})::Tuple{Vector{Int}, Vector{LocalConstraint}}\n\nPropagates the Forbidden constraint. It removes the rules from the domain that would complete the forbidden tree.\n\nwarning: Warning\nThe Forbidden constraint makes use of LocalConstraints to make sure that constraints are also enforced in the future when the context of a Hole changes. Therefore, Forbidden can only be used in implementations that keep track of the LocalConstraints and propagate them at the right moments.\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints.propagate-Tuple{ForbiddenPath, Grammar, GrammarContext, Vector{Int64}, Union{Nothing, HoleReference}}","page":"HerbConstraints.jl","title":"HerbConstraints.propagate","text":"Propagates the ForbiddenPath constraint. It removes the elements from the domain that would complete the forbidden sequence.\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints.propagate-Tuple{HerbConstraints.LocalOneOf, Grammar, GrammarContext, Vector{Int64}, Union{Nothing, HoleReference}}","page":"HerbConstraints.jl","title":"HerbConstraints.propagate","text":"Propagates the LocalOneOf constraint. It enforces that at least one of its given constraints hold.\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints.propagate-Tuple{LocalForbidden, Grammar, GrammarContext, Vector{Int64}, Union{Nothing, HoleReference}}","page":"HerbConstraints.jl","title":"HerbConstraints.propagate","text":"Propagates the LocalForbidden constraint. It removes rules from the domain that would make the RuleNode at the given path match the pattern defined by the MatchNode.\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints.propagate-Tuple{LocalOrdered, Grammar, GrammarContext, Vector{Int64}, Union{Nothing, HoleReference}}","page":"HerbConstraints.jl","title":"HerbConstraints.propagate","text":"Propagates the LocalOrdered constraint. It removes rules from the domain that would violate the order of variables as defined in the constraint.\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints.propagate-Tuple{OneOf, Grammar, GrammarContext, Vector{Int64}, Union{Nothing, HoleReference}}","page":"HerbConstraints.jl","title":"HerbConstraints.propagate","text":"Propagates the OneOf constraint. It enforces that at least one of its given constraints hold.\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints.propagate-Tuple{Ordered, Grammar, GrammarContext, Vector{Int64}, Union{Nothing, HoleReference}}","page":"HerbConstraints.jl","title":"HerbConstraints.propagate","text":"propagate(c::Ordered, g::Grammar, context::GrammarContext, domain::Vector{Int})::Tuple{Vector{Int}, Vector{LocalConstraint}}\n\nPropagates the Ordered constraint. Any rule that violates the order as defined by the contraint is removed from the domain.\n\nwarning: Warning\nThe Ordered constraint makes use of LocalConstraints to make sure that constraints are also enforced in the future when the context of a Hole changes. Therefore, Ordered can only be used in implementations that keep track of the LocalConstraints and propagate them at the right moments.\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints.propagate-Tuple{RequireOnLeft, Grammar, GrammarContext, Vector{Int64}, Union{Nothing, HoleReference}}","page":"HerbConstraints.jl","title":"HerbConstraints.propagate","text":"Propagates the RequireOnLeft constraint. It removes every element from the domain that does not have a necessary predecessor in the left subtree.\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#Index","page":"HerbConstraints.jl","title":"Index","text":"","category":"section"},{"location":"HerbConstraints/","page":"HerbConstraints.jl","title":"HerbConstraints.jl","text":"","category":"page"},{"location":"tutorials/advanced_search/#Getting-started","page":"Advanced Search Procedures","title":"Getting started","text":"","category":"section"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"You can either paste this code into the Julia REPL or into a seperate file, e.g. get_started.jl followed by running julia get_started.jl.","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"To start, we import the necessary packages.","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"using HerbGrammar, HerbData, HerbSearch, HerbInterpret","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"We start with the same simple grammar from the main file get_started.md.","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"g = @cfgrammar begin\n Number = |(1:2)\n Number = x\n Number = Number + Number\n Number = Number * Number\nend","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"We use a simple problem.","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":" problem = Problem([IOExample(Dict(:x => x), 2x+1) for x ∈ 1:5])","category":"page"},{"location":"tutorials/advanced_search/#Parameters","page":"Advanced Search Procedures","title":"Parameters","text":"","category":"section"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"We can use a search strategy, where we can specify different parameters. For example, by setting the max_depth, we limit the depth of the search. In the next example, we can see the effect of the depth on the number of allocations considered. ","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"solution = @time search(g, problem, :Number, max_depth=3)\n>>> 0.003284 seconds (50.08 k allocations: 2.504 MiB)\nprintln(solution)\n>>> (x + 1) + x\n\nsolution = @time search(g, problem, :Number, max_depth=6)\n>>> 0.005696 seconds (115.28 k allocations: 5.910 MiB)\nprintln(solution)\n>>> (1 + x) + x","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"Another parameter to use is max_enumerations, which limits the number of programs that can be tested at evaluation. We can see the number of enumerations necessary to solve a simple problem in the next example.","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"for i in range(1, 50)\n println(i, \" enumerations\")\n solution = @time search(g, problem, :Number, max_enumerations=i)\n println(solution)\nend\n\n>>> ....\n>>> 23 enums: nothing\n>>> 0.010223 seconds (117.01 k allocations: 5.935 MiB, 44.23% gc time)\n>>> 24 enums: (1 + x) + x\n>>> 0.005305 seconds (117.01 k allocations: 5.935 MiB)\n>>> 25 enums: (1 + x) + x\n>>> 0.005381 seconds (117.01 k allocations: 5.935 MiB)\n>>> ...","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"We see that only when i >= 24, there is a result, after that, increasing i does not have any effect on the number of allocations. ","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"A final parameter we consider here is allow_evaluation_errors, which is false by default. When this is set to true, the program will still run even when an exception is thrown during evaluation. To see the effect of this, we create a new grammar. We can also retrieve the error together with the solution from the search method.","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"g = @cfgrammar begin\n Number = 1\n List = []\n Index = List[Number]\nend\n\nproblem = Problem([IOExample(Dict(), x) for x ∈ 1:5])\nsolution = search(g, problem, :Index, max_depth=2, allow_evaluation_errors=true)\nprintln(solution)\n>>> nothing","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"There is also another search method called search_best which return both the solution and the possible error. The method returns the best program found so far. In this case, we can also see the error:","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"solution, error = search_best(g, problem, :Index, max_depth=2, allow_evaluation_errors=true)\nprintln(solution)\n>>> nothing\nprintln(error)\n>>> 9223372036854775807 # or: typemax(Int)","category":"page"},{"location":"tutorials/advanced_search/#Search-methods","page":"Advanced Search Procedures","title":"Search methods","text":"","category":"section"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"We now show examples of using different search procedures.","category":"page"},{"location":"tutorials/advanced_search/#Breadth-First-Search","page":"Advanced Search Procedures","title":"Breadth-First Search","text":"","category":"section"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"The breadth-first search will first enumerate all possible programs at the same depth before considering a program with a depth of one more. A tree of the grammar is returned with programs ordered in increasing sizes. We can first collect the programs that have a max-depth of 2 and a max_size of infinite (integer maximum value), where the starting symbol is of type Real. This function uses a default heuristic 'left-most first', such that the left-most child in the tree is always explored first. ","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"g1 = @cfgrammar begin\n Real = 1 | 2\n Real = Real * Real\nend\nprograms = collect(get_bfs_enumerator(g1, 2, typemax(Int), :Real))","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"We can test that this function returns the correct functions and all functions. ","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"answer_programs = [\n RuleNode(1),\n RuleNode(2),\n RuleNode(3, [RuleNode(1), RuleNode(1)]),\n RuleNode(3, [RuleNode(1), RuleNode(2)]),\n RuleNode(3, [RuleNode(2), RuleNode(1)]),\n RuleNode(3, [RuleNode(2), RuleNode(2)])\n]\n\nprintln(all(p ∈ programs for p ∈ answer_programs))\n>>> true","category":"page"},{"location":"tutorials/advanced_search/#Depth-First-Search","page":"Advanced Search Procedures","title":"Depth-First Search","text":"","category":"section"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"In depth-first search, we first explore a certain branch of the search tree till the max_depth or a correct program is reached before we consider the next branch. ","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"g1 = @cfgrammar begin\nReal = 1 | 2\nReal = Real * Real\nend\nprograms = collect(get_dfs_enumerator(g1, 2, typemax(Int), :Real))\nprintln(programs)\n>>> RuleNode[1,, 3{1,1}, 3{1,2}, 3{2,1}, 3{2,2}, 2,]","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"get_dfs_enumerator also has a default left-most heuristic and we consider what the difference is in output. ","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"g1 = @cfgrammar begin\n Real = 1 | 2\n Real = Real * Real\nend\nprograms = collect(get_dfs_enumerator(g1, 2, typemax(Int), :Real, heuristic_rightmost))\nprintln(programs)\n>>> RuleNode[1,, 3{1,1}, 3{2,1}, 3{1,2}, 3{2,2}, 2,]","category":"page"},{"location":"tutorials/advanced_search/#Metropolis-Hastings","page":"Advanced Search Procedures","title":"Metropolis-Hastings","text":"","category":"section"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"One of the stochastic search methods that is implemented is Metropolis-Hastings (MH), which samples from a distribution of programs based on the grammar. For more information on MH, see for example this webpage.","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"For the example below, we use this grammar and helper function.","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"grammar = @csgrammar begin\n X = |(1:5)\n X = X * X\n X = X + X\n X = X - X\n X = x\nend\nfunction create_problem(f, range=20)\n examples = [IOExample(Dict(:x => x), f(x)) for x ∈ 1:range]\n return Problem(examples), examples\nend","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"The below example uses a simple arithmetic example. As the search method is stochastic, different programs may be returned, as shown below.","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"e = Meta.parse(\"x -> x * x + 4\")\nproblem, examples = create_problem(eval(e))\nenumerator = get_mh_enumerator(examples, mean_squared_error)\nprogram, cost = search_best(grammar, problem, :X, enumerator=enumerator, error_function=mse_error_function, max_depth=3)\n\n>>> (:(x * x - (1 - 5)), 0)\n>>> (:(4 + x * x), 0)\n>>> (:(x * x + 4), 0)","category":"page"},{"location":"tutorials/advanced_search/#Very-Large-Scale-Neighbourhood-Search","page":"Advanced Search Procedures","title":"Very Large Scale Neighbourhood Search","text":"","category":"section"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"The second implemented stochastic search method is VLSN, which search for a local optimum in the neighbourhood. For more information, see this article.","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"Given the same grammar as before, we can try with some simple examples.","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"e = Meta.parse(\"x -> 10\")\nmax_depth = 2\nproblem, examples = create_problem(eval(e))\nenumerator = get_vlsn_enumerator(examples, mean_squared_error, max_depth)\nprogram, cost = search_best(grammar, problem, :X, enumerator=enumerator, error_function=mse_error_function, max_depth=max_depth)\n>>> (:(5 + 5), 0)","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"e = Meta.parse(\"x -> x\")\nmax_depth = 1\nproblem, examples = create_problem(eval(e))\nenumerator = get_vlsn_enumerator(examples, mean_squared_error, max_depth)\nprogram, cost = search_best(grammar, problem, :X, enumerator=enumerator, error_function=mse_error_function, max_depth=max_depth)\n>>> (:x, 0)","category":"page"},{"location":"tutorials/advanced_search/#Simulated-Annealing","page":"Advanced Search Procedures","title":"Simulated Annealing","text":"","category":"section"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"The third stochastic search method is called simulated annealing, is another hill-climbing method to find local optima. For more information, see this page.","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"We try the example from earlier, but now we can additionally define the initial_temperature of the algorithm, which is 1 by default. Two possible answers to the program are given as well.","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"e = Meta.parse(\"x -> x * x + 4\")\ninitial_temperature = 2\nproblem, examples = create_problem(eval(e))\nenumerator = get_sa_enumerator(examples, mean_squared_error, initial_temperature)\nprogram, cost = search_best(grammar, problem, :X, enumerator=enumerator, error_function=mse_error_function, max_depth=3)\n>>> (:(4 + x * x), 0)\n>>> (:(x * x + (5 - 1)), 0)","category":"page"},{"location":"tutorials/advanced_search/#Genetic-Search","page":"Advanced Search Procedures","title":"Genetic Search","text":"","category":"section"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"Genetic search is a type of evolutionary algorithm, which will simulate the process of natural selection and return the 'fittest' program of the population. For more information, see here.","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"We show the example of finding a lambda function.","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"e = Meta.parse(\"x -> 3 * x * x + (x + 2)\")\nproblem, examples = create_problem(eval(e))\nenumerator = get_genetic_enumerator(examples, \n initial_population_size = 10,\n mutation_probability = 0.8,\n maximum_initial_population_depth = 3,\n)\nprogram, cost = search_best(grammar, problem, :X, enumerator=enumerator, error_function=mse_error_function, max_depth=nothing, max_time=20)\n>>> (:(((((x - 5) + x) + x * x) + 1) + (((((2 + x * x) + 3) + x * x) + 1) - ((x - x) + x))), 0)\n>>> (:(x * 1 + (x * ((x + x) + x * 1) + (1 + 1) * 1)), 0)\n>>> (:((((x + x) + x) + 2) * x + ((x - x) + (2 - x))), 0)","category":"page"},{"location":"tutorials/advanced_search/#Other-functionality","page":"Advanced Search Procedures","title":"Other functionality","text":"","category":"section"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"Finally, we showcase two other functionalities of HerbSearch, sampling and heuristics.","category":"page"},{"location":"tutorials/advanced_search/#Sampling","page":"Advanced Search Procedures","title":"Sampling","text":"","category":"section"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"Sampling is implemented for the different stochastic search methods.","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"We consider here a simple grammar, which gives different programs for different search depths.","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"grammar = @cfgrammar begin \n A = B | C | F\n F = G\n C = D\n D = E\nend\n\n# A->B (depth 1) or A->F->G (depth 2) or A->C->D->E (depth 3)\n\n# For depth ≤ 1 the only option is A->B\nexpression = rand(RuleNode, grammar, :A, 1)\n@assert rulenode2expr(expression, grammar) in [:B,:C,:F]\n\n# For depth ≤ 2 the two options are A->B (depth 1) and A->B->G| A->C->G | A->F->G (depth 2)\nexpression = rand(RuleNode, grammar, :A, 2)\n@assert rulenode2expr(expression,grammar) in [:B,:C,:F,:G]","category":"page"},{"location":"tutorials/advanced_search/#Heuristics","page":"Advanced Search Procedures","title":"Heuristics","text":"","category":"section"},{"location":"tutorials/advanced_search/#Examples-with-larger-domains-and-constraints","page":"Advanced Search Procedures","title":"Examples with larger domains and constraints","text":"","category":"section"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"Here, we showcase a few examples using a more complicated grammar and a few types of constraints","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"# Expects to return a program equivalent to 1 + (1 - x) = 2 - x\n\ng₁ = @csgrammar begin\n Element = |(1 : 3) # 1 - 3\n Element = Element + Element # 4\n Element = 1 - Element # 5\n Element = x # 6\nend\n\naddconstraint!(g₁, ComesAfter(6, [5]))\n\nexamples = [\n IOExample(Dict(:x => 0), 2),\n IOExample(Dict(:x => 1), 1),\n IOExample(Dict(:x => 2), 0)\n]\nproblem = Problem(examples)\nsolution = search(g₁, problem, :Element, max_depth=3)\n\n@assert test_with_input(SymbolTable(g₁), solution, Dict(:x => -2)) == 4\n\n# Expects to return a program equivalent to 4 + x * (x + 3 + 3) = x^2 + 6x + 4\n\ng₂ = @csgrammar begin\n Element = Element + Element + Element # 1\n Element = Element + Element * Element # 2\n Element = x # 3\n Element = |(3 : 5) # 4\nend\n\n# Restrict .. + x * x\naddconstraint!(g₂, Forbidden(MatchNode(2, [MatchVar(:x), MatchNode(3), MatchNode(3)])))\n# Restrict 4 and 5 in lower level\naddconstraint!(g₂, ForbiddenPath([2, 1, 5]))\naddconstraint!(g₂, ForbiddenPath([2, 1, 6]))\n\nexamples = [\n IOExample(Dict(:x => 1), 11)\n IOExample(Dict(:x => 2), 20)\n IOExample(Dict(:x => -1), -1)\n]\nproblem = Problem(examples)\nsolution = search(g₂, problem, :Element)\n\n@assert test_with_input(SymbolTable(g₂), solution, Dict(:x => 0)) == 4\n\n# Expects to return a program equivalent to (1 - (((1 - x) - 1) - 1)) - 1 = x + 1\n\ng₃ = @csgrammar begin\n Element = |(1 : 20) # 1 - 20\n Element = Element - 1 # 21\n Element = 1 - Element # 22\n Element = x # 23\nend\n\naddconstraint!(g₃, ComesAfter(23, [22, 21]))\naddconstraint!(g₃, ComesAfter(22, [21]))\n\nexamples = [\n IOExample(Dict(:x => 1), 2)\n IOExample(Dict(:x => 10), 11)\n]\nproblem = Problem(examples)\nsolution = search(g₃, problem, :Element)\n\n@assert test_with_input(SymbolTable(g₃), solution, Dict(:x => 0)) == 1\n@assert test_with_input(SymbolTable(g₃), solution, Dict(:x => 100)) == 101\n\n# Expects to return a program equivalent to 18 + 4x\n\ng₄ = @csgrammar begin\n Element = |(0 : 20) # 1 - 20\n Element = Element + Element + Element # 21\n Element = Element + Element * Element # 22\n Element = x # 23\nend\n\n# Enforce ordering on + +\naddconstraint!(g₄, Ordered(\n MatchNode(21, [MatchVar(:x), MatchVar(:y), MatchVar(:z)]),\n [:x, :y, :z]\n))\n\nexamples = [\n IOExample(Dict(:x => 1), 22),\n IOExample(Dict(:x => 0), 18),\n IOExample(Dict(:x => -1), 14)\n]\nproblem = Problem(examples)\nsolution = search(g₄, problem, :Element)\n\n@assert test_with_input(SymbolTable(g₄), solution, Dict(:x => 100)) == 418\n\n# Expects to return a program equivalent to (x == 2) ? 1 : (x + 2)\n\ng₅ = @csgrammar begin\n Element = Number # 1\n Element = Bool # 2\n\n Number = |(1 : 3) # 3-5\n \n Number = Number + Number # 6\n Bool = Number ≡ Number # 7\n Number = x # 8\n \n Number = Bool ? Number : Number # 9\n Bool = Bool ? Bool : Bool # 10\nend\n\n# Forbid ? = ?\naddconstraint!(g₅, Forbidden(MatchNode(7, [MatchVar(:x), MatchVar(:x)])))\n# Order =\naddconstraint!(g₅, Ordered(MatchNode(7, [MatchVar(:x), MatchVar(:y)]), [:x, :y]))\n# Order +\naddconstraint!(g₅, Ordered(MatchNode(6, [MatchVar(:x), MatchVar(:y)]), [:x, :y]))\n\nexamples = [\n IOExample(Dict(:x => 0), 2)\n IOExample(Dict(:x => 1), 3)\n IOExample(Dict(:x => 2), 1)\n]\nproblem = Problem(examples)\nsolution = search(g₅, problem, :Element)\n\n@assert test_with_input(SymbolTable(g₅), solution, Dict(:x => 3)) == 5","category":"page"},{"location":"HerbInterpret/#HerbInterpret_docs","page":"HerbInterpret.jl","title":"HerbInterpret.jl Documentation","text":"","category":"section"},{"location":"HerbInterpret/","page":"HerbInterpret.jl","title":"HerbInterpret.jl","text":"CurrentModule=HerbInterpret","category":"page"},{"location":"HerbInterpret/","page":"HerbInterpret.jl","title":"HerbInterpret.jl","text":"Modules = [HerbInterpret]\nOrder = [:type, :const, :macro, :function]","category":"page"},{"location":"HerbInterpret/#HerbInterpret.evaluate_program-Tuple{RuleNode, Vector{<:Example}, Grammar, Function}","page":"HerbInterpret.jl","title":"HerbInterpret.evaluate_program","text":"evaluate_program(program::RuleNode, examples::Vector{<:Example}, grammar::Grammar, evaluation_function::Function)\n\nRuns a program on the examples and returns tuples of actual desired output and the program's output\n\n\n\n\n\n","category":"method"},{"location":"HerbInterpret/#HerbInterpret.execute_on_examples-Tuple{Dict{Symbol, Any}, Any, Vector{Dict{Symbol, Any}}}","page":"HerbInterpret.jl","title":"HerbInterpret.execute_on_examples","text":"execute_on_examples(tab::SymbolTable, expr::Any, example_inputs::Vector{Dict{Symbol, Any}})::Vector{Any}\n\nExecutes a given expression on a set of inputs and returns the respective outputs. WARNING: This function throws exceptions that are caused in the given expression. These exceptions have to be handled by the caller of this function.\n\n\n\n\n\n","category":"method"},{"location":"HerbInterpret/#HerbInterpret.interpret-Tuple{Dict{Symbol, Any}, Any}","page":"HerbInterpret.jl","title":"HerbInterpret.interpret","text":"interpret(tab::SymbolTable, ex::Expr)\n\nEvaluates an expression without compiling it. Uses AST and symbol lookups. Only supports :call and :(=) expressions at the moment.\n\nExample usage:\n\ntab = SymbolTable(:f => f, :x => x)\nex = :(f(x))\ninterpret(tab, ex)\n\nWARNING: This function throws exceptions that are caused in the given expression. These exceptions have to be handled by the caller of this function.\n\n\n\n\n\n","category":"method"},{"location":"HerbInterpret/#HerbInterpret.test_all_examples-Tuple{Dict{Symbol, Any}, Any, Vector{Example}}","page":"HerbInterpret.jl","title":"HerbInterpret.test_all_examples","text":"test_all_examples(tab::SymbolTable, expr::Any, examples::Vector{Example})::Vector{Bool}\n\nRuns the interpreter on all examples with the given input table and expression. The symbol table defines everything (functions, symbols) that are not input variables to the program to be synthesised. Returns a list of true/false values indicating if the expression satisfies the corresponding example. WARNING: This function throws exceptions that are caused in the given expression. These exceptions have to be handled by the caller of this function.\n\n\n\n\n\n","category":"method"},{"location":"HerbInterpret/#HerbInterpret.test_examples-Tuple{Dict{Symbol, Any}, Any, Vector{Example}}","page":"HerbInterpret.jl","title":"HerbInterpret.test_examples","text":"test_examples(tab::SymbolTable, expr::Any, examples::Vector{Example})::Bool\n\nEvaluates all examples and returns true iff all examples pass. Shortcircuits as soon as an example is found for which the program doesn't work. Returns false if one of the examples produces an error.\n\n\n\n\n\n","category":"method"},{"location":"HerbInterpret/#HerbInterpret.test_with_input-Tuple{Dict{Symbol, Any}, Any, Dict}","page":"HerbInterpret.jl","title":"HerbInterpret.test_with_input","text":"test_with_input(tab::SymbolTable, expr::Any, input::Dict)\n\nInterprets an expression or symbol with the given symboltable and the input. WARNING: This function throws exceptions that are caused in the given expression. These exceptions have to be handled by the caller of this function.\n\n\n\n\n\n","category":"method"},{"location":"HerbInterpret/#Index","page":"HerbInterpret.jl","title":"Index","text":"","category":"section"},{"location":"HerbInterpret/","page":"HerbInterpret.jl","title":"HerbInterpret.jl","text":"","category":"page"},{"location":"tutorials/getting_started_with_herb/#Search","page":"A more verbose getting started with Herb.jl","title":"Search","text":"","category":"section"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"This notebook describes how you can search a program space as defined by a grammar. Specifically, we will look at example-based search, where the goal is to find a program that is able to transform the inputs of every example to the corresponding output.","category":"page"},{"location":"tutorials/getting_started_with_herb/#Setup","page":"A more verbose getting started with Herb.jl","title":"Setup","text":"","category":"section"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"First, we start with the setup. We need to access to all the function in the Herb.jl framework.","category":"page"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"using Herb","category":"page"},{"location":"tutorials/getting_started_with_herb/#Defining-the-program-space","page":"A more verbose getting started with Herb.jl","title":"Defining the program space","text":"","category":"section"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"Next, we start by creating a grammar. We define a context-free grammar which is just a simple set of production rules for defining combinations of terminal symbols (in our case real numbers). ","category":"page"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"Contrary, we could define a context-sensitive grammar, when the production rules only hold in a certain context. However, for more information on this, please see example2_defining_grammars.ipynb.","category":"page"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"For now, we specify a simple grammar for dealing with integers and explain all the rules individually:","category":"page"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"First, we specify our interval [0:9] on real numbers and also constrain them to be integer.\nThen, we can also use the variable x to hold an integer.\nThe third rule determines we can add two integers.\nThe fourth rule determines we can subtract an integer from another.\nFinally, we also allow the multiplication of two integers.","category":"page"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"If you run this cell, you can see all the rules rolled out.","category":"page"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"g = Herb.HerbGrammar.@cfgrammar begin\n Real = |(0:9)\n Real = x\n Real = Real + Real\n Real = Real - Real\n Real = Real * Real\nend","category":"page"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"1: Real = 0\n2: Real = 1\n3: Real = 2\n4: Real = 3\n5: Real = 4\n6: Real = 5\n7: Real = 6\n8: Real = 7\n9: Real = 8\n10: Real = 9\n11: Real = x\n12: Real = Real + Real\n13: Real = Real - Real\n14: Real = Real * Real","category":"page"},{"location":"tutorials/getting_started_with_herb/#Defining-the-problem","page":"A more verbose getting started with Herb.jl","title":"Defining the problem","text":"","category":"section"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"As mentioned before, we are looking at example-based search. This means that the problem is defined by a set of input-output examples. A single example hence consists of an input and an output. The input is defined as a dictionary, with a value assigned to each variable in the grammar. It is important to write the variable name as a Symbol instead of a string. A Symbol in Julia is written with a colon prefix, i.e. :x. The output of the input-output example is just a single value for this specific grammar, but could possibly relate to e.g. arrays of values, too.","category":"page"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"In the cell below we automatically generate some examples for x assigning values 1-5.","category":"page"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"# Create input-output examples\nexamples = [Herb.HerbData.IOExample(Dict(:x => x), 3x+5) for x ∈ 1:5]","category":"page"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"5-element Vector{Main.Herb.HerbData.IOExample}:\n Main.Herb.HerbData.IOExample(Dict{Symbol, Any}(:x => 1), 8)\n Main.Herb.HerbData.IOExample(Dict{Symbol, Any}(:x => 2), 11)\n Main.Herb.HerbData.IOExample(Dict{Symbol, Any}(:x => 3), 14)\n Main.Herb.HerbData.IOExample(Dict{Symbol, Any}(:x => 4), 17)\n Main.Herb.HerbData.IOExample(Dict{Symbol, Any}(:x => 5), 20)","category":"page"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"Now that we have some input-output examples, we can define the problem. Next to the examples, a problem also contains a name meant to link to the file path, which can be used to keep track of current examples. For now, this is irrelevant, and you can give the program any name you like.","category":"page"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"problem = Herb.HerbData.Problem(examples, \"example\")","category":"page"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"Main.Herb.HerbData.Problem(Main.Herb.HerbData.Example[Main.Herb.HerbData.IOExample(Dict{Symbol, Any}(:x => 1), 8), Main.Herb.HerbData.IOExample(Dict{Symbol, Any}(:x => 2), 11), Main.Herb.HerbData.IOExample(Dict{Symbol, Any}(:x => 3), 14), Main.Herb.HerbData.IOExample(Dict{Symbol, Any}(:x => 4), 17), Main.Herb.HerbData.IOExample(Dict{Symbol, Any}(:x => 5), 20)], \"example\")","category":"page"},{"location":"tutorials/getting_started_with_herb/#Searching","page":"A more verbose getting started with Herb.jl","title":"Searching","text":"","category":"section"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"Now that we have defined the search space and the goal of the search, we can start the search. ","category":"page"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"Of course, our problem is underdefined as there might be multiple programs that satisfy our examples. Let us consider the case where we also have a ternary if-then-else operator and standard boolean operators in our grammar: we could synthesize the program x ≤ 5 ? 3x+5 : 0. This program satisfies all our examples, but we don't expect it to generalize very well.","category":"page"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"In general, we assume that a smaller program is more general than a larger program. Therefore, we search for the smallest program in our grammar that satisfies our examples. This can be done using a breadth-first search over the program/search space.","category":"page"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"This search is very basic; it makes use of an enumeration technique, where we enumerate programs one-by-one until we find a program that matches our examples. The search procedure has a built-in default evaluator to verify the candidate programs with the given input. The search procedure also has a built-in search procedure using breadth-first search. ","category":"page"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"So, we only need to give our grammar and the problem to our search procedure, along with a starting Symbol, in our case a Real.","category":"page"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"Herb.HerbSearch.search(g, problem, :Real)","category":"page"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":":(x * 3 + 5)","category":"page"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"As you can see, the search procedure found the correct program!","category":"page"},{"location":"tutorials/getting_started_with_herb/#Defining-the-search-procedure","page":"A more verbose getting started with Herb.jl","title":"Defining the search procedure","text":"","category":"section"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"In the previous case, we used the built-ins of the search procedure. However, we can also give a custom enumerator to the search procedure and define a few more values.","category":"page"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"We first define a new problem to test with, we are looking for the programs that can compute the value167. We immediately pass the examples to the problem and then set up the new search.","category":"page"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"Search is done by passing the grammar, the problem and the starting point like before. We now also specify the enumeration function to be used, and now we use depth-first search. Then, we give the maximum depth of the programs we want to search for (3), the maximum number of nodes in the Abstract Syntax Tree that exists during search (10), and the maximum time in seconds allowed for the search.","category":"page"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"problem = Herb.HerbData.Problem([Herb.HerbData.IOExample(Dict(:x => x), 168) for x ∈ 1:5], \"example2\")\nexpr = Herb.HerbSearch.search(g, problem, :Real, enumerator=Herb.HerbSearch.get_dfs_enumerator, max_depth=4, max_size=30, max_time=180)\nprint(expr)","category":"page"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"nothing","category":"page"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"We see that our synthesizer can find a program to construct the value 168, though a fun experiment would be trying to get the value 167, what do you think would happen? If you want you can try below.","category":"page"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"In any case, this concludes our first introduction to the Herb.jl program synthesis framework. You can see more examples in this repository, or explore yourself. Enjoy!","category":"page"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"problem = Herb.HerbData.Problem([Herb.HerbData.IOExample(Dict(:x => x), 167) for x ∈ 1:5], \"example2\")\nexpr = Herb.HerbSearch.search(g, problem, :Real)\nprintln(expr)","category":"page"},{"location":"get_started/#Getting-Started","page":"Getting Started","title":"Getting Started","text":"","category":"section"},{"location":"get_started/","page":"Getting Started","title":"Getting Started","text":"You can either paste this code into the Julia REPL or into a seperate file, e.g. get_started.jl followed by julia get_started.jl.","category":"page"},{"location":"get_started/","page":"Getting Started","title":"Getting Started","text":"To begin, we need to import all needed packages using","category":"page"},{"location":"get_started/","page":"Getting Started","title":"Getting Started","text":"using HerbGrammar, HerbData, HerbSearch, HerbInterpret","category":"page"},{"location":"get_started/","page":"Getting Started","title":"Getting Started","text":"To define a program synthesis problem, we need a grammar and specification. ","category":"page"},{"location":"get_started/","page":"Getting Started","title":"Getting Started","text":"First, the grammar can be constructed using the @cfgrammar macro included in HerbGrammar. Here we describe a simple integer arithmetic example, that can add and multiply an input variable x or the integers 1,2, using","category":"page"},{"location":"get_started/","page":"Getting Started","title":"Getting Started","text":"g = @cfgrammar begin\n Number = |(1:2)\n Number = x\n Number = Number + Number\n Number = Number * Number\nend","category":"page"},{"location":"get_started/","page":"Getting Started","title":"Getting Started","text":"Second, the problem specification can be provided using e.g. input/output examples using HerbData. Inputs are provided as a Dict assigning values to variables, and outputs as arbitrary values. The problem itself is then a list of IOExamples using","category":"page"},{"location":"get_started/","page":"Getting Started","title":"Getting Started","text":"problem = Problem([IOExample(Dict(:x => x), 2x+1) for x ∈ 1:5])","category":"page"},{"location":"get_started/","page":"Getting Started","title":"Getting Started","text":"The problem is given now, let us search for a solution with HerbSearch. For now we will just use the default parameters searching for a satisfying program over the grammar, given the problem and a starting symbol using","category":"page"},{"location":"get_started/","page":"Getting Started","title":"Getting Started","text":"solution = search(g, problem, :Number, max_depth=3)\nprintln(solution)","category":"page"},{"location":"get_started/","page":"Getting Started","title":"Getting Started","text":"There are various ways to adapt the search technique to your needs. Please have a look at the search documentation.","category":"page"},{"location":"get_started/","page":"Getting Started","title":"Getting Started","text":"Eventually, we want to test our solution on some other inputs using HerbInterpret. We transform our grammar g to a Julia expression with Symboltable(g), add our solution and the input, assigning the value 6 to the variable x.","category":"page"},{"location":"get_started/","page":"Getting Started","title":"Getting Started","text":"output = test_with_input(SymbolTable(g), solution, Dict(:x => 6))\nprintln(output)","category":"page"},{"location":"get_started/","page":"Getting Started","title":"Getting Started","text":"Just like that we tackled (almost) all modules of Herb.jl.","category":"page"},{"location":"get_started/#Where-to-go-from-here?","page":"Getting Started","title":"Where to go from here?","text":"","category":"section"},{"location":"get_started/","page":"Getting Started","title":"Getting Started","text":"See our other tutorials!","category":"page"},{"location":"get_started/#The-full-code-example","page":"Getting Started","title":"The full code example","text":"","category":"section"},{"location":"get_started/","page":"Getting Started","title":"Getting Started","text":"using HerbSearch, HerbData, HerbInterpret, HerbGrammar\n\n# define our very simple context-free grammar\n# Can add and multiply an input variable x or the integers 1,2.\ng = @cfgrammar begin\n Number = |(1:2)\n Number = x\n Number = Number + Number\n Number = Number * Number\nend\n\nproblem = Problem([IOExample(Dict(:x => x), 2x+1) for x ∈ 1:5])\nsolution = search(g, problem, :Number, max_depth=3)\n\ntest_with_input(SymbolTable(g), solution, Dict(:x => 6))","category":"page"},{"location":"HerbData/#HerbData_docs","page":"HerbCore.jl","title":"HerbData.jl Documentation","text":"","category":"section"},{"location":"HerbData/","page":"HerbCore.jl","title":"HerbCore.jl","text":"CurrentModule=HerbData","category":"page"},{"location":"HerbData/","page":"HerbCore.jl","title":"HerbCore.jl","text":"Modules = [HerbData]\nOrder = [:type, :const, :macro, :function]","category":"page"},{"location":"HerbData/#HerbData.IOExample","page":"HerbCore.jl","title":"HerbData.IOExample","text":"struct IOExample <: Example\n\nAn input-output example. input is a Dict of {Symbol,Any} where the symbol represents a variable in a program. output can be anything.\n\n\n\n\n\n","category":"type"},{"location":"HerbData/#HerbData.IOPExample","page":"HerbCore.jl","title":"HerbData.IOPExample","text":"struct IOPExample <: Example\n\nAn input-output example with an associated program. ex is an IOExample. program is a program of arbitrary form. Please note that this is a pure container, and thus does not guarantee any checks on the validity of the program.\n\n\n\n\n\n","category":"type"},{"location":"HerbData/#HerbData.Problem","page":"HerbCore.jl","title":"HerbData.Problem","text":"struct Problem\n\nProgram synthesis problem defined with a vector of Examples\n\n\n\n\n\n","category":"type"},{"location":"HerbData/#HerbData.read_IOPexamples-Tuple{AbstractString}","page":"HerbCore.jl","title":"HerbData.read_IOPexamples","text":"read_IOPexamples(filepath::AbstractString)::Vector{Tuple{IOPExample}\n\nReads serialized IO + program examples from disk after type checking.\n\n\n\n\n\n","category":"method"},{"location":"HerbData/#HerbData.read_IOexamples-Tuple{AbstractString}","page":"HerbCore.jl","title":"HerbData.read_IOexamples","text":"read_IOexamples(filepath::AbstractString)::Vector{IOExample}\n\nReads serialized IO examples from disk after type checking.\n\n\n\n\n\n","category":"method"},{"location":"HerbData/#HerbData.readdata-Tuple{AbstractString, Function}","page":"HerbCore.jl","title":"HerbData.readdata","text":"readdata(directory::AbstractString, lineparser::Function)::Vector{Problem}\n\nReads all files in the given directory and parses them line by line into an ExampleProblem using the given lineparser.\n\nTODO: Turn this into an iterator that doesn't load all data into memory at initialization.\n\n\n\n\n\n","category":"method"},{"location":"HerbData/#HerbData.readfile-Tuple{AbstractString, Function}","page":"HerbCore.jl","title":"HerbData.readfile","text":"readfile(filepath::AbstractString, lineparser::Function)::Problem\n\nReads a file and parses every non-empty line using the line parser.\n\n\n\n\n\n","category":"method"},{"location":"HerbData/#HerbData.write_IOPexamples-Tuple{AbstractString, Vector{IOPExample}}","page":"HerbCore.jl","title":"HerbData.write_IOPexamples","text":"write_IOPexamples(filepath::AbstractString, examples::Vector{Tuple{IOExample, Any}})\n\nWrites IO examples and the corresponding programs to disk by serializing them into a file using HDF5 checking for and appending the .xiop.\n\n\n\n\n\n","category":"method"},{"location":"HerbData/#HerbData.write_IOexamples-Tuple{AbstractString, Vector{IOExample}}","page":"HerbCore.jl","title":"HerbData.write_IOexamples","text":"write_IOexamples(filepath::AbstractString, examples::Vector{IOExample})\n\nWrites IO examples to disk by serializing them into a file using HDF5 checking for and appending the .xio file ending.\n\n\n\n\n\n","category":"method"},{"location":"HerbData/#Index","page":"HerbCore.jl","title":"Index","text":"","category":"section"},{"location":"HerbData/","page":"HerbCore.jl","title":"HerbCore.jl","text":"","category":"page"},{"location":"tutorials/defining_grammars/#Defining-Grammars-in-Herb.jl-using-HerbGrammar","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl using HerbGrammar","text":"","category":"section"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"The program space in Herb.jl is defined using a grammar. This notebook demonstrates how such a grammar can be created. There are multiple kinds of grammars, but they can all be defined in a very similar way.","category":"page"},{"location":"tutorials/defining_grammars/#Setup","page":"Defining Grammars in Herb.jl","title":"Setup","text":"","category":"section"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"using HerbGrammar, HerbConstraints","category":"page"},{"location":"tutorials/defining_grammars/#Creating-a-simple-grammar","page":"Defining Grammars in Herb.jl","title":"Creating a simple grammar","text":"","category":"section"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"This cell contains a very simple arithmetic grammar. The grammar is defined using the @cfgrammar macro. This macro converts the grammar definition in the form of a Julia expression into Herb's internal grammar representation. Macro's are executed during compilation. If you want to load a grammar during execution, have a look at the HerbGrammar.expr2cfgrammar function.","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"g₁ = HerbGrammar.@cfgrammar begin\n Int = 1\n Int = 2\n Int = 3\n Int = Int * Int\n Int = Int + Int\nend","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"1: Int = 1\n2: Int = 2\n3: Int = 3\n4: Int = Int * Int\n5: Int = Int + Int","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"Defining every integer one-by-one can be quite tedious. Therefore, it is also possible to use the following syntax that makes use of a Julia iterator:","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"g₂ = HerbGrammar.@cfgrammar begin\n Int = |(0:9)\n Int = Int * Int\n Int = Int + Int\nend","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"1: Int = 0\n2: Int = 1\n3: Int = 2\n4: Int = 3\n5: Int = 4\n6: Int = 5\n7: Int = 6\n8: Int = 7\n9: Int = 8\n10: Int = 9\n11: Int = Int * Int\n12: Int = Int + Int","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"You can do the same with lists:","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"g₃ = HerbGrammar.@cfgrammar begin\n Int = |([0, 2, 4, 6, 8])\n Int = Int * Int\n Int = Int + Int\nend","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"1: Int = 0\n2: Int = 2\n3: Int = 4\n4: Int = 6\n5: Int = 8\n6: Int = Int * Int\n7: Int = Int + Int","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"Variables can also be added to the grammar by just using the variable name:","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"g₄ = HerbGrammar.@cfgrammar begin\n Int = |(0:9)\n Int = Int * Int\n Int = Int + Int\n Int = x\nend","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"1: Int = 0\n2: Int = 1\n3: Int = 2\n4: Int = 3\n5: Int = 4\n6: Int = 5\n7: Int = 6\n8: Int = 7\n9: Int = 8\n10: Int = 9\n11: Int = Int * Int\n12: Int = Int + Int\n13: Int = x","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"Grammars can also work with functions. After all, + and * are just infix operators for Julia's identically-named functions. You can use functions that are provided by Julia, or functions that you wrote yourself:","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"f(a) = a + 1\n\ng₅ = HerbGrammar.@cfgrammar begin\n Int = |(0:9)\n Int = Int * Int\n Int = Int + Int\n Int = f(Int)\n Int = x\nend","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"1: Int = 0\n2: Int = 1\n3: Int = 2\n4: Int = 3\n5: Int = 4\n6: Int = 5\n7: Int = 6\n8: Int = 7\n9: Int = 8\n10: Int = 9\n11: Int = Int * Int\n12: Int = Int + Int\n13: Int = f(Int)\n14: Int = x","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"Similarly, we can also define the operator times (x) manually.","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"×(a, b) = a * b\n\ng₆ = HerbGrammar.@cfgrammar begin\n Int = |(0:9)\n Int = a\n Int = Int + Int\n Int = Int × Int\nend","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"1: Int = 0\n2: Int = 1\n3: Int = 2\n4: Int = 3\n5: Int = 4\n6: Int = 5\n7: Int = 6\n8: Int = 7\n9: Int = 8\n10: Int = 9\n11: Int = a\n12: Int = Int + Int\n13: Int = Int × Int","category":"page"},{"location":"tutorials/defining_grammars/#Working-with-grammars","page":"Defining Grammars in Herb.jl","title":"Working with grammars","text":"","category":"section"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"If you want to implement something using these grammars, it is useful to know about the functions that you can use to manipulate grammars and extract information. This section is not necessarily complete, but it aims to give an overview of the most important functions. ","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"It is recommended to also read up on Julia metaprogramming if you are not already familiar with that.","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"One of the most important things about grammars is that each rule has an index associated with it:","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"g₇ = HerbGrammar.@cfgrammar begin\n Int = |(0:9)\n Int = Int + Int\n Int = Int * Int\n Int = x\nend\n\ncollect(enumerate(g₇.rules))","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"13-element Vector{Tuple{Int64, Any}}:\n (1, 0)\n (2, 1)\n (3, 2)\n (4, 3)\n (5, 4)\n (6, 5)\n (7, 6)\n (8, 7)\n (9, 8)\n (10, 9)\n (11, :(Int + Int))\n (12, :(Int * Int))\n (13, :x)","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"We can use this index to extract information from the grammar.","category":"page"},{"location":"tutorials/defining_grammars/#isterminal","page":"Defining Grammars in Herb.jl","title":"isterminal","text":"","category":"section"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"isterminal returns true if a rule is terminal, i.e. it cannot be expanded. For example, rule 1 is terminal, but rule 11 is not, since it contains the non-terminal symbol :Int. ","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"HerbGrammar.isterminal(g₇, 1)","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"true","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"HerbGrammar.isterminal(g₇, 11)","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"false","category":"page"},{"location":"tutorials/defining_grammars/#return_type","page":"Defining Grammars in Herb.jl","title":"return_type","text":"","category":"section"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"This function is rather obvious; it returns the non-terminal symbol that corresponds to a certain rule. The return type for all rules in our grammar is :Int.","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"HerbGrammar.return_type(g₇, 11)","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":":Int","category":"page"},{"location":"tutorials/defining_grammars/#child_types","page":"Defining Grammars in Herb.jl","title":"child_types","text":"","category":"section"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"child_types returns the types of the nonterminal children of a rule in a vector. If you just want to know how many children a rule has, and not necessarily which types they have, you can use nchildren","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"HerbGrammar.child_types(g₇, 11)","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"2-element Vector{Symbol}:\n :Int\n :Int","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"HerbGrammar.nchildren(g₇, 11)","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"2","category":"page"},{"location":"tutorials/defining_grammars/#nonterminals","page":"Defining Grammars in Herb.jl","title":"nonterminals","text":"","category":"section"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"The nonterminals function can be used to obtain a list of all nonterminals in the grammar.","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"HerbGrammar.nonterminals(g₇)","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"1-element Vector{Symbol}:\n :Int","category":"page"},{"location":"tutorials/defining_grammars/#Adding-rules","page":"Defining Grammars in Herb.jl","title":"Adding rules","text":"","category":"section"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"It is also possible to add rules to a grammar during execution. This can be done using the add_rule! function. As with most functions in Julia that end with an exclamation mark, this function modifies its argument (the grammar).","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"A rule can be provided in the same syntax as is used in the grammar definition. The rule should be of the Expr type, which is a built-in type for representing expressions. An easy way of creating Expr values in Julia is to encapsulate it in brackets and use a colon as prefix:","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"HerbGrammar.add_rule!(g₇, :(Int = Int - Int))","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"1: Int = 0\n2: Int = 1\n3: Int = 2\n4: Int = 3\n5: Int = 4\n6: Int = 5\n7: Int = 6\n8: Int = 7\n9: Int = 8\n10: Int = 9\n11: Int = Int + Int\n12: Int = Int * Int\n13: Int = x\n14: Int = Int - Int","category":"page"},{"location":"tutorials/defining_grammars/#Removing-rules","page":"Defining Grammars in Herb.jl","title":"Removing rules","text":"","category":"section"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"It is also possible to remove rules in Herb.jl, however, this is a bit more involved. As said before, rules have an index associated with them. The internal representation of programs that are defined by the grammar makes use of those indices for efficiency. Blindly removing a rule would shift the indices of other rules, and this could mean that existing programs get a different meaning or become invalid. ","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"Therefore, there are two functions for removing rules:","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"remove_rule! removes a rule from the grammar, but fills its place with a placeholder. Therefore, the indices stay the same, and only programs that use the removed rule become invalid.\ncleanup_removed_rules! removes all placeholders and shifts the indices of the other rules.","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"HerbGrammar.remove_rule!(g₇, 11)","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"1: Int = 0\n2: Int = 1\n3: Int = 2\n4: Int = 3\n5: Int = 4\n6: Int = 5\n7: Int = 6\n8: Int = 7\n9: Int = 8\n10: Int = 9\n11: nothing = nothing\n12: Int = Int * Int\n13: Int = x\n14: Int = Int - Int","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"HerbGrammar.cleanup_removed_rules!(g₇)","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"1: Int = 0\n2: Int = 1\n3: Int = 2\n4: Int = 3\n5: Int = 4\n6: Int = 5\n7: Int = 6\n8: Int = 7\n9: Int = 8\n10: Int = 9\n11: Int = Int * Int\n12: Int = x\n13: Int = Int - Int","category":"page"},{"location":"tutorials/defining_grammars/#Context-sensitive-grammars","page":"Defining Grammars in Herb.jl","title":"Context-sensitive grammars","text":"","category":"section"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"Context-sensitive grammars allow additional constraints to be added with respect to context-free grammars. The syntax for defining a context-sensitive grammar is identical to defining a context-sensitive grammar:","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"g₈ = HerbGrammar.@csgrammar begin\n Int = |(0:9)\n Int = Int + Int\n Int = Int * Int\n Int = x\nend","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"1: Int = 0\n2: Int = 1\n3: Int = 2\n4: Int = 3\n5: Int = 4\n6: Int = 5\n7: Int = 6\n8: Int = 7\n9: Int = 8\n10: Int = 9\n11: Int = Int + Int\n12: Int = Int * Int\n13: Int = x","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"Constraints can be added using the addconstraint! function, which takes a context-sensitive grammar and a constraint and adds the constraint to the grammar. Currently, Herb.jl only has propagators constraints. These constraints each have a corresponding propagate function that removes all options that violate that constraint from the domain. At the moment, there are three propagator constraints:","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"ComesAfter(rule, predecessors): It is only possible to use rule rule when predecessors are in its path to the root.\nForbidden(sequence): Forbids the derivation specified as a path in an expression tree.\nOrdered(order): Rules have to be used in the specified order. That is, rule at index K can only be used if rules at indices [1...K-1] are used in the left subtree of the current expression.","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"Below, an example is given of a context-sensitive grammar with a ComesAfter constraint:","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"HerbGrammar.addconstraint!(g₈, HerbConstraints.ComesAfter(1, [9]))","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"1-element Vector{Main.HerbCore.Constraint}:\n Main.HerbConstraints.ComesAfter(1, [9])","category":"page"},{"location":"tutorials/defining_grammars/#Probabilistic-grammars","page":"Defining Grammars in Herb.jl","title":"Probabilistic grammars","text":"","category":"section"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"Herb.jl also supports probabilistic grammars. These grammars allow the user to assign a probability to each rule in the grammar. A probabilistic grammar can be defined in a very similar way to a standard grammar, but has some slightly different syntax:","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"g₉ = HerbGrammar.@pcfgrammar begin\n 0.4 : Int = |(0:9)\n 0.2 : Int = Int + Int\n 0.1 : Int = Int * Int\n 0.3 : Int = x\nend\n\nfor r ∈ 1:length(g₃.rules)\n p = HerbGrammar.probability(g₈, r)\n\n println(\"$p : $r\")\nend","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"0.07692307692307693 : 1\n0.07692307692307693 : 2\n0.07692307692307693 : 3\n0.07692307692307693 : 4\n0.07692307692307693 : 5\n0.07692307692307693 : 6\n0.07692307692307693 : 7\n\n\n┌ Warning: Requesting probability in a non-probabilistic grammar.\n│ Uniform distribution is assumed.\n└ @ Main.HerbGrammar d:\\GitHub\\HerbGrammar.jl\\src\\grammar_base.jl:155\n┌ Warning: Requesting probability in a non-probabilistic grammar.\n│ Uniform distribution is assumed.\n└ @ Main.HerbGrammar d:\\GitHub\\HerbGrammar.jl\\src\\grammar_base.jl:155\n┌ Warning: Requesting probability in a non-probabilistic grammar.\n│ Uniform distribution is assumed.\n└ @ Main.HerbGrammar d:\\GitHub\\HerbGrammar.jl\\src\\grammar_base.jl:155\n┌ Warning: Requesting probability in a non-probabilistic grammar.\n│ Uniform distribution is assumed.\n└ @ Main.HerbGrammar d:\\GitHub\\HerbGrammar.jl\\src\\grammar_base.jl:155\n┌ Warning: Requesting probability in a non-probabilistic grammar.\n│ Uniform distribution is assumed.\n└ @ Main.HerbGrammar d:\\GitHub\\HerbGrammar.jl\\src\\grammar_base.jl:155\n┌ Warning: Requesting probability in a non-probabilistic grammar.\n│ Uniform distribution is assumed.\n└ @ Main.HerbGrammar d:\\GitHub\\HerbGrammar.jl\\src\\grammar_base.jl:155\n┌ Warning: Requesting probability in a non-probabilistic grammar.\n│ Uniform distribution is assumed.\n└ @ Main.HerbGrammar d:\\GitHub\\HerbGrammar.jl\\src\\grammar_base.jl:155","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"The numbers before each rule represent the probability assigned to that rule. The total probability for each return type should add up to 1.0. If this isn't the case, Herb.jl will normalize the probabilities.","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"If a single line in the grammar definition represents multiple rules, such as 0.4 : Int = |(0:9), the probability will be evenly divided over all these rules.","category":"page"},{"location":"tutorials/defining_grammars/#File-writing","page":"Defining Grammars in Herb.jl","title":"File writing","text":"","category":"section"},{"location":"tutorials/defining_grammars/#Saving-and-loading-context-free-grammars","page":"Defining Grammars in Herb.jl","title":"Saving & loading context-free grammars","text":"","category":"section"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"If you want to store a grammar on the disk, you can use the store_cfg, read_cfg and functions to store and read grammars respectively. The store_cfg grammar can also be used to store probabilistic grammars. Reading probabilistic grammars can be done using read_pcfg. The stored grammar files can also be opened using a text editor to be modified, as long as the contents of the file doesn't violate the syntax for defining grammars.","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"HerbGrammar.store_cfg(\"demo.txt\", g₇)","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"HerbGrammar.read_cfg(\"demo.txt\")","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"1: Int = 0\n2: Int = 1\n3: Int = 2\n4: Int = 3\n5: Int = 4\n6: Int = 5\n7: Int = 6\n8: Int = 7\n9: Int = 8\n10: Int = 9\n11: Int = Int * Int\n12: Int = x\n13: Int = Int - Int","category":"page"},{"location":"tutorials/defining_grammars/#Saving-and-loading-context-sensitive-grammars","page":"Defining Grammars in Herb.jl","title":"Saving & loading context-sensitive grammars","text":"","category":"section"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"Saving and loading context-sensitive grammars is very similar to how it is done with context-free grammars. The only difference is that an additional file is created for the constraints. The file that contains the grammars can be edited and can also be read using the reader for context-free grammars. The file that contains the constraints cannot be edited.","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"HerbGrammar.store_csg(\"demo.grammar\", \"demo.constraints\", g₈)\ng₈, g₈.constraints","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"(1: Int = 0\n2: Int = 1\n3: Int = 2\n4: Int = 3\n5: Int = 4\n6: Int = 5\n7: Int = 6\n8: Int = 7\n9: Int = 8\n10: Int = 9\n11: Int = Int + Int\n12: Int = Int * Int\n13: Int = x\n, Main.HerbCore.Constraint[Main.HerbConstraints.ComesAfter(1, [9])])","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"g₉ = HerbGrammar.read_csg(\"demo.grammar\", \"demo.constraints\")\ng₉, g₉.constraints","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"(1: Int = 0\n2: Int = 1\n3: Int = 2\n4: Int = 3\n5: Int = 4\n6: Int = 5\n7: Int = 6\n8: Int = 7\n9: Int = 8\n10: Int = 9\n11: Int = Int + Int\n12: Int = Int * Int\n13: Int = x\n, Main.HerbCore.Constraint[Main.HerbConstraints.ComesAfter(1, [9])])","category":"page"},{"location":"","page":"Herb.jl","title":"Herb.jl","text":"CurrentModule=Herb","category":"page"},{"location":"#[Herb.jl](https://github.com/Herb-AI/Herb.jl)","page":"Herb.jl","title":"Herb.jl","text":"","category":"section"},{"location":"","page":"Herb.jl","title":"Herb.jl","text":"A library for defining and efficiently solving program synthesis tasks in Julia.","category":"page"},{"location":"#Why-Herb.jl?","page":"Herb.jl","title":"Why Herb.jl?","text":"","category":"section"},{"location":"","page":"Herb.jl","title":"Herb.jl","text":"When writing research software we almost always investigate highly specific properties or algorithms of our domain, leading to us building the tools from scratch over and over again. The very same holds for the field of program synthesis: Tools are hard to run, benchmarks are hard to get and prepare, and its hard to adapt our existing code to a novel idea.","category":"page"},{"location":"","page":"Herb.jl","title":"Herb.jl","text":"Herb.jl will take care of this for you and helps you defining, solving and extending your program synthesis problems.","category":"page"},{"location":"","page":"Herb.jl","title":"Herb.jl","text":"Herb.jl provides...","category":"page"},{"location":"","page":"Herb.jl","title":"Herb.jl","text":"a unified and universal framework for program synthesis\nHerb.jl allows you to describe all sorts of program synthesis problems using context-free grammars\na number of state-of-the-art benchmarks and solvers already implemented and usable out-of-the-box","category":"page"},{"location":"","page":"Herb.jl","title":"Herb.jl","text":"Herb.jl's sub-packages provide fast and easily extendable implementations of ","category":"page"},{"location":"","page":"Herb.jl","title":"Herb.jl","text":"various static and dynamic search strategies,\nlearning search strategies, sampling techniques and more,\nconstraint formulation and propagation, \neasy grammar formulation and usage,\nwide-range of usable program interpreters and languages + the possibility to use your own, and \nefficient data formulation.","category":"page"},{"location":"#Why-Julia?","page":"Herb.jl","title":"Why Julia?","text":"","category":"section"},{"location":"","page":"Herb.jl","title":"Herb.jl","text":"Julia is a perfect fit for program synthesis due to numerous reasons. Starting from scientific reasons like speed of execution and composability over to practical reasons like speed of writing Julia code. For a full ode on why to use Julia, please see the WhyJulia manifesto.","category":"page"},{"location":"#Sub-Modules","page":"Herb.jl","title":"Sub-Modules","text":"","category":"section"},{"location":"","page":"Herb.jl","title":"Herb.jl","text":"Herb's functionality is distributed among several sub-packages:","category":"page"},{"location":"","page":"Herb.jl","title":"Herb.jl","text":"HerbCore.jl: The core of Herb.jl defining core concepts to avoid circular dependencies.\nHerbGrammar.jl:\nHerbData.jl:\nHerbInterpret.jl:\nHerbSearch.jl:\nHerbConstraints.jl:","category":"page"},{"location":"#Basics","page":"Herb.jl","title":"Basics","text":"","category":"section"},{"location":"","page":"Herb.jl","title":"Herb.jl","text":"Pages = [\"install.md\", \"get_started.md\", \"concepts.md\"]","category":"page"},{"location":"#Advanced-content","page":"Herb.jl","title":"Advanced content","text":"","category":"section"},{"location":"","page":"Herb.jl","title":"Herb.jl","text":"","category":"page"}] +[{"location":"install/#Installation-Guide","page":"Installation Guide","title":"Installation Guide","text":"","category":"section"},{"location":"install/","page":"Installation Guide","title":"Installation Guide","text":"Before installing Herb.jl, ensure that you have a running Julia distribution installed (Julia version 1.7 and above were tested). ","category":"page"},{"location":"install/","page":"Installation Guide","title":"Installation Guide","text":"Thanks to Julia's package management, installing Herb.jl is very straighforward. Activate the default Julia REPL using","category":"page"},{"location":"install/","page":"Installation Guide","title":"Installation Guide","text":"julia","category":"page"},{"location":"install/","page":"Installation Guide","title":"Installation Guide","text":"or from within one of your projects using","category":"page"},{"location":"install/","page":"Installation Guide","title":"Installation Guide","text":"julia --project=.","category":"page"},{"location":"install/","page":"Installation Guide","title":"Installation Guide","text":"From the Julia REPL run ","category":"page"},{"location":"install/","page":"Installation Guide","title":"Installation Guide","text":"]\nadd Herb","category":"page"},{"location":"install/","page":"Installation Guide","title":"Installation Guide","text":"or instead running","category":"page"},{"location":"install/","page":"Installation Guide","title":"Installation Guide","text":"import Pkg\nPkg.add(\"Herb\")","category":"page"},{"location":"install/","page":"Installation Guide","title":"Installation Guide","text":"which will both install all dependencies automatically.","category":"page"},{"location":"install/","page":"Installation Guide","title":"Installation Guide","text":"For later convenience we can also add the respective dependencies to our project, so that we do not have to write Herb.HerbGrammar every time.","category":"page"},{"location":"install/","page":"Installation Guide","title":"Installation Guide","text":"] add HerbConstraints HerbCore HerbData HerbInterpret HerbGrammar HerbSearch","category":"page"},{"location":"install/","page":"Installation Guide","title":"Installation Guide","text":"And just like this you are done! Welcome to Herb.jl!","category":"page"},{"location":"concepts/#Architecture-and-core-concepts","page":"Architecture and core concepts","title":"Architecture and core concepts","text":"","category":"section"},{"location":"HerbSearch/#HerbSearch_docs","page":"HerbSearch.jl","title":"HerbSearch.jl Documentation","text":"","category":"section"},{"location":"HerbSearch/","page":"HerbSearch.jl","title":"HerbSearch.jl","text":"CurrentModule=HerbSearch","category":"page"},{"location":"HerbSearch/","page":"HerbSearch.jl","title":"HerbSearch.jl","text":"Modules = [HerbSearch]\nOrder = [:type, :const, :macro, :function]","category":"page"},{"location":"HerbSearch/#HerbSearch.ContextSensitivePriorityEnumerator","page":"HerbSearch.jl","title":"HerbSearch.ContextSensitivePriorityEnumerator","text":"mutable struct ContextSensitivePriorityEnumerator <: ExpressionIterator\n\nEnumerates a context-free grammar starting at Symbol sym with respect to the grammar up to a given depth and a given size. The exploration is done using the given priority function for derivations, and the expand function for discovered nodes.\n\n\n\n\n\n","category":"type"},{"location":"HerbSearch/#HerbSearch.ExpandFailureReason","page":"HerbSearch.jl","title":"HerbSearch.ExpandFailureReason","text":"@enum ExpandFailureReason limit_reached=1 already_complete=2\n\nRepresentation of the different reasons why expanding a partial tree failed. Currently, there are two possible causes of the expansion failing:\n\nlimit_reached: The depth limit or the size limit of the partial tree would be violated by the expansion\nalready_complete: There is no hole left in the tree, so nothing can be expanded.\n\n\n\n\n\n","category":"type"},{"location":"HerbSearch/#HerbSearch.ExpressionIterator","page":"HerbSearch.jl","title":"HerbSearch.ExpressionIterator","text":"abstract type ExpressionIterator\n\nAbstract super-type for all possible enumerators.\n\n\n\n\n\n","category":"type"},{"location":"HerbSearch/#HerbSearch.GeneticSearchIterator","page":"HerbSearch.jl","title":"HerbSearch.GeneticSearchIterator","text":"GeneticSearchIterator{FitnessFunction,CrossOverFunction,MutationFunction,SelectParentsFunction,EvaluationFunction} <: ExpressionIterator\n\nDefines an ExpressionIterator using genetic search. \n\nConsists of:\n\ngrammar::ContextSensitiveGrammar: the grammar to search over\nexamples::Vector{<:Example}: a collection of examples defining the specification \nfitness::FitnessFunction: assigns a numerical value (fitness score) to each individual based on how closely it meets the desired objective\ncross_over::CrossOverFunction: combines the program from two parent individuals to create one or more offspring individuals\nmutation!::MutationFunction: mutates the program of an invididual\nselect_parents::SelectParentsFunction: selects two parents for the crossover\nevaluation_function::EvaluationFunction: interpreter to evaluate the individual programs\nstart_symbol::Symbol: defines the start symbol from which the search should be started\npopulation_size::Int64: number of inviduals in the population\nmutation_probability::Float64: probability of mutation for each individual\nmaximum_initial_population_depth::Int64: maximum depth of trees when population is initialized \n\nend\n\n\n\n\n\n","category":"type"},{"location":"HerbSearch/#HerbSearch.PriorityQueueItem","page":"HerbSearch.jl","title":"HerbSearch.PriorityQueueItem","text":"struct PriorityQueueItem\n\nRepresents an item in the priority enumerator priority queue. An item contains of:\n\ntree: A partial AST\nsize: The size of the tree. This is a cached value which prevents having to traverse the entire tree each time the size is needed.\nconstraints: The local constraints that apply to this tree. These constraints are enforced each time the tree is modified.\n\n\n\n\n\n","category":"type"},{"location":"HerbSearch/#HerbSearch.PriorityQueueItem-Tuple{AbstractRuleNode, Int64}","page":"HerbSearch.jl","title":"HerbSearch.PriorityQueueItem","text":"PriorityQueueItem(tree::AbstractRuleNode, size::Int)\n\nConstructs PriorityQueueItem given only a tree and the size, but no constraints.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.PropagateResult","page":"HerbSearch.jl","title":"HerbSearch.PropagateResult","text":"@enum PropagateResult tree_complete=1 tree_incomplete=2 tree_infeasible=3\n\nRepresentation of the possible results of a constraint propagation. At the moment there are three possible outcomes:\n\ntree_complete: The propagation was applied successfully and the tree does not contain any holes anymore. Thus no constraints can be applied anymore.\ntree_incomplete: The propagation was applied successfully and the tree does contain more holes. Thus more constraints may be applied to further prune the respective domains.\ntree_infeasible: The propagation was succesful, but there are holes with empty domains. Hence, the tree is now infeasible.\n\n\n\n\n\n","category":"type"},{"location":"HerbSearch/#HerbSearch.StochasticSearchEnumerator","page":"HerbSearch.jl","title":"HerbSearch.StochasticSearchEnumerator","text":"Base.@kwdef struct StochasticSearchEnumerator <: ExpressionIterator\n\nA unified struct for the algorithms Metropolis Hastings, Very Large Scale Neighbourhood and Simulated Annealing. Each algorithm implements neighbourhood propose accept and temperature functions. Below the signiture of all this function is shown\n\nSignatures\n\n\n\nReturns a node location from the program that is the neighbourhood. It can also return other information using dict\n\nneighbourhood(program::RuleNode, grammar::Grammar) -> (loc::NodeLocation, dict::Dict)\n\n\n\nProposes a list of programs using the location provided by neighbourhood and the dict.\n\npropose(current_program, loc::NodeLocation, grammar::Grammar, max_depth::Int64, dict::Dict) -> Iter[RuleNode]\n\n\n\nBased on the current program and possible cost and temperature it accepts the program or not. Usually we would always want to accept better programs but we might get stuck if we do so. That is why some implementations of the accept function accept with a probability costs that are worse. cost means how different are the outcomes of the program compared to the correct outcomes. The lower the cost the better the program performs on the examples. The cost is provided by the cost_function\n\naccept(current_cost::Real, possible_cost::Real, temperature::Real) -> Bool\n\n\n\nReturns the new temperature based on the previous temperature. Higher the temperature means that the algorithm will explore more.\n\ntemperature(previous_temperature::Real) -> Real\n\n\n\nReturns the cost of the current program. It receives a list of tuples (expected, found) and gives back a cost.\n\ncost_function(outcomes::Tuple{<:Number,<:Number}[]) -> Real\n\n\n\nFields\n\ngrammar::ContextSensitiveGrammar grammar that the algorithm uses\nmax_depth::Int64 = 5 maximum depth of the program to generate\nexamples::Vector{Example} example used to check the program\nneighbourhood::Function \npropose::Function\naccept::Function\ntemperature::Function\ncost_function::Function\nstart_symbol::Symbol the start symbol of the algorithm :Real or :Int\ninitial_temperature::Real = 1 \nevaluation_function::Function that evaluates the julia expressions\n\nAn iterator over all possible expressions of a grammar up to max_depth with start symbol sym.\n\n\n\n\n\n","category":"type"},{"location":"HerbSearch/#Base.iterate-Tuple{ContextSensitivePriorityEnumerator, DataStructures.PriorityQueue}","page":"HerbSearch.jl","title":"Base.iterate","text":"Base.iterate(iter::ContextSensitivePriorityEnumerator, pq::DataStructures.PriorityQueue)\n\nDescribes the iteration for a given ContextSensitivePriorityEnumerator and a PriorityQueue over the grammar without enqueueing new items to the priority queue. Recursively returns the result for the priority queue.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#Base.iterate-Tuple{ContextSensitivePriorityEnumerator}","page":"HerbSearch.jl","title":"Base.iterate","text":"Base.iterate(iter::ContextSensitivePriorityEnumerator)\n\nDescribes the iteration for a given ContextSensitivePriorityEnumerator over the grammar. The iteration constructs a PriorityQueue first and then prunes it propagating the active constraints. Recursively returns the result for the priority queue.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#Base.iterate-Tuple{HerbSearch.GeneticSearchIterator, HerbSearch.GeneticIteratorState}","page":"HerbSearch.jl","title":"Base.iterate","text":"Base.iterate(iter::GeneticSearchIterator, current_state::GeneticIteratorState)\n\nIterates the search space using a genetic algorithm. Takes the iterator and the current state to mutate and crossover random inviduals. Returns the best program-so-far and the state of the iterator.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#Base.iterate-Tuple{HerbSearch.GeneticSearchIterator}","page":"HerbSearch.jl","title":"Base.iterate","text":"Base.iterate(iter::GeneticSearchIterator)\n\nIterates the search space using a genetic algorithm. First generates a population sampling random programs. Returns the best program-so-far, and the state of the iterator.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#Base.iterate-Tuple{HerbSearch.StochasticSearchEnumerator, HerbSearch.IteratorState}","page":"HerbSearch.jl","title":"Base.iterate","text":"Base.iterate(iter::StochasticSearchEnumerator, current_state::IteratorState)\n\nThe algorithm that constructs the iterator of StochasticSearchEnumerator. It has the following structure:\n\nget a random node location -> location,dict = neighbourhood(current_program)\ncall propose on the current program getting a list of possbile replacements in the node location \niterate through all the possible replacements and perform the replacement in the current program \naccept the new program by modifying the next_program or reject the new program\nreturn the new next_program\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#Base.rand","page":"HerbSearch.jl","title":"Base.rand","text":"rand(::Type{RuleNode}, grammar::Grammar, max_depth::Int=10)\n\nGenerates a random RuleNode of arbitrary type and maximum depth max_depth.\n\n\n\n\n\n","category":"function"},{"location":"HerbSearch/#Base.rand-2","page":"HerbSearch.jl","title":"Base.rand","text":"rand(::Type{RuleNode}, grammar::Grammar, typ::Symbol, dmap::AbstractVector{Int}, max_depth::Int=10)\n\nGenerates a random RuleNode, i.e. an expression tree, of root type typ and maximum depth max_depth guided by a depth map dmap if possible.\n\n\n\n\n\n","category":"function"},{"location":"HerbSearch/#Base.rand-3","page":"HerbSearch.jl","title":"Base.rand","text":"rand(::Type{RuleNode}, grammar::Grammar, typ::Symbol, max_depth::Int=10)\n\nGenerates a random RuleNode of return type typ and maximum depth max_depth.\n\n\n\n\n\n","category":"function"},{"location":"HerbSearch/#HerbSearch._expand-Tuple{Hole, ContextSensitiveGrammar, Int64, Int64, GrammarContext, Function, Function}","page":"HerbSearch.jl","title":"HerbSearch._expand","text":"_expand(node::Hole, grammar::ContextSensitiveGrammar, ::Int, max_holes::Int, context::GrammarContext, hole_heuristic::Function, derivation_heuristic::Function)::Union{ExpandFailureReason, Vector{TreeConstraints}}\n\nExpands a given hole that was found in _expand using the given derivation heuristic. Returns the list of discovered nodes in that order and with their respective constraints.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch._expand-Tuple{RuleNode, ContextSensitiveGrammar, Int64, Int64, GrammarContext, Function, Function}","page":"HerbSearch.jl","title":"HerbSearch._expand","text":"_expand(root::RuleNode, grammar::ContextSensitiveGrammar, max_depth::Int, max_holes::Int, context::GrammarContext, hole_heuristic::Function, derivation_heuristic::Function)::Union{ExpandFailureReason, Vector{TreeConstraints}}\n\nRecursive expand function used in multiple enumeration techniques. Expands one hole/undefined leaf of the given RuleNode tree found using the given hole heuristic. If the expansion was successful, returns a list of new trees and a list of lists of hole locations, corresponding to the holes of each newly expanded tree. Returns nothing if tree is already complete (i.e. contains no holes). Returns an empty list if the tree is partial (i.e. contains holes), but they could not be expanded because of the depth limit.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch._find_next_complete_tree-Tuple{ContextSensitiveGrammar, Int64, Int64, Function, Function, DataStructures.PriorityQueue}","page":"HerbSearch.jl","title":"HerbSearch._find_next_complete_tree","text":"_find_next_complete_tree(grammar::ContextSensitiveGrammar, max_depth::Int, max_size::Int, priority_function::Function, expand_function::Function, pq::PriorityQueue)::Union{Tuple{RuleNode, PriorityQueue}, Nothing}\n\nTakes a priority queue and returns the smallest AST from the grammar it can obtain from the queue or by (repeatedly) expanding trees that are in the queue. Returns nothing if there are no trees left within the depth limit.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.best_accept-Tuple{Real, Real, Any}","page":"HerbSearch.jl","title":"HerbSearch.best_accept","text":"best_accept(current_cost::Real, next_cost::Real, temperature)\n\nReturns true if the cost of the proposed program is smaller than the cost of the current program. Otherwise, returns false.\n\nArguments\n\ncurrent_cost::Real: the cost of the current program.\nnext_cost::Real: the cost of the proposed program.\ntemperature::Real: the temperature; not used.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.calculate_cost-Tuple{RuleNode, Function, AbstractVector{Example}, Grammar, Function}","page":"HerbSearch.jl","title":"HerbSearch.calculate_cost","text":"calculate_cost(program::RuleNode, cost_function::Function, examples::AbstractVector{Example}, grammar::Grammar, evaluation_function::Function)\n\nReturns the cost of the program using the examples and the cost_function. It first convert the program to an expression and evaluates it on all the examples using HerbInterpret.evaluate_program.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.const_temperature-Tuple{Any}","page":"HerbSearch.jl","title":"HerbSearch.const_temperature","text":"const_temperature(current_temperature)\n\nReturns the temperature unchanged. This function is used by Metropolis Hastings and Very Large Neighbourhood Search algorithms.\n\nArguments\n\ncurrent_temperature::Real: the current temperature of the search.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.constructNeighbourhood-Tuple{RuleNode, Grammar}","page":"HerbSearch.jl","title":"HerbSearch.constructNeighbourhood","text":"constructNeighbourhood(current_program::RuleNode, grammar::Grammar)\n\nThe neighbourhood node location is chosen at random. The dictionary is nothing.\n\nArguments\n\ncurrent_program::RuleNode: the current program.\ngrammar::Grammar: the grammar.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.constructNeighbourhoodRuleSubset-Tuple{RuleNode, Grammar}","page":"HerbSearch.jl","title":"HerbSearch.constructNeighbourhoodRuleSubset","text":"constructNeighbourhoodRuleSubset(current_program::RuleNode, grammar::Grammar)\n\nThe neighbourhood node location is chosen at random. The dictionary is contains one entry with key \"rule_subset\" and value of type Vector{Any} being a random subset of grammar rules.\n\nArguments\n\ncurrent_program::RuleNode: the current program.\ngrammar::Grammar: the grammar.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.count_expressions-Tuple{ExpressionIterator}","page":"HerbSearch.jl","title":"HerbSearch.count_expressions","text":"count_expressions(iter::ExpressionIterator)\n\nCounts and returns the number of possible expressions in the expression iterator. The Iterator is not modified.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.count_expressions-Tuple{Grammar, Int64, Int64, Symbol}","page":"HerbSearch.jl","title":"HerbSearch.count_expressions","text":"count_expressions(grammar::Grammar, max_depth::Int, max_size::Int, sym::Symbol)\n\nCounts and returns the number of possible expressions of a grammar up to max_depth with start symbol sym.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.crossover_swap_children_1-Tuple{RuleNode, RuleNode}","page":"HerbSearch.jl","title":"HerbSearch.crossover_swap_children_1","text":"crossover_swap_children_1(parent1::RuleNode, parent2::RuleNode)\n\nPerforms a random crossover of two parents of type RuleNode. The subprograms are swapped and only one altered parent program is returned.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.crossover_swap_children_2-Tuple{RuleNode, RuleNode}","page":"HerbSearch.jl","title":"HerbSearch.crossover_swap_children_2","text":"crossover_swap_children_2(parent1::RuleNode, parent2::RuleNode)\n\nPerforms a random crossover of two parents of type RuleNode. The subprograms are swapped and both altered parent programs are returned.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.decreasing_temperature-Tuple{Real}","page":"HerbSearch.jl","title":"HerbSearch.decreasing_temperature","text":"decreasing_temperature(percentage::Real)\n\nReturns a function that produces a temperature decreased by percentage%. This function is used by the Simmulated Annealing algorithm.\n\nArguments\n\npercentage::Real: the percentage to decrease the temperature by.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.default_error_function-Tuple{Any, Any, Any}","page":"HerbSearch.jl","title":"HerbSearch.default_error_function","text":"default_error_function(old_error, output, expected_output)\n\nDefault error function for search_best.\n\n- old_error - The existing total error\n- output - The actual output of the evaluator\n- expected_output - The expected output for the example\n\nThe default function returns 0 if the outputs match and 1 otherwise.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.default_fitness-Tuple{Any, Any}","page":"HerbSearch.jl","title":"HerbSearch.default_fitness","text":"default_fitness(program, results)\n\nDefines the default fitness function taking the program and its results. Results are a vector of tuples, where each tuple is in the form Tuple{expected_output, actual_output}. As we are looking for individuals with the highest fitness function, the error is inverted. \n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.enumerate_neighbours_propose-Tuple{Int64}","page":"HerbSearch.jl","title":"HerbSearch.enumerate_neighbours_propose","text":"enumerate_neighbours_propose(enumeration_depth::Int64)\n\nThe return function is a function that produces a list with all the subprograms with depth at most enumeration_depth.\n\nArguments\n\nenumeration_depth::Int64: the maximum enumeration depth.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.get_best_program-Tuple{Array{RuleNode}, HerbSearch.GeneticSearchIterator}","page":"HerbSearch.jl","title":"HerbSearch.get_best_program","text":"get_best_program(population::Array{RuleNode}, iter:: GeneticSearchIterator)::RuleNode\n\nReturns the best program within the population with respect to the fitness function.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.get_bfs_enumerator","page":"HerbSearch.jl","title":"HerbSearch.get_bfs_enumerator","text":"get_bfs_enumerator(grammar::ContextFreeGrammar, max_depth::Int, max_size::Int, sym::Symbol, hole_heuristic::Function=heuristic_leftmost, derivation_heuristic::Function=(a,_) -> a)::ContextSensitivePriorityEnumerator\n\nReturns a breadth-first search enumerator given a ContextFreeGrammar. Returns trees in the grammar in increasing order of size. \n\n\n\n\n\n","category":"function"},{"location":"HerbSearch/#HerbSearch.get_bfs_enumerator-2","page":"HerbSearch.jl","title":"HerbSearch.get_bfs_enumerator","text":"get_bfs_enumerator(grammar::ContextSensitiveGrammar, max_depth::Int, max_size::Int, sym::Symbol, hole_heuristic::Function=heuristic_leftmost, derivation_heuristic::Function=(a,_) -> a)::ContextSensitivePriorityEnumerator\n\nReturns a breadth-first search enumerator given a ContextSensitiveGrammar. Returns trees in the grammar in increasing order of size. \n\n\n\n\n\n","category":"function"},{"location":"HerbSearch/#HerbSearch.get_dfs_enumerator","page":"HerbSearch.jl","title":"HerbSearch.get_dfs_enumerator","text":"get_dfs_enumerator(grammar::ContextFreeGrammar, max_depth::Int, max_size::Int, sym::Symbol, hole_heuristic::Function=heuristic_leftmost, derivation_heuristic::Function=(a,_) -> a)::ContextSensitivePriorityEnumerator\n\nReturns a depth-first search enumerator given a ContextFreeGrammar. Returns trees in the grammar in decreasing order of size.\n\n\n\n\n\n","category":"function"},{"location":"HerbSearch/#HerbSearch.get_dfs_enumerator-2","page":"HerbSearch.jl","title":"HerbSearch.get_dfs_enumerator","text":"get_dfs_enumerator(grammar::ContextSensitiveGrammar, max_depth::Int, max_size::Int, sym::Symbol, hole_heuristic::Function=heuristic_leftmost, derivation_heuristic::Function=(a,_) -> a)::ContextSensitivePriorityEnumerator\n\nReturns a depth-first search enumerator given a ContextSensitiveGrammar. Returns trees in the grammar in decreasing order of size.\n\n\n\n\n\n","category":"function"},{"location":"HerbSearch/#HerbSearch.get_genetic_enumerator-Tuple{Any}","page":"HerbSearch.jl","title":"HerbSearch.get_genetic_enumerator","text":"get_genetic_enumerator(examples; fitness_function = HerbSearch.default_fitness, initial_population_size = 10, maximum_initial_population_depth = 3, mutation_probability = 0.1, cross_over = HerbSearch.crossover_swap_children_2, select_parents = HerbSearch.select_fitness_proportional_parents, evaluation_function::Function=HerbInterpret.test_with_input)\n\nReturns a GeneticSearchIterator given a grammar. The iterator is fitted against the examples provided evaluated using the fitness function. All other arguments are hyperparameters for the genetic search procedure.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.get_mh_enumerator","page":"HerbSearch.jl","title":"HerbSearch.get_mh_enumerator","text":"get_mh_enumerator(examples::AbstractArray{<:Example}, cost_function::Function, evaluation_function::Function=HerbInterpret.test_with_input)\n\nReturns an enumerator that runs according to the Metropolis Hastings algorithm.\n\nexamples : array of examples\ncost_function : cost function to evaluate the programs proposed\nevaluation_function : evaluation function that evaluates the program generated and produces an output \n\nThe propose function is randomfillpropose and the accept function is probabilistic. The temperature value of the algorithm remains constant over time. \n\n\n\n\n\n","category":"function"},{"location":"HerbSearch/#HerbSearch.get_most_likely_first_enumerator","page":"HerbSearch.jl","title":"HerbSearch.get_most_likely_first_enumerator","text":"get_most_likely_first_enumerator(grammar::ContextFreeGrammar, max_depth::Int, max_size::Int, sym::Symbol, hole_heuristic::Function=heuristic_leftmost, derivation_heuristic::Function=(a,_) -> a)::ContextSensitivePriorityEnumerator\n\nReturns an enumerator that enumerates expressions in the grammar in decreasing order of probability. Only use this function with probabilistic grammars.\n\n\n\n\n\n","category":"function"},{"location":"HerbSearch/#HerbSearch.get_most_likely_first_enumerator-2","page":"HerbSearch.jl","title":"HerbSearch.get_most_likely_first_enumerator","text":"get_most_likely_first_enumerator(grammar::ContextSensitiveGrammar, max_depth::Int, max_size::Int, sym::Symbol, hole_heuristic::Function=heuristic_leftmost, derivation_heuristic::Function=(a,_) -> a)::ContextSensitivePriorityEnumerator\n\nReturns an enumerator that enumerates expressions in the grammar in decreasing order of probability. Only use this function with probabilistic grammars.\n\n\n\n\n\n","category":"function"},{"location":"HerbSearch/#HerbSearch.get_sa_enumerator","page":"HerbSearch.jl","title":"HerbSearch.get_sa_enumerator","text":"get_sa_enumerator(examples, cost_function, initial_temperature=1, temperature_decreasing_factor = 0.99, evaluation_function::Function=HerbInterpret.test_with_input)\n\nReturns an enumerator that runs according to the Very Large Scale Neighbourhood Search algorithm.\n\nexamples : array of examples\ncost_function : cost function to evaluate the programs proposed\ninitial_temperature : the starting temperature of the algorithm\ntemperature_decreasing_factor : the decreasing factor of the temperature of the time\nevaluation_function : evaluation function that evaluates the program generated and produces an output \n\nThe propose function is random_fill_propose (the same as for Metropolis Hastings). The accept function is probabilistic but takes into account the tempeerature too.\n\n\n\n\n\n","category":"function"},{"location":"HerbSearch/#HerbSearch.get_vlsn_enumerator","page":"HerbSearch.jl","title":"HerbSearch.get_vlsn_enumerator","text":"get_vlsn_enumerator(examples, cost_function, enumeration_depth = 2, evaluation_function::Function=HerbInterpret.test_with_input)\n\nReturns an enumerator that runs according to the Very Large Scale Neighbourhood Search algorithm.\n\nexamples : array of examples\ncost_function : cost function to evaluate the programs proposed\nenumeration_depth : the enumeration depth to search for a best program at a time\nevaluation_function : evaluation function that evaluates the program generated and produces an output \n\nThe propose function consists of all possible programs of the given enumeration_depth. The accept function accepts the program with the lowest cost according to the cost_function. The temperature value of the algorithm remains constant over time. \n\n\n\n\n\n","category":"function"},{"location":"HerbSearch/#HerbSearch.heuristic_leftmost-Tuple{AbstractRuleNode, Int64}","page":"HerbSearch.jl","title":"HerbSearch.heuristic_leftmost","text":"heuristic_leftmost(node::AbstractRuleNode, max_depth::Int)::Union{ExpandFailureReason, HoleReference}\n\nDefines a heuristic over holes, where the left-most hole always gets considered first. Returns a HoleReference once a hole is found. This is the default option for enumerators.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.heuristic_random-Tuple{AbstractRuleNode, Int64}","page":"HerbSearch.jl","title":"HerbSearch.heuristic_random","text":"heuristic_random(node::AbstractRuleNode, max_depth::Int)::Union{ExpandFailureReason, HoleReference}\n\nDefines a heuristic over holes, where random holes get chosen randomly using random exploration. Returns a HoleReference once a hole is found.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.heuristic_rightmost-Tuple{AbstractRuleNode, Int64}","page":"HerbSearch.jl","title":"HerbSearch.heuristic_rightmost","text":"heuristic_rightmost(node::AbstractRuleNode, max_depth::Int)::Union{ExpandFailureReason, HoleReference}\n\nDefines a heuristic over holes, where the right-most hole always gets considered first. Returns a HoleReference once a hole is found. \n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.heuristic_smallest_domain-Tuple{AbstractRuleNode, Int64}","page":"HerbSearch.jl","title":"HerbSearch.heuristic_smallest_domain","text":"heuristic_smallest_domain(node::AbstractRuleNode, max_depth::Int)::Union{ExpandFailureReason, HoleReference}\n\nDefines a heuristic over all available holes in the unfinished AST, by considering the size of their respective domains. A domain here describes the number of possible derivations with respect to the constraints. Returns a HoleReference once a hole is found. \n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.mean_squared_error-Tuple{T} where T<:(AbstractVector{<:Tuple{Number, Number}})","page":"HerbSearch.jl","title":"HerbSearch.mean_squared_error","text":"mean_squared_error(results::AbstractVector{Tuple{<:Number,<:Number}})\n\nReturns the mean squared error of results.\n\nArguments\n\nresults<:AbstractVector{<:Tuple{Number,Number}}: the vector of tuples, where each tuple is in the form Tuple{expected_output, actual_output}.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.misclassification-Tuple{T} where T<:(AbstractVector{<:Tuple{Number, Number}})","page":"HerbSearch.jl","title":"HerbSearch.misclassification","text":"misclassification(results::AbstractVector{Tuple{<:Number,<:Number}})\n\nReturns the amount of misclassified examples, i.e. how many tuples with non-matching entries are there in results.\n\nArguments\n\nresults<:AbstractVector{<:Tuple{Number,Number}}: the vector of tuples, where each tuple is in the form Tuple{expected_output, actual_output}.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.most_likely_priority_function-Tuple{ContextSensitiveGrammar, AbstractRuleNode, Union{Real, Tuple{Vararg{Real}}}}","page":"HerbSearch.jl","title":"HerbSearch.most_likely_priority_function","text":"most_likely_priority_function(g::ContextSensitiveGrammar, tree::AbstractRuleNode, ::Union{Real, Tuple{Vararg{Real}}})\n\nCalculates logit for all possible derivations for a node in a tree and returns them.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.mse_error_function-Tuple{Any, Any, Any}","page":"HerbSearch.jl","title":"HerbSearch.mse_error_function","text":"mse_error_function(old_error, output, expected_output)\n\nMean squared error function for search_best.\n\n- old_error - The existing total error\n- output - The actual output of the evaluator\n- expected_output - The expected output for the example\n\nThe function build the mean squared error from output and expected_output`.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.mutate_random!","page":"HerbSearch.jl","title":"HerbSearch.mutate_random!","text":"mutate_random!(program::RuleNode, grammar::Grammar, max_depth::Int64 = 2)\n\nMutates the given program by inserting a randomly generated sub-program at a random location.\n\n\n\n\n\n","category":"function"},{"location":"HerbSearch/#HerbSearch.probabilistic_accept-Tuple{Real, Real, Real}","page":"HerbSearch.jl","title":"HerbSearch.probabilistic_accept","text":"probabilistic_accept(current_cost::Real, next_cost::Real, temperature::Real)\n\nProbabilistically decides whether to accept the new program (next) based on the ratio of costs (smaller is better) between the previous and new program. Returns True if the new program is accepted, False otherwise.\n\nArguments\n\ncurrent_cost::Real: the cost of the current program.\nnext_cost::Real: the cost of the proposed program.\ntemperature::Real: the temperature; not used.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.probabilistic_accept_with_temperature-Tuple{Real, Real, Real}","page":"HerbSearch.jl","title":"HerbSearch.probabilistic_accept_with_temperature","text":"probabilistic_accept_with_temperature(current_cost::Real, next_cost::Real, temperature::Real)\n\nReturns true if the cost of the proposed program is smaller than the cost of the current program. Otherwise, returns true with the probability equal to: \n\n1 (1 + exp(delta temperature))\n\nIn any other case, returns false.\n\nArguments\n\ncurrent_cost::Real: the cost of the current program.\nnext_cost::Real: the cost of the proposed program.\ntemperature::Real: the temperature of the search.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.probabilistic_accept_with_temperature_fraction-Tuple{Real, Real, Real}","page":"HerbSearch.jl","title":"HerbSearch.probabilistic_accept_with_temperature_fraction","text":"probabilistic_accept_with_temperature_fraction(current_cost::Real, program_to_consider_cost::Real, temperature::Real)\n\nProbabilistically decides whether to accept the new program (next) based on the ratio of costs (smaller is better) between the previous and new program multiplied by the temperature. Returns True if the new program is accepted, False otherwise.\n\nArguments\n\ncurrent_cost::Real: the cost of the current program.\nnext_cost::Real: the cost of the proposed program.\ntemperature::Real: the current temperature \n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.propagate_constraints","page":"HerbSearch.jl","title":"HerbSearch.propagate_constraints","text":"function propagate_constraints(root::AbstractRuleNode, grammar::ContextSensitiveGrammar, local_constraints::Set{LocalConstraint}, max_holes::Int, filled_hole::Union{HoleReference, Nothing}=nothing)::Tuple{PropagateResult, Set{LocalConstraint}}\n\nPropagates a set of local constraints recursively to all children of a given root node. As propagate_constraints gets often called when a hole was just filled, filled_hole helps keeping track to propagate the constraints to relevant nodes, e.g. children of filled_hole. max_holes makes sure that max_size of Base.iterate is not violated. The function returns the PropagateResult and the set of relevant LocalConstraints.\n\n\n\n\n\n","category":"function"},{"location":"HerbSearch/#HerbSearch.random_fill_propose-Tuple{RuleNode, NodeLoc, Grammar, Int64, AbstractVector{Int64}, Union{Nothing, Dict{String, Any}}}","page":"HerbSearch.jl","title":"HerbSearch.random_fill_propose","text":"random_fill_propose(current_program, neighbourhood_node_loc, grammar, max_depth, dict)\n\nReturns a list with only one proposed, completely random, subprogram.\n\nArguments\n\ncurrent_program::RuleNode: the current program.\nneighbourhood_node_loc::NodeLoc: the location of the program to replace.\ngrammar::Grammar: the grammar used to create programs.\nmax_depth::Int: the maximum depth of the resulting programs.\ndmap::AbstractVector{Int} : the minimum possible depth to reach for each rule\ndict::Dict{String, Any}: the dictionary with additional arguments; not used.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.search-Tuple{Grammar, Problem, Symbol}","page":"HerbSearch.jl","title":"HerbSearch.search","text":"search(g::Grammar, problem::Problem, start::Symbol; evaluator::Function=test_with_input, enumerator::Function=get_bfs_enumerator, max_depth::Union{Int, Nothing}=nothing, max_size::Union{Int, Nothing}=nothing, max_time::Union{Int, Nothing}=nothing, max_enumerations::Union{Int, Nothing}=nothing, allow_evaluation_errors::Bool=false, mod::Module=Main)::Union{Any, Nothing}\n\nSearches for a program by calling search_rulenode starting from Symbol start guided by enumerator and Grammar trying to satisfy the higher-order constraints in form of input/output examples defined in the Problem. This is the heart of the Herb's search for satisfying programs. Returns the found program when the evaluation calculated using evaluator is successful.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.search_best-Tuple{Grammar, Problem, Symbol}","page":"HerbSearch.jl","title":"HerbSearch.search_best","text":"search_best(g::Grammar, problem::Problem, start::Symbol; evaluator::Function=test_with_input, enumerator::Function=get_bfs_enumerator, error_function::Function=default_error_function, max_depth::Union{Int, Nothing}=nothing, max_size::Union{Int, Nothing}=nothing, max_time::Union{Int, Nothing}=nothing, max_enumerations::Union{Int, Nothing}=nothing, allow_evaluation_errors::Bool=false, mod::Module=Main)::Tuple{Any, Real}\n\nSearches the grammar for the program that satisfies the maximum number of examples in the problem. The evaluator should be a function that takes a SymbolTable, expression and a dictionary with input variable assignments and returns the output of the expression.\n\n- g - The grammar that defines the search space\n- problem - The problem definition with IO examples\n- start - The start symbol in the grammar\n- evaluator - The evaluation function. Takes a SymbolTable, expression and a dictionary with \n input variable assignments and returns the output of the expression.\n- enumerator - A constructor for the enumerator that should be used in the search\n- error_function - The error function. Takes the existing total error, the actual output of the evaluator \n and the expected value for the example.\n- max_depth - The maximum depth of the search\n- max_time - The maximum time allowed for the search in seconds\n- max_enumerations - The maximum number of programs to enumerate and test\n- allow_evaluation_errors - Whether the search should crash if an exception is thrown in the evaluation\n- mod - A module containing definitions for the functions in the grammar that do not exist in Main\n\nReturns a tuple with the best found program so far and the error. Can be considerably slower than search due to having to evaluate each expression on each example.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.search_rulenode-Tuple{Grammar, Problem, Symbol}","page":"HerbSearch.jl","title":"HerbSearch.search_rulenode","text":"search_rulenode(g::Grammar, problem::Problem, start::Symbol; evaluator::Function=test_with_input, enumerator::Function=get_bfs_enumerator, max_depth::Union{Int, Nothing}=nothing, max_size::Union{Int, Nothing}=nothing, max_time::Union{Int, Nothing}=nothing, max_enumerations::Union{Int, Nothing}=nothing, allow_evaluation_errors::Bool=false, mod::Module=Main)::Union{Tuple{RuleNode, Any}, Nothing}\n\nSearches the grammar for the program that satisfies the maximum number of examples in the problem.\n\n - g - The grammar that defines the search space\n - problem - The problem definition with IO examples\n - start - The start symbol in the grammar\n - evaluator - The evaluation function. Takes a SymbolTable, expression and a dictionary with \n input variable assignments and returns the output of the expression.\n - enumerator - A constructor for the enumerator that should be used in the search\n - max_depth - The maximum depth of the search\n - max_size - The maximum number of nodes for ASTs in the search\n - max_time - The maximum time allowed for the search in seconds\n - max_enumerations - The maximum number of programs to enumerate and test'\n - allow_evaluation_errors - Whether the search should crash if an exception is thrown in the evaluation\n - mod - A module containing definitions for the functions in the grammar that do not exist in Main\nReturns a tuple of the rulenode and the expression of the solution program once it has been found, \nor nothing otherwise.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.select_chromosome-Tuple{Array{RuleNode}, Array{<:Real}}","page":"HerbSearch.jl","title":"HerbSearch.select_chromosome","text":"select_chromosome(population::Array{RuleNode}, fitness_array::Array{<:Real})::RuleNode\n\nSelects a chromosome (individual) from the population based on a fitness array. The function uses a fitness-proportionate selection strategy, often referred to as \"roulette wheel\" selection. Assumes fitness_array to be normalized already.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.select_fitness_proportional_parents-Tuple{Array{RuleNode}, Array{<:Real}}","page":"HerbSearch.jl","title":"HerbSearch.select_fitness_proportional_parents","text":"select_fitness_proportional_parents(population::Array{RuleNode}, fitness_array::Array{<:Real})::Tuple{RuleNode,RuleNode}\n\nSelects two parent chromosomes (individuals) from a population based on fitness-proportionate selection. The selected parents can be used for genetic crossover in the next steps of the algorithm.\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#HerbSearch.validate_iterator-Tuple{Any}","page":"HerbSearch.jl","title":"HerbSearch.validate_iterator","text":"validate_iterator(iter)\n\nValidates the parameters of the iterator\n\n\n\n\n\n","category":"method"},{"location":"HerbSearch/#StatsBase.sample","page":"HerbSearch.jl","title":"StatsBase.sample","text":"sample(root::RuleNode, typ::Symbol, grammar::Grammar, maxdepth::Int=typemax(Int))\n\nUniformly samples a random node from the tree limited to maxdepth.\n\n\n\n\n\n","category":"function"},{"location":"HerbSearch/#StatsBase.sample-2","page":"HerbSearch.jl","title":"StatsBase.sample","text":"sample(::Type{NodeLoc}, root::RuleNode, typ::Symbol, grammar::Grammar)\n\nUniformly selects a random node in the tree of a given type, specified using its parent such that the subtree can be replaced. Returns a NodeLoc.\n\n\n\n\n\n","category":"function"},{"location":"HerbSearch/#StatsBase.sample-3","page":"HerbSearch.jl","title":"StatsBase.sample","text":"sample(::Type{NodeLoc}, root::RuleNode, maxdepth::Int=typemax(Int))\n\nUniformly selects a random node in the tree no deeper than maxdepth using reservoir sampling. Returns a NodeLoc that specifies the location using its parent so that the subtree can be replaced.\n\n\n\n\n\n","category":"function"},{"location":"HerbSearch/#StatsBase.sample-4","page":"HerbSearch.jl","title":"StatsBase.sample","text":"sample(root::RuleNode, typ::Symbol, grammar::Grammar,\n maxdepth::Int=typemax(Int))\n\nUniformly selects a random node of the given return type typ limited by maxdepth.\n\n\n\n\n\n","category":"function"},{"location":"HerbSearch/","page":"HerbSearch.jl","title":"HerbSearch.jl","text":"The HerbSearch package takes care of all operations related to searching for the desired program. This includes","category":"page"},{"location":"HerbSearch/","page":"HerbSearch.jl","title":"HerbSearch.jl","text":"the functionality to sample a certain program given a grammar,\nthe implementation of several heuristic functions,\nsearching for a program that satisfies the specification, and\nimplementations of several search algorithms in terms of how they enumerate the search space\nBreadth-First Search \nDepth-First Search \nMetropolis Hastings \nVery Large Scale Neighbourhood Search \nSimulated Annealing\nGenetic Search","category":"page"},{"location":"HerbSearch/#Index","page":"HerbSearch.jl","title":"Index","text":"","category":"section"},{"location":"HerbSearch/","page":"HerbSearch.jl","title":"HerbSearch.jl","text":"","category":"page"},{"location":"HerbCore/#HerbCore_docs","page":"HerbCore.jl Documentation","title":"HerbCore.jl Documentation","text":"","category":"section"},{"location":"HerbCore/","page":"HerbCore.jl Documentation","title":"HerbCore.jl Documentation","text":"CurrentModule=HerbCore","category":"page"},{"location":"HerbCore/","page":"HerbCore.jl Documentation","title":"HerbCore.jl Documentation","text":"Modules = [HerbCore]\nOrder = [:type, :const, :macro, :function]","category":"page"},{"location":"HerbCore/#HerbCore.AbstractRuleNode","page":"HerbCore.jl Documentation","title":"HerbCore.AbstractRuleNode","text":"AbstractRuleNode\n\nAbstract type for representing expression trees. Expression trees consist of RuleNodes and Holes.\n\nA RuleNode represents a certain production rule in the Grammar.\nA Hole is a placeholder where certain rules in the grammar still can be applied. \n\n\n\n\n\n","category":"type"},{"location":"HerbCore/#HerbCore.Constraint","page":"HerbCore.jl Documentation","title":"HerbCore.Constraint","text":"Represents a constraint for a ContextSensitiveGrammar. Concrete implementations can be found in HerbConstraints.jl.\n\n\n\n\n\n","category":"type"},{"location":"HerbCore/#HerbCore.Grammar","page":"HerbCore.jl Documentation","title":"HerbCore.Grammar","text":"Grammar\n\nAbstract type representing all grammars. It is assumed that all grammar structs have at least the following attributes:\n\nrules::Vector{Any}: A list of RHS of rules (subexpressions).\ntypes::Vector{Symbol}: A list of LHS of rules (types, all symbols).\nisterminal::BitVector: A bitvector where bit i represents whether rule i is terminal.\niseval::BitVector: A bitvector where bit i represents whether rule i is an eval rule.\nbytype::Dict{Symbol,Vector{Int}}: A dictionary that maps a type to all rules of said type.\ndomains::Dict{Symbol, BitVector}: A dictionary that maps a type to a domain bitvector. The domain bitvector has bit i set to true iff the ith rule is of this type.\nchildtypes::Vector{Vector{Symbol}}: A list of types of the children for each rule. \n\nIf a rule is terminal, the corresponding list is empty.\n\nlog_probabilities::Union{Vector{Real}, Nothing}: A list of probabilities for each rule. \n\nIf the grammar is non-probabilistic, the list can be nothing.\n\nFor concrete types, see ContextFreeGrammar and ContextSensitiveGrammar.\n\n\n\n\n\n","category":"type"},{"location":"HerbCore/#HerbCore.Hole","page":"HerbCore.jl Documentation","title":"HerbCore.Hole","text":"Hole <: AbstractRuleNode\n\nA Hole is a placeholder where certain rules from the grammar can still be applied. The domain of a Hole defines which rules can be applied. The domain is a bitvector, where the ith bit is set to true if the ith rule in the grammar can be applied. \n\n\n\n\n\n","category":"type"},{"location":"HerbCore/#HerbCore.HoleReference","page":"HerbCore.jl Documentation","title":"HerbCore.HoleReference","text":"HoleReference\n\nContains a hole and the path to the hole from the root of the tree.\n\n\n\n\n\n","category":"type"},{"location":"HerbCore/#HerbCore.RuleNode","page":"HerbCore.jl Documentation","title":"HerbCore.RuleNode","text":"RuleNode <: AbstractRuleNode\n\nA RuleNode represents a node in an expression tree. Each node corresponds to a certain rule in the Grammar. A RuleNode consists of:\n\nind: The index of the rule in the Grammar which this node is representing.\n_val: Field for storing immediately evaluated values\nchildren: The children of this node in the expression tree\n\ncompat: Compat\nEvaluate immediately functionality is not yet supported by most of Herb.jl.\n\n\n\n\n\n","category":"type"},{"location":"HerbCore/#HerbCore.RuleNode-Tuple{Int64, Any}","page":"HerbCore.jl Documentation","title":"HerbCore.RuleNode","text":"RuleNode(ind::Int, _val::Any)\n\nCreate a RuleNode for the Grammar rule with index ind, _val as immediately evaluated value and no children\n\nwarning: Warning\nOnly use this constructor if you are absolutely certain that a rule is terminal and cannot have children. Use [RuleNode(ind::Int, grammar::Grammar)] for rules that might have children. In general, Holes should be used as a placeholder when the children of a node are not yet known. \n\ncompat: Compat\nEvaluate immediately functionality is not yet supported by most of Herb.jl.\n\n\n\n\n\n","category":"method"},{"location":"HerbCore/#HerbCore.RuleNode-Tuple{Int64, Vector{AbstractRuleNode}}","page":"HerbCore.jl Documentation","title":"HerbCore.RuleNode","text":"RuleNode(ind::Int, children::Vector{AbstractRuleNode})\n\nCreate a RuleNode for the Grammar rule with index ind and children as subtrees.\n\n\n\n\n\n","category":"method"},{"location":"HerbCore/#Base.isless-Tuple{AbstractRuleNode, AbstractRuleNode}","page":"HerbCore.jl Documentation","title":"Base.isless","text":"Base.isless(rn₁::AbstractRuleNode, rn₂::AbstractRuleNode)::Bool\n\nCompares two RuleNodes. Returns true if the left RuleNode is less than the right RuleNode. Order is determined from the index of the RuleNodes. If both RuleNodes have the same index, a depth-first search is performed in both RuleNodes until nodes with a different index are found.\n\n\n\n\n\n","category":"method"},{"location":"HerbCore/#Base.length-Tuple{Hole}","page":"HerbCore.jl Documentation","title":"Base.length","text":"Base.length(root::RuleNode)\n\nReturn the number of nodes in the tree rooted at root. Holes don't count.\n\n\n\n\n\n","category":"method"},{"location":"HerbCore/#Base.length-Tuple{RuleNode}","page":"HerbCore.jl Documentation","title":"Base.length","text":"Base.length(root::RuleNode)\n\nReturn the number of nodes in the tree rooted at root. Holes don't count.\n\n\n\n\n\n","category":"method"},{"location":"HerbCore/#HerbCore.contains_hole-Tuple{RuleNode}","page":"HerbCore.jl Documentation","title":"HerbCore.contains_hole","text":"contains_hole(rn::RuleNode) = any(contains_hole(c) for c ∈ rn.children)\n\nChecks if an AbstractRuleNode tree contains a Hole.\n\n\n\n\n\n","category":"method"},{"location":"HerbCore/#HerbCore.depth-Tuple{RuleNode}","page":"HerbCore.jl Documentation","title":"HerbCore.depth","text":"depth(root::RuleNode)::Int\n\nReturn the depth of the AbstractRuleNode tree rooted at root. Holes don't count towards the depth.\n\n\n\n\n\n","category":"method"},{"location":"HerbCore/#HerbCore.get_node_at_location-Tuple{RuleNode, Vector{Int64}}","page":"HerbCore.jl Documentation","title":"HerbCore.get_node_at_location","text":"get_node_at_location(root::RuleNode, location::Vector{Int})\n\nRetrieves a RuleNode at the given location by reference. \n\n\n\n\n\n","category":"method"},{"location":"HerbCore/#HerbCore.get_rulesequence-Tuple{RuleNode, Vector{Int64}}","page":"HerbCore.jl Documentation","title":"HerbCore.get_rulesequence","text":"get_rulesequence(node::RuleNode, path::Vector{Int})\n\nExtract the derivation sequence from a path (sequence of child indices) and an AbstractRuleNode. If the path is deeper than the deepest node, it returns what it has.\n\n\n\n\n\n","category":"method"},{"location":"HerbCore/#HerbCore.node_depth-Tuple{AbstractRuleNode, AbstractRuleNode}","page":"HerbCore.jl Documentation","title":"HerbCore.node_depth","text":"node_depth(root::AbstractRuleNode, node::AbstractRuleNode)::Int\n\nReturn the depth of node for an AbstractRuleNode tree rooted at root. Depth is 1 when root == node.\n\nwarning: Warning\nnode must be a subtree of root in order for this function to work.\n\n\n\n\n\n","category":"method"},{"location":"HerbCore/#HerbCore.rulesoftype-Tuple{RuleNode, Set{Int64}}","page":"HerbCore.jl Documentation","title":"HerbCore.rulesoftype","text":"rulesoftype(node::RuleNode, ruleset::Set{Int})\n\nReturns every rule in the ruleset that is also used in the AbstractRuleNode tree.\n\n\n\n\n\n","category":"method"},{"location":"HerbCore/#HerbCore.rulesonleft-Tuple{RuleNode, Vector{Int64}}","page":"HerbCore.jl Documentation","title":"HerbCore.rulesonleft","text":"rulesonleft(expr::RuleNode, path::Vector{Int})::Set{Int}\n\nFinds all rules that are used in the left subtree defined by the path.\n\n\n\n\n\n","category":"method"},{"location":"HerbCore/#HerbCore.swap_node-Tuple{AbstractRuleNode, AbstractRuleNode, Vector{Int64}}","page":"HerbCore.jl Documentation","title":"HerbCore.swap_node","text":"swap_node(expr::AbstractRuleNode, new_expr::AbstractRuleNode, path::Vector{Int})\n\nReplace a node in expr, specified by path, with new_expr. Path is a sequence of child indices, starting from the root node.\n\n\n\n\n\n","category":"method"},{"location":"HerbCore/#HerbCore.swap_node-Tuple{RuleNode, RuleNode, Int64, RuleNode}","page":"HerbCore.jl Documentation","title":"HerbCore.swap_node","text":"swap_node(expr::RuleNode, node::RuleNode, child_index::Int, new_expr::RuleNode)\n\nReplace child i of a node, a part of larger expr, with new_expr.\n\n\n\n\n\n","category":"method"},{"location":"HerbCore/#Index","page":"HerbCore.jl Documentation","title":"Index","text":"","category":"section"},{"location":"HerbCore/","page":"HerbCore.jl Documentation","title":"HerbCore.jl Documentation","text":"","category":"page"},{"location":"HerbGrammar/#HerbGrammar_docs","page":"HerbGrammar.jl","title":"HerbGrammar.jl Documentation","text":"","category":"section"},{"location":"HerbGrammar/","page":"HerbGrammar.jl","title":"HerbGrammar.jl","text":"CurrentModule=HerbGrammar","category":"page"},{"location":"HerbGrammar/","page":"HerbGrammar.jl","title":"HerbGrammar.jl","text":"Modules = [HerbGrammar]\nOrder = [:type, :const, :macro, :function]","category":"page"},{"location":"HerbGrammar/#HerbGrammar.ContextFreeGrammar","page":"HerbGrammar.jl","title":"HerbGrammar.ContextFreeGrammar","text":"ContextFreeGrammar <: Grammar\n\nRepresents a context-free grammar and its production rules. Consists of:\n\nrules::Vector{Any}: A list of RHS of rules (subexpressions).\ntypes::Vector{Symbol}: A list of LHS of rules (types, all symbols).\nisterminal::BitVector: A bitvector where bit i represents whether rule i is terminal.\niseval::BitVector: A bitvector where bit i represents whether rule i is an eval rule.\nbytype::Dict{Symbol,Vector{Int}}: A dictionary that maps a type to all rules of said type.\ndomains::Dict{Symbol, BitVector}: A dictionary that maps a type to a domain bitvector. The domain bitvector has bit i set to true iff the ith rule is of this type.\nchildtypes::Vector{Vector{Symbol}}: A list of types of the children for each rule. If a rule is terminal, the corresponding list is empty.\nlog_probabilities::Union{Vector{Real}, Nothing}: A list of probabilities for each rule. If the grammar is non-probabilistic, the list can be nothing.\n\nUse the @cfgrammar macro to create a ContextFreeGrammar object. Use the @pcfgrammar macro to create a ContextFreeGrammar object with probabilities. For context-sensitive grammars, see ContextSensitiveGrammar.\n\n\n\n\n\n","category":"type"},{"location":"HerbGrammar/#HerbGrammar.ContextSensitiveGrammar","page":"HerbGrammar.jl","title":"HerbGrammar.ContextSensitiveGrammar","text":"ContextSensitiveGrammar <: Grammar\n\nRepresents a context-sensitive grammar. Extends Grammar with constraints.\n\nConsists of:\n\nrules::Vector{Any}: A list of RHS of rules (subexpressions).\ntypes::Vector{Symbol}: A list of LHS of rules (types, all symbols).\nisterminal::BitVector: A bitvector where bit i represents whether rule i is terminal.\niseval::BitVector: A bitvector where bit i represents whether rule i is an eval rule.\nbytype::Dict{Symbol,Vector{Int}}: A dictionary that maps a type to all rules of said type.\ndomains::Dict{Symbol, BitVector}: A dictionary that maps a type to a domain bitvector. The domain bitvector has bit i set to true iff the ith rule is of this type.\nchildtypes::Vector{Vector{Symbol}}: A list of types of the children for each rule. If a rule is terminal, the corresponding list is empty.\nlog_probabilities::Union{Vector{Real}, Nothing}: A list of probabilities for each rule. If the grammar is non-probabilistic, the list can be nothing.\nconstraints::Vector{Constraint}: A list of constraints that programs in this grammar have to abide.\n\nUse the @csgrammar macro to create a ContextSensitiveGrammar object. Use the @pcsgrammar macro to create a ContextSensitiveGrammar object with probabilities. For context-free grammars, see ContextFreeGrammar.\n\n\n\n\n\n","category":"type"},{"location":"HerbGrammar/#HerbGrammar.NodeLoc","page":"HerbGrammar.jl","title":"HerbGrammar.NodeLoc","text":"NodeLoc A helper struct that points to a node in the tree via its parent such that the child can be easily swapped out. If i is 0 the node pointed to is the root node and parent is the node itself.\n\n\n\n\n\n","category":"type"},{"location":"HerbGrammar/#HerbGrammar.SymbolTable","page":"HerbGrammar.jl","title":"HerbGrammar.SymbolTable","text":"SymbolTable(grammar::Grammar, mod::Module=Main)\n\nReturns a SymbolTable populated with a mapping from symbols in the Grammar to symbols in module mod or Main, if defined.\n\n\n\n\n\n","category":"type"},{"location":"HerbGrammar/#HerbGrammar.SymbolTable-2","page":"HerbGrammar.jl","title":"HerbGrammar.SymbolTable","text":"SymbolTable\n\nData structure for mapping terminal symbols in the Grammar to their Julia interpretation.\n\n\n\n\n\n","category":"type"},{"location":"HerbGrammar/#HerbGrammar.@cfgrammar-Tuple{Any}","page":"HerbGrammar.jl","title":"HerbGrammar.@cfgrammar","text":"@cfgrammar\n\nA macro for defining a ContextFreeGrammar. \n\nExample usage:\n\ngrammar = @cfgrammar begin\n\tR = x\n\tR = 1 | 2\n\tR = R + R\nend\n\nSyntax:\n\nLiterals: Symbols that are already defined in Julia are considered literals, such as 1, 2, or π. For example: R = 1.\nVariables: A variable is a symbol that is not a nonterminal symbol and not already defined in Julia. For example: R = x.\nFunctions: Functions and infix operators that are defined in Julia or the Main module can be used with the default evaluator. For example: R = R + R, R = f(a, b).\nCombinations: Multiple rules can be defined on a single line in the grammar definition using the | symbol. For example: R = 1 | 2 | 3.\nIterators: Another way to define multiple rules is by providing a Julia iterator after a | symbol. For example: R = |(1:9).\n\nRelated:\n\n@csgrammar uses the same syntax to create ContextSensitiveGrammars.\n@pcfgrammar uses a similar syntax to create probabilistic ContextFreeGrammars.\n\n\n\n\n\n","category":"macro"},{"location":"HerbGrammar/#HerbGrammar.@csgrammar-Tuple{Any}","page":"HerbGrammar.jl","title":"HerbGrammar.@csgrammar","text":"@csgrammar\n\nA macro for defining a ContextSensitiveGrammar. Constraints can be added afterwards using the addconstraint! function.\n\nExample usage:\n\ngrammar = @csgrammar begin\n\tR = x\n\tR = 1 | 2\n\tR = R + R\nend\n\nSyntax:\n\nLiterals: Symbols that are already defined in Julia are considered literals, such as 1, 2, or π. For example: R = 1.\nVariables: A variable is a symbol that is not a nonterminal symbol and not already defined in Julia. For example: R = x.\nFunctions: Functions and infix operators that are defined in Julia or the Main module can be used with the default evaluator. For example: R = R + R, R = f(a, b).\nCombinations: Multiple rules can be defined on a single line in the grammar definition using the | symbol. For example: R = 1 | 2 | 3.\nIterators: Another way to define multiple rules is by providing a Julia iterator after a | symbol. For example: R = |(1:9).\n\nRelated:\n\n@cfgrammar uses the same syntax to create ContextFreeGrammars.\n@pcsgrammar uses a similar syntax to create probabilistic ContextSensitiveGrammars.\n\n\n\n\n\n","category":"macro"},{"location":"HerbGrammar/#HerbGrammar.@pcfgrammar-Tuple{Any}","page":"HerbGrammar.jl","title":"HerbGrammar.@pcfgrammar","text":"@pcfgrammar\n\nA macro for defining a probabilistic ContextFreeGrammar. \n\nExample usage:\n\ngrammar = @pcfgrammar begin\n\t0.5 : R = x\n\t0.3 : R = 1 | 2\n\t0.2 : R = R + R\nend\n\nSyntax:\n\nThe syntax of rules is identical to the syntax used by @cfgrammar:\n\nLiterals: Symbols that are already defined in Julia are considered literals, such as 1, 2, or π. For example: R = 1.\nVariables: A variable is a symbol that is not a nonterminal symbol and not already defined in Julia. For example: R = x.\nFunctions: Functions and infix operators that are defined in Julia or the Main module can be used with the default evaluator. For example: R = R + R, R = f(a, b).\nCombinations: Multiple rules can be defined on a single line in the grammar definition using the | symbol. For example: R = 1 | 2 | 3.\nIterators: Another way to define multiple rules is by providing a Julia iterator after a | symbol. For example: R = |(1:9).\n\nEvery rule is also prefixed with a probability. Rules and probabilities are separated using the : symbol. If multiple rules are defined on a single line, the probability is equally divided between the rules. The sum of probabilities for all rules of a certain non-terminal symbol should be equal to 1. The probabilities are automatically scaled if this isn't the case.\n\nRelated:\n\n@pcsgrammar uses the same syntax to create probabilistic ContextSensitiveGrammars.\n@cfgrammar uses a similar syntax to create non-probabilistic ContextFreeGrammars.\n\n\n\n\n\n","category":"macro"},{"location":"HerbGrammar/#HerbGrammar.@pcsgrammar-Tuple{Any}","page":"HerbGrammar.jl","title":"HerbGrammar.@pcsgrammar","text":"@pcsgrammar\n\nA macro for defining a probabilistic ContextSensitiveGrammar. \n\nExample usage:\n\ngrammar = @pcsgrammar begin\n\t0.5 : R = x\n\t0.3 : R = 1 | 2\n\t0.2 : R = R + R\nend\n\nSyntax:\n\nThe syntax of rules is identical to the syntax used by @csgrammar:\n\nLiterals: Symbols that are already defined in Julia are considered literals, such as 1, 2, or π. For example: R = 1.\nVariables: A variable is a symbol that is not a nonterminal symbol and not already defined in Julia. For example: R = x.\nFunctions: Functions and infix operators that are defined in Julia or the Main module can be used with the default evaluator. For example: R = R + R, R = f(a, b).\nCombinations: Multiple rules can be defined on a single line in the grammar definition using the | symbol. For example: R = 1 | 2 | 3.\nIterators: Another way to define multiple rules is by providing a Julia iterator after a | symbol. For example: R = |(1:9).\n\nEvery rule is also prefixed with a probability. Rules and probabilities are separated using the : symbol. If multiple rules are defined on a single line, the probability is equally divided between the rules. The sum of probabilities for all rules of a certain non-terminal symbol should be equal to 1. The probabilities are automatically scaled if this isn't the case.\n\nRelated:\n\n@pcfgrammar uses the same syntax to create probabilistic ContextFreeGrammars.\n@csgrammar uses a similar syntax to create non-probabilistic ContextSensitiveGrammars.\n\n\n\n\n\n","category":"macro"},{"location":"HerbGrammar/#Base.get-Tuple{RuleNode, NodeLoc}","page":"HerbGrammar.jl","title":"Base.get","text":"get(root::RuleNode, loc::NodeLoc) Obtain the node pointed to by loc.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#Base.insert!-Tuple{RuleNode, NodeLoc, RuleNode}","page":"HerbGrammar.jl","title":"Base.insert!","text":"insert!(loc::NodeLoc, rulenode::RuleNode) Replaces the subtree pointed to by loc with the given rulenode.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.add_rule!-Tuple{Grammar, Expr}","page":"HerbGrammar.jl","title":"HerbGrammar.add_rule!","text":"add_rule!(g::Grammar, e::Expr)\n\nAdds a rule to the grammar. \n\nUsage:\n\n\tadd_rule!(grammar, :(\"Real = Real + Real\"))\n\nThe syntax is identical to the syntax of @csgrammar and @cfgrammar, but only single rules are supported.\n\nwarning: Warning\nCalls to this function are ignored if a rule is already in the grammar.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.addconstraint!-Tuple{ContextSensitiveGrammar, Constraint}","page":"HerbGrammar.jl","title":"HerbGrammar.addconstraint!","text":"addconstraint!(grammar::ContextSensitiveGrammar, c::Constraint)\n\nAdds a Constraint to a ContextSensitiveGrammar.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.cfg2csg-Tuple{ContextFreeGrammar}","page":"HerbGrammar.jl","title":"HerbGrammar.cfg2csg","text":"cfg2csg(g::ContextFreeGrammar)::ContextSensitiveGrammar\n\nConverts a ContextFreeGrammar to a ContextSensitiveGrammar without any Constraints.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.child_types-Tuple{Grammar, Int64}","page":"HerbGrammar.jl","title":"HerbGrammar.child_types","text":"child_types(grammar::Grammar, rule_index::Int)\n\nReturns the types of the children (nonterminals) of the production rule at rule_index.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.child_types-Tuple{Grammar, RuleNode}","page":"HerbGrammar.jl","title":"HerbGrammar.child_types","text":"child_types(grammar::Grammar, node::RuleNode)\n\nReturns the list of child types (nonterminal symbols) in the production rule used by node.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.cleanup_removed_rules!-Tuple{Grammar}","page":"HerbGrammar.jl","title":"HerbGrammar.cleanup_removed_rules!","text":"cleanup_removed_rules!(g::Grammar)\n\nRemoves any placeholders for previously deleted rules. This means that indices get shifted.\n\nwarning: Warning\nWhen indices are shifted, this grammar can no longer be used to interpret AbstractRuleNode trees created before the call to this function. These trees become meaningless. \n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.clearconstraints!-Tuple{ContextSensitiveGrammar}","page":"HerbGrammar.jl","title":"HerbGrammar.clearconstraints!","text":"Clear all constraints from the grammar\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.containedin-Tuple{Vector, Vector}","page":"HerbGrammar.jl","title":"HerbGrammar.containedin","text":"containedin(vec1::Vector, vec2::Vector)\n\nChecks if elements of vec1 are contained in vec2 in the same order (possibly with elements in between)\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.contains_returntype","page":"HerbGrammar.jl","title":"HerbGrammar.contains_returntype","text":"contains_returntype(node::RuleNode, grammar::Grammar, sym::Symbol, maxdepth::Int=typemax(Int))\n\nReturns true if the tree rooted at node contains at least one node at depth less than maxdepth with the given return type or nonterminal symbol.\n\n\n\n\n\n","category":"function"},{"location":"HerbGrammar/#HerbGrammar.expr2cfgrammar-Tuple{Expr}","page":"HerbGrammar.jl","title":"HerbGrammar.expr2cfgrammar","text":"expr2cfgrammar(ex::Expr)::ContextFreeGrammar\n\nA function for converting an Expr to a ContextFreeGrammar. If the expression is hardcoded, you should use the @cfgrammar macro. Only expressions in the correct format (see @cfgrammar) can be converted.\n\nExample usage:\n\ngrammar = expr2cfgrammar(\n\tbegin\n\t\tR = x\n\t\tR = 1 | 2\n\t\tR = R + R\n\tend\n)\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.expr2csgrammar-Tuple{Expr}","page":"HerbGrammar.jl","title":"HerbGrammar.expr2csgrammar","text":"expr2csgrammar(ex::Expr)::ContextSensitiveGrammar\n\nA function for converting an Expr to a ContextSensitiveGrammar. If the expression is hardcoded, you should use the @csgrammar macro. Only expressions in the correct format (see @csgrammar) can be converted.\n\nExample usage:\n\ngrammar = expr2csgrammar(\n\tbegin\n\t\tR = x\n\t\tR = 1 | 2\n\t\tR = R + R\n\tend\n)\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.expr2pcfgrammar-Tuple{Expr}","page":"HerbGrammar.jl","title":"HerbGrammar.expr2pcfgrammar","text":"Function for converting an Expr to a ContextFreeGrammar with probabilities. If the expression is hardcoded, you should use the @pcfgrammar macro. Only expressions in the correct format (see @pcfgrammar) can be converted.\n\nExample usage:\n\ngrammar = expr2pcsgrammar(\n\tbegin\n\t\t0.5 : R = x\n\t\t0.3 : R = 1 | 2\n\t\t0.2 : R = R + R\n\tend\n)\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.expr2pcsgrammar-Tuple{Expr}","page":"HerbGrammar.jl","title":"HerbGrammar.expr2pcsgrammar","text":"Function for converting an Expr to a ContextSensitiveGrammar with probabilities. If the expression is hardcoded, you should use the @pcsgrammar macro. Only expressions in the correct format (see @pcsgrammar) can be converted.\n\nExample usage:\n\ngrammar = expr2pcsgrammar(\n\tbegin\n\t\t0.5 : R = x\n\t\t0.3 : R = 1 | 2\n\t\t0.2 : R = R + R\n\tend\n)\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.get_childtypes-Tuple{Any, AbstractVector{Symbol}}","page":"HerbGrammar.jl","title":"HerbGrammar.get_childtypes","text":"get_childtypes(rule::Any, types::AbstractVector{Symbol})\n\nReturns the child types/nonterminals of a production rule.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.get_domain-Tuple{Grammar, Symbol}","page":"HerbGrammar.jl","title":"HerbGrammar.get_domain","text":"get_domain(g::Grammar, type::Symbol)::BitVector\n\nReturns the domain for the hole of a certain type as a BitVector of the same length as the number of rules in the grammar. Bit i is set to true iff rule i is of type type.\n\ninfo: Info\nSince this function can be intensively used when exploring a program space defined by a grammar, the outcomes of this function are precomputed and stored in the domains field in a Grammar.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.get_domain-Tuple{Grammar, Vector{Int64}}","page":"HerbGrammar.jl","title":"HerbGrammar.get_domain","text":"get_domain(g::Grammar, rules::Vector{Int})::BitVector\n\nTakes a domain rules defined as a vector of ints and converts it to a domain defined as a BitVector.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.get_node_at_location-Tuple{RuleNode, Vector{Int64}}","page":"HerbGrammar.jl","title":"HerbGrammar.get_node_at_location","text":"get_node_at_location(root::RuleNode, location::Vector{Int})\n\nRetrieves a RuleNode at the given location by reference. \n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.get_rulesequence-Tuple{RuleNode, Vector{Int64}}","page":"HerbGrammar.jl","title":"HerbGrammar.get_rulesequence","text":"get_rulesequence(node::RuleNode, path::Vector{Int})\n\nExtract the derivation sequence from a path (sequence of child indices) and an AbstractRuleNode. If the path is deeper than the deepest node, it returns what it has.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.has_children-Tuple{RuleNode}","page":"HerbGrammar.jl","title":"HerbGrammar.has_children","text":"has_children(node::RuleNode)\n\nReturns true if node has children\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.iscomplete-Tuple{Grammar, RuleNode}","page":"HerbGrammar.jl","title":"HerbGrammar.iscomplete","text":"iscomplete(grammar::Grammar, node::RuleNode)\n\nReturns true if the expression represented by the RuleNode is a complete expression, meaning that it is fully defined and doesn't have any Holes.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.iseval-Tuple{Any}","page":"HerbGrammar.jl","title":"HerbGrammar.iseval","text":"iseval(rule)\n\nReturns true if the rule is the special evaluate immediately function, i.e., _()\n\ncompat: Compat\nevaluate immediately functionality is not yet supported by most of Herb.jl\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.iseval-Tuple{Grammar, Int64}","page":"HerbGrammar.jl","title":"HerbGrammar.iseval","text":"iseval(grammar::Grammar, index::Int)::Bool\n\nReturns true if the production rule at rule_index contains the special _() eval function.\n\ncompat: Compat\nevaluate immediately functionality is not yet supported by most of Herb.jl\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.iseval-Tuple{Grammar}","page":"HerbGrammar.jl","title":"HerbGrammar.iseval","text":"iseval(grammar::Grammar)::Bool\n\nReturns true if any production rules in grammar contain the special _() eval function.\n\ncompat: Compat\nevaluate immediately functionality is not yet supported by most of Herb.jl\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.isprobabilistic-Tuple{Grammar}","page":"HerbGrammar.jl","title":"HerbGrammar.isprobabilistic","text":"isprobabilistic(grammar::Grammar)::Bool\n\nFunction returns whether a Grammar is probabilistic.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.isterminal-Tuple{Any, AbstractVector{Symbol}}","page":"HerbGrammar.jl","title":"HerbGrammar.isterminal","text":"isterminal(rule::Any, types::AbstractVector{Symbol})\n\nReturns true if the rule is terminal, i.e., it does not contain any of the types in the provided vector. For example, :(x) is terminal, and :(1+1) is terminal, but :(Real + Real) is typically not.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.isterminal-Tuple{Grammar, Int64}","page":"HerbGrammar.jl","title":"HerbGrammar.isterminal","text":"isterminal(grammar::Grammar, rule_index::Int)::Bool\n\nReturns true if the production rule at rule_index is terminal, i.e., does not contain any nonterminal symbols.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.isterminal-Tuple{Grammar, RuleNode}","page":"HerbGrammar.jl","title":"HerbGrammar.isterminal","text":"isterminal(grammar::Grammar, node::RuleNode)::Bool\n\nReturns true if the production rule used by node is terminal, i.e., does not contain any nonterminal symbols.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.isvariable-Tuple{Grammar, RuleNode}","page":"HerbGrammar.jl","title":"HerbGrammar.isvariable","text":"isvariable(grammar::Grammar, node::RuleNode)::Bool\n\nReturns true if the rule used by node represents a variable.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.log_probability-Tuple{Grammar, Int64}","page":"HerbGrammar.jl","title":"HerbGrammar.log_probability","text":"log_probability(grammar::Grammar, index::Int)::Real\n\nReturns the log probability for the rule at index in the grammar.\n\nwarning: Warning\nIf the grammar is not probabilistic, a warning is displayed, and a uniform probability is assumed.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.max_arity-Tuple{Grammar}","page":"HerbGrammar.jl","title":"HerbGrammar.max_arity","text":"max_arity(grammar::Grammar)::Int\n\nReturns the maximum arity (number of children) over all production rules in the Grammar.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.mindepth-Tuple{Grammar, Symbol, AbstractVector{Int64}}","page":"HerbGrammar.jl","title":"HerbGrammar.mindepth","text":"mindepth(grammar::Grammar, typ::Symbol, dmap::AbstractVector{Int})\n\nReturns the minimum depth achievable for a given nonterminal symbol. The minimum depth is the depth of the lowest tree that can be made using typ as a start symbol. dmap can be obtained from mindepth_map.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.mindepth_map-Tuple{Grammar}","page":"HerbGrammar.jl","title":"HerbGrammar.mindepth_map","text":"mindepth_map(grammar::Grammar)\n\nReturns the minimum depth achievable for each production rule in the Grammar. In other words, this function finds the depths of the lowest trees that can be made using each of the available production rules as a root.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.nchildren-Tuple{Grammar, Int64}","page":"HerbGrammar.jl","title":"HerbGrammar.nchildren","text":"nchildren(grammar::Grammar, rule_index::Int)::Int\n\nReturns the number of children (nonterminals) of the production rule at rule_index.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.nchildren-Tuple{Grammar, RuleNode}","page":"HerbGrammar.jl","title":"HerbGrammar.nchildren","text":"nchildren(grammar::Grammar, node::RuleNode)::Int\n\nReturns the number of children in the production rule used by node.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.nonterminals-Tuple{Grammar}","page":"HerbGrammar.jl","title":"HerbGrammar.nonterminals","text":"nonterminals(grammar::Grammar)::Vector{Symbol}\n\nReturns a list of the nonterminals or types in the Grammar.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.probability-Tuple{Grammar, Int64}","page":"HerbGrammar.jl","title":"HerbGrammar.probability","text":"probability(grammar::Grammar, index::Int)::Real\n\nReturn the probability for a rule in the grammar. Use log_probability whenever possible.\n\nwarning: Warning\nIf the grammar is not probabilistic, a warning is displayed, and a uniform probability is assumed.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.read_cfg-Tuple{AbstractString}","page":"HerbGrammar.jl","title":"HerbGrammar.read_cfg","text":"read_cfg(filepath::AbstractString)::ContextFreeGrammar\n\nReads a ContextFreeGrammar from the file provided in filepath.\n\ndanger: Danger\nOnly open trusted grammars. Parts of the grammar can be passed to Julia's eval function. \n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.read_csg-Tuple{AbstractString, AbstractString}","page":"HerbGrammar.jl","title":"HerbGrammar.read_csg","text":"read_csg(grammarpath::AbstractString, constraintspath::AbstractString)::ContextSensitiveGrammar\n\nReads a ContextSensitiveGrammar from the files at grammarpath and constraintspath. The grammar path may also point to a ContextFreeGrammar.\n\ndanger: Danger\nOnly open trusted grammars. Parts of the grammar can be passed to Julia's eval function. \n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.read_pcfg-Tuple{AbstractString}","page":"HerbGrammar.jl","title":"HerbGrammar.read_pcfg","text":"read_pcfg(filepath::AbstractString)::ContextFreeGrammar\n\nReads a probabilistic ContextFreeGrammar from a file provided in filepath.\n\ndanger: Danger\nOnly open trusted grammars. Parts of the grammar can be passed to Julia's eval function. \n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.read_pcsg-Tuple{AbstractString, AbstractString}","page":"HerbGrammar.jl","title":"HerbGrammar.read_pcsg","text":"read_pcsg(grammarpath::AbstractString, constraintspath::AbstractString)::ContextSensitiveGrammar\n\nReads a probabilistic ContextSensitiveGrammar from the files at grammarpath and constraintspath. The grammar path may also point to a ContextFreeGrammar.\n\ndanger: Danger\nOnly open trusted grammars. Parts of the grammar can be passed to Julia's eval function. \n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.remove_rule!-Tuple{Grammar, Int64}","page":"HerbGrammar.jl","title":"HerbGrammar.remove_rule!","text":"remove_rule!(g::Grammar, idx::Int)\n\nRemoves the rule corresponding to idx from the grammar. In order to avoid shifting indices, the rule is replaced with nothing, and all other data structures are updated accordingly.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.return_type-Tuple{Grammar, Int64}","page":"HerbGrammar.jl","title":"HerbGrammar.return_type","text":"return_type(grammar::Grammar, rule_index::Int)::Symbol\n\nReturns the type of the production rule at rule_index.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.return_type-Tuple{Grammar, RuleNode}","page":"HerbGrammar.jl","title":"HerbGrammar.return_type","text":"return_type(grammar::Grammar, node::RuleNode)\n\nGives the return type or nonterminal symbol in the production rule used by node.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.root_node_loc-Tuple{RuleNode}","page":"HerbGrammar.jl","title":"HerbGrammar.root_node_loc","text":"rootnodeloc(root::RuleNode) Returns a NodeLoc pointing to the root node.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.rulenode2expr-Tuple{RuleNode, Grammar}","page":"HerbGrammar.jl","title":"HerbGrammar.rulenode2expr","text":"rulenode2expr(rulenode::RuleNode, grammar::Grammar)\n\nConverts a RuleNode into a Julia expression corresponding to the rule definitions in the grammar. The returned expression can be evaluated with Julia semantics using eval().\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.rulenode_log_probability-Tuple{RuleNode, Grammar}","page":"HerbGrammar.jl","title":"HerbGrammar.rulenode_log_probability","text":"Calculates the log probability associated with a rulenode in a probabilistic grammar.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.rulesoftype-Tuple{RuleNode, Grammar, Symbol, RuleNode}","page":"HerbGrammar.jl","title":"HerbGrammar.rulesoftype","text":"rulesoftype(node::RuleNode, grammar::Grammar, ruletype::Symbol, ignoreNode::RuleNode)\n\nReturns every rule of nonterminal symbol ruletype that is also used in the AbstractRuleNode tree, but not in the ignoreNode subtree.\n\nwarning: Warning\nThe ignoreNode must be a subtree of node for it to have an effect.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.rulesoftype-Tuple{RuleNode, Grammar, Symbol}","page":"HerbGrammar.jl","title":"HerbGrammar.rulesoftype","text":"rulesoftype(node::RuleNode, grammar::Grammar, ruletype::Symbol)\n\nReturns every rule of nonterminal symbol ruletype that is also used in the AbstractRuleNode tree.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.rulesoftype-Tuple{RuleNode, Set{Int64}, RuleNode}","page":"HerbGrammar.jl","title":"HerbGrammar.rulesoftype","text":"rulesoftype(node::RuleNode, ruleset::Set{Int}, ignoreNode::RuleNode)\n\nReturns every rule in the ruleset that is also used in the AbstractRuleNode tree, but not in the ignoreNode subtree.\n\nwarning: Warning\nThe ignoreNode must be a subtree of node for it to have an effect.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.rulesonleft-Tuple{RuleNode, Vector{Int64}}","page":"HerbGrammar.jl","title":"HerbGrammar.rulesonleft","text":"rulesonleft(expr::RuleNode, path::Vector{Int})::Set{Int}\n\nFinds all rules that are used in the left subtree defined by the path.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.store_cfg-Tuple{AbstractString, ContextFreeGrammar}","page":"HerbGrammar.jl","title":"HerbGrammar.store_cfg","text":"store_cfg(filepath::AbstractString, grammar::ContextFreeGrammar)\n\nWrites a ContextFreeGrammar to the file provided by filepath.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.store_csg-Tuple{AbstractString, AbstractString, ContextSensitiveGrammar}","page":"HerbGrammar.jl","title":"HerbGrammar.store_csg","text":"store_csg(grammarpath::AbstractString, constraintspath::AbstractString, g::ContextSensitiveGrammar)\n\nWrites a ContextSensitiveGrammar to the files at grammarpath and constraintspath. The grammarpath file will contain a ContextSensitiveGrammar definition, and the constraintspath file will contain the Constraints of the ContextSensitiveGrammar.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.subsequenceof-Tuple{Vector{Int64}, Vector{Int64}}","page":"HerbGrammar.jl","title":"HerbGrammar.subsequenceof","text":"subsequenceof(vec1::Vector{Int}, vec2::Vector{Int})\n\nChecks if vec1 is a subsequence of vec2.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.swap_node-Tuple{AbstractRuleNode, AbstractRuleNode, Vector{Int64}}","page":"HerbGrammar.jl","title":"HerbGrammar.swap_node","text":"swap_node(expr::AbstractRuleNode, new_expr::AbstractRuleNode, path::Vector{Int})\n\nReplace a node in expr, specified by path, with new_expr. Path is a sequence of child indices, starting from the root node.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#HerbGrammar.swap_node-Tuple{RuleNode, RuleNode, Int64, RuleNode}","page":"HerbGrammar.jl","title":"HerbGrammar.swap_node","text":"swap_node(expr::RuleNode, node::RuleNode, child_index::Int, new_expr::RuleNode)\n\nReplace child i of a node, a part of larger expr, with new_expr.\n\n\n\n\n\n","category":"method"},{"location":"HerbGrammar/#Index","page":"HerbGrammar.jl","title":"Index","text":"","category":"section"},{"location":"HerbGrammar/","page":"HerbGrammar.jl","title":"HerbGrammar.jl","text":"","category":"page"},{"location":"HerbConstraints/#HerbConstraints_docs","page":"HerbConstraints.jl","title":"HerbConstraints.jl Documentation","text":"","category":"section"},{"location":"HerbConstraints/","page":"HerbConstraints.jl","title":"HerbConstraints.jl","text":"CurrentModule=HerbConstraints","category":"page"},{"location":"HerbConstraints/","page":"HerbConstraints.jl","title":"HerbConstraints.jl","text":"Modules = [HerbConstraints]\nOrder = [:type, :const, :macro, :function]","category":"page"},{"location":"HerbConstraints/#HerbConstraints.AbstractMatchNode","page":"HerbConstraints.jl","title":"HerbConstraints.AbstractMatchNode","text":"abstract type AbstractMatchNode\n\nTree structure to which rulenode trees can be matched. Consists of MatchNodes, which can match a specific RuleNode, and MatchVars, which is a variable that can be filled in with any RuleNode.\n\n\n\n\n\n","category":"type"},{"location":"HerbConstraints/#HerbConstraints.ComesAfter","page":"HerbConstraints.jl","title":"HerbConstraints.ComesAfter","text":"ComesAfter <: PropagatorConstraint\n\nA ComesAfter constraint is a PropagatorConstraint containing the following:\n\nrule::Int: A reference to a rule in the grammar\npredecessors: A list of rules in the grammar\n\nThis Constraint enforces that the rule can only be applied if every rule in predecessors is used in the path from the root of the tree to the current hole in the order that they are given. Even though the rules must be in order, there might be other rules inbetween.\n\nFor example, consider the tree 1(a, 2(b, 3(c, d)))):\n\nComesAfter(4, [2, 3]) would enforce that rule 4 can only be used if 2 and 3 are used in the path from the root in that order. Therefore, only hole c and d can be filled with 4.\nComesAfter(4, [1, 3]) also allows c and d to be filled, since 1 and 3 are still used in the correct order. It does not matter that 2 is also used in the path to the root.\nComesAfter(4, [3, 2]) does not allow any hole to be filled with 4, since either the predecessors are either not in the path or in the wrong order for each of the holes. \n\n\n\n\n\n","category":"type"},{"location":"HerbConstraints/#HerbConstraints.ComesAfter-Tuple{Int64, Int64}","page":"HerbConstraints.jl","title":"HerbConstraints.ComesAfter","text":"ComesAfter(rule::Int, predecessor::Int)\n\nCreates a ComesAfter constraint with only a single predecessor. \n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints.Forbidden","page":"HerbConstraints.jl","title":"HerbConstraints.Forbidden","text":"Forbidden <: PropagatorConstraint\n\nThis [PropagatorConstraint] forbids any subtree that matches the pattern given by tree to be generated. A pattern is a tree of AbstractMatchNodes. Such a node can either be a MatchNode, which contains a rule index corresponding to the rule index in the Grammar and the appropriate number of children, similar to RuleNodes. It can also contain a MatchVar, which contains a single identifier symbol. A MatchVar can match any subtree, but if there are multiple instances of the same variable in the pattern, the matched subtrees must be identical. Any rule in the domain that makes the match attempt successful is removed.\n\nFor example, consider the tree 1(a, 2(b, 3(c, 4)))):\n\nForbidden(MatchNode(3, [MatchNode(5), MatchNode(4)])) forbids c to be filled with 5.\nForbidden(MatchNode(3, [MatchVar(:v), MatchNode(4)])) forbids c to be filled, since a [MatchVar] can match any rule, thus making the match attempt successful for the entire domain of c. Therefore, this tree invalid.\nForbidden(MatchNode(3, [MatchVar(:v), MatchVar(:v)])) forbids c to be filled with 4, since that would make both assignments to v equal, which causes a successful match.\n\nwarning: Warning\nThe Forbidden constraint makes use of LocalConstraints to make sure that constraints are also enforced in the future when the context of a Hole changes. Therefore, Forbidden can only be used in implementations that keep track of the LocalConstraints and propagate them at the right moments.\n\n\n\n\n\n","category":"type"},{"location":"HerbConstraints/#HerbConstraints.ForbiddenPath","page":"HerbConstraints.jl","title":"HerbConstraints.ForbiddenPath","text":"ForbiddenPath <: PropagatorConstraint\n\nA [PropagatorConstraint] that forbids a certain derivation sequence. sequence defines the forbidden sequence. Each rule that would complete the sequence when expanding a Hole in an AbstractRuleNode tree is removed from the domain. The derivation sequence is the path from the root to the hole.\n\nFor example, consider the tree 1(a, 2(b, 3(c, d)))):\n\nForbiddenPath([1, 2, 4]) enforces that rule 4 cannot be applied at b, since it completes the sequence. However, it can be applied at a, c and d.\nForbiddenPath([3, 1]) enforces that rule 1 cannot be applied at c or d.\n\n\n\n\n\n","category":"type"},{"location":"HerbConstraints/#HerbConstraints.GrammarContext","page":"HerbConstraints.jl","title":"HerbConstraints.GrammarContext","text":"mutable struct GrammarContext\n\nStructure used to track the context. Contains: \t- the expression being modified \t- the path to the hole that is being expanded, represented as a sequence of child indices. \t e.g., [2, 1] would point to the first child of the second child of the root. \t- a vector with local constraints that should be propagated upon expansion.\n\n\n\n\n\n","category":"type"},{"location":"HerbConstraints/#HerbConstraints.LocalConstraint","page":"HerbConstraints.jl","title":"HerbConstraints.LocalConstraint","text":"abstract type LocalConstraint <: Constraint\n\nAbstract type representing all local constraints. Local constraints correspond to a specific (partial) AbstractRuleNode tree. Each local constraint contains a path to a specific location in the tree. Each local constraint has an implementation of a propagate-function that takes\n\nthe LocalConstraint\na Grammar\na GrammarContext, which most importantly contains the tree and the location in the tree where propagation should take place.\nThe domain which the propagate-function prunes. \n\nThe propagate-function returns a tuple containing\n\nThe pruned domain\nA list of new LocalConstraints\n\nwarning: Warning\nBy default, LocalConstraints are only propagated once. Constraints that have to be propagated more frequently should return themselves in the list of new local constraints.\n\n\n\n\n\n","category":"type"},{"location":"HerbConstraints/#HerbConstraints.LocalForbidden","page":"HerbConstraints.jl","title":"HerbConstraints.LocalForbidden","text":"LocalForbidden\n\nForbids the a subtree that matches the MatchNode tree to be generated at the location provided by the path. Use a Forbidden constraint for enforcing this throughout the entire search space.\n\n\n\n\n\n","category":"type"},{"location":"HerbConstraints/#HerbConstraints.LocalOneOf","page":"HerbConstraints.jl","title":"HerbConstraints.LocalOneOf","text":"Meta-constraint that enforces the disjunction of its given constraints.\n\n\n\n\n\n","category":"type"},{"location":"HerbConstraints/#HerbConstraints.LocalOrdered","page":"HerbConstraints.jl","title":"HerbConstraints.LocalOrdered","text":"Enforces an order over two or more subtrees that fill the variables specified in order when the pattern is applied at the location given by path. Use an Ordered constraint for enforcing this throughout the entire search space.\n\n\n\n\n\n","category":"type"},{"location":"HerbConstraints/#HerbConstraints.MatchFail","page":"HerbConstraints.jl","title":"HerbConstraints.MatchFail","text":"@enum MatchFail hardfail softfail\n\nThis enum is used for distinguishing between two types of failures when trying to match a RuleNode either with another RuleNode or with an AbstractMatchNode\n\nHardfail means that there is no match, and there is no way to fill in the holes to get a match.\nSoftfail means that there is no match, but there might be a way to fill the holes that results in a match.\n\n\n\n\n\n","category":"type"},{"location":"HerbConstraints/#HerbConstraints.MatchNode","page":"HerbConstraints.jl","title":"HerbConstraints.MatchNode","text":"struct MatchNode <: AbstractMatchNode\n\nMatch a specific rulenode, where the grammar rule index is rule_ind and children matches the children of the RuleNode. Example usage:\n\nMatchNode(3, [MatchNode(1), MatchNode(2)])\n\nThis matches RuleNode(3, [RuleNode(1), RuleNode(2)])\n\n\n\n\n\n","category":"type"},{"location":"HerbConstraints/#HerbConstraints.MatchVar","page":"HerbConstraints.jl","title":"HerbConstraints.MatchVar","text":"struct MatchVar <: AbstractMatchNode\n\nMatches anything and assigns it to a variable. The ForbiddenTree constraint will not match if identical variable symbols match to different trees. Example usage:\n\nMatchNode(3, [MatchVar(:x), MatchVar(:x)])\n\nThis matches RuleNode(3, [RuleNode(1), RuleNode(1)]), RuleNode(3, [RuleNode(2), RuleNode(2)]), etc.\n\n\n\n\n\n","category":"type"},{"location":"HerbConstraints/#HerbConstraints.OneOf","page":"HerbConstraints.jl","title":"HerbConstraints.OneOf","text":"Meta-constraint that enforces the disjunction of its given constraints.\n\n\n\n\n\n","category":"type"},{"location":"HerbConstraints/#HerbConstraints.Ordered","page":"HerbConstraints.jl","title":"HerbConstraints.Ordered","text":"Ordered <: PropagatorConstraint\n\nA PropagatorConstraint that enforces a specific order in MatchVar assignments in the pattern defined by tree. A pattern is a tree of AbstractMatchNodes. Such a node can either be a MatchNode, which contains a rule index corresponding to the rule index in the Grammar and the appropriate number of children, similar to RuleNodes. It can also contain a MatchVar, which contains a single identifier symbol. A MatchVar can match any subtree, but if there are multiple instances of the same variable in the pattern, the matched subtrees must be identical.\n\nThe order defines an order between the variable assignments. For example, if the order is [x, y], the constraint will require the assignment to x to be less than or equal to the assignment to y. The order is recursively defined by RuleNode indices. For more information, see Base.isless(rn₁::AbstractRuleNode, rn₂::AbstractRuleNode).\n\nFor example, consider the tree 1(a, 2(b, 3(c, 4)))):\n\nOrdered(MatchNode(3, [MatchVar(:v), MatchVar(:w)]), [:v, :w]) removes every rule with an index of 5 or greater from the domain of c, since that would make the index of the assignment to v greater than the index of the assignment to w, violating the order.\nOrdered(MatchNode(3, [MatchVar(:v), MatchVar(:w)]), [:w, :v]) removes every rule with an index of 4 or less from the domain of c, since that would make the index of the assignment to v less than the index of the assignment to w, violating the order.\n\nwarning: Warning\nThe Ordered constraint makes use of LocalConstraints to make sure that constraints are also enforced in the future when the context of a Hole changes. Therefore, Ordered can only be used in implementations that keep track of the LocalConstraints and propagate them at the right moments.\n\n\n\n\n\n","category":"type"},{"location":"HerbConstraints/#HerbConstraints.PropagatorConstraint","page":"HerbConstraints.jl","title":"HerbConstraints.PropagatorConstraint","text":"PropagatorConstraint <: Constraint\n\nAbstract type representing all propagator constraints. Each propagator constraint has an implementation of a propagate-function that takes\n\nthe PropagatorConstraint\na Grammar\na GrammarContext, which most importantly contains the tree and the location in the tree where propagation should take place.\nThe domain which the propagate-function prunes. \n\nThe propagate-function returns a tuple containing\n\nThe pruned domain\nA list of new LocalConstraints\n\n\n\n\n\n","category":"type"},{"location":"HerbConstraints/#HerbConstraints.RequireOnLeft","page":"HerbConstraints.jl","title":"HerbConstraints.RequireOnLeft","text":"Rules have to be used in the specified order. That is, rule at index K can only be used if rules at indices [1...K-1] are used in the left subtree of the current expression\n\n\n\n\n\n","category":"type"},{"location":"HerbConstraints/#HerbConstraints.@csgrammar_annotated-Tuple{Any}","page":"HerbConstraints.jl","title":"HerbConstraints.@csgrammar_annotated","text":"@csgrammar_annotated Define an annotated grammar and return it as a ContextSensitiveGrammar. Allows for adding optional annotations per rule. As well as that, allows for adding optional labels per rule, which can be referenced in annotations. Syntax is backwards-compatible with @csgrammar. Examples:\n\ng₁ = @csgrammar_annotated begin\n Element = 1\n Element = x\n Element = Element + Element := commutative\n Element = Element * Element := (commutative, transitive)\nend\n\ng₁ = @csgrammar_annotated begin\n Element = 1\n Element = x\n Element = Element + Element := forbidden_path([3, 1])\n Element = Element * Element := (commutative, transitive)\nend\n\ng₁ = @csgrammar_annotated begin\n one:: Element = 1\n variable:: Element = x\n addition:: Element = Element + Element := (\n commutative,\n transitive,\n forbidden_path([:addition, :one]) || forbidden_path([:one, :variable])\n )\n multiplication:: Element = Element * Element := (commutative, transitive)\nend\n\n\n\n\n\n","category":"macro"},{"location":"HerbConstraints/#Base.show-Tuple{IO, MatchNode}","page":"HerbConstraints.jl","title":"Base.show","text":"Base.show(io::IO, node::MatchNode; separator=\",\", last_child::Bool=true)\n\nPrints a found MatchNode given an and the respective children to IO. \n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#Base.show-Tuple{IO, MatchVar}","page":"HerbConstraints.jl","title":"Base.show","text":"Base.show(io::IO, node::MatchVar; separator=\",\", last_child::Bool=true)\n\nPrints a matching variable assignment described by MatchVar to IO.\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints._matchnode2expr","page":"HerbConstraints.jl","title":"HerbConstraints._matchnode2expr","text":"_matchnode2expr(expr::Expr, pattern::MatchNode, grammar::Grammar, j=0)\n\nInternal function for matchnode2expr, recursively iterating over a matched pattern and converting it to an expression. This is primarily useful for pretty-printing a pattern. Returns the corresponding expression and the current child index.\n\n\n\n\n\n","category":"function"},{"location":"HerbConstraints/#HerbConstraints._matchnode2expr-2","page":"HerbConstraints.jl","title":"HerbConstraints._matchnode2expr","text":"_matchnode2expr(typ::Symbol, pattern::MatchNode, grammar::Grammar, j=0)\n\nInternal function for matchnode2expr, returning the matched translated symbol. This is primarily useful for pretty-printing a pattern. Returns the corresponding expression, i.e. the variable name and the current child index.\n\n\n\n\n\n","category":"function"},{"location":"HerbConstraints/#HerbConstraints._matchnode2expr-3","page":"HerbConstraints.jl","title":"HerbConstraints._matchnode2expr","text":"_matchnode2expr(expr::Expr, pattern::MatchVar, grammar::Grammar, j=0)\n\nInternal function for matchnode2expr, recursively iterating over a matched variable and converting it to an expression. This is primarily useful for pretty-printing a pattern. Returns the corresponding expression and the current child index.\n\n\n\n\n\n","category":"function"},{"location":"HerbConstraints/#HerbConstraints._matchnode2expr-4","page":"HerbConstraints.jl","title":"HerbConstraints._matchnode2expr","text":"_matchnode2expr(typ::Symbol, pattern::MatchVar, grammar::Grammar, j=0)\n\nInternal function for matchnode2expr. This is primarily useful for pretty-printing a pattern. Returns the corresponding expression, i.e. the variable name and the current child index.\n\n\n\n\n\n","category":"function"},{"location":"HerbConstraints/#HerbConstraints._pattern_match-Tuple{RuleNode, MatchNode, Dict{Symbol, AbstractRuleNode}}","page":"HerbConstraints.jl","title":"HerbConstraints._pattern_match","text":"_pattern_match(rn::RuleNode, mn::MatchNode, vars::Dict{Symbol, AbstractRuleNode})::Union{Nothing, MatchFail}\n\nTries to match RuleNode rn with MatchNode mn. Modifies the variable assignment dictionary vars. Returns nothing if the match is successful. If the match is unsuccessful, it returns whether it is a softfail or hardfail (see MatchFail docstring)\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints._pattern_match-Tuple{RuleNode, MatchVar, Dict{Symbol, AbstractRuleNode}}","page":"HerbConstraints.jl","title":"HerbConstraints._pattern_match","text":"_pattern_match(rn::RuleNode, mv::MatchVar, vars::Dict{Symbol, AbstractRuleNode})::Union{Nothing, MatchFail}\n\nMatching RuleNode rn with MatchVar mv. If the variable is already assigned, the rulenode is matched with the specific variable value. Returns nothing if the match is succesful. If the match is unsuccessful, it returns whether it is a softfail or hardfail (see MatchFail docstring)\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints._pattern_match_with_hole-Tuple{Hole, MatchNode, Vector{Int64}, Dict{Symbol, AbstractRuleNode}}","page":"HerbConstraints.jl","title":"HerbConstraints._pattern_match_with_hole","text":"_pattern_match_with_hole(::Hole, mn::MatchNode, hole_location::Vector{Int}, ::Dict{Symbol, AbstractRuleNode})::Union{Int, Symbol, MatchFail, Tuple{Symbol, Vector{Int}}}\n\nMatches the Hole with the given MatchNode. \n\nTODO check this behaviour?\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints._pattern_match_with_hole-Tuple{Hole, MatchVar, Vector{Int64}, Dict{Symbol, AbstractRuleNode}}","page":"HerbConstraints.jl","title":"HerbConstraints._pattern_match_with_hole","text":"_pattern_match_with_hole(::Hole, mn::MatchNode, hole_location::Vector{Int}, ::Dict{Symbol, AbstractRuleNode})::Union{Int, Symbol, MatchFail, Tuple{Symbol, Vector{Int}}}\n\nMatches the Hole with the given MatchVar, similar to _pattern_match_with_hole.\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints._pattern_match_with_hole-Tuple{RuleNode, MatchNode, Vector{Int64}, Dict{Symbol, AbstractRuleNode}}","page":"HerbConstraints.jl","title":"HerbConstraints._pattern_match_with_hole","text":"_pattern_match_with_hole(rn::RuleNode, mn::MatchNode, hole_location::Vector{Int}, vars::Dict{Symbol, AbstractRuleNode})::Union{Int, Symbol, MatchFail, Tuple{Symbol, Vector{Int}}}\n\nTries to match RuleNode rn with MatchNode mn and fill in the domain of the hole at hole_location. Returns if match is successful either:\n\nThe id for the node which fills the hole\nA symbol for the variable that fills the hole\nA tuple containing:\nThe variable that matched (the subtree containing) the hole\nThe location of the hole in this subtree\n\nIf the match is unsuccessful, it returns:\n\nhardfail if there are no holes that can be filled in such a way that the match will become succesful\nsoftfail if the match could become successful if the holes are filled in a certain way\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints._pattern_match_with_hole-Tuple{RuleNode, MatchVar, Vector{Int64}, Dict{Symbol, AbstractRuleNode}}","page":"HerbConstraints.jl","title":"HerbConstraints._pattern_match_with_hole","text":"_pattern_match_with_hole(rn::RuleNode, mv::MatchVar, hole_location::Vector{Int}, vars::Dict{Symbol, AbstractRuleNode})::Union{Int, Symbol, MatchFail, Tuple{Symbol, Vector{Int}}}\n\nTries to match RuleNode rn with MatchVar mv and fill in the domain of the hole at hole_location. If the variable name is already assigned in vars, the rulenode is matched with the hole. Otherwise the variable and the hole location are returned.\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints._rulenode_compare-Tuple{RuleNode, RuleNode}","page":"HerbConstraints.jl","title":"HerbConstraints._rulenode_compare","text":"Returns -1 if rn₁ < rn₂\nReturns 0 if rn₁ == rn₂ \nReturns 1 if rn₁ > rn₂\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints._rulenode_match_with_hole-Tuple{RuleNode, RuleNode, Vector{Int64}}","page":"HerbConstraints.jl","title":"HerbConstraints._rulenode_match_with_hole","text":"_rulenode_match_with_hole(rn₁::RuleNode, rn₂::RuleNode, hole_location::Vector{Int})::Union{Int, MatchFail}\n\nMatches two rulenodes. Returns how to fill in the hole in rn₁ to make it match rn₂ if:\n\nrn₁ has a single hole at the provided location\nrn₂ doesn't have any holes\nrn₁ matches rn₂ apart from the single hole location.\n\nIf the match fails, it returns whether it is a softfail or a hardfail (see MatchFail docstring)\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints.addparent!-Tuple{GrammarContext, Int64}","page":"HerbConstraints.jl","title":"HerbConstraints.addparent!","text":"addparent!(context::GrammarContext, parent::Int)\n\nAdds a parent to the context. The parent is defined by the grammar rule id.\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints.annotation2constraint-Tuple{Any, Int64, Vector{String}}","page":"HerbConstraints.jl","title":"HerbConstraints.annotation2constraint","text":"Converts an annotation to a constraint. commutative: creates an Ordered constraint transitive: creates an (incorrect) Forbidden constraint forbidden_path(path::Vector{Union{Symbol, Int}}): creates a ForbiddenPath constraint with the original rule included ... || ...: creates a OneOf constraint (also works with ... || ... || ... et cetera, though not very performant)\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints.check_tree-Tuple{ComesAfter, Grammar, AbstractRuleNode}","page":"HerbConstraints.jl","title":"HerbConstraints.check_tree","text":"Checks if the given tree abides the constraint.\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints.check_tree-Tuple{Forbidden, Grammar, RuleNode}","page":"HerbConstraints.jl","title":"HerbConstraints.check_tree","text":"check_tree(c::Forbidden, g::Grammar, tree::RuleNode)::Bool\n\nChecks if the given AbstractRuleNode tree abides the Forbidden constraint.\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints.check_tree-Tuple{ForbiddenPath, Grammar, AbstractRuleNode}","page":"HerbConstraints.jl","title":"HerbConstraints.check_tree","text":"Checks if the given tree abides the constraint.\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints.check_tree-Tuple{HerbConstraints.Condition, Grammar, RuleNode}","page":"HerbConstraints.jl","title":"HerbConstraints.check_tree","text":"Checks if the given tree abides the constraint.\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints.check_tree-Tuple{OneOf, Grammar, AbstractRuleNode}","page":"HerbConstraints.jl","title":"HerbConstraints.check_tree","text":"Checks if the given tree abides the constraint.\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints.check_tree-Tuple{Ordered, Grammar, RuleNode}","page":"HerbConstraints.jl","title":"HerbConstraints.check_tree","text":"check_tree(c::Ordered, g::Grammar, tree::RuleNode)::Bool\n\nChecks if the given AbstractRuleNode tree abides the Ordered constraint.\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints.check_tree-Tuple{RequireOnLeft, Grammar, AbstractRuleNode}","page":"HerbConstraints.jl","title":"HerbConstraints.check_tree","text":"Checks if the given tree abides the constraint.\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints.copy_and_insert-Tuple{GrammarContext, Int64}","page":"HerbConstraints.jl","title":"HerbConstraints.copy_and_insert","text":"copy_and_insert(old_context::GrammarContext, parent::Int)\n\nCopies the given context and insert the parent in the node location.\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints.make_smaller_or_equal-Tuple{RuleNode, RuleNode, Vector{Int64}, Vector{Int64}}","page":"HerbConstraints.jl","title":"HerbConstraints.make_smaller_or_equal","text":"Filters the domain of the hole at hole_location in rn₁ to make rn₁ be ordered before rn₂. Returns the filtered domain, and a boolean indicating if this constraint can be deleted.\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints.matchnode2expr-Tuple{MatchNode, Grammar}","page":"HerbConstraints.jl","title":"HerbConstraints.matchnode2expr","text":"matchnode2expr(pattern::MatchNode, grammar::Grammar)\n\nConverts a MatchNode tree into a Julia expression. This is primarily useful for pretty-printing a pattern. Returns the corresponding expression.\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints.matchnode2expr-Tuple{MatchVar, Grammar}","page":"HerbConstraints.jl","title":"HerbConstraints.matchnode2expr","text":"matchnode2expr(pattern::MatchVar, grammar::Grammar)\n\nConverts a MatchVar into an expression by returning the variable directly. This is primarily useful for pretty-printing a pattern.\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints.propagate-Tuple{Forbidden, Grammar, GrammarContext, Vector{Int64}, Union{Nothing, HoleReference}}","page":"HerbConstraints.jl","title":"HerbConstraints.propagate","text":"propagate(c::Forbidden, g::Grammar, context::GrammarContext, domain::Vector{Int})::Tuple{Vector{Int}, Vector{LocalConstraint}}\n\nPropagates the Forbidden constraint. It removes the rules from the domain that would complete the forbidden tree.\n\nwarning: Warning\nThe Forbidden constraint makes use of LocalConstraints to make sure that constraints are also enforced in the future when the context of a Hole changes. Therefore, Forbidden can only be used in implementations that keep track of the LocalConstraints and propagate them at the right moments.\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints.propagate-Tuple{ForbiddenPath, Grammar, GrammarContext, Vector{Int64}, Union{Nothing, HoleReference}}","page":"HerbConstraints.jl","title":"HerbConstraints.propagate","text":"Propagates the ForbiddenPath constraint. It removes the elements from the domain that would complete the forbidden sequence.\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints.propagate-Tuple{HerbConstraints.LocalOneOf, Grammar, GrammarContext, Vector{Int64}, Union{Nothing, HoleReference}}","page":"HerbConstraints.jl","title":"HerbConstraints.propagate","text":"Propagates the LocalOneOf constraint. It enforces that at least one of its given constraints hold.\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints.propagate-Tuple{LocalForbidden, Grammar, GrammarContext, Vector{Int64}, Union{Nothing, HoleReference}}","page":"HerbConstraints.jl","title":"HerbConstraints.propagate","text":"Propagates the LocalForbidden constraint. It removes rules from the domain that would make the RuleNode at the given path match the pattern defined by the MatchNode.\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints.propagate-Tuple{LocalOrdered, Grammar, GrammarContext, Vector{Int64}, Union{Nothing, HoleReference}}","page":"HerbConstraints.jl","title":"HerbConstraints.propagate","text":"Propagates the LocalOrdered constraint. It removes rules from the domain that would violate the order of variables as defined in the constraint.\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints.propagate-Tuple{OneOf, Grammar, GrammarContext, Vector{Int64}, Union{Nothing, HoleReference}}","page":"HerbConstraints.jl","title":"HerbConstraints.propagate","text":"Propagates the OneOf constraint. It enforces that at least one of its given constraints hold.\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints.propagate-Tuple{Ordered, Grammar, GrammarContext, Vector{Int64}, Union{Nothing, HoleReference}}","page":"HerbConstraints.jl","title":"HerbConstraints.propagate","text":"propagate(c::Ordered, g::Grammar, context::GrammarContext, domain::Vector{Int})::Tuple{Vector{Int}, Vector{LocalConstraint}}\n\nPropagates the Ordered constraint. Any rule that violates the order as defined by the contraint is removed from the domain.\n\nwarning: Warning\nThe Ordered constraint makes use of LocalConstraints to make sure that constraints are also enforced in the future when the context of a Hole changes. Therefore, Ordered can only be used in implementations that keep track of the LocalConstraints and propagate them at the right moments.\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#HerbConstraints.propagate-Tuple{RequireOnLeft, Grammar, GrammarContext, Vector{Int64}, Union{Nothing, HoleReference}}","page":"HerbConstraints.jl","title":"HerbConstraints.propagate","text":"Propagates the RequireOnLeft constraint. It removes every element from the domain that does not have a necessary predecessor in the left subtree.\n\n\n\n\n\n","category":"method"},{"location":"HerbConstraints/#Index","page":"HerbConstraints.jl","title":"Index","text":"","category":"section"},{"location":"HerbConstraints/","page":"HerbConstraints.jl","title":"HerbConstraints.jl","text":"","category":"page"},{"location":"tutorials/advanced_search/#Getting-started","page":"Advanced Search Procedures","title":"Getting started","text":"","category":"section"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"You can either paste this code into the Julia REPL or into a seperate file, e.g. get_started.jl followed by running julia get_started.jl.","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"To start, we import the necessary packages.","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"using HerbGrammar, HerbData, HerbSearch, HerbInterpret","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"We start with the same simple grammar from the main file get_started.md.","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"g = @cfgrammar begin\n Number = |(1:2)\n Number = x\n Number = Number + Number\n Number = Number * Number\nend","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"We use a simple problem.","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":" problem = Problem([IOExample(Dict(:x => x), 2x+1) for x ∈ 1:5])","category":"page"},{"location":"tutorials/advanced_search/#Parameters","page":"Advanced Search Procedures","title":"Parameters","text":"","category":"section"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"We can use a search strategy, where we can specify different parameters. For example, by setting the max_depth, we limit the depth of the search. In the next example, we can see the effect of the depth on the number of allocations considered. ","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"solution = @time search(g, problem, :Number, max_depth=3)\n>>> 0.003284 seconds (50.08 k allocations: 2.504 MiB)\nprintln(solution)\n>>> (x + 1) + x\n\nsolution = @time search(g, problem, :Number, max_depth=6)\n>>> 0.005696 seconds (115.28 k allocations: 5.910 MiB)\nprintln(solution)\n>>> (1 + x) + x","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"Another parameter to use is max_enumerations, which limits the number of programs that can be tested at evaluation. We can see the number of enumerations necessary to solve a simple problem in the next example.","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"for i in range(1, 50)\n println(i, \" enumerations\")\n solution = @time search(g, problem, :Number, max_enumerations=i)\n println(solution)\nend\n\n>>> ....\n>>> 23 enums: nothing\n>>> 0.010223 seconds (117.01 k allocations: 5.935 MiB, 44.23% gc time)\n>>> 24 enums: (1 + x) + x\n>>> 0.005305 seconds (117.01 k allocations: 5.935 MiB)\n>>> 25 enums: (1 + x) + x\n>>> 0.005381 seconds (117.01 k allocations: 5.935 MiB)\n>>> ...","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"We see that only when i >= 24, there is a result, after that, increasing i does not have any effect on the number of allocations. ","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"A final parameter we consider here is allow_evaluation_errors, which is false by default. When this is set to true, the program will still run even when an exception is thrown during evaluation. To see the effect of this, we create a new grammar. We can also retrieve the error together with the solution from the search method.","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"g = @cfgrammar begin\n Number = 1\n List = []\n Index = List[Number]\nend\n\nproblem = Problem([IOExample(Dict(), x) for x ∈ 1:5])\nsolution = search(g, problem, :Index, max_depth=2, allow_evaluation_errors=true)\nprintln(solution)\n>>> nothing","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"There is also another search method called search_best which return both the solution and the possible error. The method returns the best program found so far. In this case, we can also see the error:","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"solution, error = search_best(g, problem, :Index, max_depth=2, allow_evaluation_errors=true)\nprintln(solution)\n>>> nothing\nprintln(error)\n>>> 9223372036854775807 # or: typemax(Int)","category":"page"},{"location":"tutorials/advanced_search/#Search-methods","page":"Advanced Search Procedures","title":"Search methods","text":"","category":"section"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"We now show examples of using different search procedures.","category":"page"},{"location":"tutorials/advanced_search/#Breadth-First-Search","page":"Advanced Search Procedures","title":"Breadth-First Search","text":"","category":"section"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"The breadth-first search will first enumerate all possible programs at the same depth before considering a program with a depth of one more. A tree of the grammar is returned with programs ordered in increasing sizes. We can first collect the programs that have a max-depth of 2 and a max_size of infinite (integer maximum value), where the starting symbol is of type Real. This function uses a default heuristic 'left-most first', such that the left-most child in the tree is always explored first. ","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"g1 = @cfgrammar begin\n Real = 1 | 2\n Real = Real * Real\nend\nprograms = collect(get_bfs_enumerator(g1, 2, typemax(Int), :Real))","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"We can test that this function returns the correct functions and all functions. ","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"answer_programs = [\n RuleNode(1),\n RuleNode(2),\n RuleNode(3, [RuleNode(1), RuleNode(1)]),\n RuleNode(3, [RuleNode(1), RuleNode(2)]),\n RuleNode(3, [RuleNode(2), RuleNode(1)]),\n RuleNode(3, [RuleNode(2), RuleNode(2)])\n]\n\nprintln(all(p ∈ programs for p ∈ answer_programs))\n>>> true","category":"page"},{"location":"tutorials/advanced_search/#Depth-First-Search","page":"Advanced Search Procedures","title":"Depth-First Search","text":"","category":"section"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"In depth-first search, we first explore a certain branch of the search tree till the max_depth or a correct program is reached before we consider the next branch. ","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"g1 = @cfgrammar begin\nReal = 1 | 2\nReal = Real * Real\nend\nprograms = collect(get_dfs_enumerator(g1, 2, typemax(Int), :Real))\nprintln(programs)\n>>> RuleNode[1,, 3{1,1}, 3{1,2}, 3{2,1}, 3{2,2}, 2,]","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"get_dfs_enumerator also has a default left-most heuristic and we consider what the difference is in output. ","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"g1 = @cfgrammar begin\n Real = 1 | 2\n Real = Real * Real\nend\nprograms = collect(get_dfs_enumerator(g1, 2, typemax(Int), :Real, heuristic_rightmost))\nprintln(programs)\n>>> RuleNode[1,, 3{1,1}, 3{2,1}, 3{1,2}, 3{2,2}, 2,]","category":"page"},{"location":"tutorials/advanced_search/#Metropolis-Hastings","page":"Advanced Search Procedures","title":"Metropolis-Hastings","text":"","category":"section"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"One of the stochastic search methods that is implemented is Metropolis-Hastings (MH), which samples from a distribution of programs based on the grammar. For more information on MH, see for example this webpage.","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"For the example below, we use this grammar and helper function.","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"grammar = @csgrammar begin\n X = |(1:5)\n X = X * X\n X = X + X\n X = X - X\n X = x\nend\nfunction create_problem(f, range=20)\n examples = [IOExample(Dict(:x => x), f(x)) for x ∈ 1:range]\n return Problem(examples), examples\nend","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"The below example uses a simple arithmetic example. As the search method is stochastic, different programs may be returned, as shown below.","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"e = Meta.parse(\"x -> x * x + 4\")\nproblem, examples = create_problem(eval(e))\nenumerator = get_mh_enumerator(examples, mean_squared_error)\nprogram, cost = search_best(grammar, problem, :X, enumerator=enumerator, error_function=mse_error_function, max_depth=3)\n\n>>> (:(x * x - (1 - 5)), 0)\n>>> (:(4 + x * x), 0)\n>>> (:(x * x + 4), 0)","category":"page"},{"location":"tutorials/advanced_search/#Very-Large-Scale-Neighbourhood-Search","page":"Advanced Search Procedures","title":"Very Large Scale Neighbourhood Search","text":"","category":"section"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"The second implemented stochastic search method is VLSN, which search for a local optimum in the neighbourhood. For more information, see this article.","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"Given the same grammar as before, we can try with some simple examples.","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"e = Meta.parse(\"x -> 10\")\nmax_depth = 2\nproblem, examples = create_problem(eval(e))\nenumerator = get_vlsn_enumerator(examples, mean_squared_error, max_depth)\nprogram, cost = search_best(grammar, problem, :X, enumerator=enumerator, error_function=mse_error_function, max_depth=max_depth)\n>>> (:(5 + 5), 0)","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"e = Meta.parse(\"x -> x\")\nmax_depth = 1\nproblem, examples = create_problem(eval(e))\nenumerator = get_vlsn_enumerator(examples, mean_squared_error, max_depth)\nprogram, cost = search_best(grammar, problem, :X, enumerator=enumerator, error_function=mse_error_function, max_depth=max_depth)\n>>> (:x, 0)","category":"page"},{"location":"tutorials/advanced_search/#Simulated-Annealing","page":"Advanced Search Procedures","title":"Simulated Annealing","text":"","category":"section"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"The third stochastic search method is called simulated annealing, is another hill-climbing method to find local optima. For more information, see this page.","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"We try the example from earlier, but now we can additionally define the initial_temperature of the algorithm, which is 1 by default. Two possible answers to the program are given as well.","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"e = Meta.parse(\"x -> x * x + 4\")\ninitial_temperature = 2\nproblem, examples = create_problem(eval(e))\nenumerator = get_sa_enumerator(examples, mean_squared_error, initial_temperature)\nprogram, cost = search_best(grammar, problem, :X, enumerator=enumerator, error_function=mse_error_function, max_depth=3)\n>>> (:(4 + x * x), 0)\n>>> (:(x * x + (5 - 1)), 0)","category":"page"},{"location":"tutorials/advanced_search/#Genetic-Search","page":"Advanced Search Procedures","title":"Genetic Search","text":"","category":"section"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"Genetic search is a type of evolutionary algorithm, which will simulate the process of natural selection and return the 'fittest' program of the population. For more information, see here.","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"We show the example of finding a lambda function.","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"e = Meta.parse(\"x -> 3 * x * x + (x + 2)\")\nproblem, examples = create_problem(eval(e))\nenumerator = get_genetic_enumerator(examples, \n initial_population_size = 10,\n mutation_probability = 0.8,\n maximum_initial_population_depth = 3,\n)\nprogram, cost = search_best(grammar, problem, :X, enumerator=enumerator, error_function=mse_error_function, max_depth=nothing, max_time=20)\n>>> (:(((((x - 5) + x) + x * x) + 1) + (((((2 + x * x) + 3) + x * x) + 1) - ((x - x) + x))), 0)\n>>> (:(x * 1 + (x * ((x + x) + x * 1) + (1 + 1) * 1)), 0)\n>>> (:((((x + x) + x) + 2) * x + ((x - x) + (2 - x))), 0)","category":"page"},{"location":"tutorials/advanced_search/#Other-functionality","page":"Advanced Search Procedures","title":"Other functionality","text":"","category":"section"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"Finally, we showcase two other functionalities of HerbSearch, sampling and heuristics.","category":"page"},{"location":"tutorials/advanced_search/#Sampling","page":"Advanced Search Procedures","title":"Sampling","text":"","category":"section"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"Sampling is implemented for the different stochastic search methods.","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"We consider here a simple grammar, which gives different programs for different search depths.","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"grammar = @cfgrammar begin \n A = B | C | F\n F = G\n C = D\n D = E\nend\n\n# A->B (depth 1) or A->F->G (depth 2) or A->C->D->E (depth 3)\n\n# For depth ≤ 1 the only option is A->B\nexpression = rand(RuleNode, grammar, :A, 1)\n@assert rulenode2expr(expression, grammar) in [:B,:C,:F]\n\n# For depth ≤ 2 the two options are A->B (depth 1) and A->B->G| A->C->G | A->F->G (depth 2)\nexpression = rand(RuleNode, grammar, :A, 2)\n@assert rulenode2expr(expression,grammar) in [:B,:C,:F,:G]","category":"page"},{"location":"tutorials/advanced_search/#Heuristics","page":"Advanced Search Procedures","title":"Heuristics","text":"","category":"section"},{"location":"tutorials/advanced_search/#Examples-with-larger-domains-and-constraints","page":"Advanced Search Procedures","title":"Examples with larger domains and constraints","text":"","category":"section"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"Here, we showcase a few examples using a more complicated grammar and a few types of constraints","category":"page"},{"location":"tutorials/advanced_search/","page":"Advanced Search Procedures","title":"Advanced Search Procedures","text":"# Expects to return a program equivalent to 1 + (1 - x) = 2 - x\n\ng₁ = @csgrammar begin\n Element = |(1 : 3) # 1 - 3\n Element = Element + Element # 4\n Element = 1 - Element # 5\n Element = x # 6\nend\n\naddconstraint!(g₁, ComesAfter(6, [5]))\n\nexamples = [\n IOExample(Dict(:x => 0), 2),\n IOExample(Dict(:x => 1), 1),\n IOExample(Dict(:x => 2), 0)\n]\nproblem = Problem(examples)\nsolution = search(g₁, problem, :Element, max_depth=3)\n\n@assert test_with_input(SymbolTable(g₁), solution, Dict(:x => -2)) == 4\n\n# Expects to return a program equivalent to 4 + x * (x + 3 + 3) = x^2 + 6x + 4\n\ng₂ = @csgrammar begin\n Element = Element + Element + Element # 1\n Element = Element + Element * Element # 2\n Element = x # 3\n Element = |(3 : 5) # 4\nend\n\n# Restrict .. + x * x\naddconstraint!(g₂, Forbidden(MatchNode(2, [MatchVar(:x), MatchNode(3), MatchNode(3)])))\n# Restrict 4 and 5 in lower level\naddconstraint!(g₂, ForbiddenPath([2, 1, 5]))\naddconstraint!(g₂, ForbiddenPath([2, 1, 6]))\n\nexamples = [\n IOExample(Dict(:x => 1), 11)\n IOExample(Dict(:x => 2), 20)\n IOExample(Dict(:x => -1), -1)\n]\nproblem = Problem(examples)\nsolution = search(g₂, problem, :Element)\n\n@assert test_with_input(SymbolTable(g₂), solution, Dict(:x => 0)) == 4\n\n# Expects to return a program equivalent to (1 - (((1 - x) - 1) - 1)) - 1 = x + 1\n\ng₃ = @csgrammar begin\n Element = |(1 : 20) # 1 - 20\n Element = Element - 1 # 21\n Element = 1 - Element # 22\n Element = x # 23\nend\n\naddconstraint!(g₃, ComesAfter(23, [22, 21]))\naddconstraint!(g₃, ComesAfter(22, [21]))\n\nexamples = [\n IOExample(Dict(:x => 1), 2)\n IOExample(Dict(:x => 10), 11)\n]\nproblem = Problem(examples)\nsolution = search(g₃, problem, :Element)\n\n@assert test_with_input(SymbolTable(g₃), solution, Dict(:x => 0)) == 1\n@assert test_with_input(SymbolTable(g₃), solution, Dict(:x => 100)) == 101\n\n# Expects to return a program equivalent to 18 + 4x\n\ng₄ = @csgrammar begin\n Element = |(0 : 20) # 1 - 20\n Element = Element + Element + Element # 21\n Element = Element + Element * Element # 22\n Element = x # 23\nend\n\n# Enforce ordering on + +\naddconstraint!(g₄, Ordered(\n MatchNode(21, [MatchVar(:x), MatchVar(:y), MatchVar(:z)]),\n [:x, :y, :z]\n))\n\nexamples = [\n IOExample(Dict(:x => 1), 22),\n IOExample(Dict(:x => 0), 18),\n IOExample(Dict(:x => -1), 14)\n]\nproblem = Problem(examples)\nsolution = search(g₄, problem, :Element)\n\n@assert test_with_input(SymbolTable(g₄), solution, Dict(:x => 100)) == 418\n\n# Expects to return a program equivalent to (x == 2) ? 1 : (x + 2)\n\ng₅ = @csgrammar begin\n Element = Number # 1\n Element = Bool # 2\n\n Number = |(1 : 3) # 3-5\n \n Number = Number + Number # 6\n Bool = Number ≡ Number # 7\n Number = x # 8\n \n Number = Bool ? Number : Number # 9\n Bool = Bool ? Bool : Bool # 10\nend\n\n# Forbid ? = ?\naddconstraint!(g₅, Forbidden(MatchNode(7, [MatchVar(:x), MatchVar(:x)])))\n# Order =\naddconstraint!(g₅, Ordered(MatchNode(7, [MatchVar(:x), MatchVar(:y)]), [:x, :y]))\n# Order +\naddconstraint!(g₅, Ordered(MatchNode(6, [MatchVar(:x), MatchVar(:y)]), [:x, :y]))\n\nexamples = [\n IOExample(Dict(:x => 0), 2)\n IOExample(Dict(:x => 1), 3)\n IOExample(Dict(:x => 2), 1)\n]\nproblem = Problem(examples)\nsolution = search(g₅, problem, :Element)\n\n@assert test_with_input(SymbolTable(g₅), solution, Dict(:x => 3)) == 5","category":"page"},{"location":"HerbInterpret/#HerbInterpret_docs","page":"HerbInterpret.jl","title":"HerbInterpret.jl Documentation","text":"","category":"section"},{"location":"HerbInterpret/","page":"HerbInterpret.jl","title":"HerbInterpret.jl","text":"CurrentModule=HerbInterpret","category":"page"},{"location":"HerbInterpret/","page":"HerbInterpret.jl","title":"HerbInterpret.jl","text":"Modules = [HerbInterpret]\nOrder = [:type, :const, :macro, :function]","category":"page"},{"location":"HerbInterpret/#HerbInterpret.evaluate_program-Tuple{RuleNode, Vector{<:Example}, Grammar, Function}","page":"HerbInterpret.jl","title":"HerbInterpret.evaluate_program","text":"evaluate_program(program::RuleNode, examples::Vector{<:Example}, grammar::Grammar, evaluation_function::Function)\n\nRuns a program on the examples and returns tuples of actual desired output and the program's output\n\n\n\n\n\n","category":"method"},{"location":"HerbInterpret/#HerbInterpret.execute_on_examples-Tuple{Dict{Symbol, Any}, Any, Vector{Dict{Symbol, Any}}}","page":"HerbInterpret.jl","title":"HerbInterpret.execute_on_examples","text":"execute_on_examples(tab::SymbolTable, expr::Any, example_inputs::Vector{Dict{Symbol, Any}})::Vector{Any}\n\nExecutes a given expression on a set of inputs and returns the respective outputs. WARNING: This function throws exceptions that are caused in the given expression. These exceptions have to be handled by the caller of this function.\n\n\n\n\n\n","category":"method"},{"location":"HerbInterpret/#HerbInterpret.interpret-Tuple{Dict{Symbol, Any}, Any}","page":"HerbInterpret.jl","title":"HerbInterpret.interpret","text":"interpret(tab::SymbolTable, ex::Expr)\n\nEvaluates an expression without compiling it. Uses AST and symbol lookups. Only supports :call and :(=) expressions at the moment.\n\nExample usage:\n\ntab = SymbolTable(:f => f, :x => x)\nex = :(f(x))\ninterpret(tab, ex)\n\nWARNING: This function throws exceptions that are caused in the given expression. These exceptions have to be handled by the caller of this function.\n\n\n\n\n\n","category":"method"},{"location":"HerbInterpret/#HerbInterpret.test_all_examples-Tuple{Dict{Symbol, Any}, Any, Vector{Example}}","page":"HerbInterpret.jl","title":"HerbInterpret.test_all_examples","text":"test_all_examples(tab::SymbolTable, expr::Any, examples::Vector{Example})::Vector{Bool}\n\nRuns the interpreter on all examples with the given input table and expression. The symbol table defines everything (functions, symbols) that are not input variables to the program to be synthesised. Returns a list of true/false values indicating if the expression satisfies the corresponding example. WARNING: This function throws exceptions that are caused in the given expression. These exceptions have to be handled by the caller of this function.\n\n\n\n\n\n","category":"method"},{"location":"HerbInterpret/#HerbInterpret.test_examples-Tuple{Dict{Symbol, Any}, Any, Vector{Example}}","page":"HerbInterpret.jl","title":"HerbInterpret.test_examples","text":"test_examples(tab::SymbolTable, expr::Any, examples::Vector{Example})::Bool\n\nEvaluates all examples and returns true iff all examples pass. Shortcircuits as soon as an example is found for which the program doesn't work. Returns false if one of the examples produces an error.\n\n\n\n\n\n","category":"method"},{"location":"HerbInterpret/#HerbInterpret.test_with_input-Tuple{Dict{Symbol, Any}, Any, Dict}","page":"HerbInterpret.jl","title":"HerbInterpret.test_with_input","text":"test_with_input(tab::SymbolTable, expr::Any, input::Dict)\n\nInterprets an expression or symbol with the given symboltable and the input. WARNING: This function throws exceptions that are caused in the given expression. These exceptions have to be handled by the caller of this function.\n\n\n\n\n\n","category":"method"},{"location":"HerbInterpret/#Index","page":"HerbInterpret.jl","title":"Index","text":"","category":"section"},{"location":"HerbInterpret/","page":"HerbInterpret.jl","title":"HerbInterpret.jl","text":"","category":"page"},{"location":"tutorials/getting_started_with_herb/#Search","page":"A more verbose getting started with Herb.jl","title":"Search","text":"","category":"section"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"This notebook describes how you can search a program space as defined by a grammar. Specifically, we will look at example-based search, where the goal is to find a program that is able to transform the inputs of every example to the corresponding output.","category":"page"},{"location":"tutorials/getting_started_with_herb/#Setup","page":"A more verbose getting started with Herb.jl","title":"Setup","text":"","category":"section"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"First, we start with the setup. We need to access to all the function in the Herb.jl framework.","category":"page"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"using Herb","category":"page"},{"location":"tutorials/getting_started_with_herb/#Defining-the-program-space","page":"A more verbose getting started with Herb.jl","title":"Defining the program space","text":"","category":"section"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"Next, we start by creating a grammar. We define a context-free grammar which is just a simple set of production rules for defining combinations of terminal symbols (in our case real numbers). ","category":"page"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"Contrary, we could define a context-sensitive grammar, when the production rules only hold in a certain context. However, for more information on this, please see example2_defining_grammars.ipynb.","category":"page"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"For now, we specify a simple grammar for dealing with integers and explain all the rules individually:","category":"page"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"First, we specify our interval [0:9] on real numbers and also constrain them to be integer.\nThen, we can also use the variable x to hold an integer.\nThe third rule determines we can add two integers.\nThe fourth rule determines we can subtract an integer from another.\nFinally, we also allow the multiplication of two integers.","category":"page"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"If you run this cell, you can see all the rules rolled out.","category":"page"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"g = Herb.HerbGrammar.@cfgrammar begin\n Real = |(0:9)\n Real = x\n Real = Real + Real\n Real = Real - Real\n Real = Real * Real\nend","category":"page"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"1: Real = 0\n2: Real = 1\n3: Real = 2\n4: Real = 3\n5: Real = 4\n6: Real = 5\n7: Real = 6\n8: Real = 7\n9: Real = 8\n10: Real = 9\n11: Real = x\n12: Real = Real + Real\n13: Real = Real - Real\n14: Real = Real * Real","category":"page"},{"location":"tutorials/getting_started_with_herb/#Defining-the-problem","page":"A more verbose getting started with Herb.jl","title":"Defining the problem","text":"","category":"section"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"As mentioned before, we are looking at example-based search. This means that the problem is defined by a set of input-output examples. A single example hence consists of an input and an output. The input is defined as a dictionary, with a value assigned to each variable in the grammar. It is important to write the variable name as a Symbol instead of a string. A Symbol in Julia is written with a colon prefix, i.e. :x. The output of the input-output example is just a single value for this specific grammar, but could possibly relate to e.g. arrays of values, too.","category":"page"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"In the cell below we automatically generate some examples for x assigning values 1-5.","category":"page"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"# Create input-output examples\nexamples = [Herb.HerbData.IOExample(Dict(:x => x), 3x+5) for x ∈ 1:5]","category":"page"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"5-element Vector{Main.Herb.HerbData.IOExample}:\n Main.Herb.HerbData.IOExample(Dict{Symbol, Any}(:x => 1), 8)\n Main.Herb.HerbData.IOExample(Dict{Symbol, Any}(:x => 2), 11)\n Main.Herb.HerbData.IOExample(Dict{Symbol, Any}(:x => 3), 14)\n Main.Herb.HerbData.IOExample(Dict{Symbol, Any}(:x => 4), 17)\n Main.Herb.HerbData.IOExample(Dict{Symbol, Any}(:x => 5), 20)","category":"page"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"Now that we have some input-output examples, we can define the problem. Next to the examples, a problem also contains a name meant to link to the file path, which can be used to keep track of current examples. For now, this is irrelevant, and you can give the program any name you like.","category":"page"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"problem = Herb.HerbData.Problem(examples, \"example\")","category":"page"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"Main.Herb.HerbData.Problem(Main.Herb.HerbData.Example[Main.Herb.HerbData.IOExample(Dict{Symbol, Any}(:x => 1), 8), Main.Herb.HerbData.IOExample(Dict{Symbol, Any}(:x => 2), 11), Main.Herb.HerbData.IOExample(Dict{Symbol, Any}(:x => 3), 14), Main.Herb.HerbData.IOExample(Dict{Symbol, Any}(:x => 4), 17), Main.Herb.HerbData.IOExample(Dict{Symbol, Any}(:x => 5), 20)], \"example\")","category":"page"},{"location":"tutorials/getting_started_with_herb/#Searching","page":"A more verbose getting started with Herb.jl","title":"Searching","text":"","category":"section"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"Now that we have defined the search space and the goal of the search, we can start the search. ","category":"page"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"Of course, our problem is underdefined as there might be multiple programs that satisfy our examples. Let us consider the case where we also have a ternary if-then-else operator and standard boolean operators in our grammar: we could synthesize the program x ≤ 5 ? 3x+5 : 0. This program satisfies all our examples, but we don't expect it to generalize very well.","category":"page"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"In general, we assume that a smaller program is more general than a larger program. Therefore, we search for the smallest program in our grammar that satisfies our examples. This can be done using a breadth-first search over the program/search space.","category":"page"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"This search is very basic; it makes use of an enumeration technique, where we enumerate programs one-by-one until we find a program that matches our examples. The search procedure has a built-in default evaluator to verify the candidate programs with the given input. The search procedure also has a built-in search procedure using breadth-first search. ","category":"page"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"So, we only need to give our grammar and the problem to our search procedure, along with a starting Symbol, in our case a Real.","category":"page"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"Herb.HerbSearch.search(g, problem, :Real)","category":"page"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":":(x * 3 + 5)","category":"page"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"As you can see, the search procedure found the correct program!","category":"page"},{"location":"tutorials/getting_started_with_herb/#Defining-the-search-procedure","page":"A more verbose getting started with Herb.jl","title":"Defining the search procedure","text":"","category":"section"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"In the previous case, we used the built-ins of the search procedure. However, we can also give a custom enumerator to the search procedure and define a few more values.","category":"page"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"We first define a new problem to test with, we are looking for the programs that can compute the value167. We immediately pass the examples to the problem and then set up the new search.","category":"page"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"Search is done by passing the grammar, the problem and the starting point like before. We now also specify the enumeration function to be used, and now we use depth-first search. Then, we give the maximum depth of the programs we want to search for (3), the maximum number of nodes in the Abstract Syntax Tree that exists during search (10), and the maximum time in seconds allowed for the search.","category":"page"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"problem = Herb.HerbData.Problem([Herb.HerbData.IOExample(Dict(:x => x), 168) for x ∈ 1:5], \"example2\")\nexpr = Herb.HerbSearch.search(g, problem, :Real, enumerator=Herb.HerbSearch.get_dfs_enumerator, max_depth=4, max_size=30, max_time=180)\nprint(expr)","category":"page"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"nothing","category":"page"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"We see that our synthesizer can find a program to construct the value 168, though a fun experiment would be trying to get the value 167, what do you think would happen? If you want you can try below.","category":"page"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"In any case, this concludes our first introduction to the Herb.jl program synthesis framework. You can see more examples in this repository, or explore yourself. Enjoy!","category":"page"},{"location":"tutorials/getting_started_with_herb/","page":"A more verbose getting started with Herb.jl","title":"A more verbose getting started with Herb.jl","text":"problem = Herb.HerbData.Problem([Herb.HerbData.IOExample(Dict(:x => x), 167) for x ∈ 1:5], \"example2\")\nexpr = Herb.HerbSearch.search(g, problem, :Real)\nprintln(expr)","category":"page"},{"location":"get_started/#Getting-Started","page":"Getting Started","title":"Getting Started","text":"","category":"section"},{"location":"get_started/","page":"Getting Started","title":"Getting Started","text":"You can either paste this code into the Julia REPL or into a seperate file, e.g. get_started.jl followed by julia get_started.jl.","category":"page"},{"location":"get_started/","page":"Getting Started","title":"Getting Started","text":"To begin, we need to import all needed packages using","category":"page"},{"location":"get_started/","page":"Getting Started","title":"Getting Started","text":"using HerbGrammar, HerbData, HerbSearch, HerbInterpret","category":"page"},{"location":"get_started/","page":"Getting Started","title":"Getting Started","text":"To define a program synthesis problem, we need a grammar and specification. ","category":"page"},{"location":"get_started/","page":"Getting Started","title":"Getting Started","text":"First, the grammar can be constructed using the @cfgrammar macro included in HerbGrammar. Here we describe a simple integer arithmetic example, that can add and multiply an input variable x or the integers 1,2, using","category":"page"},{"location":"get_started/","page":"Getting Started","title":"Getting Started","text":"g = @cfgrammar begin\n Number = |(1:2)\n Number = x\n Number = Number + Number\n Number = Number * Number\nend","category":"page"},{"location":"get_started/","page":"Getting Started","title":"Getting Started","text":"Second, the problem specification can be provided using e.g. input/output examples using HerbData. Inputs are provided as a Dict assigning values to variables, and outputs as arbitrary values. The problem itself is then a list of IOExamples using","category":"page"},{"location":"get_started/","page":"Getting Started","title":"Getting Started","text":"problem = Problem([IOExample(Dict(:x => x), 2x+1) for x ∈ 1:5])","category":"page"},{"location":"get_started/","page":"Getting Started","title":"Getting Started","text":"The problem is given now, let us search for a solution with HerbSearch. For now we will just use the default parameters searching for a satisfying program over the grammar, given the problem and a starting symbol using","category":"page"},{"location":"get_started/","page":"Getting Started","title":"Getting Started","text":"solution = search(g, problem, :Number, max_depth=3)\nprintln(solution)","category":"page"},{"location":"get_started/","page":"Getting Started","title":"Getting Started","text":"There are various ways to adapt the search technique to your needs. Please have a look at the search documentation.","category":"page"},{"location":"get_started/","page":"Getting Started","title":"Getting Started","text":"Eventually, we want to test our solution on some other inputs using HerbInterpret. We transform our grammar g to a Julia expression with Symboltable(g), add our solution and the input, assigning the value 6 to the variable x.","category":"page"},{"location":"get_started/","page":"Getting Started","title":"Getting Started","text":"output = test_with_input(SymbolTable(g), solution, Dict(:x => 6))\nprintln(output)","category":"page"},{"location":"get_started/","page":"Getting Started","title":"Getting Started","text":"Just like that we tackled (almost) all modules of Herb.jl.","category":"page"},{"location":"get_started/#Where-to-go-from-here?","page":"Getting Started","title":"Where to go from here?","text":"","category":"section"},{"location":"get_started/","page":"Getting Started","title":"Getting Started","text":"See our other tutorials!","category":"page"},{"location":"get_started/#The-full-code-example","page":"Getting Started","title":"The full code example","text":"","category":"section"},{"location":"get_started/","page":"Getting Started","title":"Getting Started","text":"using HerbSearch, HerbData, HerbInterpret, HerbGrammar\n\n# define our very simple context-free grammar\n# Can add and multiply an input variable x or the integers 1,2.\ng = @cfgrammar begin\n Number = |(1:2)\n Number = x\n Number = Number + Number\n Number = Number * Number\nend\n\nproblem = Problem([IOExample(Dict(:x => x), 2x+1) for x ∈ 1:5])\nsolution = search(g, problem, :Number, max_depth=3)\n\ntest_with_input(SymbolTable(g), solution, Dict(:x => 6))","category":"page"},{"location":"HerbData/#HerbData_docs","page":"HerbCore.jl","title":"HerbData.jl Documentation","text":"","category":"section"},{"location":"HerbData/","page":"HerbCore.jl","title":"HerbCore.jl","text":"CurrentModule=HerbData","category":"page"},{"location":"HerbData/","page":"HerbCore.jl","title":"HerbCore.jl","text":"Modules = [HerbData]\nOrder = [:type, :const, :macro, :function]","category":"page"},{"location":"HerbData/#HerbData.IOExample","page":"HerbCore.jl","title":"HerbData.IOExample","text":"struct IOExample <: Example\n\nAn input-output example. input is a Dict of {Symbol,Any} where the symbol represents a variable in a program. output can be anything.\n\n\n\n\n\n","category":"type"},{"location":"HerbData/#HerbData.IOPExample","page":"HerbCore.jl","title":"HerbData.IOPExample","text":"struct IOPExample <: Example\n\nAn input-output example with an associated program. ex is an IOExample. program is a program of arbitrary form. Please note that this is a pure container, and thus does not guarantee any checks on the validity of the program.\n\n\n\n\n\n","category":"type"},{"location":"HerbData/#HerbData.Problem","page":"HerbCore.jl","title":"HerbData.Problem","text":"struct Problem\n\nProgram synthesis problem defined with a vector of Examples\n\n\n\n\n\n","category":"type"},{"location":"HerbData/#HerbData.read_IOPexamples-Tuple{AbstractString}","page":"HerbCore.jl","title":"HerbData.read_IOPexamples","text":"read_IOPexamples(filepath::AbstractString)::Vector{Tuple{IOPExample}\n\nReads serialized IO + program examples from disk after type checking.\n\n\n\n\n\n","category":"method"},{"location":"HerbData/#HerbData.read_IOexamples-Tuple{AbstractString}","page":"HerbCore.jl","title":"HerbData.read_IOexamples","text":"read_IOexamples(filepath::AbstractString)::Vector{IOExample}\n\nReads serialized IO examples from disk after type checking.\n\n\n\n\n\n","category":"method"},{"location":"HerbData/#HerbData.readdata-Tuple{AbstractString, Function}","page":"HerbCore.jl","title":"HerbData.readdata","text":"readdata(directory::AbstractString, lineparser::Function)::Vector{Problem}\n\nReads all files in the given directory and parses them line by line into an ExampleProblem using the given lineparser.\n\nTODO: Turn this into an iterator that doesn't load all data into memory at initialization.\n\n\n\n\n\n","category":"method"},{"location":"HerbData/#HerbData.readfile-Tuple{AbstractString, Function}","page":"HerbCore.jl","title":"HerbData.readfile","text":"readfile(filepath::AbstractString, lineparser::Function)::Problem\n\nReads a file and parses every non-empty line using the line parser.\n\n\n\n\n\n","category":"method"},{"location":"HerbData/#HerbData.write_IOPexamples-Tuple{AbstractString, Vector{IOPExample}}","page":"HerbCore.jl","title":"HerbData.write_IOPexamples","text":"write_IOPexamples(filepath::AbstractString, examples::Vector{Tuple{IOExample, Any}})\n\nWrites IO examples and the corresponding programs to disk by serializing them into a file using HDF5 checking for and appending the .xiop.\n\n\n\n\n\n","category":"method"},{"location":"HerbData/#HerbData.write_IOexamples-Tuple{AbstractString, Vector{IOExample}}","page":"HerbCore.jl","title":"HerbData.write_IOexamples","text":"write_IOexamples(filepath::AbstractString, examples::Vector{IOExample})\n\nWrites IO examples to disk by serializing them into a file using HDF5 checking for and appending the .xio file ending.\n\n\n\n\n\n","category":"method"},{"location":"HerbData/#Index","page":"HerbCore.jl","title":"Index","text":"","category":"section"},{"location":"HerbData/","page":"HerbCore.jl","title":"HerbCore.jl","text":"","category":"page"},{"location":"tutorials/defining_grammars/#Defining-Grammars-in-Herb.jl-using-HerbGrammar","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl using HerbGrammar","text":"","category":"section"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"The program space in Herb.jl is defined using a grammar. This notebook demonstrates how such a grammar can be created. There are multiple kinds of grammars, but they can all be defined in a very similar way.","category":"page"},{"location":"tutorials/defining_grammars/#Setup","page":"Defining Grammars in Herb.jl","title":"Setup","text":"","category":"section"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"using HerbGrammar, HerbConstraints","category":"page"},{"location":"tutorials/defining_grammars/#Creating-a-simple-grammar","page":"Defining Grammars in Herb.jl","title":"Creating a simple grammar","text":"","category":"section"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"This cell contains a very simple arithmetic grammar. The grammar is defined using the @cfgrammar macro. This macro converts the grammar definition in the form of a Julia expression into Herb's internal grammar representation. Macro's are executed during compilation. If you want to load a grammar during execution, have a look at the HerbGrammar.expr2cfgrammar function.","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"g₁ = HerbGrammar.@cfgrammar begin\n Int = 1\n Int = 2\n Int = 3\n Int = Int * Int\n Int = Int + Int\nend","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"1: Int = 1\n2: Int = 2\n3: Int = 3\n4: Int = Int * Int\n5: Int = Int + Int","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"Defining every integer one-by-one can be quite tedious. Therefore, it is also possible to use the following syntax that makes use of a Julia iterator:","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"g₂ = HerbGrammar.@cfgrammar begin\n Int = |(0:9)\n Int = Int * Int\n Int = Int + Int\nend","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"1: Int = 0\n2: Int = 1\n3: Int = 2\n4: Int = 3\n5: Int = 4\n6: Int = 5\n7: Int = 6\n8: Int = 7\n9: Int = 8\n10: Int = 9\n11: Int = Int * Int\n12: Int = Int + Int","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"You can do the same with lists:","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"g₃ = HerbGrammar.@cfgrammar begin\n Int = |([0, 2, 4, 6, 8])\n Int = Int * Int\n Int = Int + Int\nend","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"1: Int = 0\n2: Int = 2\n3: Int = 4\n4: Int = 6\n5: Int = 8\n6: Int = Int * Int\n7: Int = Int + Int","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"Variables can also be added to the grammar by just using the variable name:","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"g₄ = HerbGrammar.@cfgrammar begin\n Int = |(0:9)\n Int = Int * Int\n Int = Int + Int\n Int = x\nend","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"1: Int = 0\n2: Int = 1\n3: Int = 2\n4: Int = 3\n5: Int = 4\n6: Int = 5\n7: Int = 6\n8: Int = 7\n9: Int = 8\n10: Int = 9\n11: Int = Int * Int\n12: Int = Int + Int\n13: Int = x","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"Grammars can also work with functions. After all, + and * are just infix operators for Julia's identically-named functions. You can use functions that are provided by Julia, or functions that you wrote yourself:","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"f(a) = a + 1\n\ng₅ = HerbGrammar.@cfgrammar begin\n Int = |(0:9)\n Int = Int * Int\n Int = Int + Int\n Int = f(Int)\n Int = x\nend","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"1: Int = 0\n2: Int = 1\n3: Int = 2\n4: Int = 3\n5: Int = 4\n6: Int = 5\n7: Int = 6\n8: Int = 7\n9: Int = 8\n10: Int = 9\n11: Int = Int * Int\n12: Int = Int + Int\n13: Int = f(Int)\n14: Int = x","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"Similarly, we can also define the operator times (x) manually.","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"×(a, b) = a * b\n\ng₆ = HerbGrammar.@cfgrammar begin\n Int = |(0:9)\n Int = a\n Int = Int + Int\n Int = Int × Int\nend","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"1: Int = 0\n2: Int = 1\n3: Int = 2\n4: Int = 3\n5: Int = 4\n6: Int = 5\n7: Int = 6\n8: Int = 7\n9: Int = 8\n10: Int = 9\n11: Int = a\n12: Int = Int + Int\n13: Int = Int × Int","category":"page"},{"location":"tutorials/defining_grammars/#Working-with-grammars","page":"Defining Grammars in Herb.jl","title":"Working with grammars","text":"","category":"section"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"If you want to implement something using these grammars, it is useful to know about the functions that you can use to manipulate grammars and extract information. This section is not necessarily complete, but it aims to give an overview of the most important functions. ","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"It is recommended to also read up on Julia metaprogramming if you are not already familiar with that.","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"One of the most important things about grammars is that each rule has an index associated with it:","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"g₇ = HerbGrammar.@cfgrammar begin\n Int = |(0:9)\n Int = Int + Int\n Int = Int * Int\n Int = x\nend\n\ncollect(enumerate(g₇.rules))","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"13-element Vector{Tuple{Int64, Any}}:\n (1, 0)\n (2, 1)\n (3, 2)\n (4, 3)\n (5, 4)\n (6, 5)\n (7, 6)\n (8, 7)\n (9, 8)\n (10, 9)\n (11, :(Int + Int))\n (12, :(Int * Int))\n (13, :x)","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"We can use this index to extract information from the grammar.","category":"page"},{"location":"tutorials/defining_grammars/#isterminal","page":"Defining Grammars in Herb.jl","title":"isterminal","text":"","category":"section"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"isterminal returns true if a rule is terminal, i.e. it cannot be expanded. For example, rule 1 is terminal, but rule 11 is not, since it contains the non-terminal symbol :Int. ","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"HerbGrammar.isterminal(g₇, 1)","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"true","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"HerbGrammar.isterminal(g₇, 11)","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"false","category":"page"},{"location":"tutorials/defining_grammars/#return_type","page":"Defining Grammars in Herb.jl","title":"return_type","text":"","category":"section"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"This function is rather obvious; it returns the non-terminal symbol that corresponds to a certain rule. The return type for all rules in our grammar is :Int.","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"HerbGrammar.return_type(g₇, 11)","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":":Int","category":"page"},{"location":"tutorials/defining_grammars/#child_types","page":"Defining Grammars in Herb.jl","title":"child_types","text":"","category":"section"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"child_types returns the types of the nonterminal children of a rule in a vector. If you just want to know how many children a rule has, and not necessarily which types they have, you can use nchildren","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"HerbGrammar.child_types(g₇, 11)","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"2-element Vector{Symbol}:\n :Int\n :Int","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"HerbGrammar.nchildren(g₇, 11)","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"2","category":"page"},{"location":"tutorials/defining_grammars/#nonterminals","page":"Defining Grammars in Herb.jl","title":"nonterminals","text":"","category":"section"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"The nonterminals function can be used to obtain a list of all nonterminals in the grammar.","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"HerbGrammar.nonterminals(g₇)","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"1-element Vector{Symbol}:\n :Int","category":"page"},{"location":"tutorials/defining_grammars/#Adding-rules","page":"Defining Grammars in Herb.jl","title":"Adding rules","text":"","category":"section"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"It is also possible to add rules to a grammar during execution. This can be done using the add_rule! function. As with most functions in Julia that end with an exclamation mark, this function modifies its argument (the grammar).","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"A rule can be provided in the same syntax as is used in the grammar definition. The rule should be of the Expr type, which is a built-in type for representing expressions. An easy way of creating Expr values in Julia is to encapsulate it in brackets and use a colon as prefix:","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"HerbGrammar.add_rule!(g₇, :(Int = Int - Int))","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"1: Int = 0\n2: Int = 1\n3: Int = 2\n4: Int = 3\n5: Int = 4\n6: Int = 5\n7: Int = 6\n8: Int = 7\n9: Int = 8\n10: Int = 9\n11: Int = Int + Int\n12: Int = Int * Int\n13: Int = x\n14: Int = Int - Int","category":"page"},{"location":"tutorials/defining_grammars/#Removing-rules","page":"Defining Grammars in Herb.jl","title":"Removing rules","text":"","category":"section"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"It is also possible to remove rules in Herb.jl, however, this is a bit more involved. As said before, rules have an index associated with them. The internal representation of programs that are defined by the grammar makes use of those indices for efficiency. Blindly removing a rule would shift the indices of other rules, and this could mean that existing programs get a different meaning or become invalid. ","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"Therefore, there are two functions for removing rules:","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"remove_rule! removes a rule from the grammar, but fills its place with a placeholder. Therefore, the indices stay the same, and only programs that use the removed rule become invalid.\ncleanup_removed_rules! removes all placeholders and shifts the indices of the other rules.","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"HerbGrammar.remove_rule!(g₇, 11)","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"1: Int = 0\n2: Int = 1\n3: Int = 2\n4: Int = 3\n5: Int = 4\n6: Int = 5\n7: Int = 6\n8: Int = 7\n9: Int = 8\n10: Int = 9\n11: nothing = nothing\n12: Int = Int * Int\n13: Int = x\n14: Int = Int - Int","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"HerbGrammar.cleanup_removed_rules!(g₇)","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"1: Int = 0\n2: Int = 1\n3: Int = 2\n4: Int = 3\n5: Int = 4\n6: Int = 5\n7: Int = 6\n8: Int = 7\n9: Int = 8\n10: Int = 9\n11: Int = Int * Int\n12: Int = x\n13: Int = Int - Int","category":"page"},{"location":"tutorials/defining_grammars/#Context-sensitive-grammars","page":"Defining Grammars in Herb.jl","title":"Context-sensitive grammars","text":"","category":"section"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"Context-sensitive grammars allow additional constraints to be added with respect to context-free grammars. The syntax for defining a context-sensitive grammar is identical to defining a context-sensitive grammar:","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"g₈ = HerbGrammar.@csgrammar begin\n Int = |(0:9)\n Int = Int + Int\n Int = Int * Int\n Int = x\nend","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"1: Int = 0\n2: Int = 1\n3: Int = 2\n4: Int = 3\n5: Int = 4\n6: Int = 5\n7: Int = 6\n8: Int = 7\n9: Int = 8\n10: Int = 9\n11: Int = Int + Int\n12: Int = Int * Int\n13: Int = x","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"Constraints can be added using the addconstraint! function, which takes a context-sensitive grammar and a constraint and adds the constraint to the grammar. Currently, Herb.jl only has propagators constraints. These constraints each have a corresponding propagate function that removes all options that violate that constraint from the domain. At the moment, there are three propagator constraints:","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"ComesAfter(rule, predecessors): It is only possible to use rule rule when predecessors are in its path to the root.\nForbidden(sequence): Forbids the derivation specified as a path in an expression tree.\nOrdered(order): Rules have to be used in the specified order. That is, rule at index K can only be used if rules at indices [1...K-1] are used in the left subtree of the current expression.","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"Below, an example is given of a context-sensitive grammar with a ComesAfter constraint:","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"HerbGrammar.addconstraint!(g₈, HerbConstraints.ComesAfter(1, [9]))","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"1-element Vector{Main.HerbCore.Constraint}:\n Main.HerbConstraints.ComesAfter(1, [9])","category":"page"},{"location":"tutorials/defining_grammars/#Probabilistic-grammars","page":"Defining Grammars in Herb.jl","title":"Probabilistic grammars","text":"","category":"section"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"Herb.jl also supports probabilistic grammars. These grammars allow the user to assign a probability to each rule in the grammar. A probabilistic grammar can be defined in a very similar way to a standard grammar, but has some slightly different syntax:","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"g₉ = HerbGrammar.@pcfgrammar begin\n 0.4 : Int = |(0:9)\n 0.2 : Int = Int + Int\n 0.1 : Int = Int * Int\n 0.3 : Int = x\nend\n\nfor r ∈ 1:length(g₃.rules)\n p = HerbGrammar.probability(g₈, r)\n\n println(\"$p : $r\")\nend","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"0.07692307692307693 : 1\n0.07692307692307693 : 2\n0.07692307692307693 : 3\n0.07692307692307693 : 4\n0.07692307692307693 : 5\n0.07692307692307693 : 6\n0.07692307692307693 : 7\n\n\n┌ Warning: Requesting probability in a non-probabilistic grammar.\n│ Uniform distribution is assumed.\n└ @ Main.HerbGrammar d:\\GitHub\\HerbGrammar.jl\\src\\grammar_base.jl:155\n┌ Warning: Requesting probability in a non-probabilistic grammar.\n│ Uniform distribution is assumed.\n└ @ Main.HerbGrammar d:\\GitHub\\HerbGrammar.jl\\src\\grammar_base.jl:155\n┌ Warning: Requesting probability in a non-probabilistic grammar.\n│ Uniform distribution is assumed.\n└ @ Main.HerbGrammar d:\\GitHub\\HerbGrammar.jl\\src\\grammar_base.jl:155\n┌ Warning: Requesting probability in a non-probabilistic grammar.\n│ Uniform distribution is assumed.\n└ @ Main.HerbGrammar d:\\GitHub\\HerbGrammar.jl\\src\\grammar_base.jl:155\n┌ Warning: Requesting probability in a non-probabilistic grammar.\n│ Uniform distribution is assumed.\n└ @ Main.HerbGrammar d:\\GitHub\\HerbGrammar.jl\\src\\grammar_base.jl:155\n┌ Warning: Requesting probability in a non-probabilistic grammar.\n│ Uniform distribution is assumed.\n└ @ Main.HerbGrammar d:\\GitHub\\HerbGrammar.jl\\src\\grammar_base.jl:155\n┌ Warning: Requesting probability in a non-probabilistic grammar.\n│ Uniform distribution is assumed.\n└ @ Main.HerbGrammar d:\\GitHub\\HerbGrammar.jl\\src\\grammar_base.jl:155","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"The numbers before each rule represent the probability assigned to that rule. The total probability for each return type should add up to 1.0. If this isn't the case, Herb.jl will normalize the probabilities.","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"If a single line in the grammar definition represents multiple rules, such as 0.4 : Int = |(0:9), the probability will be evenly divided over all these rules.","category":"page"},{"location":"tutorials/defining_grammars/#File-writing","page":"Defining Grammars in Herb.jl","title":"File writing","text":"","category":"section"},{"location":"tutorials/defining_grammars/#Saving-and-loading-context-free-grammars","page":"Defining Grammars in Herb.jl","title":"Saving & loading context-free grammars","text":"","category":"section"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"If you want to store a grammar on the disk, you can use the store_cfg, read_cfg and functions to store and read grammars respectively. The store_cfg grammar can also be used to store probabilistic grammars. Reading probabilistic grammars can be done using read_pcfg. The stored grammar files can also be opened using a text editor to be modified, as long as the contents of the file doesn't violate the syntax for defining grammars.","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"HerbGrammar.store_cfg(\"demo.txt\", g₇)","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"HerbGrammar.read_cfg(\"demo.txt\")","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"1: Int = 0\n2: Int = 1\n3: Int = 2\n4: Int = 3\n5: Int = 4\n6: Int = 5\n7: Int = 6\n8: Int = 7\n9: Int = 8\n10: Int = 9\n11: Int = Int * Int\n12: Int = x\n13: Int = Int - Int","category":"page"},{"location":"tutorials/defining_grammars/#Saving-and-loading-context-sensitive-grammars","page":"Defining Grammars in Herb.jl","title":"Saving & loading context-sensitive grammars","text":"","category":"section"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"Saving and loading context-sensitive grammars is very similar to how it is done with context-free grammars. The only difference is that an additional file is created for the constraints. The file that contains the grammars can be edited and can also be read using the reader for context-free grammars. The file that contains the constraints cannot be edited.","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"HerbGrammar.store_csg(\"demo.grammar\", \"demo.constraints\", g₈)\ng₈, g₈.constraints","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"(1: Int = 0\n2: Int = 1\n3: Int = 2\n4: Int = 3\n5: Int = 4\n6: Int = 5\n7: Int = 6\n8: Int = 7\n9: Int = 8\n10: Int = 9\n11: Int = Int + Int\n12: Int = Int * Int\n13: Int = x\n, Main.HerbCore.Constraint[Main.HerbConstraints.ComesAfter(1, [9])])","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"g₉ = HerbGrammar.read_csg(\"demo.grammar\", \"demo.constraints\")\ng₉, g₉.constraints","category":"page"},{"location":"tutorials/defining_grammars/","page":"Defining Grammars in Herb.jl","title":"Defining Grammars in Herb.jl","text":"(1: Int = 0\n2: Int = 1\n3: Int = 2\n4: Int = 3\n5: Int = 4\n6: Int = 5\n7: Int = 6\n8: Int = 7\n9: Int = 8\n10: Int = 9\n11: Int = Int + Int\n12: Int = Int * Int\n13: Int = x\n, Main.HerbCore.Constraint[Main.HerbConstraints.ComesAfter(1, [9])])","category":"page"},{"location":"","page":"Herb.jl","title":"Herb.jl","text":"CurrentModule=Herb","category":"page"},{"location":"#[Herb.jl](https://github.com/Herb-AI/Herb.jl)","page":"Herb.jl","title":"Herb.jl","text":"","category":"section"},{"location":"","page":"Herb.jl","title":"Herb.jl","text":"A library for defining and efficiently solving program synthesis tasks in Julia.","category":"page"},{"location":"#Why-Herb.jl?","page":"Herb.jl","title":"Why Herb.jl?","text":"","category":"section"},{"location":"","page":"Herb.jl","title":"Herb.jl","text":"When writing research software we almost always investigate highly specific properties or algorithms of our domain, leading to us building the tools from scratch over and over again. The very same holds for the field of program synthesis: Tools are hard to run, benchmarks are hard to get and prepare, and its hard to adapt our existing code to a novel idea.","category":"page"},{"location":"","page":"Herb.jl","title":"Herb.jl","text":"Herb.jl will take care of this for you and helps you defining, solving and extending your program synthesis problems.","category":"page"},{"location":"","page":"Herb.jl","title":"Herb.jl","text":"Herb.jl provides...","category":"page"},{"location":"","page":"Herb.jl","title":"Herb.jl","text":"a unified and universal framework for program synthesis\nHerb.jl allows you to describe all sorts of program synthesis problems using context-free grammars\na number of state-of-the-art benchmarks and solvers already implemented and usable out-of-the-box","category":"page"},{"location":"","page":"Herb.jl","title":"Herb.jl","text":"Herb.jl's sub-packages provide fast and easily extendable implementations of ","category":"page"},{"location":"","page":"Herb.jl","title":"Herb.jl","text":"various static and dynamic search strategies,\nlearning search strategies, sampling techniques and more,\nconstraint formulation and propagation, \neasy grammar formulation and usage,\nwide-range of usable program interpreters and languages + the possibility to use your own, and \nefficient data formulation.","category":"page"},{"location":"#Why-Julia?","page":"Herb.jl","title":"Why Julia?","text":"","category":"section"},{"location":"","page":"Herb.jl","title":"Herb.jl","text":"Julia is a perfect fit for program synthesis due to numerous reasons. Starting from scientific reasons like speed of execution and composability over to practical reasons like speed of writing Julia code. For a full ode on why to use Julia, please see the WhyJulia manifesto.","category":"page"},{"location":"#Sub-Modules","page":"Herb.jl","title":"Sub-Modules","text":"","category":"section"},{"location":"","page":"Herb.jl","title":"Herb.jl","text":"Herb's functionality is distributed among several sub-packages:","category":"page"},{"location":"","page":"Herb.jl","title":"Herb.jl","text":"HerbCore.jl: The core of Herb.jl defining core concepts to avoid circular dependencies.\nHerbGrammar.jl:\nHerbData.jl:\nHerbInterpret.jl:\nHerbSearch.jl:\nHerbConstraints.jl:","category":"page"},{"location":"#Basics","page":"Herb.jl","title":"Basics","text":"","category":"section"},{"location":"","page":"Herb.jl","title":"Herb.jl","text":"Pages = [\"install.md\", \"get_started.md\", \"concepts.md\"]","category":"page"},{"location":"#Advanced-content","page":"Herb.jl","title":"Advanced content","text":"","category":"section"},{"location":"","page":"Herb.jl","title":"Herb.jl","text":"","category":"page"}] } diff --git a/dev/tutorials/advanced_search/index.html b/dev/tutorials/advanced_search/index.html index 1cfd9cb..fab5a5f 100644 --- a/dev/tutorials/advanced_search/index.html +++ b/dev/tutorials/advanced_search/index.html @@ -246,4 +246,4 @@ problem = Problem(examples) solution = search(g₅, problem, :Element) -@assert test_with_input(SymbolTable(g₅), solution, Dict(:x => 3)) == 5 +@assert test_with_input(SymbolTable(g₅), solution, Dict(:x => 3)) == 5 diff --git a/dev/tutorials/defining_grammars/index.html b/dev/tutorials/defining_grammars/index.html index f314034..56f7a45 100644 --- a/dev/tutorials/defining_grammars/index.html +++ b/dev/tutorials/defining_grammars/index.html @@ -250,4 +250,4 @@ 11: Int = Int + Int 12: Int = Int * Int 13: Int = x -, Main.HerbCore.Constraint[Main.HerbConstraints.ComesAfter(1, [9])]) +, Main.HerbCore.Constraint[Main.HerbConstraints.ComesAfter(1, [9])]) diff --git a/dev/tutorials/getting_started_with_herb/index.html b/dev/tutorials/getting_started_with_herb/index.html index 37be8e4..96d0f51 100644 --- a/dev/tutorials/getting_started_with_herb/index.html +++ b/dev/tutorials/getting_started_with_herb/index.html @@ -28,4 +28,4 @@ expr = Herb.HerbSearch.search(g, problem, :Real, enumerator=Herb.HerbSearch.get_dfs_enumerator, max_depth=4, max_size=30, max_time=180) print(expr)
      nothing

      We see that our synthesizer can find a program to construct the value 168, though a fun experiment would be trying to get the value 167, what do you think would happen? If you want you can try below.

      In any case, this concludes our first introduction to the Herb.jl program synthesis framework. You can see more examples in this repository, or explore yourself. Enjoy!

      problem = Herb.HerbData.Problem([Herb.HerbData.IOExample(Dict(:x => x), 167) for x ∈ 1:5], "example2")
 expr = Herb.HerbSearch.search(g, problem, :Real)
-println(expr)
      +println(expr)