Skip to content
This repository has been archived by the owner on Aug 14, 2024. It is now read-only.

Set cores #23

Merged
merged 16 commits into from
Oct 20, 2022
28 changes: 16 additions & 12 deletions _episodes/02-snakefiles.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,11 +81,15 @@ By default, Snakemake looks for a file called `Snakefile`, and we can run
Snakemake as follows:

~~~
snakemake
snakemake --cores 1
~~~
{: .language-bash}

By default, Snakemake tells us what it's doing as it executes actions:
The number of cores to use is a required argument for versions 5.11 and
above of Snakemake. It is possible to use more cores, but a single core
bkmgit marked this conversation as resolved.
Show resolved Hide resolved
is sufficient for now. It is also possible to indicate the number of
cores using `-c`, which will be used going forward. By default,
Snakemake tells us what it's doing as it executes actions:

~~~
Provided cores: 1
Expand Down Expand Up @@ -123,7 +127,7 @@ The first 5 lines of `isles.dat` should look exactly like before.
> using the `-s` flag. For example, if our Snakefile is named `MyOtherSnakefile`:
>
> ~~~
> snakemake -s MyOtherMakefile
> snakemake -c 1 -s MyOtherSnakefile
> ~~~
>{: .language-bash}
>
Expand Down Expand Up @@ -166,7 +170,7 @@ its dependency:
If we run Snakemake again,

~~~
snakemake
snakemake -c 1
~~~
{: .language-bash}

Expand Down Expand Up @@ -225,7 +229,7 @@ rule count_words_abyss:
If we run Snakemake,

~~~
snakemake
snakemake -c 1
~~~
{: .language-bash}

Expand All @@ -242,7 +246,7 @@ and this rule is already up-to-date. We need to explicitly tell Snakemake we
want to build `abyss.dat`:

~~~
snakemake abyss.dat
snakemake -c 1 abyss.dat
~~~
{: .language-bash}

Expand Down Expand Up @@ -279,7 +283,7 @@ Finished job 0.
> If we ask Snakemake to build a file which does not have a
> rule in our Snakefile, then we get messages like:
> ~~~
> $ snakemake what.dat
> $ snakemake -c 1 what.dat
> MissingRuleException:
> No rule to produce what.dat (if you use input functions make sure that they
> don't raise unexpected exceptions).
Expand Down Expand Up @@ -308,7 +312,7 @@ remove the data files whether or not they exist. If we run Snakemake and specify
this target:

~~~
snakemake clean
snakemake -c 1 clean
~~~
{: .language-bash}

Expand Down Expand Up @@ -369,7 +373,7 @@ to trigger the build of its dependencies, if needed.
If we run,

~~~
snakemake dats
snakemake -c 1 dats
~~~
{: .language-bash}

Expand Down Expand Up @@ -415,7 +419,7 @@ If we run `dats` again, then snakemake will see that the dependencies
no actions, there is `nothing to be done`:

~~~
snakemake dats
snakemake -c 1 dats
~~~
{: .language-bash}

Expand Down Expand Up @@ -461,8 +465,8 @@ prints out commands instead of actually executing them. Very useful for
debugging!

~~~
snakemake clean
snakemake -n -p isles.dat
snakemake -c 1 clean
snakemake -c 1 -n -p isles.dat
~~~
{: .language-bash}

Expand Down
12 changes: 6 additions & 6 deletions _episodes/03-wildcards.md
Original file line number Diff line number Diff line change
Expand Up @@ -119,7 +119,7 @@ Let's update our text files and re-run our rule:

~~~
touch books/*.txt
snakemake results.txt
snakemake -c 1 results.txt
~~~
{: .language-bash}

Expand Down Expand Up @@ -176,7 +176,7 @@ Finished job 0.
>
> ~~~
> touch *.dat
> snakemake results.txt
> snakemake -c 1 results.txt
> ~~~
> {: .language-bash}
>
Expand All @@ -196,7 +196,7 @@ Finished job 0.
> >
> > ~~~
> > touch books/*.txt
> > snakemake results.txt
> > snakemake -c 1 results.txt
> > ~~~
> >{: .language-bash}
> >
Expand Down Expand Up @@ -267,7 +267,7 @@ Let's mark `wordcount.py` as updated, and re-run the pipeline:

~~~
touch wordcount.py
snakemake
snakemake -c 1
~~~
{: .language-bash}

Expand Down Expand Up @@ -317,7 +317,7 @@ files. However, it turns out we don't have to! Let's see what happens to

~~~
touch wordcount.py
snakemake results.txt
snakemake -c 1 results.txt
~~~
{: .language-bash}

Expand Down Expand Up @@ -374,7 +374,7 @@ steps.
>
> ~~~
> touch books/last.txt
> snakemake results.txt
> snakemake -c 1 results.txt
> ~~~
> {: .language-bash}
>
Expand Down
8 changes: 4 additions & 4 deletions _episodes/04-patterns.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,8 +48,8 @@ Let's test the new pattern rule. We use the -p option to show that it is
running things correctly:

~~~
snakemake clean
snakemake -p dats
snakemake -c 1 clean
snakemake -c 1 -p dats
~~~
{: .language-bash}

Expand All @@ -58,7 +58,7 @@ to build individual `.dat` targets as before, and that our new rule will work
no matter what stem is being matched.

~~~
snakemake -p sierra.dat
snakemake -c 1 -p sierra.dat
~~~
{: .language-bash}

Expand Down Expand Up @@ -98,7 +98,7 @@ Finished job 0.
> such as `snakemake clean`, you cannot execute a pattern rule this way:
>
> ~~~
> snakemake count_words
> snakemake -c 1 count_words
> ~~~
> {:.language-bash}
> ~~~
Expand Down
21 changes: 11 additions & 10 deletions _episodes/05-snakemake-python.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ rule zipf_test:
~~~
{: .language-python}

After updating your rule, run `snakemake clean` and `snakemake -p` to confirm
After updating your rule, run `snakemake -c 1 clean` and `snakemake -c 1 -p` to confirm
that the pipeline still works.

> ## Named Dependencies
Expand Down Expand Up @@ -94,7 +94,7 @@ one place to update the list of files to process.
> Update your Snakefile with the `DATS` global variable.
>
> Try recreating both the `dats` and `results.txt` targets
> (run `snakemake clean` in between).
> (run `snakemake -c 1 clean` in between).
>
> > ## Solution
> >
Expand All @@ -121,10 +121,10 @@ rule zipf_test:
~~~
{: .language-python}

Now let's clean up our workspace with `snakemake clean`:
Now let's clean up our workspace with `snakemake -c 1 clean`:

~~~
snakemake clean
snakemake -c 1 clean
~~~
{: .language-bash}

Expand All @@ -148,7 +148,7 @@ Finished job 0.
Now let's re-run the pipeline...

~~~
snakemake
snakemake -c 1
~~~
{: .language-bash}

Expand Down Expand Up @@ -202,7 +202,7 @@ Finished job 0.
Let's do a dry-run:

~~~
snakemake -n
snakemake -c 1 -n
~~~
{: .language-bash}

Expand Down Expand Up @@ -392,7 +392,7 @@ Upon execution of the corresponding rule, Snakemake runs our Python code
in the `run:` block:

~~~
snakemake --quiet print_book_names
snakemake -c 1 print_book_names --quiet rules
~~~
{: .language-bash}

Expand All @@ -417,10 +417,11 @@ Finished job 0.
~~~
{: .output}

> ## Note the `--quiet` option
> ## Note the `--quiet rules` option
>
> `--quiet` or `-q` suppresses a lot of the rule progress output from Snakemake.
> This can be useful when you just want to see your own output.
> `--quiet rules` or `-q rules` suppresses a lot of the rule progress output from Snakemake.
> This can be useful when you just want to see your own output. Other choices are
> `-q all` and `-q progress`.
{:.callout}

[ref-dependency]: ../reference#dependency
Expand Down
12 changes: 6 additions & 6 deletions _episodes/06-remaining-stages.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,14 +114,14 @@ directory.
> build a specific file with a command like:
>
> ~~~
> snakemake last.dat
> snakemake -c 1 last.dat
> ~~~
> {:.language-bash}
>
> After moving the location of `dat` files, the correct command is:
>
> ~~~
> snakemake dats/last.dat
> snakemake -c 1 dats/last.dat
> ~~~
> {:.language-bash}
>
Expand Down Expand Up @@ -174,7 +174,7 @@ directory.
## Default Rules

The default rule is the rule that Snakemake runs if you don't specify a rule
on the command-line (e.g.: if you just run `snakemake`).
on the command-line (e.g.: if you just run `snakemake -c 1`).

The default rule is simply the first rule in a Snakefile. While the default
rule can be anything you like, it is common practice to call the default rule
Expand Down Expand Up @@ -228,8 +228,8 @@ It is common practice to have a `clean` rule that deletes all intermediate
and generated files, taking your workflow back to a blank slate.

We already have a `clean` rule, so now is a good time to check that it
removes all intermediate and output files. First do a `snakemake all` followed
by `snakemake clean`. Then check to see if any output files remain and add them
removes all intermediate and output files. First do a `snakemake -c 1 all` followed
by `snakemake -c 1 clean`. Then check to see if any output files remain and add them
to the clean rule if required.

## Creating an Archive
Expand All @@ -252,7 +252,7 @@ In this case, we will create an archive tar file.
> * The archive should contain all `dat` files, plots, and the
> Zipf summary table (`results.txt`).
> * Update `all` to expect `zipf_analysis.tar.gz` as input.
> * Remove the archive when `snakemake clean` is called.
> * Remove the archive when `snakemake -c 1 clean` is called.
>
> The syntax to create an archive is:
> ~~~
Expand Down
Loading