Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rename commandline intro, and make it work for all operating systems #1038

Merged
merged 12 commits into from
Oct 27, 2023
5 changes: 2 additions & 3 deletions dataladhandbook_support/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,5 @@ def setup(app):
directives.setup(app)


from ._version import get_versions
__version__ = get_versions()['version']
del get_versions
from . import _version
__version__ = _version.get_versions()['version']
331 changes: 247 additions & 84 deletions dataladhandbook_support/_version.py

Large diffs are not rendered by default.

5 changes: 2 additions & 3 deletions docs/basics/101-102-populate.rst
Original file line number Diff line number Diff line change
Expand Up @@ -386,9 +386,8 @@ Well done! Your ``DataLad-101`` dataset and its history are slowly growing.
.. [#f1] ``tree`` is a Unix command to list file system content. If it is not yet installed,
you can get it with your native package manager (e.g., ``apt``, ``brew``, or conda).
For example, if you use OSX, ``brew install tree`` will get you this tool.
On Windows, if you have the Miniconda-based installation described in :ref:`install`, you can install the ``m2-base`` package (``conda install m2-base``), which contains tree along with many other Unix-like commands.
Note that this tree works slightly different than its Unix equivalent - it will only display directories, not files, and it doesn't accept common options or flags.
It will also display *hidden* directories, i.e., those that start with a ``.`` (dot).
Windows has its own ``tree`` command.
Note that this ``tree`` works slightly different than its Unix equivalent - by default, it will only display directories, not files, and the command options it accepts are either ``/f`` (display file names) or ``/a`` (change display of subdirectories to text instead of graphic characters).

.. [#f2] ``wget`` is a Unix command for non-interactively downloading files from the
web. If it is not yet installed, you can get it with your native package manager (e.g.,
Expand Down
5 changes: 2 additions & 3 deletions docs/basics/101-105-install.rst
Original file line number Diff line number Diff line change
Expand Up @@ -139,10 +139,9 @@ chapters in this handbook will demonstrate how useful this information can be.

Here is the repository structure:

.. windows-wit:: tree -d may fail
.. windows-wit:: use tree

If you have installed :term:`conda`\s ``m2-base`` package for access to Unix commands such as tree, you will have the tree command.
However, this version of tree does not support the use of any command flags, so please just run ``tree`` instead of ``tree -d``.
The Windows version of tree requires different parametrization, so please run ``tree`` instead of ``tree -d``.

.. runrecord:: _examples/DL-101-105-103
:language: console
Expand Down
4 changes: 2 additions & 2 deletions docs/basics/101-115-symlinks.rst
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,8 @@ We'll take a look together, using the ``books/`` directory as an example:

.. windows-wit:: This will look different to you

First of all, the ``tree`` equivalent provided by :term:`conda`\s ``m2-base`` package doesn't list individual files, only directories.
And, secondly, even if you list the individual files (e.g., with ``ls -l``), you would not see the :term:`symlink`\s shown below.
First of all, the Windows ``tree`` command lists only directories by default, unless you parametrize it with ``/f``.
And, secondly, even if you list the individual files, you would not see the :term:`symlink`\s shown below.
Due to insufficient support of symlinks on Windows, git-annex does not use them.
Please read on for a basic understanding of how git-annex usually works -- a Windows Wit at the end of this section will then highlight the difference in functionality on Windows.

Expand Down
8 changes: 4 additions & 4 deletions docs/glossary.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,8 @@ Glossary
.. glossary::

absolute path
The complete path from the root of the file system. Absolute paths always start with ``/``.
Example: ``/home/user/Pictures/xkcd-webcomics/530.png``. See also :term:`relative path`.
The complete path from the root of the file system. On Unix-like systems, absolute paths always start with ``/``, and on Windows systems, they start with a ``/`` (likely prefixed by a disk identifier).
Examples: ``/home/user/Pictures/xkcd-webcomics/530.png``, ``C:\Users\user\Pictures\xkcd-webcomics\530.png``. See also :term:`relative path`.

adjusted branch
git-annex concept: a special :term:`branch` in a dataset.
Expand Down Expand Up @@ -302,8 +302,8 @@ Glossary
Git concept. A "Git Reference", typically shortened to "ref", is a text file containing a :term:`commit` :term:`shasum` as a human-readable reference to a specific version of your dataset or Git repository. Thanks to refs, Git users do not need to memorize or type shasums when switching between dataset states, and can use simple names instead: For example, a :term:`branch` such as ``main`` is a ref, and a :term:`tag` is one, too. In both cases, those refs are text files that contain the shasum of the commit at the tip of a branch, or the shasum of the commit you added the tag to. Refs are organized in the directory ``.git/refs`` and Git commands and configurations can use refs to perform updating operations or determine their behavior. More details can be found at `at git-scm.com <https://git-scm.com/book/en/v2/Git-Internals-Git-References>`_

relative path
A path related to the present working directory. Relative paths never start with ``/``.
Example: ``../Pictures/xkcd-webcomics/530.png``. See also :term:`absolute path`.
A path related to the present working directory. Relative paths never start with ``/`` or ``\``.
Examples on Unix and Windows: ``../Pictures/xkcd-webcomics/530.png``, ``..\Pictures\xkcd-webcomics\530.png``. See also :term:`absolute path`.

remote
Git-terminology: A repository (and thus also :term:`DataLad dataset`) that a given repository
Expand Down
114 changes: 86 additions & 28 deletions docs/intro/howto.rst
Original file line number Diff line number Diff line change
@@ -1,45 +1,45 @@
.. _howto:

*********************
General prerequisites
*********************
****************
The Command Line
****************

.. index:: ! terminal, ! shell, ! command Line

While there is a graphical user interface for DataLad (the :term:`DataLad Gooey`), this handbook will first and foremost focus on teaching DataLad concepts without the overhead of a user-interface, using DataLad's most powerful interface on the *command line*.
This means that the code examples in the handbook show no buttons to click on, but a set of commands and options users type into their *terminal*.
If you are not used to working with command-line arguments, DataLad can appear intimidating.
Luckily, the set of possible commands is limited, and even without prior experience with a shell, one can get used to it fairly quickly.

This chapter aims at providing novices with general basics about the shell, common Unix
commands, and some general file system facts.
commands and their Windows equivalent, and some general file system facts.
This chapter is also a place to return to and (re-)read if you come across a
non-DataLad command or principle you want to remind yourself of.
If you are already familiar with the shell and know the difference between an absolute
and a relative path, you can safely skip this chapter and continue to the :ref:`DataLad Basics <basics-intro>`.

Almost all of this chapter is based on parts of a wonderful lab documentation
Alex Waite wrote.

The Command Line
================
The shell (sometimes also called a terminal, console, or CLI) is an interactive,
text based interface. If you have used Matlab or IPython, then you are already familiar
with the basics of a command line interface.
While there is a graphical user interface for DataLad (the :term:`DataLad Gooey`), this handbook will first and foremost focus on teaching DataLad concepts without the overhead of a user-interface, using DataLad's most powerful interface on the *command line*.
This means that the code examples in the handbook show no buttons to click on, but a set of commands and options users type into their *terminal*.
If you are not used to working with command-line tools, DataLad can appear intimidating.
Luckily, the set of possible commands is limited, and even without prior experience with a shell, one can get used to it fairly quickly.

.. figure:: ../artwork/src/img/shell.png
:width: 50%
:alt: A z-shell on a Debian system

A terminal window in a standard desktop environment.

The shell (also called a terminal, console, or CLI) is an interactive,
text based interface. If you have used Matlab or IPython, then you are already familiar
with the basics of a command line interface.
On Unix-like systems (e.g., running Linux or macos), the shell application is usually called "terminal".
On Windows systems, several different interfaces exist: The "CMD" Command Prompt and the Powershell are natively installed, and the Git Bash (provided by Git for Windows) or Anaconda prompt CLI (provided by Anaconda or Miniconda) can come with the installation of the respective software tool.
We recommend using CMD, or, if you have them installed already and prefer them over the CMD, the Git Bash or Anaconda prompt.
As later parts in this section will show, shells under Windows may use different commands than shells under Linux and macos systems.

Command Syntax
==============

Interactions with the shell take the form of commands, text-based instructions to your computer.
Commands are case sensitive and follow the syntax of: ``command [options...] <arguments...>``.
Whenever you see some example code in the code snippets of this book, make sure
that you capitalize exactly as shown if you try it out yourself.
The options modify the behavior of the program, and are usually preceded by ``-`` or ``--``.
The options modify the behavior of the program, and are usually preceded by ``-`` or ``--`` on Unix-like systems.
In this example


Expand Down Expand Up @@ -72,17 +72,20 @@ to list the size of a file in a *human-readable* format, supply the short option
$ ls -l --human-readable output.txt

Every command has many of those options (often called "flags") that modify their behavior.
On Windows, options of native Windows commands can be preceded by a ``/`` instead of dashes, for example ``dir /p output.txt``.
There are too many to even consider memorizing. Remember the ones you use often,
and the rest you will lookup in their documentation or via your favorite search engine.
DataLad commands naturally also come with many options, and in the next chapters
and later examples you will get to see many of them.

Basic Commands
==============
The following commands can appear in our examples or are generally useful to know:
They can help you to *explore and navigate* in your file system (``cd``, ``ls``),
copy, move, or remove files (``cp``, ``mv``, ``rm``), or create new directories
(``mkdir``).
The following commands, split between Unix-like (e.g., Linux, macos) and Windows environments, can appear in our examples or are generally useful to know:
They can help you to *explore and navigate* in your file system, copy, move, or remove files, or create new directories.
Note that the Git Bash on Windows emulates a Unix environment in which you could use Unix commands despite being on Windows.

Unix-like systems or environments
"""""""""""""""""""""""""""""""""

``ls -lah <folder>``
list the contents of a folder, including hidden files (-a), and all their information (-l);
Expand All @@ -103,6 +106,33 @@ copy, move, or remove files (``cp``, ``mv``, ``rm``), or create new directories
create a folder
``rmdir <folder>``
delete an empty folder
``export NAME=Value``
Set an :term:`environment variable` ``NAME`` to ``Value`` for your current terminal session

Windows systems
"""""""""""""""

``dir <folder>``
list the contents of a folder including hidden files
``cd <folder>``
change to another folder
``copy <from> <to>``
copy a file
``ren <oldname> <newname>``
rename a file or folder
``mv <from> <to>``
move a file or folder
``del <file>``
delete a file
``rmdir /s <folder>``
delete a folder and its contents (``/s``)
``md <folder>``
create a folder
``rmdir <folder>``
delete an empty folder
``set NAME=Value``
Set an :term:`environment variable` ``NAME`` to ``Value`` for your current terminal session


The Prompt
==========
Expand All @@ -123,23 +153,35 @@ output of those commands.

Paths
=====
Let's say I want to create a new folder in my home folder,

Paths look different on Unix-like and Windows systems.
Most prominently, the *path separators*, i.e., the symbol distinguishing directories in path, are back slashes (``\``) on Windows and front slashes (``/``) on Unix-like systems.
On Windows systems, paths are also usually prefixed with a "disk designator" such as ``C:`` or ``d:``.
Let's say I want to create a new folder in my home folder on a Unix system,
I can run the following command:

.. code-block:: bash

$ mkdir /home/me/awesome_datalad_project

And that works. ``/home/me/awesome_datalad_project`` is what is called an *absolute*
path. Absolute paths *always* start with a ``/``, and define the folder's location
with no ambiguity.
If I want to do the same in Window's CMD, I'd do

.. code-block::

$ mkdir C:\Users\me\awesome_datalad_project

And that both works on the respective system. ``/home/me/awesome_datalad_project`` and ``\Users\me\awesome_datalad_project`` are what is called an *absolute*
path.
Absolute paths *always* start with a ``/`` (on Unix-like systems) or a ``\`` (on Windows systems), and define the folder's location with no ambiguity.

However, much like in spoken language, using someone's full proper name every
time would be exhausting, and thus pronouns are used.

This shorthand is called *relative* paths, because they are defined (wait for it...)
*relative* to your current location on the file system. Relative paths *never* start
with a ``/``.
with a ``/`` or ``\``.
For example, ``myfile.txt`` is a relative path to the file ``myfile.txt`` in the current directory on Unix-like systems.
The Windows equivalent is to ``myfile.txt`` in the current directory on the ``C`` disk is ``C:myfile.txt``.

Unix knows a few shortcuts to refer to file system related directories, and you will
come across them often. Whenever you see a ``.``, ``..``, or ``~`` in a DataLad command,
Expand Down Expand Up @@ -209,6 +251,14 @@ However, since I want to go back to my home folder, it's much faster to run:

$ cd ~

Windows similarly knows the ``.`` and ``..`` shortcuts, but can not handle the ``~`` shortcut.
In order to quickly get home, you could use

.. code-block::

$ cd %userprofile%

More information on Windows paths can be found `here <https://learn.microsoft.com/en-us/windows/win32/fileio/naming-a-file>`_.

Text Editors
============
Expand All @@ -231,6 +281,14 @@ this is the breakdown:
``emacs``
Powerful; tons of features; written in Lisp; huge ecosystem; advanced learning curve.

The text editors above are all command-line editors.
They will open up directly in your terminal.

While those text editors can also be installed on Windows, command-line editors are rarely used on Windows.
Git for Windows might set :term:`vim` as the default editor upon installation, which can require some acclimatization.
A good graphical alternative is Notepad++, a powerful Windows-native text editor.
You may either be able to configure this during the installation, of afterwards by running ``git config core.editor notepad``.


Shells
======
Expand Down Expand Up @@ -305,4 +363,4 @@ Pressing Tab again will list the matching options (``Documents`` and ``Downloads
A visual example of tab-completion in action:


**That's it -- equipped with the basics of Unix, you are good to go on your DataLad adventure!**
**That's it -- equipped with the basics of the command line, you are good to go on your DataLad adventure!**
7 changes: 0 additions & 7 deletions docs/intro/installation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -394,13 +394,6 @@ To update an existing installation with conda, use:

$ conda update -c conda-forge datalad

.. windows-wit:: Install Unix command-line tools on Windows with Conda

On Windows, many Unix command-line tools such as ``cp`` that a frequently
used in this handbook are not available by default. You can get a good set
of tools by installing :term:`conda`\s ``m2-base`` package via ``conda
install m2-base``.

The `DataLad installer`_ also supports setting up a Conda environment, in case
a suitable Python version is already available.

Expand Down
Loading
Loading