backup: improve reliability and documentation

* allow starting backup from any folder
* automatically run using sudo
  (to never fail due to ./backup owned by root)
* run post_backup.sh after 'docker-compose start', to minimize downtime
  when e.g. uploading backups to a remote server
* chown all of ./backup to the appropriate user

Fixes SensorsIot#566

docs/Basic_setup/Backup-and-Restore.md

@@ -4,55 +4,64 @@ This page explains how to use the backup and restore functionality of IOTstack.
 ## Backup
 The backup command can be executed from IOTstack's menu, or from a cronjob.
 
-### Running backup
-To ensure that all your data is saved correctly, the stack should be brought down. This is mainly due to databases potentially being in a state that could cause data loss.
+To ensure that all your data is saved correctly, the stack has to be stopped.
+This is mainly due to databases potentially being in a state that could cause
+data loss.
 
 There are 2 ways to run backups:
 
 * From the menu: `Backup and Restore` > `Run backup`
 * Running the following command: `bash ./scripts/backup.sh`
 
-The command that's run from the command line can also be executed from a cronjob:
+The command that's run from the command line can also be executed from crontab:
 
-```0 2 * * * cd /home/pi/IOTstack && /bin/bash ./scripts/backup.sh```
+```
+0 2 * * * cd /home/pi/IOTstack && /bin/bash ./scripts/backup.sh
+```
 
 The current directory of bash must be in IOTstack's directory, to ensure that it can find the relative paths of the files it's meant to back up. In the example above, it's assume that it's inside the `pi` user's home directory.
 
-### Arguments
-```
-./scripts/backup.sh {TYPE=3} {USER=$(whoami)}
+**Usage:**
+``` console
+$ ~/IOTstack/scripts/backup.sh {TYPE}
 ```
 
-* Types:
-  * 1 = Backup with Date
-    * A tarball file will be created that contains the date and time the backup was started, in the filename.
-  * 2 = Rolling Date
-    * A tarball file will be created that contains the day of the week (0-6) the backup was started, in the filename.
-    * If a tarball already exists with the same name, it will be overwritten.
-  * 3 = Both
-* User:
-    This parameter only becomes active if run as root. This script will default to the current logged in user
-      If this parameter is not supplied when run as root, the script will ask for the username as input
+**Types:**
+
+1.  Backup with Date. Backup to filename with the date and time the backup was
+    started.
+2.  Rolling Date. Filename will based on the day of the week (0-6) the backup
+    was started.  New backups will overwrite old ones.
+3.  Both
+
+You can find the backups in the `~/IOTstack/backups/` folder. With rolling
+being in `~/IOTstack/backups/rolling/` and date backups in
+`~/IOTstack/backups/backup/`. Log files can also be found in the logs/
+directory.
 
-Backups:
+**Examples:**
 
-  * You can find the backups in the ./backups/ folder. With rolling being in ./backups/rolling/ and date backups in ./backups/backup/
-  * Log files can also be found in the ./backups/logs/ directory.
+-   Regular date & timestamped backup into ~/IOTstack/backup/backups:
 
-### Examples:
+    `~/IOTstack/scripts/backup.sh 1`
 
-  * `./scripts/backup.sh`
-  * `./scripts/backup.sh 3`
+-   Either these commands will produce backups of both types:
 
-Either of these will run both backups.
+    `~/IOTstack/scripts/backup.sh` or<br />
+    `~/IOTstack/scripts/backup.sh 3`
 
-  * `./scripts/backup.sh 2`
 
-This will only produce a backup in the rollowing folder. It will be called 'backup_XX.tar.gz' where XX is the current day of the week (as an int)
+-   Produce a backup into ~/IOTstack/backup/rolling/. It will be called
+    'backup_XX.tar.gz' where XX is the current day of the week (as an int):
 
-  * `sudo bash ./scripts/backup.sh 2 pi`
+    `~/IOTstack/scripts/backup.sh 2`
 
-This will only produce a backup in the rollowing folder and change all the permissions to the 'pi' user.
+-   (expert use) Usually the backup should be executed without sudo using your
+    regular user, as this will automatically produce backups with correct
+    permissions.  This will only produce a backup in the rolling folder and
+    change all the permissions to the 'pi' user:
+
+    `sudo bash ~/IOTstack/scripts/backup.sh 2 pi`
 
 ## Restore
 There are 2 ways to run a restore:
@@ -64,7 +73,7 @@ There are 2 ways to run a restore:
 
 *Note*: It is suggested that you test that your backups can be restored after initially setting up, and anytime you add or remove a service. Major updates to services can also break backups.
 
-### Arguments
+**Usage:**
 ```
 ./scripts/restore.sh {FILENAME=backup.tar.gz} {noask}
 ```
@@ -76,35 +85,126 @@ The restore script takes 2 arguments:
 ## Pre and post script hooks
 The script checks if there are any pre and post back up hooks to execute commands. Both of these files will be included in the backup, and have also been added to the `.gitignore` file, so that they will not be touched when IOTstack updates.
 
-### Prebackup script hook
-The prebackup hook script is executed before any compression happens and before anything is written to the temporary backup manifest file (`./.tmp/backup-list_{{NAME}}.txt`). It can be used to prepare any services (such as databases that IOTstack isn't aware of) for backing up.
+Both of these scripts will be run as root.
+
+### Pre-backup script hook
+The pre-backup hook script is executed before any compression happens and before anything is written to the temporary backup manifest file (`./.tmp/backup-list_{{NAME}}.txt`). It can be used to prepare any services (such as databases that IOTstack isn't aware of) for backing up.
 
-To use it, simple create a `./pre_backup.sh` file in IOTstack's main directory. It will be executed next time a backup runs.
+To use it, simple create the `~/IOTstack/pre_backup.sh` file. It will be executed next time a backup runs.
 
-### Postbackup script hook
-The postbackup hook script is executed after the tarball file has been written to disk, and before the final backup log information is written to disk.
+### Post-backup script hook { #post-backup }
+The post-backup hook script is executed after the backup tarball file has been
+written to disk and the stack has been started back up. Any output will be
+included in the backup log file.
 
-To use it, simple create a `./post_backup.sh` file in IOTstack's main directory. It will be executed after the next time a backup runs.
+To use it, simple create the `~/IOTstack/post_backup.sh` file. It will be
+executed after the next time a backup runs. It will be provided the backup
+.tar.gz as its first argument.
+
+This is useful for triggering transfer of the backup to a cloud or another
+server, see below for possible third party integrations as to what is possible.
 
 ### Post restore script hook
 The post restore hook script is executed after all files have been extracted and written to disk. It can be used to apply permissions that your custom services may require.
 
 To use it, simple create a `./post_restore.sh` file in IOTstack's main directory. It will be executed after a restore happens.
 
 ## Third party integration
-This section explains how to backup your files with 3rd party software.
+This section explains how you could backup your files to different
+integrations. Actual initiation of the transfer of the backup has to be done
+using the [post backup](#post-backup) hook described above.
+
+### IOTstackBackup
+
+[IOTstackBackup](https://github.com/Paraphraser/IOTstackBackup) is a project
+aiming to provide a sophisticated all-in-on solution: on-line backups of
+databases and configurable transfers to remotes without writing your own
+scripts.
 
 ### Dropbox
-Coming soon.
+Dropbox-Uploader is a great utility to easily upload data from your Pi to the cloud. https://magpi.raspberrypi.org/articles/dropbox-raspberry-pi. It can be installed from the Menu under Backups.
+
+#### Troubleshoot: Token added incorrectly or install aborted at the token screen
+
+Run `~/Dropbox-Uploader/dropbox_uploader.sh unlink` and if you have added it key then it will prompt you to confirm its removal. If no key was found it will ask you for a new key.
+
+Confirm by running `~/Dropbox-Uploader/dropbox_uploader.sh` it should ask you for your key if you removed it or show you the following prompt if it has the key:
+
+``` console
+ $ ~/Dropbox-Uploader/dropbox_uploader.sh
+Dropbox Uploader v1.0
+Andrea Fabrizi - [email protected]
+
+Usage: /home/pi/Dropbox-Uploader/dropbox_uploader.sh [PARAMETERS] COMMAND...
+
+Commands:
+	 upload   <LOCAL_FILE/DIR ...>  <REMOTE_FILE/DIR>
+	 download <REMOTE_FILE/DIR> [LOCAL_FILE/DIR]
+	 delete   <REMOTE_FILE/DIR>
+	 move     <REMOTE_FILE/DIR> <REMOTE_FILE/DIR>
+	 copy     <REMOTE_FILE/DIR> <REMOTE_FILE/DIR>
+	 mkdir    <REMOTE_DIR>
+....
+
+```
+
+Ensure you **are not** running as sudo as this will store your api in the /root directory as `/root/.dropbox_uploader`
+
+If you ran the command with sudo the remove the old token file if it exists with either `sudo rm /root/.dropbox_uploader` or `sudo ~/Dropbox-Uploader/dropbox_uploader.sh unlink`
+
+
 
 ### Google Drive
-Coming soon.
+rclone is a program uploading to Google Drive. Install it from the menu then
+see [here](
+https://medium.com/@artur.klauser/mounting-google-drive-on-raspberry-pi-f5002c7095c2)
+for these sections:
+
+* Getting a Google Drive Client ID
+* Setting up the Rclone Configuration
+
+Note: When naming the service in `rclone config` ensure to call it "gdrive"
+
+**The Auto-mounting instructions for the drive in the link don't work on Rasbian**. Auto-mounting of the drive isn't necessary for the backup script.
 
-### rsync
-Coming soon.
+If you want your Google Drive to mount on every boot then follow the instructions at the bottom of the wiki page
 
-### Duplicati
-Coming soon.
+#### Auto-mounting Google Drive
 
-### SFTP
-Coming soon.
+To enable rclone to mount on boot you will need to make a user service. Run the following commands
+
+``` console
+$ mkdir -p ~/.config/systemd/user
+$ nano ~/.config/systemd/user/gdrive.service
+```
+Copy the following code into the editor, save and exit
+
+```
+[Unit]
+Description=rclone: Remote FUSE filesystem for cloud storage
+Documentation=man:rclone(1)
+
+[Service]
+Type=notify
+ExecStartPre=/bin/mkdir -p %h/mnt/gdrive
+ExecStart= \
+  /usr/bin/rclone mount \
+  --fast-list \
+  --vfs-cache-mode writes \
+  gdrive: %h/mnt/gdrive
+
+[Install]
+WantedBy=default.target
+```
+enable it to start on boot with: (no sudo)
+``` console
+$ systemctl --user enable gdrive.service
+```
+start with
+``` console
+$ systemctl --user start gdrive.service
+```
+if you no longer want it to start on boot then type:
+``` console
+$ systemctl --user disable gdrive.service
+```

mkdocs.yml

@@ -61,6 +61,7 @@ extra_javascript:
   - javascript/fix-codeblock-console-copy-button.js
 
 markdown_extensions:
+  - attr_list
   - pymdownx.highlight:
       pygments_lang_class: true
   - admonition