quiz-data - Update documentation

classes/import_quiz.php

@@ -261,7 +261,8 @@ public function get_curl_request($wsurl): curl_request {
 
     /**
      * Get quiz data from file, convert question file locations to ids
-     * and then import to Moodle.
+     * and then import to Moodle. This needs to be called twice - once
+     * to create the quiz and then again to add the questions.
      *
      * @return void
      */

cli/importquizstructuretomoodle.php

@@ -78,6 +78,7 @@
         'default' => null,
         'variable' => 'coursename',
         'valuerequired' => true,
+        'hidden' => true,
     ],
     [
         'longopt' => 'instanceid',
@@ -86,6 +87,7 @@
         'default' => null,
         'variable' => 'instanceid',
         'valuerequired' => true,
+        'hidden' => true,
     ],
     [
         'longopt' => 'token',

cli/importwholecoursetomoodle.php

@@ -85,23 +85,24 @@
     [
         'longopt' => 'coursename',
         'shortopt' => 'c',
-        'description' => 'Unique course name for course or module context.',
+        'description' => 'Unique course name.',
         'default' => null,
         'variable' => 'coursename',
         'valuerequired' => true,
     ],
     [
         'longopt' => 'modulename',
         'shortopt' => 'm',
-        'description' => 'Unique (within course) module name for module context.',
+        'description' => 'Not used',
         'default' => null,
         'variable' => 'modulename',
         'valuerequired' => true,
+        'hidden' => true
     ],
     [
         'longopt' => 'instanceid',
         'shortopt' => 'n',
-        'description' => 'Numerical id of the course, module of course category.',
+        'description' => 'Numerical id of the course.',
         'default' => null,
         'variable' => 'instanceid',
         'valuerequired' => true,

doc/createwholecourserepo.md

@@ -0,0 +1,41 @@
+# Creating a repo for a course and all its quizzes
+
+## Prerequisites
+- Set up the [webserver](webservicesetup.md) on the Moodle instance.
+- Set up your [local machine](localsetup.md).
+
+## Exporting
+- From the commandline in the `cli` folder run `createwholecourserepo.php`. There are a number of options you can input. List them all with `php createwholecourserepo.php -h`. You can use -shortname or --longname on the command line followed by a space and the value.
+
+|Short|Long|Required value|
+|-|-|-|
+|i|moodleinstance|Key of Moodle instance in $moodleinstances to use. Should match end of instance URL.|
+|r|rootdirectory|Directory on user's computer containing repos.|
+|d|directory|Directory of repo on user's computer containing "top" folder, relative to root directory.|
+|s|subcategory|Relative subcategory of repo to actually export.|
+|c|coursename|Unique course name for course or module context.
+|q|questioncategoryid|Numerical id of subcategory to actually export.
+|n|instanceid|Numerical id of the course category.
+|t|token|Security token for webservice.
+|h|help|
+|x|ignorecat|Regex of categories to ignore - add an extra leading / for Windows.
+
+This is very similar to [`createrepo.php`](createrepo.md) but normally Gitsync retrieves questions within a Moodle context, returning all or a subselection of question categories with the repo directory structure matching the category structure in Moodle. Courses and quizzes are in separate contexts, however. `createwholecourserepo.php` keep quizzes with their parent course in a single repo.
+
+You can use all the same command line arguments as a basic `createrepo` to narrow down the questions exported. You will need to supply either the course name or its ID. You will also need to supply the destination directory for the course - this should be a child directory of the main repo directory.
+
+Example:
+
+Assume you have correct information in config.php, i.e. the Moodle URL in `$moodleinstances`, and the webservice token stored in `$token`, the default moodle instance in `$instance` and the local root directory for your question files in `$rootdirectory`.
+
+Assume you have a course called "Scratch" in Moodle. You would like all questions from the "top" level, and all sub-categories, to become files in a sub-directory "scratch-wholecourse/scratch-course" of your local `$rootdirectory` directory. You would also like to export the questions belonging to all the course's quiz contexts and the structures of those quizzes into "scratch-wholecourse".  
+
+Create and initialise the "scratch-wholecourse" directory using `git init scratch-wholecourse` then run `php createwholecourserepo.php`.
+
+`php createwholecourserepo.php -c "Scratch" -d "scratch-wholecourse/scratch-course" `
+
+Along with the "scratch-course" directory, there should be a directory created by the script for each of the quizzes with names in the format "scratch-course_quiz_sanitised-quiz-name".
+
+### On failure
+
+- If the script fails, discard changes in the repository (e.g. with the `git reset` command) and run the `php createwholecourserepo.php` again once the issue has been dealt with.

doc/exportquizstructurefrommoodle.md

@@ -0,0 +1,36 @@
+# Updating a quiz repo with the latest version of a the quiz structure
+
+## Prerequisites
+- Set up the [webserver](webservicesetup.md) on the Moodle instance.
+- Set up your [local machine](localsetup.md).
+- Import a quiz repo into Moodle using `importrepotomoodle.php` or create a quiz context repo from Moodle using `createrepo.php`.
+
+## Exporting
+- From the commandline in the `cli` folder run `exportquizstructurefrommoodle.php`. There are a number of options you can input. List them all with `php exportquizstructurefrommoodle.php -h`. You can use -shortname or --longname on the command line followed by a space and the value.
+
+|Short|Long|Required value|
+|-|-|-|
+|i|moodleinstance|Key of Moodle instance in $moodleinstances to use. Should match end of instance URL.|
+|r|rootdirectory|Directory on user's computer containing repos.|
+|p|nonquizmanifestpath|Filepath of manifest file of another context relative to root directory.|
+|f|quizmanifestpath|Filepath of quiz manifest file relative to root directory.|
+|t|token|Security token for webservice.|
+|h|help|
+
+Examples:
+
+`php exportquizstructurefrommoodle.php -r "C:\question_repos" -f "quiz_1\instance2_module_stack1_quiz-1_question_manifest.json"`
+
+The manifest file should have been created by createrepo.php if the repo was initially created by exporting questions from Moodle or importrepotomoodle.php if questions were initially imported into Moodle from an existing repo.
+
+The structure of the quiz will be extracted into a file with a name in the format quiz-name_quiz.json. The file contains the layout of the quiz and relative file paths to the questions (`quizfilepath`).
+
+If the quiz contains questions from a context other than its own (most-likely the course context), you will need to include the path to the manifest of the other context (--nonquizmanifestpath). Those questions will have a relative filepath for the other repo (`nonquizfilepath`).
+
+The questions must either be in the quiz context or the context of the nonquizmanifest. If you have questions in a third location, the export will fail.
+
+If you update the quiz structure in Moodle you can export the structure again and the structure file in the repo will be updated.
+
+### On failure
+
+- If the script fails, discard changes in the repository (e.g. with the `git reset` command) and run the `php exportquizstructurefrommoodle.php` again once the issue has been dealt with.

doc/exportwholecoursefrommoodle.md

@@ -0,0 +1,36 @@
+# Creating a repo for a course and all its quizzes from Moodle
+
+## Prerequisites
+- Set up the [webserver](webservicesetup.md) on the Moodle instance.
+- Set up your [local machine](localsetup.md).
+- Import a repo into Moodle using `importwholecoursetomoodle.php` or create a repo from Moodle using `createwholecourserepo.php`.
+
+## Exporting
+- From the commandline in the `cli` folder run `exportwholecoursefrommoodle.php`. There are a number of options you can input. List them all with `php exportwholecoursefrommoodle.php -h`. You can use -shortname or --longname on the command line followed by a space and the value.
+
+|Short|Long|Required value|
+|-|-|-|
+|i|moodleinstance|Key of Moodle instance in $moodleinstances to use. Should match end of instance URL.|
+|r|rootdirectory|Directory on user's computer containing repos.|
+|f|manifestpath|Filepath of manifest file relative to root directory.|
+|s|subcategory|Relative subcategory of repo to actually export.|
+|q|questioncategoryid|Numerical id of subcategory to actually export.
+|t|token|Security token for webservice.|
+|h|help|
+|x|ignorecat|Regex of categories to ignore - add an extra leading / for Windows.
+
+Examples:
+
+This is very similar to [`exportrepofrommoodle.php`](exportrepofrommoodle.md) but normally Gitsync retrieves questions within a Moodle context, returning all or a subselection of question categories with the repo directory structure matching the category structure in Moodle. Courses and quizzes are in separate contexts, however. `exportwholecoursefrommoodle.php` keep quizzes with their parent course in a single repo.
+
+`php exportwholecoursefrommoodle.php -f "scratch-wholecourse\scratch-course\moodle1_course_course-1_question_manifest.json"`
+
+The manifest file should have been created by `createwholecourserepo.php` if the repo was initially created by exporting questions from Moodle or `importwholecoursetomoodle.php` if questions were initially imported into Moodle from an existing repo. When creating a whole course manifest, additional information is stored in the manifest linking the local directories of the quizzes to the instances of the quizzes in Moodle.
+
+If you only want to export a certain question category (and its subcategories) within the course you will need to supply the category's name relative to the 'top' category e.g. 'category 1/subcategory 2'. Alternatively you can supply the questioncategoryid which is available in the URL ('&category=XXX') when browsing the category in the question bank.
+
+Export will only be possible if there are no uncommitted changes in the repo. After the export, the manifest will be tidied to remove any entries where the question is no longer in the Moodle. (The manifest is the link between your repo and Moodle and you can't link to something which isn't there.)
+
+### On failure
+
+- If the script fails, discard changes in the repository (e.g. with the `git reset` command) and run the `php exportwholecoursefrommoodle.php` again once the issue has been dealt with. All questions will be exported afresh.

doc/importquizstructuretomoodle.md

@@ -0,0 +1,35 @@
+# Importing a quiz structure into Moodle
+
+## Prerequisites
+- Set up the [webserver](webservicesetup.md) on the Moodle instance.
+- Set up your [local machine](localsetup.md).
+- Create a blank quiz in Moodle and then import any questions using `importrepotomoodle.php`.
+
+## Importing
+- From the commandline in the `cli` folder run `importquizstructuretomoodle.php`. There are a number of options you can input. List them all with `php importquizstructuretomoodle.php -h`. You can use -shortname or --longname on the command line followed by a space and the value.
+
+|Short|Long|Required value|
+|-|-|-|
+|i|moodleinstance|Key of Moodle instance in $moodleinstances to use. Should match end of instance URL.|
+|r|rootdirectory|Directory on user's computer containing repos.|
+|p|nonquizmanifestpath|Filepath of manifest file of another context relative to root directory.|
+|f|quizmanifestpath|Filepath of quiz manifest file relative to root directory.|
+|a|quizdatapath|Filepath of quiz data file relative to root directory.
+|t|token|Security token for webservice.|
+|h|help|
+
+Examples:
+
+`php importquizstructuretomoodle.php -r "C:\question_repos" -f "quiz_1\instance2_module_stack1_quiz-1_question_manifest.json"`
+
+The manifest files should have been created by createrepo.php if the repo was initially created by exporting questions from Moodle or importrepotomoodle.php if questions were initially imported into Moodle from an existing repo.
+
+The quizdata file contains the structure of the quiz and the locations of the questions, the quizmanifest and nonquizmanifest link those locations to matching questions in Moodle. You must supply at least one of `quizmanifestpath` and `quizdatapath`. If you supply just one, Gitsync will calculate the location and name of the other based and its normal naming convention. If the quiz contains questions from a context other than its own (most-likely the course context), you will need to include the path to the manifest of the other context (`--nonquizmanifestpath`). In the quizdata file, those questions will have a relative filepath for the other repo (`nonquizfilepath`).
+
+If you have multiple quiz structure files, you will need to supply both `quizmanifestpath` and `quizdatapath`. (This is if you're using the same questions in different quizzes in Moodle but only want to have one question repo.)
+
+If you update the quiz structure in Moodle you can export the structure again and the structure file in the repo will be updated. If you update the quiz structure in the repo and want to update Moodle, create a new quiz and start again.
+
+### On failure
+
+- If the script fails, delete the quiz in Moodle and start again.

doc/importwholecoursetomoodle.md

@@ -0,0 +1,71 @@
+# Importing a course and all its quizzes into Moodle
+
+## Prerequisites
+- Set up the [webserver](webservicesetup.md) on the Moodle instance.
+- Set up your [local machine](localsetup.md).
+
+## Importing
+This is very similar to `importrepotomoodle.php` but normally Gitsync retrieves questions within a Moodle context, returning all or a subselection of question categories with the repo directory structure matching the category structure in Moodle. Courses and quizzes are in separate contexts, however. `importwholecoursetomoodle.php` and the matching repo creation and export tools keep quizzes with their parent course in a single repo.
+
+- Create a repository where the top-most directory contains question directories for the course and each of its quizzes. Within each of these context directories, store your questions with a folder hierarchy akin to question categories i.e. top/category/subcategory. Alternativelty, use `createwholecourserepo.php` to create a repo from existing questions on Moodle.
+- Each category below top should have a `gitsync_category.xml` file containing a 'category question' with the details of he category. See the `testrepoparent` folder in this repository for an example.
+- If you have created a repository (or copied one) make sure you have a `.gitignore` file and it ignores gitsync's manifest and temporary files.  
+`touch .gitignore`  
+`printf '%s\n' '**/*_question_manifest.json' '**/*_manifest_update.tmp' >> .gitignore`  
+Commit this update.  
+(This will be done automatically when using `createrepo.php`.)
+- From the commandline in the `cli` folder run `importwholecoursetomoodle.php`. There are a number of options you can input. List them all with `php importwholecoursetomoodle.php -h`. You can use -shortname or --longname on the command line followed by a space and the value.
+
+|Short|Long|Required value|
+|-|-|-|
+|i|moodleinstance|Key of Moodle instance in  moodleinstances to use. Should match end of instance URL.|
+|r|rootdirectory|Directory on user's computer containing repos.|
+|f|manifestpath|Filepath of manifest file relative to root directory.|
+|d|directory|Directory of repo on users computer containing "top" folder, relative to root directory.|
+|s|subdirectory|Relative subdirectory of repo to actually import.|
+|c|coursename|Unique course name.
+|n|instanceid|Numerical id of the course.
+|t|token|Security token for webservice.
+|h|help|
+|x|ignorecat|Regex of categories to ignore - add an extra leading / for Windows.
+
+Examples:
+
+Example 1:
+To update the questions in Moodle from the repo, point the script to your course manifest file. (Quiz structures will not be updated.)
+`php importrepotomoodle.php -f "scratch-wholecourse/scratch-course"`
+
+Example 2:
+To import course context questions, quiz context questions and quiz structures into a course for the first time, you will need to point the script to the course directory and the course to import into (using either coursename or id).
+`php importwholecoursetomoodle.php -r "C:\question_repos" -d "scratch-wholecourse/scratch-course" -c "Course 2"`
+
+## Scenario 1: Importing questions into Moodle from an existing repo
+
+e.g. You have exported questions from one Moodle instance to create the repo and you want to import them into a different instance
+
+Importing will create a manifest file specific to the Moodle instance and context in each of the context directories of the repo. This links files in the repo to specific questionbankentries in the Moodle instance.
+
+You will need to specify the course you want to import the questions and quizzes into. You will need to supply the course name. Alternatively, the instanceid of the course can be supplied. This is available from the URL while browsing the course in Moodle ('?id=XXX').
+
+If you only want to import a certain question category (and its subcategories) within the course you will need to supply the path of the corresponding folder within your repo relative to the 'top' category e.g. 'category-1/subcategory-2'.
+
+## Scenario 2: Re-importing questions into Moodle when the manifest files already exist
+
+You should specify the course manifest file path and context will be extracted from that. You can still enter a subdirectory to only re-import some of the questions.
+
+Import will only be possible if there are not updates to the questions in Moodle which haven't been exported.
+
+Only questions that have changed in the repo since the last import will be imported to Moodle (to avoid creating a new version in Moodle when nothing has changed).
+
+Quiz structure will not be updated.
+
+## Deletion
+
+A check will be run to see if there are questions in the manifest for each context that do not have a file in the repo. These will be listed.
+
+A check will be run to see if there are questions in the course and quizzes in Moodle that are not in the manifest. These will be listed.
+
+To delete the questions from Moodle and tidy the manifest, run `deletefrommoodle.php` for the course or an individual quiz.
+
+### On failure
+- If the script fails, it can be safely run again once the issue has been dealt with. Pending updates to the manifest file are stored in a temporary file in the root directory and these will be picked up at the start of the new run, avoiding multiple new versions of a question being created in Moodle.

doc/usinggit.md

@@ -7,6 +7,11 @@ It is recommended you at least skim the whole of this document before attempting
 
 For detailed information on what happens at each stage of the process, see [Process Details](processdetails.md).
 
+If you want to track a course and all its quizzes in a single repo, you will need to use different scripts that function in a similar way to the single context scripts:
+- Creating a repo - [createwholecourserepo.php](createwholecourserepo.md)
+- Importing questions to Moodle - [importwholecoursetomoodle.php](importwholecoursetomoodle.md)
+- Exporting questions from Moodle - [exportwholecoursefrommoodle.php](exportwholecoursefrommoodle.md)
+
 ## Maintaining a one-to-one link between a Moodle instance and a Git repo
 
 ### Creating a Git repo from questions in Moodle