Skip to content

Commit

Permalink
Merge pull request #129 from SpiritLevel/markdown-refresh
Browse files Browse the repository at this point in the history 
ENHANCEMENT Add dev flag to keep full git repositories to enable local development
  • Loading branch information
@camfindlay
camfindlay committed Mar 28, 2016
2 parents 7a33b97 + a398687 commit 2f29194
Show file tree
Hide file tree
Showing 3 changed files with 110 additions and 98 deletions.
141 changes: 71 additions & 70 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,88 +1,89 @@
# doc.silverstripe.org

This is the source code powering http://docs.silverstripe.org. It
primarily consists of the SilverStripe
This repository contains the source code powering SilverStripe's developer documentation website https://docs.silverstripe.org.

The source code here consists primarily of the SilverStripe
[framework](https://github.com/silverstripe/silverstripe-framework)
and the [docsviewer](https://github.com/silverstripe/silverstripe-docsviewer)
module with minimal configuration.

For adding functionality or editing the style of the documentation see the
[docsviewer](http://github.com/silverstripe/silverstripe-docsviewer) module.

## Development

To set up a test instance:

* Clone this repository to a LAMP server.
* Install [Composer](http://docs.silverstripe.org/en/getting_started/composer)
* Install [sake](https://docs.silverstripe.org/en/developer_guides/cli/).
* After installing composer run `composer install --prefer-source` to grab the modules.
* Run the docs crontask in the browser `dev/tasks/UpdateDocsCronTask`
to download all fresh markdown documentation files and reindex them. Note: this
will take some time to run. Alternatively, you can use sake
to perform these tasks by firstly running the command `sake
dev/tasks/RefreshMarkdownTask flush=1` and secondly `sake
dev/tasks/RebuildLuceneDocsIndex flush=1`.
* Make sure to flush the cache for markdown content to show up.

## Source Documentation Files

Documentation for each module is stored on the filesystem via a full git clone
of the module to the `src/` subdirectory in this project. These checkouts are
ignored from this repository to allow for easier updating and to keep this
project small.

To update or download the source documentation at any time run the following
BuildTask command with sake:

cd /Sites/doc.silverstripe.org/
sake dev/tasks/RefreshMarkdownTask flush=1

This build task will download / update each module as listed in the
`app/_config/docs-repositories.yml` file. Running `sake
dev/tasks/RebuildLuceneDocsIndex flush=1` will then create a search
index and reindex the documentation to facilitate searching.

Once the build task has executed and downloaded the latest files,
those files are registered along with the module version the folder relates to
through the `app/_config/docsviewer.yml` file.

```yaml
DocumentationManifest:
register_entities:
-
Path: "src/framework_3.2/docs/"
Title: "Developer Documentation"
Version: "3.2"
Stable: true
DefaultEntity: true
```
**This repository does not contain the most current documentation files**.

Set `Stable: true` on the set of documentation relating the current stable version of SilverStripe.
The most current documentation files are not stored here but instead are
stored in each framework repository within a `docs` folder. For
example, the documentation for the master branch of the SilverStripe
framework is stored in
[https://github.com/silverstripe/silverstripe-framework/tree/master/docs](https://github.com/silverstripe/silverstripe-framework/tree/master/docs).
As described below in the Installation section, one must first obtain
and re-index fresh versions of the documentation files from the framework
repositories before being able to view the latest content.

For adding functionality or editing the style of the documentation, see the
[docsviewer](https://github.com/silverstripe/silverstripe-docsviewer) module.

## Contribution
## Installation

To contribute an improvement to the docs.silverstripe.org functionality or
theme, submit a pull request on GitHub. Any approved pull requests will make
their way onto the docs.silverstripe.org site in the next release.
To set up a local instance of `doc.silverstripe.org`:

The content for docs.silverstripe.org is stored in the modules
repository inside a "docs" folder (for example, the framework
documentation is stored at
[https://github.com/silverstripe/silverstripe-framework/tree/master/docs](https://github.com/silverstripe/silverstripe-framework/tree/master/docs).
* Install [Composer](https://docs.silverstripe.org/en/getting_started/composer).
* Install [sake](https://docs.silverstripe.org/en/developer_guides/cli).
* Clone this repository to a LAMP server. For example, the command
```
git clone https://github.com/silverstripe/doc.silverstripe.org path/to/webroot/ssdocs
```
will clone this repository into `path/to/webroot/ssdocs`.
* From within `path/to/webroot/ssdocs`, run the command
```
composer install --prefer-source
```
to grab the modules, in particular, the [docsviewer](http://github.com/silverstripe/silverstripe-docsviewer) module.
* From within `path/to/webroot/ssdocs`, run the command
```
sake dev/tasks/RefreshMarkdownTask flush=1
```
to get the latest documentation source markdown files. If you are interested in contributing, instead use the command
```
sake dev/tasks/RefreshMarkdownTask flush=1 dev=1
```
to get the full git repositories.
* From within `path/to/webroot/ssdocs`, run the command
```
sake dev/tasks/RebuildLuceneDocsIndex flush=1
```
to re-index the latest documentation source markdown files. Note: re-indexing will take some time.
* Make sure to flush the cache for markdown content to show up.

If you wish to edit the documentation content, submit a pull request on that
Github project. Updates to the content are synced regularly with
docs.silverstripe.org via a cron job.
## Automation

Refreshing and re-indexing the documentation markdown files can be automated. From within `path/to/webroot/ssdocs`, run the command:
```
sake dev/tasks/UpdateDocsCronTask
```
The cron job `UpdateDocsCronTask` runs the `RefreshMarkdownTask`
and `RebuildLuceneDocsIndex` tasks every day at 8PM. This can be
altered by editing `UpdateDocsCronTask.php` and changing the scheduling
function:
```
public function getSchedule() {
return "0 20 * * *"; // runs process() function every day at 8PM
}
```

## Cron job
## Contribution

The cron job `UpdateDocsCronTask` includes tasks that fetch the latest documentation for each module from git and rebuilds the search indexes.
To contribute an improvement to the https://docs.silverstripe.org functionality or
theme, submit a pull request on GitHub. Any approved pull requests will make
their way onto the https://docs.silverstripe.org site in the next release.

public function getSchedule() {
return "0 20 * * *"; // runs process() function every day at 8PM
}
If you wish to edit the documentation content, submit a pull request on that
Github project. Updated documentation content is regularly uploaded to https://docs.silverstripe.org via a cron job.

If you are adding a new version of the documentation, you must
register it in both `app/_config/docs-repositories.yml` and
`app/_config/docsviewer.yml`. To set one particular version to be the
current stable version, set `Stable: true` in
`app/_config/docsviewer.yml`. Remove the `Stable: true` setting for
all versions that are not stable.

## Deployment

Expand Down
56 changes: 30 additions & 26 deletions app/code/RefreshMarkdownTask.php
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,15 @@ class RefreshMarkdownTask extends BuildTask
*/
private static $documentation_repositories;

/**
* @var SS_HTTPRequest $request The request object the controller was called with.
*/
protected $request = null;

/**
* @var string
*/
protected $title = "Refresh markdown files";
protected $title = "RefreshMarkdownTask";

/**
* @var string
Expand All @@ -30,13 +35,11 @@ class RefreshMarkdownTask extends BuildTask
*/
public function run($request)
{
$this->printLine("refreshing markdown files...");

$this->request = $request;
$this->printLine("Refreshing markdown files...");
$repositories = $this->getRepositories();

foreach ($repositories as $repository) {
$this->cloneRepository($repository);
$this->cleanRepository($repository);
}
}

Expand Down Expand Up @@ -64,7 +67,6 @@ private function printLine($message)
* Returns the array of repos to source markdown docs from
*
* @return array
*
*/
private function getRepositories()
{
Expand All @@ -78,40 +80,42 @@ private function getRepositories()
}

/**
* @param array $repository
* Clone $repository which contains the most current documentation source markdown files
*
* @todo test this works with all modules
* @param array $repository
*/
private function cloneRepository(array $repository)
{

list($remote, $folder, $branch) = $repository;

$path = $this->getPath();

exec("mkdir -p {$path}/src");
exec("rm -rf {$path}/src/{$folder}_{$branch}");

$this->printLine("cloning " . $remote . "/" . $branch);

chdir("{$path}/src");
exec("git clone -q https://github.com/{$remote}.git {$folder}_{$branch} --depth 1 --branch {$branch} --single-branch");

chdir("{$path}/src/{$folder}_{$branch}");
}

/**
* Clears out any non markdown files stored in assets
*
* @param array $repository
*/
private function cleanRepository(array $repository)
{
$paths = array_merge(glob("*"), glob(".*"));
// If the dev=1 flag is used when RefreshMarkdownTask is run, a full git clone of the framework repository is kept
// to enable local development of framework and doc.silverstripe.org from within doc.silverstripe.org. Otherwise,
// only a shallow clone of the framework repository is made and all non-markdown files deleted, only allowing viewing
// of documentation files. Note: The --depth 1 option in the git clone command implies the option --single-branch

foreach ($paths as $path) {
if ($path !== "docs" && $path !== "." && $path !== "..") {
exec("rm -rf {$path}");
if( $this->request->getVar('dev') ) {
$this->printLine("Performing full clone of " . $remote . "/" . $branch);
exec("git clone -q https://github.com/{$remote}.git {$folder}_{$branch} --branch {$branch}");
} else {
$this->printLine("Performing shallow clone of " . $remote . "/" . $branch);
exec("git clone -q https://github.com/{$remote}.git {$folder}_{$branch} --branch {$branch} --depth 1");
$this->printLine("Deleting non-documentation framework files from shallow clone of " . $remote . "/" . $branch);
chdir("{$path}/src/{$folder}_{$branch}");
$framework_paths = array_merge(glob("*"), glob(".*"));
foreach ($framework_paths as $framework_path) {
if ( $framework_path !== "docs" && $framework_path !=="." && $framework_path !==".." ) {
exec("rm -rf {$framework_path}");
}
}
}

}

}
11 changes: 9 additions & 2 deletions app/code/UpdateDocsCronTask.php
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,11 @@

class UpdateDocsCronTask implements CronTask
{
/**
* @var SS_HTTPRequest $request The request object the controller was called with.
*/
protected $request = null;

/**
* @return string
*/
Expand All @@ -15,14 +20,16 @@ public function getSchedule()
*/
public function process()
{

$this->request = Controller::curr()->getRequest();

//refresh markdown files
$refresh_task = new RefreshMarkdownTask();
$refresh_task->run(null);
$refresh_task->run($this->request);

//reindex markdown files
$reindex_task = new RebuildLuceneDocsIndex();
$reindex_task->run(null);
$reindex_task->run($this->request);

}
}

0 comments on commit 2f29194

Please sign in to comment.