Move documentation to docs/

TSDBBench · Jul 5, 2017 · 464053e · 464053e
1 parent d741bcf
commit 464053e
Show file tree

Hide file tree

Showing 32 changed files with 179 additions and 172 deletions.
diff --git a/README.md b/README.md
@@ -9,145 +9,5 @@ Different cluster sizes and configurations as well as different workloads can be
 
 The benchmark is done with [YCSB-TS](https://github.com/TSDBBench/YCSB-TS).
 
-## Supported databases
-* Timeseries databases with a Requirement on NoSQL DBMS:
-  * [Axibase](docs/tsdb/axibase.md)
-  * [Blueflood](docs/tsdb/blueflood.md)
-  * [Databus](docs/tsdb/databus.md)
-  * [KairosDB](docs/tsdb/kairosdb.md)
-  * [NewTS](docs/tsdb/newts.md)
-  * [OpenTSDB](docs/tsdb/opentsdb.md)
-  * [Rhombus](docs/tsdb/rhombus.md)
-  * [Graphite](docs/tsdb/graphite.md)
-* Timeseries databases with no Requirement on any DBMS:
-  * [Akumuli](docs/tsdb/akumuli.md)
-  * [Druid](docs/tsdb/druid.md)
-  * [InfluxDB](docs/tsdb/influxdb.md)
-  * [Gnocchi](docs/tsdb/gnocchi.md)
-  * [Seriesly](docs/tsdb/seriesly.md)
-  * [Prometheus](docs/tsdb/prometheus.md)
-* Column-oriented DBMS
-  * [MonetDB](docs/tsdb/monetdb.md)
-  * [Kdb+](docs/tsdb/kdpplus.md)
-* Relational databases:
-  * [MySQL](docs/tsdb/mysql.md)
-  * [PostgreSQL](docs/tsdb/postgresql.md)
-* Other:
-  * [Elasticsearch](docs/tsdb/elasticsearch.md)
-  * [h5serv](docs/tsdb/h5serv.md)
-
-## Unsupported databases (WIP)
-*  [DalmatinerDB](docs/tsdb/dalmatinerdb.md)
-*  [ScyllaDB](docs/tsdb/scylladb.md)
-*  [RiakTS](docs/tsdb/riakts.md)
-*  [Redis](docs/tsdb/redis.md)
-
-## Supported Elastic Infrastrctures
-* [VMware vSphere](docs/ei/vsphere.md)
-* [OpenStack](docs/ei/openstack.md)
-* [VirtualBox](docs/ei/virtualbox.md)
-* [DigitalOcean](docs/ei/digitalocean.md)
-* [Amazon Web Services](docs/ei/aws.md)
-
-## Initial Setup of the Elastic Infrastrcture
-* This must done only once and only for one elastic infrastructure
-* All images are available at http://tsdbbench.allweathercomputing.com/bin/, can use them directly from there.
-* See the articles for the five supported elastic infrastructures for specific instructions
-    * [VMware vSphere](docs/ei/vsphere.md)
-    * [OpenStack](docs/ei/openstack.md)
-    * [VirtualBox](docs/ei/virtualbox.md)
-    * [DigitalOcean](docs/ei/digitalocean.md)
-    * [Amazon Web Services](docs/ei/aws.md)
-
-## Initial Setup of local pc or Control-VM
-* To control TSDBBench, a local pc or a Control-VM (a VM with everything preinstalled) can be used (only vSphere and OpenStack)
-1. Local PC:
-    1. Install packages:
-        ```bash
-        sudo apt-get install python-dateutil python-jinja2 python-numpy python-pandas python-flask python-redis python-requests python-six python-tornado python-werkzeug python-markupsafe python-greenlet python-zmq python-yaml python-pip wkhtmltopdf python-magic fabric vagrant zlib1g-dev zlib1g libxml2 libxml2-dev libxslt1.1 libxslt1-dev python-webcolors python-pyvmomi
-        ```
-    2. Install pip packages:
-        ```bash
-        sudo pip install bokeh python-vagrant pdfkit
-        ```
-    3. Install vagrant plugins:
-        ```bashvagrant plugin install vagrant-vsphere
-        vagrant plugin install vagrant-openstack-provider
-        ```
-    4. Reconfigure locales and make sure that en_US.UTF-8 is generated
-        ```bash
-        sudo dpkg-reconfigure locales
-        ```
-    5. Checking out & Prepairing Git Repo
-        ```bash
-        cd /path/to/some/folder/
-        git clone https://github.com/baderas/TSDBBench
-        cd TSDBBench
-        vagrant box add --name dummy dummy.box
-        copy hooks/pre-commit .git/hooks/
-        cd ..
-        ```
-    6. Edit config for the chosen elastic infrastructure (change everything that says '' for your chosen elastic infrastructure)
-        ```bash
-        cd TSDBBench
-        nano vagrant_files/vagrantconf.rb
-        nano vagrant_files/vagrantconf_db.rb
-        nano vagrant_files/vagrantconf_gen.rb
-        ```
-2. Control-VM
-    1. Create Control-VM according to [VMware vSphere](docs/ei/vsphere.md) or [OpenStack](docs/ei/openstack.md)
-    2. Login to your Control-VM
-    3. Edit config for the chosen elastic infrastructure (change everything that says '' for your chosen elastic infrastructure)
-        ```bash
-        cd TSDBBench
-        nano vagrant_files/vagrantconf.rb
-        nano vagrant_files/vagrantconf_db.rb
-        nano vagrant_files/vagrantconf_gen.rb
-        ```
-
-## Running a testworkload
- - without creation of html file:
-    ```bash
-    cd TSDBBench
-    ./TSDBBench.py -t /path/to/some/tmpfolder -f vagrant_files -d mysql_cl1_rf1 --provider 'vsphere' -w "testworkloada" -l```
- - with creation of html file:
-    ```bash
-    cd TSDBBench
-    ./TSDBBench.py -t /path/to/some/tmpfolder -f vagrant_files -d mysql_cl1_rf1 --provider 'vsphere' -w "testworkloada" -l -m```
- - with creation of html files and multiple databases:
-    ```bash
-    cd TSDBBench
-    ./TSDBBench.py -t /path/to/some/tmpfolder -f vagrant_files -d mysql_cl1_rf1 postgresql_cl1_rf1 --provider 'vsphere' -w "testworkloada" -l -m --provider "vsphere"```
-
-## Creating html files (when not using -m)
- - Creating a html file from a ycsb_*.log file:
-    ```bash
-    cd TSDBBench
-    ./ProcessYcsbLog.py -f some_ycsb_logfile.log
-    ```
- - Creating a html file from a ycsb_*.ydc file:
-    ```bash
-    cd TSDBBench
-    ./ProcessYcsbLog.py -f some_ycsb_logfile.ydc
-    ```
- - Creating a combined html file a set of from a ycsb_*.ydc/.log files:
-    ```bash
-    cd TSDBBench
-    ./ProcessYcsbLog.py -f some_ycsb_logfile1.ydc ome_ycsb_logfile2.log ome_ycsb_logfile3.ydc ...
-    ```
-
-## Additional Information
-* Everything was tested and used on Debian Jessie x64, but should work on Ubuntu.
-    * Ubuntu has different package names for a lot of the packages, you need to find and change them
-* Logfiles/Benchmark Results are stored compressed as .ydc Files 
-
-## Development Information
-* Development specific details on databases and elastic infrastructures can be found on their specific files (see links at the beginning)
-* [Adding a New Database](docs/dev/adding_database.md)
-
-## Related Work
-* [Comparison of TSDBs (Andreas Bader)](http://www2.informatik.uni-stuttgart.de/cgi-bin/NCSTRL/NCSTRL_view.pl?id=DIP-3729&mod=0&engl=0&inst=FAK)
-* [Survey and Comparison of Open Source TSDBs (Andreas Bader, Oliver Kopp, Michael Falkenthal)](http://www2.informatik.uni-stuttgart.de/cgi-bin/NCSTRL/NCSTRL_view.pl?id=INPROC-2017-06&mod=0&engl=0&inst=IPVS)
-* [Ultimate Comparison of TSDBs](https://tsdbbench.github.io/Ultimate-TSDB-Comparison/)
-* [TSDBBench Box at atlas](https://atlas.hashicorp.com/TSDBBench/boxes/tsdbbench_dummy.box)
-* [YCSB-TS](https://github.com/TSDBBench/YCSB-TS)
+See (docs/index.md)[docs/index.md] for a full documentation.
+This documentation is also rendered at http://tsdbbench.github.io/Overlord/.
diff --git a/docs/dev/adding_database.md b/docs/dev/adding_database.md
@@ -23,4 +23,5 @@
       * E.g., TSDBBench/vagrant_files/generator/files/databases/opentsdb_cl1_rf1.py
   3. Add a Client (if not existing) to YCSB-TS
     * See https://github.com/TSDBBench/YCSB-TS
-[Back to README.md](../../README.md)
+
+[back](../)
diff --git a/docs/ei/aws.md b/docs/ei/aws.md
@@ -7,4 +7,4 @@
 * For AWS a existing default debian image is used, so no specific image creation/preparation is required
     * As a drawback a lot of things must be installed on each run, which takes more time than using an preinstalled image
 
-[Back to README.md](../../README.md)
+[back](../../)
diff --git a/docs/ei/digitalocean.md b/docs/ei/digitalocean.md
@@ -7,4 +7,4 @@
 * For DigitalOcean a existing default debian image is used, so no specific image creation/preparation is required
     * As a drawback a lot of things must be installed on each run, which takes more time than using an preinstalled image
 
-[Back to README.md](../../README.md)
+[back](../../)
diff --git a/docs/ei/openstack.md b/docs/ei/openstack.md
@@ -43,4 +43,4 @@
 11. `./MakeDebianQcow2.py -i /path/to/outputfolder/debian-8.8.0-amd64-netinst-openstack-controlvm-autoinstall.iso -f /path/to/outputfolder/`
 12. Upload the resulting file to your Openstack Storage
 
-[Back to README.md](../../README.md)
+[back](../../)
diff --git a/docs/ei/virtualbox.md b/docs/ei/virtualbox.md
@@ -26,4 +26,4 @@
     12. Import the new Box with Vagrant: `vagrant box add tsdbbench-debian /path/to/outputfolder/debian-8.8.0-amd64-netinst-virtualbox-autoinstall.box`
 
 
-[Back to README.md](../../README.md)
+[back](../../)
diff --git a/docs/ei/vsphere.md b/docs/ei/vsphere.md
@@ -44,4 +44,4 @@
 6. `./MakeDebianIso.py -t /path/to/tmpfolder -i /path/to/another/folder/debian-8.8.0-amd64-netinst.iso -f /path/to/outputfolder/ -p preseed-vsphere-controlvm.cfg`
 7. Import the resulting iso file to your vSphere datastore, create a new VM and use the iso image to install the Control-VM on it
 
-[Back to README.md](../../README.md)
+[back](../../)
diff --git a/docs/index.md b/docs/index.md
@@ -0,0 +1,146 @@
+# Overlord
+
+This is the Python and Vagrant part of TSDBBench
+
+## Supported databases
+* Timeseries databases with a requirement on NoSQL DBMS:
+  * [Axibase](tsdb/axibase)
+  * [Blueflood](tsdb/blueflood)
+  * [Databus](tsdb/databus)
+  * [KairosDB](tsdb/kairosdb)
+  * [NewTS](tsdb/newts)
+  * [OpenTSDB](tsdb/opentsdb)
+  * [Rhombus](tsdb/rhombus)
+  * [Graphite](tsdb/graphite)
+* Timeseries databases with no Requirement on any DBMS:
+  * [Akumuli](tsdb/akumuli)
+  * [Druid](tsdb/druid)
+  * [InfluxDB](tsdb/influxdb)
+  * [Gnocchi](tsdb/gnocchi)
+  * [Seriesly](tsdb/seriesly)
+  * [Prometheus](tsdb/prometheus)
+* Column-oriented DBMS
+  * [MonetDB](tsdb/monetdb)
+  * [Kdb+](tsdb/kdpplus)
+* Relational databases:
+  * [MySQL](tsdb/mysql)
+  * [PostgreSQL](tsdb/postgresql)
+* Other:
+  * [Elasticsearch](tsdb/elasticsearch)
+  * [h5serv](tsdb/h5serv)
+
+## Unsupported databases (WIP)
+*  [DalmatinerDB](tsdb/dalmatinerdb)
+*  [ScyllaDB](tsdb/scylladb)
+*  [RiakTS](tsdb/riakts)
+*  [Redis](tsdb/redis)
+
+## Supported Elastic Infrastructures
+* [VMware vSphere](ei/vsphere)
+* [OpenStack](ei/openstack)
+* [VirtualBox](ei/virtualbox)
+* [DigitalOcean](ei/digitalocean)
+* [Amazon Web Services](ei/aws)
+
+## Initial Setup of the Elastic Infrastructure
+* This must done only once and only for one elastic infrastructure
+* All images are available at http://tsdbbench.allweathercomputing.com/bin/, can use them directly from there.
+* See the articles for the five supported elastic infrastructures for specific instructions
+    * [VMware vSphere](ei/vsphere)
+    * [OpenStack](ei/openstack)
+    * [VirtualBox](ei/virtualbox)
+    * [DigitalOcean](ei/digitalocean)
+    * [Amazon Web Services](ei/aws)
+
+## Initial Setup of local pc or Control-VM
+* To control TSDBBench, a local pc or a Control-VM (a VM with everything preinstalled) can be used (only vSphere and OpenStack)
+1. Local PC:
+    1. Install packages:
+        ```bash
+        sudo apt-get install python-dateutil python-jinja2 python-numpy python-pandas python-flask python-redis python-requests python-six python-tornado python-werkzeug python-markupsafe python-greenlet python-zmq python-yaml python-pip wkhtmltopdf python-magic fabric vagrant zlib1g-dev zlib1g libxml2 libxml2-dev libxslt1.1 libxslt1-dev python-webcolors python-pyvmomi
+        ```
+    2. Install pip packages:
+        ```bash
+        sudo pip install bokeh python-vagrant pdfkit
+        ```
+    3. Install vagrant plugins:
+        ```bashvagrant plugin install vagrant-vsphere
+        vagrant plugin install vagrant-openstack-provider
+        ```
+    4. Reconfigure locales and make sure that en_US.UTF-8 is generated
+        ```bash
+        sudo dpkg-reconfigure locales
+        ```
+    5. Checking out & Prepairing Git Repo
+        ```bash
+        cd /path/to/some/folder/
+        git clone https://github.com/baderas/TSDBBench
+        cd TSDBBench
+        vagrant box add --name dummy dummy.box
+        copy hooks/pre-commit .git/hooks/
+        cd ..
+        ```
+    6. Edit config for the chosen elastic infrastructure (change everything that says '' for your chosen elastic infrastructure)
+        ```bash
+        cd TSDBBench
+        nano vagrant_files/vagrantconf.rb
+        nano vagrant_files/vagrantconf_db.rb
+        nano vagrant_files/vagrantconf_gen.rb
+        ```
+2. Control-VM
+    1. Create Control-VM according to [VMware vSphere](ei/vsphere) or [OpenStack](ei/openstack)
+    2. Login to your Control-VM
+    3. Edit config for the chosen elastic infrastructure (change everything that says '' for your chosen elastic infrastructure)
+        ```bash
+        cd TSDBBench
+        nano vagrant_files/vagrantconf.rb
+        nano vagrant_files/vagrantconf_db.rb
+        nano vagrant_files/vagrantconf_gen.rb
+        ```
+
+## Running a testworkload
+ - without creation of html file:
+    ```bash
+    cd TSDBBench
+    ./TSDBBench.py -t /path/to/some/tmpfolder -f vagrant_files -d mysql_cl1_rf1 --provider 'vsphere' -w "testworkloada" -l```
+ - with creation of html file:
+    ```bash
+    cd TSDBBench
+    ./TSDBBench.py -t /path/to/some/tmpfolder -f vagrant_files -d mysql_cl1_rf1 --provider 'vsphere' -w "testworkloada" -l -m```
+ - with creation of html files and multiple databases:
+    ```bash
+    cd TSDBBench
+    ./TSDBBench.py -t /path/to/some/tmpfolder -f vagrant_files -d mysql_cl1_rf1 postgresql_cl1_rf1 --provider 'vsphere' -w "testworkloada" -l -m --provider "vsphere"```
+
+## Creating html files (when not using -m)
+ - Creating a html file from a ycsb_*.log file:
+    ```bash
+    cd TSDBBench
+    ./ProcessYcsbLog.py -f some_ycsb_logfile.log
+    ```
+ - Creating a html file from a ycsb_*.ydc file:
+    ```bash
+    cd TSDBBench
+    ./ProcessYcsbLog.py -f some_ycsb_logfile.ydc
+    ```
+ - Creating a combined html file a set of from a ycsb_*.ydc/.log files:
+    ```bash
+    cd TSDBBench
+    ./ProcessYcsbLog.py -f some_ycsb_logfile1.ydc ome_ycsb_logfile2.log ome_ycsb_logfile3.ydc ...
+    ```
+
+## Additional Information
+* Everything was tested and used on Debian Jessie x64, but should work on Ubuntu.
+    * Ubuntu has different package names for a lot of the packages, you need to find and change them
+* Logfiles/Benchmark Results are stored compressed as .ydc Files 
+
+## Development Information
+* Development specific details on databases and elastic infrastructures can be found on their specific files (see links at the beginning)
+* [Adding a New Database](dev/adding_database)
+
+## Related Work
+* [Comparison of TSDBs (Andreas Bader)](http://www2.informatik.uni-stuttgart.de/cgi-bin/NCSTRL/NCSTRL_view.pl?id=DIP-3729&mod=0&engl=0&inst=FAK)
+* [Survey and Comparison of Open Source TSDBs (Andreas Bader, Oliver Kopp, Michael Falkenthal)](http://www2.informatik.uni-stuttgart.de/cgi-bin/NCSTRL/NCSTRL_view.pl?id=INPROC-2017-06&mod=0&engl=0&inst=IPVS)
+* [Ultimate Comparison of TSDBs](https://tsdbbench.github.io/Ultimate-TSDB-Comparison/)
+* [TSDBBench Box at atlas](https://atlas.hashicorp.com/TSDBBench/boxes/tsdbbench_dummy.box)
+* [YCSB-TS](https://github.com/TSDBBench/YCSB-TS)
diff --git a/docs/tsdb/akumuli.md b/docs/tsdb/akumuli.md
@@ -4,4 +4,4 @@
 
 * https://github.com/akumuli/Akumuli
 
-[Back to README.md](../../README.md)
+[back](../../)
diff --git a/docs/tsdb/axibase.md b/docs/tsdb/axibase.md
@@ -4,4 +4,4 @@
 
 * https://axibase.com/
 
-[Back to README.md](../../README.md)
+[back](../../)
diff --git a/docs/tsdb/blueflood.md b/docs/tsdb/blueflood.md
@@ -30,4 +30,4 @@
 * Go on with the [10 Minuten Guide](https://github.com/rackerlabs/blueflood/wiki/10minuteguide)
 
 
-[Back to README.md](../../README.md)
+[back](../../)
diff --git a/docs/tsdb/dalmatinerdb.md b/docs/tsdb/dalmatinerdb.md
@@ -28,4 +28,4 @@
   * vi etc/dalmatinerdb.conf # check the settings and adjust if needed
   * ./bin/ddb start
 
-[Back to README.md](../../README.md)
+[back](../../)
diff --git a/docs/tsdb/databus.md b/docs/tsdb/databus.md
@@ -4,4 +4,4 @@
 
 * https://github.com/linkedin/databus
 
-[Back to README.md](../../README.md)
+[back](../../)
diff --git a/docs/tsdb/druid.md b/docs/tsdb/druid.md
@@ -88,4 +88,4 @@
 * https://github.com/boneill42/druid-vagrant
 * http://www.datascienceassn.org/sites/default/files/users/user1/DruidDataIngest_0.6.62.pdf
 
-[Back to README.md](../../README.md)
+[back](../../)
diff --git a/docs/tsdb/elasticsearch.md b/docs/tsdb/elasticsearch.md
@@ -4,4 +4,4 @@
 
 * https://www.elastic.co/de/products/elasticsearch
 
-[Back to README.md](../../README.md)
+[back](../../)
diff --git a/docs/tsdb/gnocchi.md b/docs/tsdb/gnocchi.md
@@ -4,4 +4,4 @@
 
 * http://gnocchi.xyz/
 
-[Back to README.md](../../README.md)
+[back](../../)
diff --git a/docs/tsdb/graphite.md b/docs/tsdb/graphite.md
@@ -4,4 +4,4 @@
 
 * http://graphite.readthedocs.io/en/latest/
 
-[Back to README.md](../../README.md)
+[back](../../)
diff --git a/docs/tsdb/h5serv.md b/docs/tsdb/h5serv.md
@@ -70,4 +70,4 @@
 * http://www.pytables.org/usersguide/libref/structured_storage.html#tables.Table.where
 * https://code.google.com/archive/p/numexpr/
 
-[Back to README.md](../../README.md)
+[back](../../)
diff --git a/docs/tsdb/influxdb.md b/docs/tsdb/influxdb.md
@@ -20,4 +20,4 @@
     * see https://github.com/influxdb/influxdb/issues/2702
     * Fix: Using seconds instead of millisceonds, checking if the range does not exceed 100000 seconds
 
-[Back to README.md](../../README.md)
+[back](../../)
Original file line number	Diff line number	Diff line change
Expand Up		@@ -26,4 +26,4 @@
		12. Import the new Box with Vagrant: `vagrant box add tsdbbench-debian /path/to/outputfolder/debian-8.8.0-amd64-netinst-virtualbox-autoinstall.box`


		[Back to README.md](../../README.md)
		[back](../../)
Original file line number	Diff line number	Diff line change
Expand Up		@@ -4,4 +4,4 @@

		* https://github.com/akumuli/Akumuli

		[Back to README.md](../../README.md)
		[back](../../)
Original file line number	Diff line number	Diff line change
Expand Up		@@ -4,4 +4,4 @@

		* https://axibase.com/

		[Back to README.md](../../README.md)
		[back](../../)
Original file line number	Diff line number	Diff line change
Expand Up		@@ -30,4 +30,4 @@
		* Go on with the [10 Minuten Guide](https://github.com/rackerlabs/blueflood/wiki/10minuteguide)


		[Back to README.md](../../README.md)
		[back](../../)
Original file line number	Diff line number	Diff line change
Expand Up		@@ -4,4 +4,4 @@

		* https://github.com/linkedin/databus

		[Back to README.md](../../README.md)
		[back](../../)
Original file line number	Diff line number	Diff line change
Expand Up		@@ -4,4 +4,4 @@

		* https://www.elastic.co/de/products/elasticsearch

		[Back to README.md](../../README.md)
		[back](../../)
Original file line number	Diff line number	Diff line change
Expand Up		@@ -4,4 +4,4 @@

		* http://gnocchi.xyz/

		[Back to README.md](../../README.md)
		[back](../../)
Original file line number	Diff line number	Diff line change
Expand Up		@@ -4,4 +4,4 @@

		* http://graphite.readthedocs.io/en/latest/

		[Back to README.md](../../README.md)
		[back](../../)