-
Notifications
You must be signed in to change notification settings - Fork 4
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Fixing & Updating Readme; Adding documentation (Merging Readme branch)
- Loading branch information
Showing
31 changed files
with
844 additions
and
117 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,135 +1,151 @@ | ||
| # TSDBBench: Overlord | ||
| [](https://travis-ci.org/TSDBBench/Overlord) | ||
|
|
||
| The Python and Vagrant part of TSDBBench | ||
|
|
||
| The VMs are running on vSphere or Openstack, but you can choose to run TSDBBench.py on your local pc or on a Control VM. | ||
|
|
||
| ## Setup Elastic Infrastructure | ||
| For your elastic infrastructure you need image(s) as bases for vagrant to derive generator and tsdb vms from. You can choose wheter you want to use vSohere or Openstack, you don't need both. | ||
|
|
||
| 1. Create/Copy images: | ||
| - All images are available at http://nemarcontrolvm.iaas.uni-stuttgart.de/isos/, you can use them directly from there (no need to download to your local pc!). | ||
| - for vSphere: | ||
| - Copy http://nemarcontrolvm.iaas.uni-stuttgart.de/isos/debian-8.1.0-amd64-netinst-vsphere-autoinstall.iso to your vSphere datastore | ||
| - for Openstack: | ||
| - Copy http://nemarcontrolvm.iaas.uni-stuttgart.de/isos/debian-8.1.0-amd64-netinst-openstack-autoinstall.qcow2 to your images in Openstack | ||
| - If you still want to create own images (changed preseed file e.g.) (run on your local pc): | ||
| - for vSphere: | ||
|
|
||
| cd /path/to/another/folder/ | ||
| wget http://cdimage.debian.org/debian-cd/8.2.0/amd64/iso-cd/debian-8.2.0-amd64-netinst.iso | ||
| cd /path/to/some/folder/ | ||
| git clone https://github.com/baderas/TSDBBench | ||
| cd TSDBBench | ||
| ./MakeDebianIso.py -t /path/to/tmpfolder -i /path/to/another/folder/debian-8.2.0-amd64-netinst.iso -f /path/to/outputfolder/ -p preseed-vsphere.cfg | ||
| - for Openstack | ||
|
|
||
| sudo apt-get install qemu-kvm libvirt-bin | ||
| sudo gpasswd -a $USER kvm | ||
| sudo gpasswd -a $USER libvirt | ||
| sudo reboot (or logout and login) | ||
| cd /path/to/another/folder/ | ||
| wget http://cdimage.debian.org/debian-cd/8.2.0/amd64/iso-cd/debian-8.2.0-amd64-netinst.iso | ||
| cd /path/to/some/folder/ | ||
| git clone https://github.com/baderas/TSDBBench | ||
| cd TSDBBench | ||
| ./MakeDebianIso.py -t /path/to/tmpfolder -i /path/to/another/folder/debian-8.2.0-amd64-netinst.iso -f /path/to/outputfolder/ -p preseed-openstack.cfg | ||
| ./MakeDebianQcow2.py -i /path/to/outputfolder/debian-8.2.0-amd64-netinst-openstack-autoinstall.iso -f /path/to/outputfolder/ | ||
| upload the resulting file to your Openstack Storage | ||
| 2. (Only for vSphere) Create templates (run on your local pc): | ||
|
|
||
| cd /path/to/some/folder/ | ||
| git clone https://github.com/baderas/TSDBBench | ||
| cd TSDBBench | ||
| nano vagrant_files/vagrantconf.rb (add everything that says '') | ||
| nano vagrant_files/vagrantconf_db.rb (add everything that says '') | ||
| nano vagrant_files/vagrantconf_gen.rb (add everything that says '') | ||
| ./pySetupVsphere.py -f /path/to/some/folder/TSDBBench/vagrant_files/ | ||
|
|
||
| ## Setup | ||
| Everything was tested and used on Debian Jessie x64, but should work on Ubuntu. But Ubuntu has different package names for a lot of the packages, you need to find and change them! | ||
| ### Choice 1: local pc with openstack | ||
| 1. Install packages: | ||
|
|
||
| # Overlord | ||
| [](https://travis-ci.org/TSDBBench/TSDBBench) | ||
|
|
||
| The Python and Vagrant part of TSDBBench that automatically setups and benchmarks tims series databases (TSDBs). | ||
|
|
||
| The benchmarking is done with VMs that are automatically setup by Vagrant and are running on one of the five supported elastic infrastructures (EIs). | ||
|
|
||
| Different cluster sizes and configurations as well as different workloads can be used to benchmark one or more TSDBs. | ||
|
|
||
| The benchmark is done with [YCSB-TS](https://github.com/TSDBBench/YCSB-TS). | ||
|
|
||
| ## Supported databases | ||
| * Timeseries databases with a Requirement on NoSQL DBMS: | ||
| * [Axibase](docs/tsdb/axibase.md) | ||
| * [Blueflood](docs/tsdb/blueflood.md) | ||
| * [Databus](docs/tsdb/databus.md) | ||
| * [KairosDB](docs/tsdb/kairosdb.md) | ||
| * [NewTS](docs/tsdb/newts.md) | ||
| * [OpenTSDB](docs/tsdb/opentsdb.md) | ||
| * [Rhombus](docs/tsdb/rhombus.md) | ||
| * [Graphite](docs/tsdb/graphite.md) | ||
| * Timeseries databases with no Requirement on any DBMS: | ||
| * [Akumuli](docs/tsdb/akumuli.md) | ||
| * [Druid](docs/tsdb/druid.md) | ||
| * [InfluxDB](docs/tsdb/influxdb.md) | ||
| * [Gnocchi](docs/tsdb/gnocchi.md) | ||
| * [Seriesly](docs/tsdb/seriesly.md) | ||
| * [Prometheus](docs/tsdb/prometheus.md) | ||
| * Column-oriented DBMS | ||
| * [MonetDB](docs/tsdb/monetdb.md) | ||
| * [Kdb+](docs/tsdb/kdpplus.md) | ||
| * Relational databases: | ||
| * [MySQL](docs/tsdb/mysql.md) | ||
| * [PostgreSQL](docs/tsdb/postgresql.md) | ||
| * Other: | ||
| * [Elasticsearch](docs/tsdb/elasticsearch.md) | ||
| * [h5serv](docs/tsdb/h5serv.md) | ||
|
|
||
| ## Unsupported databases (WIP) | ||
| * [DalmatinerDB](docs/tsdb/dalmatinerdb.md) | ||
| * [ScyllaDB](docs/tsdb/scylladb.md) | ||
| * [RiakTS](docs/tsdb/riakts.md) | ||
| * [Redis](docs/tsdb/redis.md) | ||
|
|
||
| ## Supported Elastic Infrastrctures | ||
| * [VMware vSphere](docs/ei/vsphere.md) | ||
| * [OpenStack](docs/ei/openstack.md) | ||
| * [VirtualBox](docs/ei/virtualbox.md) | ||
| * [DigitalOcean](docs/ei/digitalocean.md) | ||
| * [Amazon Web Services](docs/ei/aws.md) | ||
|
|
||
| ## Initial Setup of the Elastic Infrastrcture | ||
| * This must done only once and only for one elastic infrastructure | ||
| * All images are available at http://tsdbbench.allweathercomputing.com/bin/, can use them directly from there. | ||
| * See the articles for the five supported elastic infrastructures for specific instructions | ||
| * [VMware vSphere](docs/ei/vsphere.md) | ||
| * [OpenStack](docs/ei/openstack.md) | ||
| * [VirtualBox](docs/ei/virtualbox.md) | ||
| * [DigitalOcean](docs/ei/digitalocean.md) | ||
| * [Amazon Web Services](docs/ei/aws.md) | ||
|
|
||
| ## Initial Setup of local pc or Control-VM | ||
| * To control TSDBBench, a local pc or a Control-VM (a VM with everything preinstalled) can be used (only vSphere and OpenStack) | ||
| 1. Local PC: | ||
| 1. Install packages: | ||
| ```bash | ||
| sudo apt-get install python-dateutil python-jinja2 python-numpy python-pandas python-flask python-redis python-requests python-six python-tornado python-werkzeug python-markupsafe python-greenlet python-zmq python-yaml python-pip wkhtmltopdf python-magic fabric vagrant zlib1g-dev zlib1g libxml2 libxml2-dev libxslt1.1 libxslt1-dev python-webcolors python-pyvmomi | ||
| 2. Install pip packages: | ||
|
|
||
| ``` | ||
| 2. Install pip packages: | ||
| ```bash | ||
| sudo pip install bokeh python-vagrant pdfkit | ||
| 3. Install vagrant plugins: | ||
|
|
||
| vagrant plugin install vagrant-vsphere | ||
| ``` | ||
| 3. Install vagrant plugins: | ||
| ```bashvagrant plugin install vagrant-vsphere | ||
| vagrant plugin install vagrant-openstack-provider | ||
| 4. Reconfigure locales and make sure that en_US.UTF-8 is generated | ||
|
|
||
| ``` | ||
| 4. Reconfigure locales and make sure that en_US.UTF-8 is generated | ||
| ```bash | ||
| sudo dpkg-reconfigure locales | ||
| 5. Checking out & Prepairing Git Repo | ||
|
|
||
| ``` | ||
| 5. Checking out & Prepairing Git Repo | ||
| ```bash | ||
| cd /path/to/some/folder/ | ||
| git clone https://github.com/baderas/TSDBBench | ||
| cd TSDBBench | ||
| vagrant box add TSDBBench/tsdbbench_dummy.box --provider openstack | ||
| vagrant box add --name dummy dummy.box | ||
| copy hooks/pre-commit .git/hooks/ | ||
| cd .. | ||
| 6. Edit Config (change everything that says '') | ||
|
|
||
| cd /path/to/some/folder/TSDBBench | ||
| ``` | ||
| 6. Edit config for the chosen elastic infrastructure (change everything that says '' for your chosen elastic infrastructure) | ||
| ```bash | ||
| cd TSDBBench | ||
| nano vagrant_files/vagrantconf.rb | ||
| nano vagrant_files/vagrantconf_db.rb | ||
| nano vagrant_files/vagrantconf_gen.rb | ||
| ### Choice 2: Control VM | ||
| - Coming soon... | ||
| ``` | ||
| 2. Control-VM | ||
| 1. Create Control-VM according to [VMware vSphere](docs/ei/vsphere.md) or [OpenStack](docs/ei/openstack.md) | ||
| 2. Login to your Control-VM | ||
| 3. Edit config for the chosen elastic infrastructure (change everything that says '' for your chosen elastic infrastructure) | ||
| ```bash | ||
| cd TSDBBench | ||
| nano vagrant_files/vagrantconf.rb | ||
| nano vagrant_files/vagrantconf_db.rb | ||
| nano vagrant_files/vagrantconf_gen.rb | ||
| ``` | ||
|
|
||
| ## Running a testworkload | ||
| - without creating html file: | ||
|
|
||
| cd /path/to/some/folder/TSDBBench | ||
| ./TSDBBench.py -t /path/to/tmpfolder -f /path/to/some/folder/TSDBBench/vagrant_files -d mysql1 -l --provider 'vsphere' -w "testworkloada" | ||
| - with creating html file: | ||
|
|
||
| cd /path/to/some/folder/TSDBBench | ||
| ./TSDBBench.py -t /path/to/tmpfolder -f /path/to/some/folder/TSDBBench/vagrant_files -d mysql1 -l --provider 'vsphere' -w "testworkloada" -m | ||
|
|
||
| - without creation of html file: | ||
| ```bash | ||
| cd TSDBBench | ||
| ./TSDBBench.py -t /path/to/some/tmpfolder -f vagrant_files -d mysql_cl1_rf1 --provider 'vsphere' -w "testworkloada" -l``` | ||
| - with creation of html file: | ||
| ```bash | ||
| cd TSDBBench | ||
| ./TSDBBench.py -t /path/to/some/tmpfolder -f vagrant_files -d mysql_cl1_rf1 --provider 'vsphere' -w "testworkloada" -l -m``` | ||
| - with creation of html files and multiple databases: | ||
| ```bash | ||
| cd TSDBBench | ||
| ./TSDBBench.py -t /path/to/some/tmpfolder -f vagrant_files -d mysql_cl1_rf1 postgresql_cl1_rf1 --provider 'vsphere' -w "testworkloada" -l -m --provider "vsphere"``` | ||
|
|
||
| ## Creating html files (when not using -m) | ||
| - Creating a html file from a ycsb_*.log file: | ||
|
|
||
| cd /path/to/some/folder/TSDBBench | ||
| ./ProcessYcsbLog.py -f some_ycsb_logfile.log | ||
| ```bash | ||
| cd TSDBBench | ||
| ./ProcessYcsbLog.py -f some_ycsb_logfile.log | ||
| ``` | ||
| - Creating a html file from a ycsb_*.ydc file: | ||
|
|
||
| cd /path/to/some/folder/TSDBBench | ||
| ./ProcessYcsbLog.py -f some_ycsb_logfile.ydc | ||
|
|
||
| ```bash | ||
| cd TSDBBench | ||
| ./ProcessYcsbLog.py -f some_ycsb_logfile.ydc | ||
| ``` | ||
| - Creating a combined html file a set of from a ycsb_*.ydc/.log files: | ||
|
|
||
| cd /path/to/some/folder/TSDBBench | ||
| ./ProcessYcsbLog.py -f some_ycsb_logfile1.ydc ome_ycsb_logfile2.log ome_ycsb_logfile3.ydc ... | ||
| ## Steps to add a new TSDB: | ||
| for this example consider your new tsdb would be opentsdb: | ||
| 1. Add Vagrantfiles: | ||
| 1. Create at least one folder in /path/to/some/folder/TSDBBench/vagrant_files | ||
| - e.g. opentsdb1 | ||
| 2. Create at least one Vagrantfile in this new folder | ||
| - named same as the folder but with _ + number + .vagrant as suffix | ||
| - e.g. /path/to/some/folder/TSDBBench/vagrant_files/opentsdb1/opentsdb1_0 | ||
| 3. In this Vagrantfile put deployment tasks like installing and configuring | ||
| - but nothing where you need to know IPs from other nodes of the cluster (that comes later) | ||
| 4. Use basic scripts as much as possible | ||
| - see /path/to/some/folder/TSDBBench/vagrant_files/basic | ||
| 2. Add the python part | ||
| 1. Add a python file to /path/to/some/folder/TSDBBench/databases | ||
| - named like the folder + .py (.e.g. opentsdb1.py) | ||
| 2. In this file you add: | ||
| 1. Deployment tasks that requires IPs from all nodes of the cluster | ||
| 2. Checks that the database is running | ||
| 3. Some basic db configs (everything that starts with db_) | ||
| 3. Look at other database python files, there are comments that explain every field in every file | ||
| 3. Testing | ||
| - You need to copy the database files to /path/to/some/folder/TSDBBench/vagrant_files/generator/files/databases/ | ||
| - You can run either run "hooks/pre-commit" or "git commit -a" (runs the hook as well if setup correctly). | ||
|
|
||
| ## Related Projects | ||
| * [VPS Benchmarks](http://www.vpsbenchmarks.com/) - measurement of VPS performance | ||
| ```bash | ||
| cd TSDBBench | ||
| ./ProcessYcsbLog.py -f some_ycsb_logfile1.ydc ome_ycsb_logfile2.log ome_ycsb_logfile3.ydc ... | ||
| ``` | ||
|
|
||
| ## Additional Information | ||
| * Everything was tested and used on Debian Jessie x64, but should work on Ubuntu. | ||
| * Ubuntu has different package names for a lot of the packages, you need to find and change them | ||
| * Logfiles/Benchmark Results are stored compressed as .ydc Files | ||
|
|
||
| ## Development Information | ||
| * Development specific details on databases and elastic infrastructures can be found on their specific files (see links at the beginning) | ||
| * [Adding a New Database](dev/adding_database.md) | ||
|
|
||
| ## Related Work | ||
| * [Comparison of TSDBs (Andreas Bader)](http://www2.informatik.uni-stuttgart.de/cgi-bin/NCSTRL/NCSTRL_view.pl?id=DIP-3729&mod=0&engl=0&inst=FAK) | ||
| * [Survey and Comparison of Open Source TSDBs (Andreas Bader, Oliver Kopp, Michael Falkenthal)](http://www2.informatik.uni-stuttgart.de/cgi-bin/NCSTRL/NCSTRL_view.pl?id=INPROC-2017-06&mod=0&engl=0&inst=IPVS) | ||
| * [Ultimate Comparison of TSDBs](https://tsdbbench.github.io/Ultimate-TSDB-Comparison/) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,23 @@ | ||
| # Adding a New Database | ||
| * for this example consider your new tsdb would be opentsdb: | ||
| 1. Add Vagrantfiles: | ||
| 1. Create at least one folder in TSDBBench/vagrant_files | ||
| * E.g. opentsdb_cl1_rf1 (cl = cluster size, rf = replication factor) | ||
| 2. Create at least one Vagrantfile in this new folder | ||
| * Named same as the folder but with _ + number + .vagrant as suffix | ||
| * Each file is one VM that is created and provisioned | ||
| * E.g. TSDBBench/vagrant_files/opentsdb_cl1_rf1/opentsdb_cl1_rf1_0.vagrant | ||
| 3. In this Vagrantfile put deployment tasks like installing and configuring | ||
| * But nothing where you need to know IPs from other nodes of the cluster (that comes in a later step) | ||
| * Can't be done with Vagrant | ||
| * Consider files in basic folder that can be reused | ||
| * Eee TSDBBench/vagrant_files/basic/opentsdb.rb for example | ||
| 2. Add the python part | ||
| 1. Add a python file to TSDBBench/vagrant_files/generator/files/databases/ | ||
| * Named like the folder + .py (.e.g. opentsdb_cl1_rf1.py) | ||
| 2. In this file add: | ||
| 1. Deployment tasks that requires IPs from all nodes of the cluster | ||
| 2. Checks that the database is running | ||
| 3. Some basic db configs (everything that starts with db_) | ||
| 3. Look at other database python files, there are comments that explain every field in every file | ||
| * E.g., TSDBBench/vagrant_files/generator/files/databases/opentsdb_cl1_rf1.py |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| # Amazon Web Services (AWS) | ||
|
|
||
| ## Links | ||
| * https://aws.amazon.com | ||
|
|
||
| ## Information | ||
| * For AWS a existing default debian image is used, so no specific image creation/preparation is required | ||
| * As a drawback a lot of things must be installed on each run, which takes more time than using an preinstalled image | ||
|
|
||
| [Back to README.md](../../README.md) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| # DigitalOcean | ||
|
|
||
| ## Links | ||
| * https://www.digitalocean.com/ | ||
|
|
||
| ## Information | ||
| * For DigitalOcean a existing default debian image is used, so no specific image creation/preparation is required | ||
| * As a drawback a lot of things must be installed on each run, which takes more time than using an preinstalled image | ||
|
|
||
| [Back to README.md](../../README.md) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,46 @@ | ||
| # OpenStack | ||
|
|
||
| ## Links | ||
| * https://www.openstack.org/ | ||
|
|
||
| ## Information | ||
| * for OpenStack qcow2 images are sued | ||
| * the images are created from the autoninstalling iso files | ||
| * the iso files for OpenStack have only minor differences to those of vSphere, see the corresponding preseed files. | ||
|
|
||
| ## Initial Elastic Infrastructure Setup | ||
| 1. Copy http://tsdbbench.allweathercomputing.com/bin/debian-8.8.0-amd64-netinst-vsphere-autoinstall.qcow2 to your images in Openstack | ||
| 2. (optional) If you want a Control-VM, do the following: | ||
| 3. Copy http://tsdbbench.allweathercomputing.com/bin/debian-8.8.0-amd64-netinst-vsphere-controlvm-autoinstall.qcow2 to your images in OpenStack | ||
| 4. Create a new VM using the Control-VM image | ||
|
|
||
| ## Creating Your Own Images (Generator/Database) | ||
| * If you want to create your own images, do the following (example on Debian Jessie): | ||
| 1. `sudo apt-get install qemu-kvm libvirt-bin` | ||
| 2. `sudo gpasswd -a $USER kvm` | ||
| 3. `sudo gpasswd -a $USER libvirt` | ||
| 4. `sudo reboot` (or logout and login) | ||
| 5. `cd /path/to/another/folder/` | ||
| 6. `wget http://cdimage.debian.org/debian-cd/8.8.0/amd64/iso-cd/debian-8.8.0-amd64-netinst.iso` | ||
| 7. `cd /path/to/some/folder/` | ||
| 8. `git clone https://github.com/baderas/TSDBBench` | ||
| 9. `cd TSDBBench` | ||
| 10. `./MakeDebianIso.py -t /path/to/tmpfolder -i /path/to/another/folder/debian-8.8.0-amd64-netinst.iso -f /path/to/outputfolder/ -p preseed-openstack.cfg` | ||
| 11. `./MakeDebianQcow2.py -i /path/to/outputfolder/debian-8.8.0-amd64-netinst-openstack-autoinstall.iso -f /path/to/outputfolder/` | ||
| 12. Upload the resulting file to your Openstack Storage | ||
|
|
||
| ## Creating your own Control-VM Image | ||
| 1. `sudo apt-get install qemu-kvm libvirt-bin` | ||
| 2. `sudo gpasswd -a $USER kvm` | ||
| 3. `sudo gpasswd -a $USER libvirt` | ||
| 4. `sudo reboot` (or logout and login) | ||
| 5. `cd /path/to/another/folder/` | ||
| 6. `wget http://cdimage.debian.org/debian-cd/8.8.0/amd64/iso-cd/debian-8.8.0-amd64-netinst.iso` | ||
| 7. `cd /path/to/some/folder/` | ||
| 8. `git clone https://github.com/baderas/TSDBBench` | ||
| 9. `cd TSDBBench` | ||
| 10. `./MakeDebianIso.py -t /path/to/tmpfolder -i /path/to/another/folder/debian-8.8.0-amd64-netinst.iso -f /path/to/outputfolder/ -p preseed-openstack-controlvm.cfg` | ||
| 11. `./MakeDebianQcow2.py -i /path/to/outputfolder/debian-8.8.0-amd64-netinst-openstack-controlvm-autoinstall.iso -f /path/to/outputfolder/` | ||
| 12. Upload the resulting file to your Openstack Storage | ||
|
|
||
| [Back to README.md](../../README.md) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| # Oracle VirtualBox | ||
|
|
||
| ## Links | ||
| * https://www.virtualbox.org/ | ||
|
|
||
| ## Information | ||
| * Virtualbox was used as first locally running development plattform | ||
| * A specific BaseBox must exist in Vagrant (comparable to templates in vSphere or qcow2 images in OpenStack) | ||
| * Steps how to create this BaseBox are missing | ||
|
|
||
| [Back to README.md](../../README.md) |
Oops, something went wrong.