Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Vagrant, CI and Postman additions #21

Open
wants to merge 87 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
87 commits
Select commit Hold shift + click to select a range
3c4ab01
Added Vagrantfile and resources to install a SciPass in a VM
chrissmall22 Nov 13, 2014
d4f9885
Removed unused puppet templates
chrissmall22 Nov 13, 2014
081613e
Removed OVS compile since it is currently broken in OVS
chrissmall22 Nov 13, 2014
174735c
Added SciPass git and ryu dependancies
chrissmall22 Nov 13, 2014
d90a3e6
Added Mininet SciPass XML config file
chrissmall22 Nov 13, 2014
6dceb23
w
chrissmall22 Nov 13, 2014
124cb5c
Added postman API ref
chrissmall22 Nov 13, 2014
33a514c
Added Postman and Ryu startup info to Readme
chrissmall22 Nov 13, 2014
722e528
Fixed mininet config in Readme
chrissmall22 Nov 13, 2014
3feca36
Added ryu compile from source to puppet manifests
chrissmall22 Nov 13, 2014
837f6d2
Created tests based on Mininet ids and interfaces
chrissmall22 Nov 13, 2014
499c176
Fixed scipass directory in puppet manifest
chrissmall22 Nov 13, 2014
5e9cbe0
Added .idea to .gitignore
chrissmall22 Nov 13, 2014
a2fd3eb
Merge branch 'odl' of git://github.com/chrissmall22/SciPass into odl
chrissmall22 Nov 13, 2014
254d6b2
Use https for git clones in manifests
chrissmall22 Nov 13, 2014
1d80883
Merge branch 'odl' of git://github.com/chrissmall22/SciPass into odl
chrissmall22 Nov 13, 2014
82bcbc2
Added to coverage files to .gitignore
chrissmall22 Nov 13, 2014
b7ba57f
Removed SciPass-mininet.py
chrissmall22 Nov 14, 2014
d657915
Merge branch 'odl' of git://github.com/chrissmall22/SciPass into odl
chrissmall22 Nov 14, 2014
f557107
Mininet test changes
chrissmall22 Nov 14, 2014
5bd5bce
Fix mininet test port numbers
chrissmall22 Nov 14, 2014
baa8f1b
Moved port numbers in SciPassTest_mininet/py
chrissmall22 Nov 14, 2014
26c3b29
One more mininet test port change
chrissmall22 Nov 14, 2014
75250a7
ODL initial checkin
chrissmall22 Nov 14, 2014
0a775eb
Added OpeyDaylight module
chrissmall22 Nov 15, 2014
87b7edc
Created HPVAN class
chrissmall22 Nov 20, 2014
9b183fe
Removed some Ryu dependant code form HPVAN
chrissmall22 Nov 20, 2014
cc3fc9b
Removed more unused code from HPVAN
chrissmall22 Nov 20, 2014
0dd70fc
Use metaclasses for HPVAN
chrissmall22 Nov 21, 2014
e0c5f67
Deleted OpenDayLight
chrissmall22 Nov 21, 2014
36b44ae
Added Circle.yml
chrissmall22 Nov 21, 2014
65818f9
Fixed typo in circle.yml
chrissmall22 Nov 21, 2014
b2462ba
Added build_dir to circle.yml
chrissmall22 Nov 21, 2014
a38586e
Added requirements.txt
chrissmall22 Nov 21, 2014
02ef431
Added requirements.txt to python dir to see if Circlci will find it
chrissmall22 Nov 21, 2014
9107efa
More requirements added
chrissmall22 Nov 21, 2014
822da0d
Added libxml2 to requirements
chrissmall22 Nov 21, 2014
7a07d69
Added apt-get install python-libxml2 to circle.yml
chrissmall22 Nov 21, 2014
8770766
Added sudo apt-get install
chrissmall22 Nov 21, 2014
71c1ec8
More circle.yml
chrissmall22 Nov 21, 2014
5a9e442
Added coveralls badge to Readme
Nov 21, 2014
bb31f96
Added .travis.yml file
chrissmall22 Nov 21, 2014
ffe82bc
Travis-ci badge in Readme
chrissmall22 Nov 21, 2014
64e0670
Fixed Travis image link
Nov 21, 2014
33a01bf
Changed builds to 2.7 and 3.3 Python in travis.yml
chrissmall22 Nov 21, 2014
e27cb0e
Added 2.6 and changed the coverage call in the MakeFile
chrissmall22 Nov 21, 2014
c10f379
Add Coveralls install to .travis.yml
chrissmall22 Nov 21, 2014
02d8dfc
Fixed typo in travis.yml
chrissmall22 Nov 21, 2014
91e2520
Removed Python 3.3 because ipaddr module won't build
chrissmall22 Nov 21, 2014
e20c79e
Removed lxml from requirements
chrissmall22 Nov 21, 2014
ee67d30
Added libxml2-dev
chrissmall22 Nov 22, 2014
084c4b8
Added python-dev and libxstl-dev to travis
chrissmall22 Nov 22, 2014
8c32ee5
More python dependanicies for the travis file
chrissmall22 Nov 22, 2014
ed6dcab
Added python-lxml to travis.yml
chrissmall22 Nov 22, 2014
e99a2c4
Remove packages to try to debug libxml2 issues
chrissmall22 Nov 22, 2014
454a749
Remove the python-libxml2 dependancy
chrissmall22 Nov 22, 2014
eef49d5
ipddr version change
chrissmall22 Nov 22, 2014
fe39893
Build libxml python from source in .travis.yml
chrissmall22 Nov 22, 2014
b74b700
Fixed cd in .travis.xml
chrissmall22 Nov 22, 2014
836f9d4
Moved coverage call to .travis.yml file
chrissmall22 Nov 22, 2014
d633886
Moved HPVAN back to a metaclass
chrissmall22 Nov 22, 2014
690ae2d
Added test_struct file to test class reorg
chrissmall22 Nov 23, 2014
4944b6e
Fixed circle.yml
chrissmall22 Nov 23, 2014
5a866d5
Trying new HPVAN setup with init call
chrissmall22 Nov 23, 2014
afacb60
Fixed SciPassRest start_rest call
chrissmall22 Nov 23, 2014
6f09129
Removed libslt2 from circl.yml dependancies
chrissmall22 Nov 23, 2014
fd8e9b8
Added REST calls
chrissmall22 Nov 23, 2014
9fea916
Added new REST interface test
chrissmall22 Nov 24, 2014
6313cec
Moved to nosetests in travis instead of coverage
chrissmall22 Nov 24, 2014
9066949
Added Python 3.x to travis builds
chrissmall22 Nov 24, 2014
6b1ea1b
Added WebOb to requirements
chrissmall22 Nov 24, 2014
858eb7b
Added oslo,config to requirements
chrissmall22 Nov 24, 2014
c40b88d
Fixed oslo.config requirements.txt entry
chrissmall22 Nov 24, 2014
8c5ff25
Removed 3.x Python builds from travis because libxml2 bindings don't …
chrissmall22 Nov 24, 2014
fe61f4b
Removed HP-VAN and ODL related files from master branch
chrissmall22 Nov 24, 2014
8bd208f
Moved back to coverage tests instead of nosetests in .travis.yml
chrissmall22 Nov 25, 2014
489314f
Removed HPVAN from SciTest_mininet.py
chrissmall22 Nov 25, 2014
59218a7
Pointed Coveralls badge at master branch
Nov 25, 2014
b020ba7
Moved Vagrant manifest to use the GlobalNOC git repository
chrissmall22 Nov 25, 2014
a626b2c
Vagrant VM pulls from GlobalNOC source
chrissmall22 Nov 25, 2014
e0b5496
Use python -m coverage so this works on both RH and ubuntu
chrissmall22 Nov 25, 2014
67e9af3
Use sudo when installing ryu in Vagrant VM
chrissmall22 Nov 25, 2014
23c7f9a
Added tox to requirements
chrissmall22 Nov 25, 2014
aeca39a
Fixed path for copying SciPass.xml file
chrissmall22 Nov 26, 2014
4cf62e8
Deleted test-OpenDayLight.py
chrissmall22 Nov 26, 2014
840ba6c
Makefile tests for mininet and to allow it to run on Windows
chrissmall22 Dec 11, 2014
9e25631
Merge branch 'master' of https://github.com/chrissmall22/SciPass
chrissmall22 Dec 11, 2014
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1 +1,7 @@
*.pyc
*.pyc
.vagrant/*
.idea/*
*,cover
coverage.xml
python/.coverage
python/htmlcov
29 changes: 29 additions & 0 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
language: python

branches:
only:
- master
- hp-van

python:
- "2.6"
- "2.7"


before_install:
- sudo apt-get update -qq
- sudo apt-get install -y libxml2-dev python-libxml2
- wget ftp://xmlsoft.org/libxml2/python/libxml2-python-2.6.15.tar.gz
- tar -xzvf libxml2-python-2.6.15.tar.gz
- cd libxml2-python-2.6.15 && python setup.py install
install:
- cd ..
- pip install -r python/requirements.txt
- pip install coveralls

script:
- cd python
- coverage run --source=./ --omit=__init__.py,*Test.py,t/SciPass.py t/SciPassTest_mininet.py; coverage report -m; python -m coverage xml;coverage annotate; coverage html;

after_success:
coveralls
7 changes: 5 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,13 @@ clean:
rm -rf dist

test:
cd python; coverage run --source=./ --omit=__init__.py,*Test.py,t/SciPass.py t/Test.py; coverage report -m; coverage xml;coverage annotate; coverage html;
cd python; python -m coverage run --source=./ --omit=__init__.py,*Test.py,t/SciPass.py t/SciPassTest.py; python -m coverage report -m; python -m coverage xml;python -m coverage annotate; python -m coverage html;

test_mininet:
cd python; python -m coverage run --source=./ --omit=__init__.py,*Test.py,t/SciPass.py t/SciPassTest_mininet.py; coverage report -m; python -m coverage xml;python -m coverage annotate; python -m coverage html;

dist:
rm -rf dist/$(NAME)-$(VERSION)
mkdir -p dist/$(NAME)-$(VERSION)
cp -r etc/ python/ SciPass.spec dist/$(NAME)-$(VERSION)/
cd dist; tar -czvf $(NAME)-$(VERSION).tar.gz $(NAME)-$(VERSION)/ --exclude .svn
cd dist; tar -czvf $(NAME)-$(VERSION).tar.gz $(NAME)-$(VERSION)/ --exclude .svn
51 changes: 50 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,53 @@
SciPass
=======

SciPass is a SDN powered Science DMZ and IDS Load Balancer.
[![Travis Build status](https://travis-ci.org/chrissmall22/SciPass.svg?branch=master)](https://travis-ci.org/chrissmall22/SciPass)
[![Coverage Status](https://img.shields.io/coveralls/chrissmall22/SciPass.svg)](https://coveralls.io/r/chrissmall22/SciPass?branch=master)

SciPass is a SDN powered Science DMZ and IDS Load Balancer.


VM Environment with OpenDayLight
=======

The Vagrantfile provides a quick and easy way to spin up a VM containing all the code to run
SciPass and a Mininet based test environment to test functionality.



To create the SciPass VM:

First make sure Vagrant https://www.vagrantup.com/ is installed for your OS. You will need Vagrant 1.5 to use the Vagrant Cloud boxes. You should also have Hypervisor software installed. this has been tested agqinst VirtualBox but VMWare Workstation/Fusion should work as well.

To create the VM with SciPass and ODL installed

vagrant up
vagrant ssh scipass

To start the Ryu version of SciPass

cd scipass/python
sudo mkdir /etc/SciPass
sudo cp t/etc/SciPass-mininet.xml /etc/SciPass/SciPass.xml
ryu-manager Ryu.py


To create a Mininet network to match SciPass-mininet.xml

cd mininet
sudo mn --topo single,9 --mac --switch ovsk --controller remote

Postman
=======
Install Postman for Chrome http://www.getpostman.com/

Add repository at scipass/resources/postman/SciPass-API.json.postman_collection


OpenDayLight
===========

To start up the OpenDaylight Controller

cd distribution-karaf-0.2.0-Helium
bin/karaf
37 changes: 37 additions & 0 deletions Vagrantfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# Vagrantfile API/syntax version. Don't touch unless you know what you're doing!


# This uses Vagrant Cloud for simplicity so it needs to have Vagrant 1.5
VAGRANTFILE_API_VERSION = "2"

Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|

config.vm.provision "shell", path: "resources/puppet/scripts/bootstrap.sh"

config.vm.provision "puppet" do |puppet|
puppet.hiera_config_path = "resources/puppet/hiera.yaml"
puppet.working_directory = "/vagrant/resources/puppet"
puppet.manifests_path = "resources/puppet/manifests"
puppet.manifest_file = "base.pp"
end

# Install single host with mininet and ODL

config.vm.define "scipass" do |scipass|
scipass.vm.box = "ubuntu/trusty64"
scipass.vm.hostname = "scipass"
scipass.vm.network "private_network", ip: "192.168.50.70"
scipass.vm.provider :virtualbox do |vb|
vb.memory = 4096
end
scipass.vm.provider "vmware_fusion" do |vf|
vf.vmx["memsize"] = "4096"
end
scipass.vm.provision "puppet" do |puppet|
puppet.hiera_config_path = "resources/puppet/hiera.yaml"
puppet.working_directory = "/vagrant/resources/puppet"
puppet.manifests_path = "resources/puppet/manifests"
puppet.manifest_file = "scipass.pp"
end
end
end
2 changes: 2 additions & 0 deletions python/.coveragerc
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
[report]
omit=*mock*,*libxml2*,*json*,*ipaddr*,*encodings*,*xmlrunner*,t/*.py
8 changes: 4 additions & 4 deletions python/SciPass.py
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ def good_flow(self, obj):
for name in self.config[datapath_id]:
for port in self.config[datapath_id][name]['ports']['lan']:
for prefix in port['prefixes']:
self.logger.error("Comparing" + str(new_prefix) + " " + str(prefix['prefix']))
#self.logger.error("Comparing" + str(new_prefix) + " " + str(prefix['prefix']))
if(prefix['prefix'].Contains( new_prefix )):
in_port = port
dpid = datapath_id
Expand Down Expand Up @@ -701,7 +701,7 @@ def _setupSciDMZRules(self, dpid = None, domain_name = None):
actions = []
actions.append({"type": "output",
"port": int(ports['wan'][0]['port_id'])})
self.logger.error("FW WAN -> WAN: ")
#self.logger.error("FW WAN -> WAN: ")
self.fireForwardingStateChangeHandlers( dpid = dpid,
domain = domain_name,
header = header,
Expand All @@ -717,7 +717,7 @@ def _setupSciDMZRules(self, dpid = None, domain_name = None):
actions = []
actions.append({"type": "output",
"port": int(ports['fw_wan'][0]['port_id'])})
self.logger.error("WAN -> FW WAN")
#self.logger.error("WAN -> FW WAN")
self.fireForwardingStateChangeHandlers( dpid = dpid,
domain = domain_name,
header = header,
Expand Down Expand Up @@ -766,7 +766,7 @@ def _setupInlineIDS(self, dpid = None, domain_name = None):
actions = []
actions.append({"type": "output",
"port": int(ports['lan'][0]['port_id'])})
self.logger.error("FW WAN -> WAN: ")
#self.logger.error("FW WAN -> WAN: ")
self.fireForwardingStateChangeHandlers( dpid = dpid,
domain = domain_name,
header = header,
Expand Down
8 changes: 8 additions & 0 deletions python/requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
Routes>=1.12.3
bottle>=0.10.6
coverage>=3.7.1
mock>=1.0.1
ipaddr>=2.1.0
xmlrunner>=1.7.7
WebOb>=1.1.1

Loading