Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Spalloc Server integration testing #76

Open
wants to merge 24 commits into
base: main
Choose a base branch
from
Open

Spalloc Server integration testing #76

Changes from 1 commit
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
20c5ea4
jenkins trigger
dkfellows Sep 22, 2021
14110a2
Change to use spalloc: URLs
dkfellows Nov 1, 2021
335ca56
Switch to the new server in compat mode
dkfellows Nov 1, 2021
718490d
Try to avoid using internal IP address for spalloc
dkfellows Nov 2, 2021
718e72d
Merge branch 'main' into spalloc-server
dkfellows Aug 4, 2022
004ded5
Merge branch 'spalloc-server' of
rowleya Dec 1, 2022
9d6d6cc
Try jenkins direct
rowleya Dec 2, 2022
0d8e731
Get the URL right!
rowleya Dec 2, 2022
0f6c2be
Fix missing folder
rowleya Dec 5, 2022
c8e9a74
Test one-by-one
rowleya Jan 25, 2023
3b9928d
Missing space
rowleya Jan 25, 2023
8f43237
Try a different capture method
rowleya Jan 25, 2023
3d29fd7
A different way to get output...
rowleya Jan 25, 2023
8008b2b
Revert to default
rowleya Jan 26, 2023
113a2c4
Add setproctitle so we know what test is running where
rowleya Jan 27, 2023
10afa9e
Fix install
rowleya Jan 27, 2023
2d2ceb2
Revert back for main tests
rowleya Feb 10, 2023
3259716
Try without proxy mode
rowleya Feb 10, 2023
141cf1b
Try with proxy now
rowleya Feb 10, 2023
1665b52
Merge branch 'main' into spalloc-server-jdbc
rowleya Feb 20, 2023
43ab1be
Check normal master again
rowleya Feb 24, 2023
7225665
Merge branch 'spalloc-server-jdbc' of
rowleya Feb 24, 2023
02b7cec
With proxy and new spalloc
rowleya Feb 28, 2023
85b28c2
Merge branch 'main' into spalloc-server-jdbc
rowleya Jun 6, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 21 additions & 19 deletions Jenkinsfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -134,25 +134,9 @@ pipeline {
stage('Before Script') {
steps {
// Write a sPyNNaker config file for spalloc and java use
sh 'echo "[Machine]" > ~/.spynnaker.cfg'
sh 'echo "spalloc_server = spalloc://[email protected]:22246" >> ~/.spynnaker.cfg'
sh 'echo "enable_advanced_monitor_support = True" >> ~/.spynnaker.cfg'
sh 'echo "[Java]" >> ~/.spynnaker.cfg'
sh 'echo "use_java = True" >> ~/.spynnaker.cfg'
sh 'echo "java_call=/usr/bin/java" >> ~/.spynnaker.cfg'
sh 'echo "java_properties=-Dspinnaker.parallel_tasks=10" >> ~/.spynnaker.cfg'
sh 'printf "java_spinnaker_path=" >> ~/.spynnaker.cfg'
sh 'pwd >> ~/.spynnaker.cfg'
makeConfig('~/.spynnaker.cfg')
// Write a GFE config file for spalloc and java use
sh 'echo "[Machine]" > ~/.spiNNakerGraphFrontEnd.cfg'
sh 'echo "spalloc_server = spalloc://[email protected]:22246" >> ~/.spiNNakerGraphFrontEnd.cfg'
sh 'echo "enable_advanced_monitor_support = True" >> ~/.spiNNakerGraphFrontEnd.cfg'
sh 'echo "[Java]" >> ~/.spiNNakerGraphFrontEnd.cfg'
sh 'echo "use_java = True" >> ~/.spiNNakerGraphFrontEnd.cfg'
sh 'echo "java_call=/usr/bin/java" >> ~/.spiNNakerGraphFrontEnd.cfg'
sh 'echo "java_properties=-Dspinnaker.parallel_tasks=10" >> ~/.spiNNakerGraphFrontEnd.cfg'
sh 'printf "java_spinnaker_path=" >> ~/.spiNNakerGraphFrontEnd.cfg'
sh 'pwd >> ~/.spiNNakerGraphFrontEnd.cfg'
makeConfig('~/.spiNNakerGraphFrontEnd.cfg')
// Prepare coverage
sh 'rm -f coverage.xml'
sh 'rm -f .coverage'
Expand Down Expand Up @@ -271,10 +255,28 @@ pipeline {
}
}

def makeConfig(String filename) {
//def spalloc = 'spalloc://[email protected]:22246'
def spalloc = 'spalloc://[email protected]:22246'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note - the internal IP address is used as both the Jenkins server and the machine have addresses on this internal network. Using the external IP means that traffic is likely to flow out of and in to the machine room i.e. the tests become more dependent on the networking state outside of the machine room.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, but tests fail catastrophically if I do that. I'm guessing there's some sort of iptables issue causing trouble. (Configuring /etc/hosts is better than using hardcoded IP addresses.)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This probably means that the service isn't listening on all IP addresses. I believe that you can fix this by adding:
address: 0.0.0.0
to the yml file here (or whereever the actual config comes from):
https://github.com/SpiNNakerManchester/JavaSpiNNaker/blob/spalloc-server/SpiNNaker-allocserv/src/main/resources/application.yml#L57

def javaPath = '/usr/bin/java'
def numThreads = '10'

sh 'echo "[Machine]" > ' + filename
sh 'echo "spalloc_server = ' + spalloc + '" >> ' + filename
sh 'echo "enable_advanced_monitor_support = True" >> ' + filename

sh 'echo "[Java]" >> ' + filename
sh 'echo "use_java = True" >> ' + filename
sh 'echo "java_call=' + javaPath + '" >> ' + filename
sh 'echo "java_properties=-Dspinnaker.parallel_tasks=' + numThreads + '" >> ' + filename
sh 'printf "java_spinnaker_path=" >> ' + filename
sh 'pwd >> ' + filename
}

def run_pytest(String tests, int timeout, String results, String covfile, String threads) {
def resfile = 'junit/' + results + '.xml'
covfile += '_cov.xml'
sh 'echo "<testsuite tests="0"></testsuite>" > ' + resfile
writeFile file: resfile, text: '<testsuite tests="0"></testsuite>'
sh 'py.test ' + tests +
' -rs -n ' + threads + ' --forked --show-progress --cov-config=.coveragerc --cov-branch ' +
'--cov spynnaker8 --cov spynnaker --cov spinn_front_end_common --cov pacman ' +
Expand Down