- Task 1 : Register on services and get your IDs (10')
- Task 2 : Setup the environment (30')
- Task 3 : Fork this GitHub repo (10')
- Task 4 : Understand the project structure (10')
- Task 5 : Develop the Java code according to the executable specification (20')
- Task 6 : Submit your results (10')
- Task 1 : Download and install jenkins (20')
- Task 2 : Configure jenkins to get source code from GitHub and invoke maven (30')
There are 3 webcasts that describe the part 1 of this lab:
- One that explains the difference between fork and clone on GitHub
- One that shows how to use maven from the command line to build and test the project.
- One that shows how to build and test from NetBeans
There are 5 webcasts that describe the part 2 of this lab:
- One that shows how to run Jenkins from the command line
- One that shows how to install the Git plugin in Jenkins
- One that shows how to let jenkins know where you have installed maven on your system
- One that shows how to configure maven and trigger a build manually (and see a failing unit test)
- One that shows how to fix the failing test, push the fix to GitHub and trigger a successful build
- If you don't have a GitHub account yet, get one here.
In this lab, we will start by installing some of the tools that we will be using during the semester.
You should use Java 8. Depending on the platform, it can be a bit tricky to figure out which JDK is actually used when you run a Java application. When you build the same project in Netbeans or on the command line, it is possible that a different version of the JDK is actually used. Keep this in mind if you encounter issues (maven might complain).
For Mac OS users, here and there, you will find useful information.
To validate your installation, you can do a java -version from the command line.
Apache maven is a software development tool that is extremely popular in the Java community. As a matter of fact, if you work in a professional environment or contribute to an open source project, there is no way that you will escape it.
While our goal is not to make you a maven expert in this course, we want you to get familiar with it. You will use maven to build, test and run your projects, so it is important that you have a high-level understanding of what it does.
If you have used Makefiles, ant scripts or Grunt.js, you are already familiar with the notion of software build automation. Simply stated, maven is used to write recipes for transforming source code into running software. It manages the entire build cycle, which includes tasks such as compilation, unit testing, integration testing, code quality checks, packaging and deployment.
We will not get into further details right now. Installing maven on your machine is easy. Just go here and follow the instructions. When you are done, make sure to update your system path so that you can invoke maven with the mvn command from a terminal, wherever your are. If you are using Windows, this page might be helpful.
When you have installed maven, you should be able to type mvn -v on the command line and see the following output:
$ mvn -v
Apache Maven 3.1.0 (893ca28a1da9d5f51ac03827af98bb730128f9f2; 2013-06-28 04:15:32+0200)
Maven home: /Users/admin/tools/maven/apache-maven-3.1.0
Java version: 1.8.0_11, vendor: Oracle Corporation
Java home: /Library/Java/JavaVirtualMachines/jdk1.8.0_11.jdk/Contents/Home/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "mac os x", version: "10.9.5", arch: "x86_64", family: "mac"
In your home directory, there is a hidden directory named .m2. It contains a file named settings.xml, which is used to configure various aspects of your maven installation. On both Unix and Windows systems, the path to the file should be as follows:
~/.m2/settings.xml
If you are unsure of the exact path to this file, type mvn -X on the command line to run maven in debug mode.
The command will produce an error because no goal was specified, but it will also output the path to the settings.xml file you are looking for.
Note the second [DEBUG] line which tells you where to find your user's settings.xml file:
$ mvn -X
Apache Maven 3.2.3 (33f8c3e1027c3ddde99d3cdebad2656a31e8fdf4; 2014-08-11T22:58:10+02:00)
Maven home: /opt/local/share/java/maven3
Java version: 1.8.0_25, vendor: Oracle Corporation
Java home: /Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre
Default locale: en_US, platform encoding: utf-8
OS name: "mac os x", version: "10.10.2", arch: "x86_64", family: "mac"
[INFO] Error stacktraces are turned on.
[DEBUG] Reading global settings from /opt/local/share/java/maven3/conf/settings.xml
[DEBUG] Reading user settings from /Users/your-user/.m2/settings.xml
[DEBUG] Using local repository at /Users/your-user/.m2/repository
[DEBUG] Using manager EnhancedLocalRepositoryManager with priority 10.0 for /Users/your-user/.m2/repository
[INFO] Scanning for projects...
[DEBUG] Extension realms for project org.apache.maven:standalone-pom:pom:1: (none)
[DEBUG] Looking up lifecyle mappings for packaging pom from ClassRealm[plexus.core, parent: null]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 0.097 s
[INFO] Finished at: 2015-02-20T10:33:16+01:00
[INFO] Final Memory: 5M/245M
[INFO] ------------------------------------------------------------------------
[ERROR] No goals have been specified for this build.
...
You have to edit this file and add your GitHub login name in a new <profile></profile> section.
If this is your first time installing or configuring maven, the file may not exist so you may have to create it yourself.
Have a look at the example below. Grab the text, replace wasadigi with your GitHub login and save the file.
<settings>
...
<profiles>
<profile>
<id>schoolPulse</id>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
<properties>
<ch.heigvd.schoolpulse.userid>wasadigi</ch.heigvd.schoolpulse.userid>
</properties>
</profile>
...
</profiles>
...
</settings>
In this course, we will be writing a lot of code, mostly in Java. We use Netbeans to prepare the labs and samples. So, if you use Netbeans, your life will be easier. If you prefer to use another IDE, that should be fine for most of the labs (we will let you know otherwise). But we will not be able to provide individual support for all IDEs, so you have to be ready to investigate issues on your own.
From what we have seen over the last few years, many students are not familiar or comfortable with a terminal and command line tools. This is particularly true for students using Windows as their main operating system. Launching cmd.exe is not something very common, and once in the terminal, it is not very easy nor practical to get work done.
If you are in this situation, then it is now time to get familiar with proper tools. This will make you a lot more productive. It will save you time during labs (not only in this course). It will allow you to do things in a cleaner and more efficient way.
You do not have to be a shell guru. Even with a couple of commands, you will quickly see the benefits. Please, please take the time to learn how to use this crucial tool.
If you are on Windows, then you have several options, including PowerShell, Gygwin, Git Bash or Babun, which seems to be gaining traction. Some of the options are described here. To be honest, I am not a Windows user, so it is difficult for me to make a recommendation and you will have to make your own experiments. But if you have not used any of these tools, take the time to pick one and learn how to use it. Otherwise, you will loose an awful lot of time and will not be able to do some of the labs (this year and next year).
Ok, now that your environment is setup, time to get to the real work. For the labs, we will be using a collaborative development workflow. You have to be familiar with git and the GitHub service. Here is a high-level description of the workflow:
-
We will create one GitHub repo for every lab. This repo will contain the instructions and a code skeleton. Often, we will provide a collection of JUnit tests that will allow us to specify, validate and grade your projects. If you try to run the JUnit tests at the beginning of the project, most of them should fail (they might not even compile). Reading the code of the tests will let you know what we expect from you (the tests are an executable specification).
-
You will fork the lab repo. Read this sentence again very carefully: we said fork and not clone. Forking the lab repo means that you will have your own copy of the lab repo hosted on GitHub, so that you can work independently. You will then clone your fork on your machine. Once again, what you are cloning is your fork, not the original repo. Got it?
-
You will then work on the assignment, write code do you tests. You should get used to commit your work on a regular basis (which means that you will have a local history). Before the submission date, you will then push your commits to your fork. You should also create a pull request, so that we know that you are done (and it's a way for us to provide feedback on the code).
-
We know where you live. After submission, we will check your forks. We have our own suite of automated tests, which we will use to evaluate your work.
If you have followed the previous instructions, then you should now have a clone of your fork on your machine, in a directory named Teaching-HEIGVD-RES-2015-Labo1. Let's check what is inside:
If you look carefully, you will see that there are 3 files named pom.xml in the directory structure. These files are used by maven and you can think of them as 3 recipes for building our application. As indicated in the diagram, we actually have 3 projects. One with the application code, one with the automated tests and one on top of them. What we are going to do is build the top-level project. Maven will automatically compile the application code, compile the test code, run the tests and give us results. If all tests pass, then Maven will also produce a .jar file.
Before opening Netbeans, let's build our software from the command line. To do that, we have to move into the top-level project, where the first pom.xml file is located and type the following command: mvn clean install. By doing that, we are asking maven to clean the workspace and to start the build process. You will see a long output on the console, starting with this:
$ cd Teaching-HEIGVD-RES-2015-Labo1/
$ ls
LICENSE Lab00App-build README.md
$ cd Lab00App-build/
$ ls
Lab00App-code Lab00App-tests pom.xml
$ mvn clean install
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO]
[INFO] Lab00App-code
[INFO] Lab00App-tests
[INFO] Lab00App-build
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Lab00App-code 1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
...
If you go back in the terminal (and that is one first example that shows how important it is to have a proper command line tool!), you will notice the following section:
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running ch.heigvd.res.lab00.ApplicationTest
Tests run: 5, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.002 sec <<< FAILURE! - in ch.heigvd.res.lab00.ApplicationTest
thereShouldBeAMethodToAddIntegers(ch.heigvd.res.lab00.ApplicationTest) Time elapsed: 0 sec <<< FAILURE!
java.lang.AssertionError: expected:<42> but was:<80>
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:834)
at org.junit.Assert.assertEquals(Assert.java:645)
at org.junit.Assert.assertEquals(Assert.java:631)
at ch.heigvd.res.lab00.ApplicationTest.thereShouldBeAMethodToAddIntegers(ApplicationTest.java:53)
Results :
Failed tests:
ApplicationTest.thereShouldBeAMethodToAddIntegers:53 expected:<42> but was:<80>
Tests run: 5, Failures: 1, Errors: 0, Skipped: 0
Very interesting: we see that 5 tests have been successfully run and that one has failed. We see that there seems to be a bug in the method that computes the sum between two integers... Indeed, if we look at the code, we see that the code of the add(int a, int b) method is wrong:
$ cat Lab00App-code/src/main/java/ch/heigvd/res/lab00/Application.java
package ch.heigvd.res.lab00;
/**
* This is a very simple class used to demonstrate the specify-implement-validate
* cycle. All methods used in the JUnit test are defined, so the test class will
* compile. There is a bug in the add method, so one of the tests is expected
* to fail.
*
* If you look at the class named ch.heigvd.res.lab00.ApplicationTest, you will
* find an executable specification for this class. The test methods specify
* the expected behavior for this class.
*
* @author Olivier Liechti
*/
public class Application {
private String message;
public Application() {
this("HEIG-VD rocks!");
}
public Application(String message) {
this.message = message;
}
public String getMessage() {
return message;
}
public int add(int a, int b) {
return a * b;
}
}After fixing the bug, we do another mvn clean install and this time, we see the following output:
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running ch.heigvd.res.lab00.ApplicationTest
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.002 sec - in ch.heigvd.res.lab00.ApplicationTest
Results :
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0
Sweet. Our implementation now is compliant with the executable specification. Well, that's because some unit tests have been commented out. Open the TrumpetTest.java file and uncomment the unit tests. Note that if you try to build the project now, it will fail. The reason is that the tests specify that you should have created interfaces and classes. You have not done that yet, and it is your assignment in this lab.
If you read the name of the test methods, you will see that they are a way to specify the expected behavior of your application (that is why we speak of executable specification):
thereShouldBeAnIInstrumentInterfaceAndATrumpetClass
itShouldBePossibleToPlayAnInstrument
aTrumpetShouldMakePouet
aTrumpetShouldBeLouderThanAFlute
aTrumpetShouldBeGolden
To get all the details of the specification, you will have to take a close look at the body of the methods, such as this one. It should be pretty straightforward to determine what you have to implement in the Lab00App-code project (do not add the application classes in the Lab00-App-tests project!).
@Test
public void aTrumpetShouldMakePouet() {
IInstrument trumpet = new Trumpet();
String sound = trumpet.play();
Assert.assertEquals("pouet", sound);
}What we have done in the terminal can also be done directly in Netbeans:
-
open the Lab00App-build project
-
right-click on the Lab00App-build node in the Project tab and select Open Required Project > Open All Projects
-
righ-click on the Lab00App-build node and select Test
You will see the following output in the IDE tabs. In the Output tab, you will see the same output that was generated when invoking maven on the command line. In the Test Results tab, you will see a user friendly visual representation of the test results.
After cloning the repo, you see 4 green tests and 1 red test. When you have finished the lab, you should see 10 green tests.
When you are done, you have to make sure that you push your changes to your fork hosted on GitHub. Indeed, to check your work, we will get the last version of your fork and run our test suite on your code.
Typically, what you will do is the following:
# We have modified several local files. We want to take a snapshot
# of files xxx.java and xx2.java and add it to the project history
git add xxx.java
git add xx2.java
# The staging area contains the 2 files. We can now create a local
# commit and provide a message
git commit -m "Implementation of xxx according to specification"
# Finally, we want to send the commit to the fork, so that it is visible
# for other users
git push origin masterLast but not least, we ask you to make a pull request. In a typical open source project, the following workflow is used by contributors:
- The contributor wants to make a contribution (create a new feature, fix a bug, contribute to the documentation), so he creates a fork.
- (optional) The contributor creates a feature branch.
- The contributor implements a new feature or fixes a bug in his fork.
- The contributor does not have write access on the upstream server (the original repo for the open source project)
- For this reason, he asks the people who have the right to commit code on the upstream server to pull changes from the fork
- On GitHub, this is not done by phone or email, but via the Web UI. On the fork home page, there is a "Pull Requests" menu. Clicking on the green button labeled New pull request initiates the process.
- The committers review the submitted commits. They can add line-by-line comments, discuss with the contributor, etc. In the end, they may accept or reject the pull request.
In our workflow, we will most likely not accept your pull requests. However, having a pull request is a nice way to notify us that you are done with the implementation. It also give us a mechanism to give you line-by-line feedback.
- Go to http://http://jenkins-ci.org/ and click on the download "Latest and greatest" link to download the .war file. Here is the direct link.
- Jenkins is a Java web application. You could deploy it in an application server (Tomcat, Glassfish, JBoss). As an alternative, Jenkins offers and embedded server, so you can run it directly with the following command:
java -jar jenkins.war
- If you look at the log produced by the command, you will see two interesting lines, which tell you where jenkins stores its configuration files and on which port it listens.
Jenkins home directory: /Users/admin/.jenkins found at: $user.home/.jenkins
mars 09, 2015 9:16:08 AM org.eclipse.jetty.util.log.JavaUtilLog info
INFO: Started [email protected]:8080
mars 09, 2015 9:16:08 AM winstone.Logger logInternal
- You should now be able to open a web browser and to log into jenkins by accessing the following url: http://localhost:8080.
-
Now that jenkins is installed, we can create a build pipeline. This is done by creating one or more jobs. Click on the New item or Create new job links.
-
When you are asked for a job name, you can specify Build lab project (commit stage). You also have to select one project type: select Maven project.
-
In the Source Code Management, you see that CVS and subversion are supported by default, but you cannot see git. You need to install a jenkins plugin for that. Click on the Jenkins icon (top-left) and click on Manage jenkins. Click on Manage Plugins, then on the Available tab. In the filter search box, type git and hit return. As you can see, there are many git and GitHub related plugins. Scroll to the bottom and select the one named Git Plugin (This plugin allows use of Git as a build SCM).
-
When you start the installation process, you will notice that Jenkins automatically downloads the required dependencies (Credentials Plugin, Git Client Plugin, SCM API Plugin and finally Git Plugin). Make sure to restart jenkins where you are done. Note that if the installation appears to be frozen, you may have to reload the page in the browser.
-
When you have installed the plugin, you can go back to your job. You do that by clicking on it in the main page and then selecting Configure in the left menu. As you can see, you have have the possibility to select Git in the Source Code Management section. Enter the URL of your GitHub fork, Add Credentials (and make sure to select them in the popup menu). You can use the default values for the rest of this section.
-
In the Build Triggers section, unselect all options. We will trigger the builds manually for now.
-
You may have to configure maven, which means that you have to tell jenkins where you have installed maven on your machine. Simply provide the path you have chosen on your local machine.
-
In the Build section, you need to indicate where the main pom.xml file is located, relative to the root of your repo. Type in the following path: ./Lab00App-build/pom.xml. In Goals and options, type clean install.
-
You are all set! You can now try to build your project by clicking on the green arrow on the right side of your job (in the main jenkins page), or by clicking on the Build now element in the left menu (on the job configuration page). When you do that, you will see that the process is asynchronous: the build is put in a queue and waits for some workers to be available (this makes sense in real projects, especially with multiple developers).
-
The color of the dot indicates whether the build was successful or not. If you click on the Console Output, you will have access to all the details (output of the maven build).
Let us imagine that we find a bug somewhere in the SoftEng-HEIGVD/Teaching-HEIGVD-RES-2015-Labo1 repo. We want to fix it and make sure that all students can get the fix in their own fork. How do we do that?
This operation cannot be done via the Web UI, but it is fairly easy and well documented. First, you have to configure an upstream server. Then, you can sync your local clone with the upstream server. Finally, you can push the update to your fork.
There are many resources available, here are a few suggestions: