DAI Lab 2: File I/O

Goals

The goals of this lab are:

• Java
  • Manipulate String objects (split, join, replace, StringBuilder, etc.).
  • Use the java.io API (InputStream, OutputStream, BufferedReader, BufferedWriter, etc.) to read and write data.
  • Read and write textual data using the appropriate character encoding.
  • Use the try-with-resources statement to properly close streams.
• Tools
  • Know how to add JUnit to a Maven project.
  • Run unit tests from the command line with Maven.
  • Run unit tests from an IDE.
  • Use the debugger of your IDE to debug your code.
• Git
  • Use the fork / pull request workflow.
  • Create and work in new branches.
  • Pull changes from a remote repository.

Introduction

In this lab you will create write a Java program that reads a text files and performs some simple transformations on it.

You will also use Git to manage your code and submit your work.

Preparation

Fork the repository

You cannot directly commit to this repository. You must therefore first create your own copy of the repository.

To do this, click on the "Fork" button in the top right corner of this page on GitHub. This will create a copy of the repository in your own GitHub account.

Now there are two repositories:

• upstream: the original repository on the DAI organization. You cannot commit to this repository.
• origin: your own copy of the repository. You can commit to this repository.

Then clone your forked repository (origin) to your computer.

Explore the code

dai-lab-fileio
├── fileioapp
│   ├── pom.xml
│   └── src
│       ├── main/java/ch/heig/dai/lab/fileio/
│       │   ├── Main.java
│       │   ├── jehrensb
│       │   │   ├── EncodingSelector.java
│       │   │   ├── FileExplorer.java
│       │   │   ├── FileReaderWriter.java
│       │   │   └── Transformer.java
│       └── test/java/ch/heig/dai/lab/fileio/
│           └── jehrensb
│               ├── EncodingSelectorTest.java
│               ├── FileExplorerTests.java
│               └── TransformerTest.java
└── README.md

The repository contains a Maven project fileioapp. We can see that it's a Maven project because of the pom.xml file.

Following the standard Maven directory layout, the project contains a src directory with two subdirectories: main/java and test/java:

• main/java for the source code of the project.
• test/java for the unit tests of the project.

The main directory contains the package ch.heig.dai.lab.fileio with the class Main. This is the main class of the project. Inside this package there is a subpackage ch.heig.dai.labo.fileio.jehrensb with classes EncodingSelector, FileExplorer, FileReaderWriter and Transformer.

The test directory mirrors this structure. It contains unit test for some of the classes in the ch.heig.dai.labo.fileio.jehrensb package.

Unit testing with Maven

The Maven pom.xml file is configured to run the unit tests when you build the project. If you open the file, you will see:

• The maven-surefire-plugin plugin: this is a plugin that runs the unit tests under the test directory when you build the application.
• The dependencies org.junit and org.junit.jupiter: these are the JUnit libraries that are used to write and run unit tests.

Maven automatically downloads these dependencies when you build the project for the first time.

Let's try this! Open a terminal, go into the ./fileioapp directory and run the following command:

mvn clean test

Maven should print "BUILD SUCCESS" as well as warnings in yellow that some tests have been skipped.

Unit testing with your IDE

You can also run the unit tests from your IDE.

Open the directory ./fileioapp in your IDE (not the root directory of the repository). Because of the pom.xmlfile your IDE should recognize the project as a Maven project and import it.

Try running the unit tests from your IDE.

Git fork / pull-request workflow

Important

Before starting to write your program, you must read the document Ch. 1-Tools/git-pr-workflow.pdf on Cyberlearn. It explains the Git fork / pull request workflow.

This is the workflow you will use to submit your work.

The fork / pull request workflow sounds long an complicated, but it is actually quite simple.

Let's try it. The task will be to create a new folder for you in the source code and test code of the project.

1. Go to the upstream repository of the instructor on GitHub and create an issue for the task you want to work on: "Create folders for <your_usename_on_github>.

2. Got to your fork repository on GitHub and click on the "Sync fork" button. This will synchronize your repository with the upstream repository and import all recent changes.

3. In your terminal, switch to the main branch of your repository. Then create a feature branch "feature-create-folders" and switch to it:

   git switch main
   git branch feature-create-folders
   git switch feature-create-folders

4. Create the new folders for you:

   • In the source code folder ./fileioapp/src/main/java/ch/heig/dai/lab/fileio/ copy the folder jehrensb to <your_usename_on_github>.
   • In the test folder ./fileioapp/src/test/java/ch/heig/dai/lab/fileio/ copy the folder jehrensb to <your_usename_on_github>.
   • In the different files that you copied, you will have to update the package names to ch.heig.dai.lab.fileio.<your_usename_on_github>.

5. Run the unit tests to make sure that everything is still working. The tests for the new folders should show up in the IDE and be executed without an error.

6. Add the changes to be committed:

   git add .

   ­

7. Use the following checklist to verify that everything is correct:

   • You've only modified your files and folders, not the files of the instructor or other students.
   • The files and folders are named correctly.
   • The package names in the files are correct.
   • You only added the requested files and folders, no additional files such as .DS_Store, Thumbs.db, etc.
   • The unit tests are still working, without any errors. ­

8. Commit and push your changes to your repository:

   git commit -m "Create folders for <your_usename_on_github>"
   git push

   Git will ask you to create a new branch on your remote repository. Use the command it suggests.

9. Go to the GitHub page of your fork and create a pull request. The pull request will be visible in the upstream repository. In the comment of the pull request, include the text "Closes #<id_of_the_issue>" (replace id_of_the_issue). This will automatically close the issue that you've created on the upstream repository. ­

10. Go see the instructor to have your pull request reviewed. ­

11. When the pull request is accepted, go to GitHub and sync your fork.

12. In a terminal, switch to the main branch of your repository and pull the changes from your repository.

    git switch main
    git pull

The changes you made in your feature branch should now be in the main branch, visible for everybody.

You can now start a new iteration at step 1 for the next tasks.

Description of the program

Before starting your tasks to develop the code, here's is a descripton of how the program works.

The program reads input files (Chuck Norris jokes) from a folder, transforms the text and writes it to an output file.

The diagram below shows how the final program should work.

sequenceDiagram
  Main ->> FileExplorer: FileExplorer(folder)
  Main ->> FileReaderWriter: FileReaderWriter()
  Main ->> EncodingSelector: EncodingSelector()
  Main ->> Transformer: Transformer(name, numWordsPerLine)

  loop
    Main ->> FileExplorer: getNewFile()
    FileExplorer -->> Main: inputFile

    Main ->> EncodingSelector: selectEncoding(inputFile)
    EncodingSelector -->> Main: encoding
    
    Main ->> FileReaderWriter: readFile(inputFile, encoding)
    FileReaderWriter -->> Main: content

    Main ->> Transformer: <transformations>(content)
    Note over Main, Transformer: For each transformation...
    Transformer -->> Main: transformedContent

    Main ->> FileReaderWriter: writeFile(outputFile, transformedContent, encoding)
  end

Loading

The program has 5 components:

FileExplorer

The FileExplorer lists the files in a folder and keeps track of the files that have already been returned. Each time the method getNewFile is called, it reads the folder and returns a file that has not been returned before.

Use the java.io.file API to get the list of files in the folder. Use the HashSet knownFiles to keep track of the files that have already been returned.

EncodingSelector

The method getEncoding returns the Charset to be used to read an input file. It does this by looking at the extension of the file name.

FileReaderWriter

It has two methods. The method readFile(file, encoding) reads the content of a file and returns it as a String. The method writeFile(file, content, encoding) writes the content to a file, using the given encoding.

Use the ...Stream and ...Reader classes from the java.io API to read and write the files.

Transformer

The Transformer class provides three methods to transform strings:

• replaceChuck: replaces the words "Chuck Norris" by another name (provided to the constructor).
• captializeWords: capitalizes the first letter of each word.
• wrapAndNumberLines: wraps the text to a given number of words per line and add a number at the beginning of each line.

What you have to do

Proceed in the following order:

1. FileExplorer
2. EncodingSelector
3. FileReaderWriter
4. Transformer

For each component, follow the pull request workflow:

1. On Github, sync your fork with the upstream repository.
2. Switch to the main branch and pull the changes from your repository.
3. Create an issue on the upstream repository
4. Create a feature branch and switch to it
5. Remove the @Disabled annotation from the unit tests of the component. Adapt the import in the unit tests to the your package name.
6. Implement the component and test it with the unit tests.
7. Go through the checklist above to verify that everything is correct.
8. When you are finished, add, commit and push your changes to your repository.
9. Create a pull request and have it reviewed by the instructor.

Recommendation for the Transformer class

In the Transformer class you have to manipulate strings. Please do not iterate over the strings as if they were arrays of characters. They are not! Use the Java standard library methods of the String class such as split, join, replace, substring, etc.

USE THE DEBUGGER!

Sorry for the blinking warning sign, but this is important!

Never ever add System.out.println(...) statements to debug your code. Learn to use the debugger of your IDE instead. It will save you a lot of time and headaches.

When a unit test fails, set a breakpoint in the unit test or the tested code and run the test in debug mode. Then step through the code to see what is happening. You will find the error much faster than with System.out.println(...).

Debugging will probably be one of the most important skills for your professional life. So start practicing now!

Develop the main class

Once the components are finished and the unit tests are passing, you can develop the main class.

The main class takes a folder name as command line argument. The folder contains the files to transform. To create this folder, unzip the file jokes.zip file anywhere on your computer, but not in the repository folder.

Then follow the pull request workflow again, i.e. create a feature branch before starting to work on the main class.

To test the main class, compile it with maven and run it with java -jar target/fileioapp-1.0.jar <folder_name> <words_per_line>.

When you are finished, add, commit and push your changes to your repository.

Submission

When you are finished, create a pull request for the last task. Your grade will be based on all your pull requests.