Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/concurrency tests #6

Merged
merged 19 commits into from
Jan 13, 2017
Merged
Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions project.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,9 @@ project.ext {
dependencies {

compile([
"org.apache.avro:avro:1.8.0",
"org.slf4j:slf4j-api:1.7.18"
"org.apache.avro:avro:1.8.1",
"org.slf4j:slf4j-api:1.7.18",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

slf4j version should become a variable since it is also used in testCompile

"com.google.code.findbugs:jsr305:3.0.1"
])

testCompile([
Expand Down
50 changes: 37 additions & 13 deletions src/main/java/io/datanerds/avropatch/Patch.java
Original file line number Diff line number Diff line change
@@ -1,30 +1,41 @@
package io.datanerds.avropatch;

import io.datanerds.avropatch.operation.DefaultSchema;
import io.datanerds.avropatch.operation.Operation;
import io.datanerds.avropatch.value.conversion.*;
import org.apache.avro.Schema;
import org.apache.avro.io.*;
import org.apache.avro.reflect.ReflectData;
import org.apache.avro.reflect.ReflectDatumReader;
import org.apache.avro.reflect.ReflectDatumWriter;
import org.apache.avro.reflect.AvroSchema;

import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import java.util.*;

/**
* This class represents a JSON PATCH operation holding a sequence of operations to apply to a given object.
*
* @see <a href="https://tools.ietf.org/html/rfc6902">https://tools.ietf.org/html/rfc6902</a>
*/
public class Patch {

public static final Schema SCHEMA = ReflectData.get().getSchema(Patch.class);
private static final DatumWriter<Patch> writer = new ReflectDatumWriter<>(SCHEMA);
private static final PatchSerializer SERIALIZER = new PatchSerializer();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is domain and (de)-serialization combined?


@AvroSchema(DefaultSchema.HEADERS)
private final Map<String, ?> headers;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't headers be a multimap?

private final List<Operation> operations;

public Patch() {
operations = new ArrayList<>();
this(new ArrayList<>(), new HashMap<>());
}

public Patch(List<Operation> operations) {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Always a good idea to declare method arguments as final (especially in case of collections) which makes it explicit to the calling function that the list would not be modified in the called method.

Copy link
Member

@dhiller dhiller Dec 19, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Disagree. The final only states that the reference can't be reassigned within the block. The list is still mutable.

Agree though on parameter declarations being implicitly final would be a good thing :) Though I think that the boilerplate doesn't satisfy the improvement, would support a source code check rather for a variable being reassigned, as i.e. Findbugs supports AFAIR.

this.operations = new ArrayList<>(operations);
this.headers = new HashMap<>();
}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not call the two-argument constructor here? This would follow the same principle as the no-argument constructor...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct :-)


public Patch(List<Operation> operations, Map<String, ?> headers) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we do a sanity check that the value instances are of a supported type?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I created issue #10 for this feature

this.operations = new ArrayList<>(operations);
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Following the pattern above, this(operations);

this.headers = new HashMap<>(headers);
}

public boolean addOperation(Operation operation) {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thread safety will become an issue here. You are using a thread unsafe list for "operations". If there are 2 competing threads one of which does a getOperations and tries to iterate on it where as the other tries to add an operation at the same time, it will result in a ConcurrentModificationException. Would be better to use either a CopyOnWriteArrayList if operations are not expected to be added too frequently or synchronize getOperations and create a copy of the list before returning. AFAIK Collections.unmodifiableList is only a shallow copy and not a deep copy, so that would not handle it.

Copy link
Member

@dhiller dhiller Dec 19, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or explicitly point out the Thread unsafety in the class documentation. This might well be desired in favor of performance as a tradeoff. I think Effective Concurrency In Java notes mentioning something like

<b>This method is not thread-safe</b>

in the java docs of a class.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See here "Document Thread-Safety"

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Originally the class wasn't designed to be thread-safe, because there would only be one thread processing a PATCH call, but since removing this convenient method results in thread-safety I will do that.

Expand All @@ -35,16 +46,29 @@ public List<Operation> getOperations() {
return Collections.unmodifiableList(operations);
}

public Map<String, ?> getHeaders() {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a concurrency issue with the map as long as the map is not open to modification. It might not be the case right now but leaves the code open to errors in case there is a write interface exposed to the map erroneously in the future. Would suggest initializing the map as immutable during initialization in the constructor.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the map should not be open to modifications

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not:

    public Map<String, ?> getHeaders() {
        return Collections.unmodifiableMap(headers);
    }

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed that it is not open to modifications to external modifiers but from a code extensibility perspective, if there is a method in this class which tries to modify this map, extra care would need to be taken to ensure that the map is made immutable then. By making it immutable now, we can future proof it so that we don't run the risk of somebody not making it immutable/thread safe while adding code modifying it in future.

return Collections.unmodifiableMap(headers);
}

public byte[] toBytes() throws IOException {
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
Encoder binaryEncoder = EncoderFactory.get().directBinaryEncoder(outputStream, null);

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this local variables binaryEncoder and outputStream can probably shortened for readability, as they are used only once and right after declaration. (see heuristic N5 in Clean Code).


writer.write(this, binaryEncoder);
SERIALIZER.writer.write(this, binaryEncoder);
return outputStream.toByteArray();
}

public static Patch of(byte[] bytes) throws IOException {
DatumReader<Patch> reader = new ReflectDatumReader<>(SCHEMA);
return reader.read(null, DecoderFactory.get().binaryDecoder(bytes, null));
return SERIALIZER.reader.read(null, DecoderFactory.get().binaryDecoder(bytes, null));

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume we are not reusing the read "Patch" object here because it is being returned to an external method as it is, correct?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes and we'd like to have thread-safety

}

private static class PatchSerializer {
final DatumWriter<Patch> writer;
final DatumReader<Patch> reader;

private PatchSerializer() {
Schema schema = AvroData.get().getSchema(Patch.class);
writer = AvroData.get().createDatumWriter(schema);
reader = AvroData.get().createDatumReader(schema);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe this.reader & this.writer ?

}
}
}
10 changes: 5 additions & 5 deletions src/main/java/io/datanerds/avropatch/operation/Add.java
Original file line number Diff line number Diff line change
Expand Up @@ -7,26 +7,26 @@
* This class represents the "add" operation of RFC 6902 'JavaScript Object Notation (JSON) Patch'.
*
* @see <a href="https://tools.ietf.org/html/rfc6902#section-4.1">https://tools.ietf.org/html/rfc6902#section-4.1</a>
* @param <T> Value type of <i>add</i> operation
* @param <T> DefaultSchema type of <i>add</i> operation
*/
public final class Add<T> implements Operation {
@AvroIgnore
public static final String op = "add";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Uppercase? Looks like a constant

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe even an enum which is used?

public final Path path;
@AvroSchema(Value.SCHEMA)
@AvroSchema(DefaultSchema.VALUE)
public final T value;

@SuppressWarnings("unused") // no arg constructor needed by Avro
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

org.apache.avro looks like fun

private Add() {
this.path = null;
this.value = null;
this(null, null);
}

/**
*
* @param path Path pointing out where the JSON value should be added
* @param value Actual value to <i>add</i> to patched object
*
* @see Value
* @see DefaultSchema
*/
public Add(Path path, T value) {
this.path = path;
Expand Down
4 changes: 2 additions & 2 deletions src/main/java/io/datanerds/avropatch/operation/Copy.java
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,9 @@ public final class Copy implements Operation {
public final Path from;
public final Path path;

@SuppressWarnings("unused") // no arg constructor needed by Avro

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no-arg

private Copy() {
from = null;
path = null;
this(null, null);
}

/**
Expand Down
84 changes: 84 additions & 0 deletions src/main/java/io/datanerds/avropatch/operation/DefaultSchema.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
package io.datanerds.avropatch.operation;

/**
* This interface holds the schema for all valid value types.
*/
public interface DefaultSchema {
String VALUE =
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Eventually move to file and load with getClass().getResource(...)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would probably make it more readable?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would not compile since the value is used in an annotation (only constants allowed there). Would need an indirection for compilation. I am adding a card to the backlog

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would not compile per se since the value is used in an annotation (only constants allowed there). Would need an indirection for compilation. I am adding a card to the backlog

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In that case generating that file from a json is probably the easiest?

"[{"
+ " \"type\": \"record\","
+ " \"name\": \"decimal\","
+ " \"fields\": [{"
+ " \"name\": \"unscaledValue\","
+ " \"type\": {"
+ " \"type\": \"bytes\","
+ " \"logicalType\": \"big-integer\""
+ " }"
+ " }, {"
+ " \"name\": \"scale\","
+ " \"type\": \"int\""
+ " }],"
+ " \"logicalType\": \"big-decimal\""
+ "}, {"
+ " \"type\": \"bytes\","
+ " \"logicalType\": \"big-integer\""
+ "}, \"boolean\", {"
+ " \"type\": \"fixed\","
+ " \"name\": \"timestamp\","
+ " \"size\": 8,"
+ " \"logicalType\": \"timestamp\""
+ "}, \"double\", \"float\", \"int\", \"long\", \"null\", \"string\", {"
+ " \"type\": \"fixed\","
+ " \"name\": \"uuid\","
+ " \"size\": 16,"
+ " \"logicalType\": \"uuid\""
+ "}, {"
+ " \"type\": \"array\","
+ " \"items\": [\"decimal\", {"
+ " \"type\": \"bytes\","
+ " \"logicalType\": \"big-integer\""
+ " }, \"boolean\", \"timestamp\", \"double\", \"float\", \"int\", \"long\", \"null\", \"string\", \"uuid\"]"
+ "}, {\n"
+ " \"type\": \"map\","
+ " \"values\": [\"decimal\", {"
+ " \"type\": \"bytes\","
+ " \"logicalType\": \"big-integer\""
+ " }, \"boolean\", \"timestamp\", \"double\", \"float\", \"int\", \"long\", \"null\", \"string\", \"uuid\"]"
+ "}]";

String HEADERS =
"{\n"
+ " \"type\": \"map\",\n"
+ " \"values\": [\"boolean\", \"double\", \"float\", \"int\", \"long\", \"string\", {\n"
+ " \"type\": \"record\",\n"
+ " \"name\": \"decimal\",\n"
+ " \"doc\": \"BigDecimal value represented via it's scale and unscaled value.\",\n"
+ " \"fields\": [{\n"
+ " \"name\": \"unscaledValue\",\n"
+ " \"type\": {\n"
+ " \"type\": \"bytes\",\n"
+ " \"logicalType\": \"big-integer\"\n"
+ " }\n"
+ " }, {\n"
+ " \"name\": \"scale\",\n"
+ " \"type\": \"int\"\n"
+ " }],\n"
+ " \"logicalType\": \"big-decimal\"\n"
+ " }, {\n"
+ " \"type\": \"bytes\",\n"
+ " \"logicalType\": \"big-integer\"\n"
+ " }, {\n"
+ " \"type\": \"fixed\",\n"
+ " \"name\": \"timestamp\",\n"
+ " \"doc\": \"Timestamp representing the number of milliseconds since January 1, 1970, 00:00:00 GMT\",\n"
+ " \"size\": 8,\n"
+ " \"logicalType\": \"timestamp\"\n"
+ " }, {\n"
+ " \"type\": \"fixed\",\n"
+ " \"name\": \"uuid\",\n"
+ " \"doc\": \"UUID serialized via two long values: It's most significant and least significant 64 bits.\",\n"
+ " \"size\": 16,\n"
+ " \"logicalType\": \"uuid\"\n"
+ " }]\n"
+ "}";
}
4 changes: 2 additions & 2 deletions src/main/java/io/datanerds/avropatch/operation/Move.java
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,9 @@ public final class Move implements Operation {
public final Path from;
public final Path path;

@SuppressWarnings("unused") // no arg constructor needed by Avro
private Move() {
this.from = null;
this.path = null;
this(null, null);
}

/**
Expand Down
2 changes: 0 additions & 2 deletions src/main/java/io/datanerds/avropatch/operation/Path.java
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
package io.datanerds.avropatch.operation;

import io.datanerds.avropatch.exception.InvalidPathException;
import org.apache.avro.reflect.AvroSchema;

import java.util.Arrays;
import java.util.Collections;
Expand All @@ -16,7 +15,6 @@ public class Path {

private static final Pattern VALID_PATTERN = Pattern.compile("[0-9a-zA-Z][0-9a-zA-Z_-]*");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to give examples of valid paths somewhere in the JavaDoc?


@AvroSchema("{\"type\": \"array\", \"items\": \"string\"}")
private final List<String> parts;

private Path() {
Expand Down
3 changes: 2 additions & 1 deletion src/main/java/io/datanerds/avropatch/operation/Remove.java
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,9 @@ public final class Remove implements Operation {
public static final String op = "remove";
public final Path path;

@SuppressWarnings("unused") // no arg constructor needed by Avro
private Remove() {
this.path = null;
this(null);
}

/**
Expand Down
8 changes: 4 additions & 4 deletions src/main/java/io/datanerds/avropatch/operation/Replace.java
Original file line number Diff line number Diff line change
Expand Up @@ -12,20 +12,20 @@ public final class Replace<T> implements Operation {
@AvroIgnore
public static final String op = "replace";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Uppercase?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why/where is this string needed anyway? Is it not redundant information?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will remove the literals completely for now. I've added them initially to comply with the JSON PATCH format. Since we're not enforcing on it we can add the literals again as soon we consider supporting JSON serialization.

public final Path path;
@AvroSchema(Value.SCHEMA)
@AvroSchema(DefaultSchema.VALUE)
public final T value;

@SuppressWarnings("unused") // no arg constructor needed by Avro
private Replace() {
this.path = null;
this.value = null;
this(null, null);
}

/**
*
* @param path Path pointing out where the JSON value should be replaced
* @param value Actual value to <i>replace</i> in patched object
*
* @see Value
* @see DefaultSchema
*/
public Replace(Path path, T value) {
this.path = path;
Expand Down
8 changes: 4 additions & 4 deletions src/main/java/io/datanerds/avropatch/operation/Test.java
Original file line number Diff line number Diff line change
Expand Up @@ -12,20 +12,20 @@ public final class Test<T> implements Operation {
@AvroIgnore
public static final String op = "test";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Uppercase?

public final Path path;
@AvroSchema(Value.SCHEMA)
@AvroSchema(DefaultSchema.VALUE)
public final T value;

@SuppressWarnings("unused") // no arg constructor needed by Avro
private Test() {
this.path = null;
this.value = null;
this(null, null);
}

/**
*
* @param path Path pointing out which JSON value should be tested against
* @param value Actual value to <i>test</i> against
*
* @see Value
* @see DefaultSchema
*/
public Test(Path path, T value) {
this.path = path;
Expand Down
23 changes: 0 additions & 23 deletions src/main/java/io/datanerds/avropatch/operation/Value.java

This file was deleted.

Loading