-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feature/concurrency tests #6
Changes from 11 commits
66fa8b6
1ff8078
9138f96
daf2c25
a477769
04d8e9c
8116216
22315e5
79d4221
65adfdf
41b78f4
f741adf
e29f313
4c0754b
52f8728
c6777a6
d5189d3
8a10dc2
b9f9eed
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,30 +1,41 @@ | ||
package io.datanerds.avropatch; | ||
|
||
import io.datanerds.avropatch.operation.DefaultSchema; | ||
import io.datanerds.avropatch.operation.Operation; | ||
import io.datanerds.avropatch.value.conversion.*; | ||
import org.apache.avro.Schema; | ||
import org.apache.avro.io.*; | ||
import org.apache.avro.reflect.ReflectData; | ||
import org.apache.avro.reflect.ReflectDatumReader; | ||
import org.apache.avro.reflect.ReflectDatumWriter; | ||
import org.apache.avro.reflect.AvroSchema; | ||
|
||
import java.io.ByteArrayOutputStream; | ||
import java.io.IOException; | ||
import java.util.ArrayList; | ||
import java.util.Collections; | ||
import java.util.List; | ||
import java.util.*; | ||
|
||
/** | ||
* This class represents a JSON PATCH operation holding a sequence of operations to apply to a given object. | ||
* | ||
* @see <a href="https://tools.ietf.org/html/rfc6902">https://tools.ietf.org/html/rfc6902</a> | ||
*/ | ||
public class Patch { | ||
|
||
public static final Schema SCHEMA = ReflectData.get().getSchema(Patch.class); | ||
private static final DatumWriter<Patch> writer = new ReflectDatumWriter<>(SCHEMA); | ||
private static final PatchSerializer SERIALIZER = new PatchSerializer(); | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why is domain and (de)-serialization combined? |
||
|
||
@AvroSchema(DefaultSchema.HEADERS) | ||
private final Map<String, ?> headers; | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Shouldn't headers be a multimap? |
||
private final List<Operation> operations; | ||
|
||
public Patch() { | ||
operations = new ArrayList<>(); | ||
this(new ArrayList<>(), new HashMap<>()); | ||
} | ||
|
||
public Patch(List<Operation> operations) { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Always a good idea to declare method arguments as final (especially in case of collections) which makes it explicit to the calling function that the list would not be modified in the called method. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Disagree. The Agree though on parameter declarations being implicitly |
||
this.operations = new ArrayList<>(operations); | ||
this.headers = new HashMap<>(); | ||
} | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why not call the two-argument constructor here? This would follow the same principle as the no-argument constructor... There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Correct :-) |
||
|
||
public Patch(List<Operation> operations, Map<String, ?> headers) { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Should we do a sanity check that the value instances are of a supported type? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I created issue #10 for this feature |
||
this.operations = new ArrayList<>(operations); | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Following the pattern above, |
||
this.headers = new HashMap<>(headers); | ||
} | ||
|
||
public boolean addOperation(Operation operation) { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Thread safety will become an issue here. You are using a thread unsafe list for "operations". If there are 2 competing threads one of which does a getOperations and tries to iterate on it where as the other tries to add an operation at the same time, it will result in a ConcurrentModificationException. Would be better to use either a CopyOnWriteArrayList if operations are not expected to be added too frequently or synchronize getOperations and create a copy of the list before returning. AFAIK Collections.unmodifiableList is only a shallow copy and not a deep copy, so that would not handle it. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Or explicitly point out the Thread unsafety in the class documentation. This might well be desired in favor of performance as a tradeoff. I think Effective Concurrency In Java notes mentioning something like
in the java docs of a class. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. See here "Document Thread-Safety" There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Originally the class wasn't designed to be thread-safe, because there would only be one thread processing a PATCH call, but since removing this convenient method results in thread-safety I will do that. |
||
|
@@ -35,16 +46,29 @@ public List<Operation> getOperations() { | |
return Collections.unmodifiableList(operations); | ||
} | ||
|
||
public Map<String, ?> getHeaders() { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Not a concurrency issue with the map as long as the map is not open to modification. It might not be the case right now but leaves the code open to errors in case there is a write interface exposed to the map erroneously in the future. Would suggest initializing the map as immutable during initialization in the constructor. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. the map should not be open to modifications There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It's not:
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Agreed that it is not open to modifications to external modifiers but from a code extensibility perspective, if there is a method in this class which tries to modify this map, extra care would need to be taken to ensure that the map is made immutable then. By making it immutable now, we can future proof it so that we don't run the risk of somebody not making it immutable/thread safe while adding code modifying it in future. |
||
return Collections.unmodifiableMap(headers); | ||
} | ||
|
||
public byte[] toBytes() throws IOException { | ||
ByteArrayOutputStream outputStream = new ByteArrayOutputStream(); | ||
Encoder binaryEncoder = EncoderFactory.get().directBinaryEncoder(outputStream, null); | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. this local variables binaryEncoder and outputStream can probably shortened for readability, as they are used only once and right after declaration. (see heuristic N5 in Clean Code). |
||
|
||
writer.write(this, binaryEncoder); | ||
SERIALIZER.writer.write(this, binaryEncoder); | ||
return outputStream.toByteArray(); | ||
} | ||
|
||
public static Patch of(byte[] bytes) throws IOException { | ||
DatumReader<Patch> reader = new ReflectDatumReader<>(SCHEMA); | ||
return reader.read(null, DecoderFactory.get().binaryDecoder(bytes, null)); | ||
return SERIALIZER.reader.read(null, DecoderFactory.get().binaryDecoder(bytes, null)); | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I assume we are not reusing the read "Patch" object here because it is being returned to an external method as it is, correct? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes and we'd like to have thread-safety |
||
} | ||
|
||
private static class PatchSerializer { | ||
final DatumWriter<Patch> writer; | ||
final DatumReader<Patch> reader; | ||
|
||
private PatchSerializer() { | ||
Schema schema = AvroData.get().getSchema(Patch.class); | ||
writer = AvroData.get().createDatumWriter(schema); | ||
reader = AvroData.get().createDatumReader(schema); | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. maybe |
||
} | ||
} | ||
} |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -7,26 +7,26 @@ | |
* This class represents the "add" operation of RFC 6902 'JavaScript Object Notation (JSON) Patch'. | ||
* | ||
* @see <a href="https://tools.ietf.org/html/rfc6902#section-4.1">https://tools.ietf.org/html/rfc6902#section-4.1</a> | ||
* @param <T> Value type of <i>add</i> operation | ||
* @param <T> DefaultSchema type of <i>add</i> operation | ||
*/ | ||
public final class Add<T> implements Operation { | ||
@AvroIgnore | ||
public static final String op = "add"; | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Uppercase? Looks like a constant There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Maybe even an enum which is used? |
||
public final Path path; | ||
@AvroSchema(Value.SCHEMA) | ||
@AvroSchema(DefaultSchema.VALUE) | ||
public final T value; | ||
|
||
@SuppressWarnings("unused") // no arg constructor needed by Avro | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||
private Add() { | ||
this.path = null; | ||
this.value = null; | ||
this(null, null); | ||
} | ||
|
||
/** | ||
* | ||
* @param path Path pointing out where the JSON value should be added | ||
* @param value Actual value to <i>add</i> to patched object | ||
* | ||
* @see Value | ||
* @see DefaultSchema | ||
*/ | ||
public Add(Path path, T value) { | ||
this.path = path; | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -13,9 +13,9 @@ public final class Copy implements Operation { | |
public final Path from; | ||
public final Path path; | ||
|
||
@SuppressWarnings("unused") // no arg constructor needed by Avro | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. no-arg |
||
private Copy() { | ||
from = null; | ||
path = null; | ||
this(null, null); | ||
} | ||
|
||
/** | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,84 @@ | ||
package io.datanerds.avropatch.operation; | ||
|
||
/** | ||
* This interface holds the schema for all valid value types. | ||
*/ | ||
public interface DefaultSchema { | ||
String VALUE = | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Eventually move to file and load with There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Would probably make it more readable? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This would not compile since the value is used in an annotation (only constants allowed there). Would need an indirection for compilation. I am adding a card to the backlog There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This would not compile per se since the value is used in an annotation (only constants allowed there). Would need an indirection for compilation. I am adding a card to the backlog There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. In that case generating that file from a json is probably the easiest? |
||
"[{" | ||
+ " \"type\": \"record\"," | ||
+ " \"name\": \"decimal\"," | ||
+ " \"fields\": [{" | ||
+ " \"name\": \"unscaledValue\"," | ||
+ " \"type\": {" | ||
+ " \"type\": \"bytes\"," | ||
+ " \"logicalType\": \"big-integer\"" | ||
+ " }" | ||
+ " }, {" | ||
+ " \"name\": \"scale\"," | ||
+ " \"type\": \"int\"" | ||
+ " }]," | ||
+ " \"logicalType\": \"big-decimal\"" | ||
+ "}, {" | ||
+ " \"type\": \"bytes\"," | ||
+ " \"logicalType\": \"big-integer\"" | ||
+ "}, \"boolean\", {" | ||
+ " \"type\": \"fixed\"," | ||
+ " \"name\": \"timestamp\"," | ||
+ " \"size\": 8," | ||
+ " \"logicalType\": \"timestamp\"" | ||
+ "}, \"double\", \"float\", \"int\", \"long\", \"null\", \"string\", {" | ||
+ " \"type\": \"fixed\"," | ||
+ " \"name\": \"uuid\"," | ||
+ " \"size\": 16," | ||
+ " \"logicalType\": \"uuid\"" | ||
+ "}, {" | ||
+ " \"type\": \"array\"," | ||
+ " \"items\": [\"decimal\", {" | ||
+ " \"type\": \"bytes\"," | ||
+ " \"logicalType\": \"big-integer\"" | ||
+ " }, \"boolean\", \"timestamp\", \"double\", \"float\", \"int\", \"long\", \"null\", \"string\", \"uuid\"]" | ||
+ "}, {\n" | ||
+ " \"type\": \"map\"," | ||
+ " \"values\": [\"decimal\", {" | ||
+ " \"type\": \"bytes\"," | ||
+ " \"logicalType\": \"big-integer\"" | ||
+ " }, \"boolean\", \"timestamp\", \"double\", \"float\", \"int\", \"long\", \"null\", \"string\", \"uuid\"]" | ||
+ "}]"; | ||
|
||
String HEADERS = | ||
"{\n" | ||
+ " \"type\": \"map\",\n" | ||
+ " \"values\": [\"boolean\", \"double\", \"float\", \"int\", \"long\", \"string\", {\n" | ||
+ " \"type\": \"record\",\n" | ||
+ " \"name\": \"decimal\",\n" | ||
+ " \"doc\": \"BigDecimal value represented via it's scale and unscaled value.\",\n" | ||
+ " \"fields\": [{\n" | ||
+ " \"name\": \"unscaledValue\",\n" | ||
+ " \"type\": {\n" | ||
+ " \"type\": \"bytes\",\n" | ||
+ " \"logicalType\": \"big-integer\"\n" | ||
+ " }\n" | ||
+ " }, {\n" | ||
+ " \"name\": \"scale\",\n" | ||
+ " \"type\": \"int\"\n" | ||
+ " }],\n" | ||
+ " \"logicalType\": \"big-decimal\"\n" | ||
+ " }, {\n" | ||
+ " \"type\": \"bytes\",\n" | ||
+ " \"logicalType\": \"big-integer\"\n" | ||
+ " }, {\n" | ||
+ " \"type\": \"fixed\",\n" | ||
+ " \"name\": \"timestamp\",\n" | ||
+ " \"doc\": \"Timestamp representing the number of milliseconds since January 1, 1970, 00:00:00 GMT\",\n" | ||
+ " \"size\": 8,\n" | ||
+ " \"logicalType\": \"timestamp\"\n" | ||
+ " }, {\n" | ||
+ " \"type\": \"fixed\",\n" | ||
+ " \"name\": \"uuid\",\n" | ||
+ " \"doc\": \"UUID serialized via two long values: It's most significant and least significant 64 bits.\",\n" | ||
+ " \"size\": 16,\n" | ||
+ " \"logicalType\": \"uuid\"\n" | ||
+ " }]\n" | ||
+ "}"; | ||
} |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,7 +1,6 @@ | ||
package io.datanerds.avropatch.operation; | ||
|
||
import io.datanerds.avropatch.exception.InvalidPathException; | ||
import org.apache.avro.reflect.AvroSchema; | ||
|
||
import java.util.Arrays; | ||
import java.util.Collections; | ||
|
@@ -16,7 +15,6 @@ public class Path { | |
|
||
private static final Pattern VALID_PATTERN = Pattern.compile("[0-9a-zA-Z][0-9a-zA-Z_-]*"); | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Do we want to give examples of valid paths somewhere in the JavaDoc? |
||
|
||
@AvroSchema("{\"type\": \"array\", \"items\": \"string\"}") | ||
private final List<String> parts; | ||
|
||
private Path() { | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -12,20 +12,20 @@ public final class Replace<T> implements Operation { | |
@AvroIgnore | ||
public static final String op = "replace"; | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Uppercase? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why/where is this string needed anyway? Is it not redundant information? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I will remove the literals completely for now. I've added them initially to comply with the JSON PATCH format. Since we're not enforcing on it we can add the literals again as soon we consider supporting JSON serialization. |
||
public final Path path; | ||
@AvroSchema(Value.SCHEMA) | ||
@AvroSchema(DefaultSchema.VALUE) | ||
public final T value; | ||
|
||
@SuppressWarnings("unused") // no arg constructor needed by Avro | ||
private Replace() { | ||
this.path = null; | ||
this.value = null; | ||
this(null, null); | ||
} | ||
|
||
/** | ||
* | ||
* @param path Path pointing out where the JSON value should be replaced | ||
* @param value Actual value to <i>replace</i> in patched object | ||
* | ||
* @see Value | ||
* @see DefaultSchema | ||
*/ | ||
public Replace(Path path, T value) { | ||
this.path = path; | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -12,20 +12,20 @@ public final class Test<T> implements Operation { | |
@AvroIgnore | ||
public static final String op = "test"; | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Uppercase? |
||
public final Path path; | ||
@AvroSchema(Value.SCHEMA) | ||
@AvroSchema(DefaultSchema.VALUE) | ||
public final T value; | ||
|
||
@SuppressWarnings("unused") // no arg constructor needed by Avro | ||
private Test() { | ||
this.path = null; | ||
this.value = null; | ||
this(null, null); | ||
} | ||
|
||
/** | ||
* | ||
* @param path Path pointing out which JSON value should be tested against | ||
* @param value Actual value to <i>test</i> against | ||
* | ||
* @see Value | ||
* @see DefaultSchema | ||
*/ | ||
public Test(Path path, T value) { | ||
this.path = path; | ||
|
This file was deleted.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
slf4j version should become a variable since it is also used in testCompile