This repository has been archived by the owner on Jun 7, 2024. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 293
Aruha 473 check size of events #515
Merged
Merged
Changes from 1 commit
Commits
Show all changes
41 commits
Select commit
Hold shift + click to select a range
9dabfa2
ARUHA-473 Reject batches with at least one event that's too large
cc50d0c
ARUHA-473 Move maximum event size to nakadi settings
761d455
ARUHA-473 Unit tests for max event size
146d6a0
ARUHA-473 Acceptance test for max event size
8849782
ARUHA-473 Fix checkstyle
a061392
ARUHA-473 Move size validation after validation, remove VALIDATING_SI…
0a791da
ARUHA-473 Move size validation inside validation
df5b675
ARUHA-473 Fix bug in test wrt event publishing step
e261cb8
ARUHA-473 Refactor string building
217be7f
ARUHA-473 Fix log string
ddd7a4f
ARUHA-473 Set capacity for StringBuilder
957938b
ARUHA-473 If an event is too large, that event is marked FAILED. Othe…
49c1846
ARUHA-473 Remove redundant Exception
cc15d1a
ARUHA-473 Explicit charset in getBytes()
3f9bf87
ARUHA-473 Refactor event size computation
6f7a450
ARUHA-473 Explicitely set Kafka consumers' fetch.message.max.bytes pr…
f315836
ARUHA-473 Add curly braces
8871dea
ARUHA-473 Explicit event size limit in swagger file
f5e6367
ARUHA-473 BatchItem constructor for String event, adding event size
d5e6f3d
ARUHA-473 Test BatchItem size with multi-byte characters
12253ed
ARUHA-473 Fix test style
30da84a
ARUHA-473 BatchFactory method to process batches as Strings
7287c40
ARUHA-473 Use String when publishing events to improve event size che…
d8af912
Merge branch 'master' into ARUHA-473-check-size-of-events
0fbaa46
ARUHA-473 Pass batch as String to publisher and fix checkstyle violat…
20aed22
ARUHA-473 Don't parse input to JSON in EventPublishingController
b9c36f8
ARUHA-473 Remove acceptance test
33bbc03
ARUHA-473 Remove unused private method
f6fcfe9
ARUHA-473 Error message for events too large includes the size of the…
f90e477
ARUHA-473 Fix tests' expected error messages for events too large
43a337a
Merge branch 'master' into ARUHA-473-check-size-of-events
bf90d6e
ARUHA-473 Fix test
0330145
ARUHA-473 Change max event size in swagger file
4cc0bcd
ARUHA-473 Fix bug when a batch is an array with only spaces.
30db7f5
ARUHA-473 Set max event size to 999,000 bytes by default
781db1e
ARUHA-473 Accept valid batches surrounded with tabs, carriage returns…
ffd729a
ARUHA-473 Refactoring
e3a4562
Merge branch 'master' into ARUHA-473-check-size-of-events
58cb450
ARUHA-473 Remove kafka max bytes property
3f2a2ac
ARUHA-473 Finish removing kafka max bytes property
57e67d3
Merge branch 'master' into ARUHA-473-check-size-of-events
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -13,10 +13,35 @@ public static List<BatchItem> from(final String events) { | |
int brackets = 0; | ||
boolean insideQuote = false; | ||
boolean escaped = false; | ||
if ((!events.startsWith("[")) || (!events.endsWith("]"))) { | ||
throw new JSONException("Array must be surrounded with square brackets"); | ||
int start = 0; | ||
final int length = events.length(); | ||
int end = length - 1; | ||
|
||
while ((events.charAt(start) == ' ' | ||
|| events.charAt(start) == '\t' | ||
|| events.charAt(start) == '\n' | ||
|| events.charAt(start) == '\r') | ||
&& start < end) { | ||
start++; | ||
} | ||
while ((events.charAt(end) == ' ' | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If this condition is extracted to its own method, then this should be refactored to reuse it. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. |
||
|| events.charAt(end) == '\t' | ||
|| events.charAt(end) == '\n' | ||
|| events.charAt(end) == '\r') | ||
&& end > start) { | ||
end--; | ||
} | ||
if (!(events.charAt(start) == '[')) { | ||
throw new JSONException(String.format("Unexpected character %s in position %d, expected '['", | ||
events.charAt(start), start)); | ||
} | ||
for (int i = 1; i < events.length() - 1; i++) { | ||
start++; | ||
if (!(events.charAt(end) == ']')) { | ||
throw new JSONException(String.format("Unexpected character %s in position %d, expected ']'", | ||
events.charAt(end), end)); | ||
} | ||
|
||
for (int i = start; i < end; i++) { | ||
if (!escaped && events.charAt(i) == '"') { | ||
if (insideQuote) { | ||
insideQuote = false; | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this condition could be extracted to it's own method, something like
isBlankCharacter
.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
or as per the test name
isEmptyCharacter
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good point, will extract
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ffd729a