Skip to content

Commit

Permalink
Initial commit of Firehose Mock (#1)
Browse files Browse the repository at this point in the history
  • Loading branch information
@ptimson
ptimson authored Feb 3, 2017
1 parent ce9f22b commit fa9e3ba
Show file tree
Hide file tree
Showing 31 changed files with 2,151 additions and 0 deletions.
96 changes: 96 additions & 0 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,6 +1,102 @@
# Firehose Mock
[![Github All Releases](https://img.shields.io/github/downloads/ptimson/firehose-mock/total.svg?style=flat)](https://github.com/ptimson/firehose-mock/releases/)
An embedded Java mock for AWS Kinesis Firehose

## Getting Started
### Create Default FirehoseMock
```java
FirehoseMock firehoseMock = FirehoseMock.defaultMock();
firehoseMock.start();
Integer port = firehoseMock.getPort();
```

### Create Custom FirehoseMock
```java
FirehoseMock firehoseMock = new FirehoseMock.Builder()
.withPort(7070) // Optional - If not supplied will use a random free port
.withAmazonS3Client(amazonS3) // Optional - If not supplied will use AmazonS3ClientBuilder.defaultClient()
.build();
firehoseMock.start();
```

## Creating Requests
You can then interact with the mock using the AWS Kinesis Firehose SDK - as you would interact with firehose normally.
However you will need to set the client's endpoint to `localhost:<port>`

### AWSFirehoseUtil
`AWSFirehoseUtil.class` is an optional helper class used to setup the client and create requests that are supported with
this mock.

#### Create Client
```java
AmazonKinesisFirehose firehoseClient = AWSFirehoseUtil.createClient("http://127.0.0.1:7070", "eu-west-1");
```

#### Create PutRecord Request
```java
PutRecordRequest putRequest = AWSFirehoseUtil.createPutRequest("myDeliveryStream", "myData");
firehoseClient.putRecord(putRequest);
```

#### Create CreateDeliveryStream Request
```java
ExtendedS3DestinationConfiguration s3StreamConfig = AWSFirehoseUtil.createS3DeliveryStream()
.withS3BucketArn("arn:myBucketArn")
.withBufferIntervalSeconds(10)
.withBufferSizeMB(1024)
.withCompressionFormat(CompressionFormat.GZIP)
.withS3Prefix("myPrefix")
.build();
CreateDeliveryStreamRequest createDeliveryStreamRequest = AWSFirehoseUtil.createDeliveryStreamRequest("myDeliverStream", s3StreamConfig);
firehoseClient.createDeliveryStream(createDeliveryStreamRequest);
```

#### Create DeleteDeliveryStream Request
```java
DeleteDeliveryStreamRequest deleteStreamRequest = AWSFirehoseUtil.deleteDeliveryStreamRequest(streamName);
firehoseClient.deleteDeliveryStream(deleteStreamRequest);
```
## Supported Requests
Firehose Mock is a WIP and so far only supports the following 3 APIs.

### PutRecord
```
{
"DeliveryStreamName": "string",
"Record": {
"Data": "myBase64EncodedString"
}
}
```
See [AWS PutRecord API Docs](http://docs.aws.amazon.com/firehose/latest/APIReference/API_PutRecord.html) for more details.

### CreateDeliveryStream
Currently FirehoseMock only supports S3 Delivery Streams with the following options:
```
{
"DeliveryStreamName": "string",
"ExtendedS3DestinationConfiguration": {
"BucketARN": "string", // Required
"BufferingHints": {
"IntervalInSeconds": 300, // Optional - Default: 300
"SizeInMBs": 5 // Optional - Default: 5
},
"CompressionFormat": "string", // Optional - Default: UNCOMPRESSED
"Prefix": "string" // Optional - Default: ""
}
}
```
See [AWS CreateDeliveryStream API Docs](http://docs.aws.amazon.com/firehose/latest/APIReference/API_CreateDeliveryStream.html) for more details.

### DeleteDeliveryStream

```
{
"DeliveryStreamName": "string"
}
```
See [AWS DeleteDeliveryStream API Docs](http://docs.aws.amazon.com/firehose/latest/APIReference/API_DeleteDeliveryStream.html) for more details.

## License
Copyright (C) 2017 Peter Timson

Expand Down
185 changes: 185 additions & 0 deletions pom.xml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,185 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>

<groupId>io.timson</groupId>
<artifactId>firehose-mock</artifactId>
<version>0.0.1</version>

<name>Firehose Mock</name>
<url>https://github.com/ptimson/firehose-mock</url>
<description>An embedded Java mock for AWS Kinesis Firehose</description>

<licenses>
<license>
<name>The Apache Software License, Version 2.0</name>
<url>http://www.apache.org/licenses/LICENSE-2.0.txt</url>
<distribution>repo</distribution>
</license>
</licenses>

<developers>
<developer>
<name>Peter Timson</name>
<id>ptimson</id>
<email>[email protected]</email>
</developer>
</developers>

<scm>
<connection>scm:git:[email protected]:ptimson/firehose-mock.git</connection>
<developerConnection>scm:git:[email protected]:ptimson/firehose-mock.git</developerConnection>
<url>[email protected]:ptimson/firehose-mock.git</url>
</scm>

<distributionManagement>
<snapshotRepository>
<id>ossrh</id>
<url>https://oss.sonatype.org/content/repositories/snapshots</url>
</snapshotRepository>
</distributionManagement>

<properties>
<aws.java.sdk.kinesis.version>1.11.84</aws.java.sdk.kinesis.version>
<aws.java.sdk.s3.version>1.11.84</aws.java.sdk.s3.version>
<commons.io.version>2.5</commons.io.version>
<hamcrest.all.version>1.3</hamcrest.all.version>
<jackson.databind.version>2.8.4</jackson.databind.version>
<java.version>1.8</java.version>
<jetty.server.version>9.4.0.v20161208</jetty.server.version>
<jetty.servlet.version>9.4.0.v20161208</jetty.servlet.version>
<junit.version>4.12</junit.version>
<maven.compiler.plugin.version>3.5.1</maven.compiler.plugin.version>
<maven.gpg.plugin.version>1.5</maven.gpg.plugin.version>
<maven.javadoc.plugin.version>2.10.4</maven.javadoc.plugin.version>
<maven.source.plugin.version>3.0.1</maven.source.plugin.version>
<mockito.all.version>1.9.5</mockito.all.version>
<nexus.staging.maven.plugin.version>1.6.7</nexus.staging.maven.plugin.version>
<snappy.java.version>1.1.3-M1</snappy.java.version>
</properties>

<dependencies>
<dependency>
<groupId>org.eclipse.jetty</groupId>
<artifactId>jetty-server</artifactId>
<version>${jetty.server.version}</version>
</dependency>
<dependency>
<groupId>org.eclipse.jetty</groupId>
<artifactId>jetty-servlet</artifactId>
<version>${jetty.servlet.version}</version>
</dependency>
<dependency>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
<version>${commons.io.version}</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<version>${jackson.databind.version}</version>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-kinesis</artifactId>
<version>${aws.java.sdk.kinesis.version}</version>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-s3</artifactId>
<version>${aws.java.sdk.s3.version}</version>
</dependency>
<dependency>
<groupId>org.xerial.snappy</groupId>
<artifactId>snappy-java</artifactId>
<version>${snappy.java.version}</version>
</dependency>

<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>${junit.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.hamcrest</groupId>
<artifactId>hamcrest-all</artifactId>
<version>${hamcrest.all.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-all</artifactId>
<version>${mockito.all.version}</version>
<scope>test</scope>
</dependency>
</dependencies>

<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>${maven.compiler.plugin.version}</version>
<configuration>
<source>${java.version}</source>
<target>${java.version}</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-source-plugin</artifactId>
<version>${maven.source.plugin.version}</version>
<executions>
<execution>
<id>attach-sources</id>
<goals>
<goal>jar</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-javadoc-plugin</artifactId>
<version>${maven.javadoc.plugin.version}</version>
<executions>
<execution>
<id>attach-javadocs</id>
<goals>
<goal>jar</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.sonatype.plugins</groupId>
<artifactId>nexus-staging-maven-plugin</artifactId>
<version>${nexus.staging.maven.plugin.version}</version>
<extensions>true</extensions>
<configuration>
<serverId>ossrh</serverId>
<nexusUrl>https://oss.sonatype.org/</nexusUrl>
<autoReleaseAfterClose>true</autoReleaseAfterClose>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-gpg-plugin</artifactId>
<version>${maven.gpg.plugin.version}</version>
<executions>
<execution>
<id>sign-artifacts</id>
<phase>verify</phase>
<goals>
<goal>sign</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>

</project>
Loading

0 comments on commit fa9e3ba

Please sign in to comment.