Introduction
Tigon is an open-source, real-time, low-latency, high-throughput stream processing framework.
Tigon is a collaborative effort between Cask Data, Inc. and AT&T that combines technologies from these companies to create a disruptive new framework to handle a diverse set of real-time streaming requirements.
Cask Data has built technology that provides scalable, reliable, and persistent high-throughput event processing with high-level Java APIs using Hadoop and HBase.
AT&T has built a streaming engine that provides massively scalable, flexible, and in-memory low-latency stream processing with a SQL-like query Language.
Together, they have combined to create Tigon.
There are many applications that can take advantage of its features:
- Ability to handle extremely large data flows;
- Exactly-once event processing using an app-level Java API with consistency, reliability, and persistence;
- Streaming database using a SQL-like language to filter, group and join data streams in-memory;
- Runs collections of queries using pipelined query plans;
- Able to transparently handle complex record routing in large parallelized implementations;
- Runs and scales as a native Apache Hadoop YARN Application;
- Reads, writes, and tightly integrates with HDFS and HBase;
- Supports a significant amount of parallelization;
- Fault-tolerance and horizontal scalability without burdening the developer;
- Enterprise security features with debugging, logging, and monitoring tools; and
- Simpler programming model, tools and UI; and
- Open-source software and development process.
For more information, see our collection of Guides and other documentation.
Tigon is supported on *NIX systems such as Linux and Macintosh OS X. It is not supported on Microsoft Windows.
To install and use Tigon and its included examples, there are a few prerequisites:
- JDK 6 or JDK 7 (required to run Tigon; note that $JAVA_HOME should be set)
- GCC
- G++
- Apache Maven 3.0+ (required to build the example applications)
Note: To run the TigonSQL Stream Engine outside of Tigon, libz, Perl 5.x, and Python 3.x are required.
Pre-compiled sources and related files can be downloaded in a zip file: tigon-developer-release-0.2.1.zip.
Once the download has completed, unzip the file in a suitable location.
To run Tigon in standalone mode:
$ run_standalone.sh <path-to-flow-jar> <flow-class-name> <run-time-args>
To run Tigon in distributed mode:
$ run_distributed.sh <zookeeper-quorum> <hdfs-namespace>
You can also build Tigon directly from the latest source code:
git clone https://github.com/caskdata/tigon.git
cd tigon
mvn clean package -DskipTests -Pdist
After the build completes, you will have a distribution of Tigon under the
tigon-distribution/target/ directory.
Take the tigon-sdk-<version>.zip file and unzip it into a suitable location.
Visit our web site for a Getting Started Guide that will guide you through installing Tigon and running an example.
Now that you've had a look at the Tigon SDK, take a look at:
- Examples, located in the
/tigon-examplesdirectory of Tigon - Online Examples (demonstrating basic features of Tigon) are located on-line
- Reference Applications:
- AdNetworkFlow: Demonstrates using Tigon to write a realtime bidding (RTB) advertisement framework
- ClickStreamFlow: Demonstrates using TigonSQL to write a webpage click stream data processing application.
- Developer Guides, located in the source distribution in
/tigon-docs/developer-guide/sourceor online
Tigon is the realtime stream processing framework used in the Cask Data Application Platform (CDAP). CDAP provides a high-level Dataset abstraction for User Data Stores and allows developers to interact with them in their flowlets. In addition to the Dataset abstraction, CDAP integrates Batch Processing with MapReduce and Spark support. By combining these with security, metrics, and an intuitive GUI, CDAP greatly simplifies Big Data application development. If you like the features and architecture of Tigon, we recommend that you take a look at CDAP.
Interested in helping to improve Tigon? We welcome all contributions, whether in filing detailed bug reports, submitting pull requests for code changes and improvements, or by asking questions and assisting others on the mailing list.
Bugs and suggestions should be made by filing an Issue.
We have a simple pull-based development model with a consensus-building phase, similar to Apache's voting process. If you’d like to help make Tigon better by adding new features, enhancing existing features, or fixing bugs, here's how to do it:
- If you are planning a large change or contribution, discuss your plans on the
cask-tigon-devmailing list first. This will help us understand your needs and best guide your solution in a way that fits the project. - Fork Tigon into your own GitHub repository.
- Create a topic branch with an appropriate name.
- Work on the code to your heart's content.
- Once you’re satisfied, create a pull request from your GitHub repo (it’s helpful if you fill in all of the description fields).
- After we review and accept your request, we’ll commit your code to the cask/tigon repository.
Thanks for helping to improve Tigon!
Tigon User Group: [email protected]
Tigon Development Discussion: [email protected]
Tigon IRC Channel #tigon on irc.freenode.net
Copyright © 2014 Cask Data, Inc.
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
Cask is a trademark of Cask Data, Inc. All rights reserved.
Apache, Apache HBase, and HBase are trademarks of The Apache Software Foundation. Used with permission. No endorsement by The Apache Software Foundation is implied by the use of these marks.