Skip to content

Commit

Permalink
Merge pull request #37 from GoogleCloudPlatform/warning3
Browse files Browse the repository at this point in the history
Add README To Use Temporary Hbase Fork
  • Loading branch information
lesv committed May 13, 2015
2 parents fe4e42a + 2906de5 commit e98f200
Show file tree
Hide file tree
Showing 3 changed files with 135 additions and 37 deletions.
8 changes: 3 additions & 5 deletions python/README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
# Python REST/Thrift Examples

# WARNING:
## These examples are temporarily broken
Bigtable REST/Thrift Gateway support is reliant on patches being merged
into the open-source HBase client. We are currently working on getting
these patches merged into the HBase release. Check back shortly for updates.

These examples demonstrate how to use the HBase client to serve as a REST or
Thrift gateway to Google Cloud Bigtable.
78 changes: 62 additions & 16 deletions python/rest/README.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# Python REST/Thrift Examples
# Python REST Examples

# WARNING:
## These examples are temporarily broken
Bigtable REST/Thrift Gateway support is reliant on patches being merged
into the open-source HBase client. We are currently working on getting
these patches merged into the HBase release. Check back shortly for updates.
This example demonstrates how to use the HBase client to serve as a
REST Gateway to Cloud Bigtable. They involve two steps: first installing
and configuring an HBase client to serve as the REST gateway, and second
installing and configuring a REST client. In this example we use a
Python REST client using the [requests](http://docs.python-requests.org/en/latest/) library.

## Cloud Bigtable Python REST Examples

Expand All @@ -13,21 +13,67 @@ calls to interact with an HBase REST gateway to Google Cloud Bigtable. It is
not an extensive library, but rather a simple demonstration of some common
operations.

## Project setup, installation, deployment, and configuration
## HBase REST Gateway setup and configuration

First follow the instructions to create a Google Cloud project, enable Cloud
Bigtable, and then the instructions on using bdutil to setup an HBase gateway.
You can download our temporary HBase client fork here:

https://docs.google.com/document/d/1k_NwGTYvInFZ_a0AVQ2LXBsdGV53bbqf_uD6586uoTc/edit
[Google HBase Release](https://github.com/GoogleCloudPlatform/cloud-bigtable-examples/releases/tag/v0.1.5)

Start the REST server in the background
****************************************************************************************************
IMPORTANT -- The HBase temporary fork is a SNAPSHOT of hbase-1.0.1 that allows users to use
HBase with Bigtable on Google Cloud Platform. These changes [1]
(https://issues.apache.org/jira/browse/HBASE-12993)
[2](https://issues.apache.org/jira/browse/HBASE-13664) have been submitted and accepted by the Apache
HBase project and once they are released we will no longer offer this TEMPORARY fork of HBase.
***************************************************************************************************

`hbase rest start`

Unless you run these scripts on the HBase instance itself, you will have to
open up the appropriate port:
************************************************************************************************
If you prefer, you can download the HBase src releases, and apply our patches.

`gcloud compute firewall-rules create hbase-rest --allow=tcp:8080`

`curl -f -O http://mirror.reverse.net/pub/apache/hbase/hbase-1.0.1/hbase-1.0.1-src.tar.gz`

`tar -xzf hbase-1.0.1-src.tar.gz`

`cd hbase-1.0.1`

`patch -p1 < fix-bigtable-rest-thrift.patch`

`MAVEN_OPTS="-Xmx2g -XX:MaxPermSize=2g" mvn install -DskipTests
assembly:single`

The release is built in hbase-assembly/target/hbase-1.0.1-bin.tar.gz
************************************************************************************************


Once you have the HBase client, instructions for installing an HBase client for
Cloud Bigtable can be found here:

https://cloud-dot-devsite.googleplex.com/bigtable/docs/installing-hbase-client

Instead of the official HBase release in the section. "Downloading required
files", you use the forked HBase binaries instead.

Then, to start the REST gateway, from the HBase release directory

`./bin/hbase rest start`

If you would like to connect to your REST gateway using your external IP on a
GCE instance, you will have to open up a firewall port.

`gcloud compute firewall-rules create <instance_name> --allow=tcp:8080`

Note the security risk of an open firewall port, and also note that you can
connect to the HBase gateway from a different GCE instance without opening up
a firewall port using the private internal IP instead of the external IP.

The internal IP can be found in the [Google Cloud Console](console.developer
.google.com) by going to Compute Engine > VM Instances and then clicking
on your instance.


## REST client setup and configuration

On a client machine of your choice, change `baseurl` to match the external IP of
the rest server.
Expand Down Expand Up @@ -55,4 +101,4 @@ or

`python put_get_with_client.py`

should both print "Done!" if all the operations succeed.
should both print "Done!" if all the operations succeed.
86 changes: 70 additions & 16 deletions python/thrift/README.md
Original file line number Diff line number Diff line change
@@ -1,26 +1,80 @@
# WARNING:
## These examples are temporarily broken
Bigtable REST/Thrift Gateway support is reliant on a few patches being merged
into the open-source HBase client. We are currently working on getting
these patches merged into the HBase release. Check back shortly for updates.
# Cloud Bigtable Python Thrift Examples

## Cloud Bigtable Python Thrift Examples
This example demonstrates how to use the HBase client to serve as a
Thrift Gateway to Cloud Bigtable. They involve two steps: first installing
and configuring an HBase client to serve as the [Apache Thrift]
(https://thrift.apache.org/) gateway, and second installing and configuring a
Thrift client. In this example, we use a Python Thrift client configured on a
GCE instance. Thrift supports many other languages besides Python, such as
NodeJS, Ruby, and C#, and can be installed on many other platforms besides a
GCE Debian instance, but the installation instructions vary by platform.

This project demonstrates how to start with a fresh GCE instance, and install
the correct system libraries to build a Thrift client for the Google Cloud
Bigtable HBase Thrift Gateway. These assume you already have the Thrift
Gateway setup and running, following the linked instructions.

While the examples here use Thrift to generate Python, Thrift can generate
code for many languages such as NodeJS, Ruby, and C#.
## Installing an HBase Thrift Gateway

## Instructions
You can download our temporary HBase client fork here:

First, follow the instructions to get a Thrift HBase server setup using bdutil.
[Google HBase Release](https://github.com/GoogleCloudPlatform/cloud-bigtable-examples/releases/tag/v0.1.5)

https://docs.google.com/document/d/1NqjtdLiY5T2bZADanbWktUQ1-qP2bDgFnOf_3lX_PbA/edit

Then provision a new instance to serve as the client:
****************************************************************************************************
IMPORTANT -- The HBase temporary fork is a SNAPSHOT of hbase-1.0.1 that allows users to use
HBase with Bigtable on Google Cloud Platform. These changes [1]
(https://issues.apache.org/jira/browse/HBASE-12993)
[2](https://issues.apache.org/jira/browse/HBASE-13664) have been submitted and accepted by the Apache
HBase project and once they are released we will no longer offer this TEMPORARY fork of HBase.
***************************************************************************************************

************************************************************************************************
If you prefer, you can download the HBase src releases, and apply our patches.


`curl -f -O http://mirror.reverse.net/pub/apache/hbase/hbase-1.0.1/hbase-1.0.1-src.tar.gz`

`tar -xzf hbase-1.0.1-src.tar.gz`

`cd hbase-1.0.1`

`patch -p1 < fix-bigtable-rest-thrift.patch`

`MAVEN_OPTS="-Xmx2g -XX:MaxPermSize=2g" mvn install -DskipTests
assembly:single`

The release is built in hbase-assembly/target/hbase-1.0.1-bin.tar.gz
************************************************************************************************


Once you have the HBase client, instructions for installing an HBase client for
Cloud Bigtable can be found here:

https://cloud-dot-devsite.googleplex.com/bigtable/docs/installing-hbase-client

Instead of the official HBase release in the section "Downloading required
files", you use the forked HBase binaries instead.

Then, to start the Thrift gateway, from the HBase release directory

`./bin/hbase thrift start`

If you would like to connect to your Thrift gateway using your external IP on a
GCE instance, you will have to open up a firewall port.

`gcloud compute firewall-rules create <instance_name> --allow=tcp:9090`

Note the security risk of an open firewall port, and also note that you can
connect to the HBase gateway from a different GCE instance without opening up
a firewall port using the private internal IP instead of the external IP.

The internal IP can be found in the [Google Cloud Console](console.developer
.google.com) by going to Compute Engine > VM Instances and then clicking
on your instance.

## Installing an HBase Thrift Client

For this example, we will install Thrift on a separate machine from our
gateway interface

First provision a new instance to serve as the client:

`$ gcloud compute instances create thrift-client`

Expand Down

0 comments on commit e98f200

Please sign in to comment.