Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(Issue #56) Support Apache Hadoop 2.2.0 #57

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

anfeng
Copy link
Contributor

@anfeng anfeng commented Nov 20, 2013

Upgrade Storm-YARN to support Apache Hadoop 2.2.0 (instead of 2.1.0-beta)

<repository>
<id>apache.snapshots</id>
<url>http://repository.apache.org/content/repositories/snapshots/</url>
</repository>
<!-- comment-out if you don't use HortonWorks Hadoop 2.2.0.2.0.6.0-76 -->
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could put these in a profile, so that instead of using comments they use a -P or a -D to trigger the profile

@goog
Copy link

goog commented Dec 17, 2013

should -D ... the default hadoop.version is 2.1.0-beta

@goog
Copy link

goog commented Dec 17, 2013

the sample test seems work with hadoop2.2.0 when following http://kingshijie.blogspot.com/2013/10/step-by-step-deploying-storm-yarn-on.html . but the Storm UI monitoring tool doesn't . @anfeng could you give me some guide?

@aperepel
Copy link

Hi, the UI worked fine for my on HDP2.0 (and in multiple clusters). Ensure
you point your browser to the right ip and port, though. Can you maybe post
more details of how it's not working for you?​

Andrew

@goog
Copy link

goog commented Dec 18, 2013

i set the storm home at the master(.151) of apache hadoop2.2.0 ,
when visit http://192.168.0.152:7070/ , Internal Server Error:

org.apache.thrift7.transport.TTransportException: java.net.ConnectException: Connection refused
    at org.apache.thrift7.transport.TSocket.open(TSocket.java:183)
    at org.apache.thrift7.transport.TFramedTransport.open(TFramedTransport.java:81)
    at backtype.storm.thrift$nimbus_client_and_conn.invoke(thrift.clj:56)
    at backtype.storm.ui.core$main_page.invoke(core.clj:126)
    at backtype.storm.ui.core$fn__7848.invoke(core.clj:741)
    at compojure.core$make_route$fn__3840.invoke(core.clj:93)
    at compojure.core$if_route$fn__3828.invoke(core.clj:39)
    at compojure.core$if_method$fn__3821.invoke(core.clj:24)
    at compojure.core$routing$fn__3846.invoke(core.clj:106)
    at clojure.core$some.invoke(core.clj:2390)
    at compojure.core$routing.doInvoke(core.clj:106)
    at clojure.lang.RestFn.applyTo(RestFn.java:139)
    at clojure.core$apply.invoke(core.clj:603)
    at compojure.core$routes$fn__3850.invoke(core.clj:111)
    at clojure.lang.Var.invoke(Var.java:415)
    at ring.middleware.reload$wrap_reload$fn__7144.invoke(reload.clj:14)
    at backtype.storm.ui.core$catch_errors$fn__7886.invoke(core.clj:798)
    at ring.adapter.jetty$proxy_handler$fn__4189.invoke(jetty.clj:16)
    at ring.adapter.jetty.proxy$org.mortbay.jetty.handler.AbstractHandler$0.handle(Unknown Source)
    at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
    at org.mortbay.jetty.Server.handle(Server.java:326)
    at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
    at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
    at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
    at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
    at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
    at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
    at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
Caused by: java.net.ConnectException: Connection refused
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
    at java.net.Socket.connect(Socket.java:579)
    at org.apache.thrift7.transport.TSocket.open(TSocket.java:178)
    ... 27 more

is it needed to install the zookeeper at hadoop master and slave nodes?
BTW: what's meaning of ''You need to install a version of storm on the hadoop gateway.'' , i have deployed storm.zip on the master node. is there something i am missing ?

@goog
Copy link

goog commented Dec 18, 2013

with the zookeeper installed , solved it .

@aperepel
Copy link

Yes, this is typically an error when ui cannot communicate with zookeeper
quorum to locate other storm components. Should be in a FAQ ;)

Andrew
On Dec 18, 2013 1:20 AM, "AsFlowers" [email protected] wrote:

with the zookeeper installed , solved it .


Reply to this email directly or view it on GitHubhttps://github.com//pull/57#issuecomment-30819707
.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants