Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Heap Space error #6

Open
asalas opened this issue Feb 27, 2018 · 0 comments
Open

Heap Space error #6

asalas opened this issue Feb 27, 2018 · 0 comments

Comments

@asalas
Copy link

asalas commented Feb 27, 2018

Hello,

I'm trying to execute pg2bq connected to a remote host, the connection is fine, but the application throws the next exception:

java.lang.OutOfMemoryError: Java heap space at java.util.concurrent.atomic.AtomicReferenceFieldUpdater.newUpdater(AtomicReferenceFieldUpdater.java:109) at java.sql.SQLException.<clinit>(SQLException.java:372) at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2119) at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:286) at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:432) at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:358) at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:171) at org.postgresql.jdbc.PgPreparedStatement.executeQuery(PgPreparedStatement.java:119) at org.apache.spark.sql.execution.datasources.jdbc.JDBCRDD.compute(JDBCRDD.scala:301) at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323) at org.apache.spark.rdd.RDD.iterator(RDD.scala:287) at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38) at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323) at org.apache.spark.rdd.RDD.iterator(RDD.scala:287) at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87) at org.apache.spark.scheduler.Task.run(Task.scala:108) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:335) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2018-02-27 18:42:46,758 [Executor task launch worker for task 0] ERROR org.apache.spark.util.SparkUncaughtExceptionHandler - Uncaught exception in thread Thread[Executor task launch worker for task 0,5,main] java.lang.OutOfMemoryError: Java heap space

Can you help me to configure correctly the usage of pg2bq to increase the memory usage?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant