Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Connection reset by peer on ubuntu 20.04 #934

Closed
mosgjig opened this issue May 5, 2020 · 10 comments
Closed

Connection reset by peer on ubuntu 20.04 #934

mosgjig opened this issue May 5, 2020 · 10 comments
Labels
question

Comments

@mosgjig
Copy link

mosgjig commented May 5, 2020 •
edited
Loading

Expected Behavior

buildDockerImage tasks work as expected, building image via docker daemon.

Current Behavior

task execution results in:
I/O exception (java.net.SocketException) caught when processing request to {}->unix://localhost:80: Connection reset by peer
This is at odds with maven builds that work fine going against docker. additionally all docker commands work fine outside of the gradle plugin.

Context

Just updated my linux os to Pop_os 20.04 which is based on ubuntu 20.04. With that update, docker now simply installs via package manager (previous iterations had you jump through hoops)...

pop-os:~$ docker version
Client:
 Version:           19.03.8
 API version:       1.40
 Go version:        go1.13.8
 Git commit:        afacb8b7f0
 Built:             Wed Mar 11 23:42:35 2020
 OS/Arch:           linux/amd64
 Experimental:      false

Server:
 Engine:
  Version:          19.03.8
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.13.8
  Git commit:       afacb8b7f0
  Built:            Wed Mar 11 22:48:33 2020
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.3.3-0ubuntu2
  GitCommit:        
 runc:
  Version:          spec: 1.0.1-dev
  GitCommit:        
 docker-init:
  Version:          0.18.0
  GitCommit:

Steps to Reproduce (for bugs)

Setup extract:

plugins {
    id 'com.bmuschko.docker-remote-api' version '6.3.0'
}
...
docker {
    registryCredentials {
        if (project.hasProperty('repo.docker.url')) {
            url.set(project.property('repo.docker.url') as String)
        }
        if (project.hasProperty('repo.docker.username')) {
            username.set(project.property('repo.docker.username') as String)
        }
        if (project.hasProperty('repo.docker.password')) {
            password.set(project.property('repo.docker.password') as String)
        }
    }
}
...
task buildDockerImage(type: DockerBuildImage, dependsOn: createDockerfile) {
    group = BasePlugin.BUILD_GROUP
    description = 'Build docker image'

    images.add(dockerImageTag)

    doFirst {
        copy {
            from(war.archiveFile)
            into Paths.get(buildDir.path, 'docker')
        }
    }
}

The above results in:

Task ':app:buildDockerImage' is not up-to-date because:
  Task.upToDateWhen is false.
Building image using context '/home/someuser/mycompany/app/build/docker'.
Using images '/mycompany/app:1.0.0-dev_dev'.
Selected ExecutorServiceProvider implementation [com.bmuschko.gradle.docker.shaded.org.glassfish.jersey.client.DefaultClientAsyncExecutorProvider] to be used for injection of executor qualified by [com.bmuschko.gradle.docker.shaded.org.glassfish.jersey.client.ClientAsyncExecutor] annotation.
Selected ScheduledExecutorServiceProvider implementation [com.bmuschko.gradle.docker.shaded.org.glassfish.jersey.client.DefaultClientBackgroundSchedulerProvider] to be used for injection of scheduler qualified by [com.bmuschko.gradle.docker.shaded.org.glassfish.jersey.client.ClientBackgroundScheduler] annotation.

I/O exception (java.net.SocketException) caught when processing request to {}->unix://localhost:80: Connection reset by peer

> Task :app:buildDockerImage FAILED
Error during callback
com.bmuschko.gradle.docker.shaded.org.apache.http.client.ClientProtocolException
	at com.bmuschko.gradle.docker.shaded.org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:187)
	at com.bmuschko.gradle.docker.shaded.org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:72)
	at com.bmuschko.gradle.docker.shaded.org.glassfish.jersey.apache.connector.ApacheConnector.apply(ApacheConnector.java:480)
	at com.bmuschko.gradle.docker.shaded.org.glassfish.jersey.client.ClientRuntime.invoke(ClientRuntime.java:296)
	at com.bmuschko.gradle.docker.shaded.org.glassfish.jersey.client.JerseyInvocation.lambda$invoke$1(JerseyInvocation.java:623)
	at com.bmuschko.gradle.docker.shaded.org.glassfish.jersey.internal.Errors.process(Errors.java:292)
	at com.bmuschko.gradle.docker.shaded.org.glassfish.jersey.internal.Errors.process(Errors.java:274)
	at com.bmuschko.gradle.docker.shaded.org.glassfish.jersey.internal.Errors.process(Errors.java:205)
	at com.bmuschko.gradle.docker.shaded.org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:390)
	at com.bmuschko.gradle.docker.shaded.org.glassfish.jersey.client.JerseyInvocation.invoke(JerseyInvocation.java:621)
	at com.bmuschko.gradle.docker.shaded.org.glassfish.jersey.client.JerseyInvocation$Builder.method(JerseyInvocation.java:432)
	at com.bmuschko.gradle.docker.shaded.org.glassfish.jersey.client.JerseyInvocation$Builder.post(JerseyInvocation.java:333)
	at com.github.dockerjava.jaxrs.async.POSTCallbackNotifier.response(POSTCallbackNotifier.java:30)
	at com.github.dockerjava.jaxrs.async.AbstractCallbackNotifier.call(AbstractCallbackNotifier.java:51)
	at java.lang.Thread.run(Thread.java:748)
Caused by: com.bmuschko.gradle.docker.shaded.org.apache.http.client.NonRepeatableRequestException: Cannot retry request with a non-repeatable request entity
	at com.bmuschko.gradle.docker.shaded.org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:108)
	at com.bmuschko.gradle.docker.shaded.org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
	at com.bmuschko.gradle.docker.shaded.org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
	... 14 more
Caused by: java.net.SocketException: Connection reset by peer
	at org.newsclub.net.unix.NativeUnixSocket.write(Native Method)
	at org.newsclub.net.unix.AFUNIXSocketImpl$AFUNIXOutputStream.write(AFUNIXSocketImpl.java:378)
	at com.bmuschko.gradle.docker.shaded.org.apache.http.impl.io.SessionOutputBufferImpl.streamWrite(SessionOutputBufferImpl.java:124)
	at com.bmuschko.gradle.docker.shaded.org.apache.http.impl.io.SessionOutputBufferImpl.flushBuffer(SessionOutputBufferImpl.java:136)
	at com.bmuschko.gradle.docker.shaded.org.apache.http.impl.io.SessionOutputBufferImpl.write(SessionOutputBufferImpl.java:167)
	at com.bmuschko.gradle.docker.shaded.org.apache.http.impl.io.ChunkedOutputStream.flushCacheWithAppend(ChunkedOutputStream.java:122)
	at com.bmuschko.gradle.docker.shaded.org.apache.http.impl.io.ChunkedOutputStream.write(ChunkedOutputStream.java:179)
	at com.bmuschko.gradle.docker.shaded.org.glassfish.jersey.message.internal.CommittingOutputStream.write(CommittingOutputStream.java:200)
	at com.bmuschko.gradle.docker.shaded.org.glassfish.jersey.message.internal.WriterInterceptorExecutor$UnCloseableOutputStream.write(WriterInterceptorExecutor.java:276)
	at com.bmuschko.gradle.docker.shaded.org.glassfish.jersey.message.internal.ReaderWriter.writeTo(ReaderWriter.java:93)
	at com.bmuschko.gradle.docker.shaded.org.glassfish.jersey.message.internal.AbstractMessageReaderWriterProvider.writeTo(AbstractMessageReaderWriterProvider.java:56)
	at com.bmuschko.gradle.docker.shaded.org.glassfish.jersey.message.internal.InputStreamProvider.writeTo(InputStreamProvider.java:82)
	at com.bmuschko.gradle.docker.shaded.org.glassfish.jersey.message.internal.InputStreamProvider.writeTo(InputStreamProvider.java:37)
	at com.bmuschko.gradle.docker.shaded.org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.invokeWriteTo(WriterInterceptorExecutor.java:242)
	at com.bmuschko.gradle.docker.shaded.org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.aroundWriteTo(WriterInterceptorExecutor.java:227)
	at com.bmuschko.gradle.docker.shaded.org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:139)
	at com.github.dockerjava.jaxrs.filter.LoggingFilter.aroundWriteTo(LoggingFilter.java:295)
	at com.bmuschko.gradle.docker.shaded.org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:139)
	at com.bmuschko.gradle.docker.shaded.org.glassfish.jersey.message.internal.MessageBodyFactory.writeTo(MessageBodyFactory.java:1116)
	at com.bmuschko.gradle.docker.shaded.org.glassfish.jersey.client.ClientRequest.doWriteEntity(ClientRequest.java:504)
	at com.bmuschko.gradle.docker.shaded.org.glassfish.jersey.client.ClientRequest.writeEntity(ClientRequest.java:486)
	at com.bmuschko.gradle.docker.shaded.org.glassfish.jersey.apache.connector.ApacheConnector$1.writeTo(ApacheConnector.java:630)
	at com.bmuschko.gradle.docker.shaded.org.apache.http.impl.execchain.RequestEntityProxy.writeTo(RequestEntityProxy.java:121)
	at com.bmuschko.gradle.docker.shaded.org.apache.http.impl.DefaultBHttpClientConnection.sendRequestEntity(DefaultBHttpClientConnection.java:156)
	at com.bmuschko.gradle.docker.shaded.org.apache.http.impl.conn.CPoolProxy.sendRequestEntity(CPoolProxy.java:152)
	at com.bmuschko.gradle.docker.shaded.org.apache.http.protocol.HttpRequestExecutor.doSendRequest(HttpRequestExecutor.java:238)
	at com.bmuschko.gradle.docker.shaded.org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:123)
	at com.bmuschko.gradle.docker.shaded.org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
	at com.bmuschko.gradle.docker.shaded.org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
	at com.bmuschko.gradle.docker.shaded.org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
	... 16 more

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':app:buildDockerImage'.
> com.bmuschko.gradle.docker.shaded.org.apache.http.client.ClientProtocolException

Your Environment

pop-os:~$ docker info
Client:
 Debug Mode: false

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 27
 Server Version: 19.03.8
 Storage Driver: overlay2
  Backing Filesystem: <unknown>
  Supports d_type: true
  Native Overlay Diff: true
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 
 runc version: 
 init version: 
 Security Options:
  apparmor
  seccomp
   Profile: default
 Kernel Version: 5.4.0-7626-generic
 Operating System: Pop!_OS 20.04 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 12
 Total Memory: 31.21GiB
 Name: pop-os
 ID: QSAE:JS3V:PLGQ:3OB3:ZZLU:YEKT:H3NL:IPLP:JDFN:D5EU:FTHN:UR25
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Username: mosgjig
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: No swap limit support

additional info:

The following tasks seem to communicate fine with daemon:

task stopDockerContainer(type: DockerStopContainer) {
    targetContainerId dockerContainerName
    onError { ex ->
        logger.error(ex.message)
    }
}
task removeDockerContainer(type: DockerRemoveContainer, dependsOn: stopDockerContainer) {
    targetContainerId dockerContainerName
    onError { ex ->
        logger.error(ex.message)
    }
}

task removeDockerImage(type: DockerRemoveImage, dependsOn: removeDockerContainer) {
    targetImageId dockerImageTag
    onError { ex ->
        logger.error(ex.message)
    }
}

Resulting in:

> Task :app:stopDockerContainer
Caching disabled for task ':app:stopDockerContainer' because:
  Caching has not been enabled for the task
Task ':app:stopDockerContainer' is not up-to-date because:
  Task has not declared any outputs despite executing actions.
Stopping container with ID 'app'.
Selected ExecutorServiceProvider implementation [com.bmuschko.gradle.docker.shaded.org.glassfish.jersey.client.DefaultClientAsyncExecutorProvider] to be used for injection of executor qualified by [com.bmuschko.gradle.docker.shaded.org.glassfish.jersey.client.ClientAsyncExecutor] annotation.
Selected ScheduledExecutorServiceProvider implementation [com.bmuschko.gradle.docker.shaded.org.glassfish.jersey.client.DefaultClientBackgroundSchedulerProvider] to be used for injection of scheduler qualified by [com.bmuschko.gradle.docker.shaded.org.glassfish.jersey.client.ClientBackgroundScheduler] annotation.
:app:stopDockerContainer (Thread[Execution worker for ':',5,main]) completed. Took 0.039 secs.
:app:removeDockerContainer (Thread[Execution worker for ':',5,main]) started.

> Task :app:removeDockerContainer
Caching disabled for task ':app:removeDockerContainer' because:
  Caching has not been enabled for the task
Task ':app:removeDockerContainer' is not up-to-date because:
  Task has not declared any outputs despite executing actions.
Removing container with ID 'app'.
Selected ExecutorServiceProvider implementation [com.bmuschko.gradle.docker.shaded.org.glassfish.jersey.client.DefaultClientAsyncExecutorProvider] to be used for injection of executor qualified by [com.bmuschko.gradle.docker.shaded.org.glassfish.jersey.client.ClientAsyncExecutor] annotation.
Selected ScheduledExecutorServiceProvider implementation [com.bmuschko.gradle.docker.shaded.org.glassfish.jersey.client.DefaultClientBackgroundSchedulerProvider] to be used for injection of scheduler qualified by [com.bmuschko.gradle.docker.shaded.org.glassfish.jersey.client.ClientBackgroundScheduler] annotation.
:app:removeDockerContainer (Thread[Execution worker for ':',5,main]) completed. Took 0.04 secs.
:app:removeDockerImage (Thread[Execution worker for ':',5,main]) started.

> Task :app:removeDockerImage
Caching disabled for task ':app:removeDockerImage' because:
  Caching has not been enabled for the task
Task ':app:removeDockerImage' is not up-to-date because:
  Task has not declared any outputs despite executing actions.
Removing image with ID '/mycompany/app:1.0.0-dev_dev'.
Selected ExecutorServiceProvider implementation [com.bmuschko.gradle.docker.shaded.org.glassfish.jersey.client.DefaultClientAsyncExecutorProvider] to be used for injection of executor qualified by [com.bmuschko.gradle.docker.shaded.org.glassfish.jersey.client.ClientAsyncExecutor] annotation.
Selected ScheduledExecutorServiceProvider implementation [com.bmuschko.gradle.docker.shaded.org.glassfish.jersey.client.DefaultClientBackgroundSchedulerProvider] to be used for injection of scheduler qualified by [com.bmuschko.gradle.docker.shaded.org.glassfish.jersey.client.ClientBackgroundScheduler] annotation.
:app:removeDockerImage (Thread[Execution worker for ':',5,main]) completed. Took 0.041 secs.
:app:processResources (Thread[Execution worker for ':',5,main]) started.
No such container: app
No such container: app
No such image: mycompany/app:1.0.0-dev_dev
@mosgjig
Copy link
Author

mosgjig commented May 5, 2020

fyi, i've already tried specifying the docker url

docker {
    url = 'unix:///var/run/docker.sock'
    registryCredentials {
        if (project.hasProperty('repo.docker.url')) {
            url.set(project.property('repo.docker.url') as String)
        }
        if (project.hasProperty('repo.docker.username')) {
            username.set(project.property('repo.docker.username') as String)
        }
        if (project.hasProperty('repo.docker.password')) {
            password.set(project.property('repo.docker.password') as String)
        }
    }
}

but received the same error:
I/O exception (java.net.SocketException) caught when processing request to {}->unix://localhost:80: Connection reset by peer

@tdjordan
Copy link

@mosgjig , Did you ever find a solution to this issue?

@bmuschko
Copy link
Owner

Maybe that NewsClub dependency is not compatible with either the Docker version or something else in the environment. Really not sure. Pop!_OS sounds little exotic. I have not experience with it or if they do something special.

@bmuschko
Copy link
Owner

Can you try again with the latest version of the plugin (v6.6.0)? You might have been affected by #944.

@mosgjig
Copy link
Author

mosgjig commented Jul 17, 2020

Can you try again with the latest version of the plugin (v6.6.0)? You might have been affected by #944.

Sure thing, i'll get you a response early next week. Thank you for the follow up!!

@mosgjig
Copy link
Author

mosgjig commented Jul 22, 2020 •
edited
Loading

Still getting the same error, but with suppressed info:

Error during callback
java.lang.RuntimeException: java.io.IOException: native write() failed : Connection reset by peer
        at com.github.dockerjava.httpclient5.ApacheDockerHttpClientImpl.execute(ApacheDockerHttpClientImpl.java:153)
        at com.github.dockerjava.httpclient5.ApacheDockerHttpClient.execute(ApacheDockerHttpClient.java:8)
        at com.github.dockerjava.core.DefaultInvocationBuilder.execute(DefaultInvocationBuilder.java:228)
        at com.github.dockerjava.core.DefaultInvocationBuilder.lambda$executeAndStream$1(DefaultInvocationBuilder.java:269)
Caused by: java.io.IOException: native write() failed : Connection reset by peer
        at com.github.dockerjava.httpclient5.UnixDomainSocket$UnixSocketOutputStream.write(UnixDomainSocket.java:319)
        at com.bmuschko.gradle.docker.shaded.org.apache.hc.core5.http.impl.io.SessionOutputBufferImpl.flushBuffer(SessionOutputBufferImpl.java:117)
        at com.bmuschko.gradle.docker.shaded.org.apache.hc.core5.http.impl.io.SessionOutputBufferImpl.write(SessionOutputBufferImpl.java:150)
        at com.bmuschko.gradle.docker.shaded.org.apache.hc.core5.http.impl.io.ChunkedOutputStream.flushCacheWithAppend(ChunkedOutputStream.java:125)
        at com.bmuschko.gradle.docker.shaded.org.apache.hc.core5.http.impl.io.ChunkedOutputStream.write(ChunkedOutputStream.java:204)
        at com.bmuschko.gradle.docker.shaded.org.apache.hc.core5.http.io.entity.InputStreamEntity.writeTo(InputStreamEntity.java:96)
        at com.bmuschko.gradle.docker.shaded.org.apache.hc.client5.http.impl.classic.RequestEntityProxy.writeTo(RequestEntityProxy.java:106)
        at com.bmuschko.gradle.docker.shaded.org.apache.hc.core5.http.impl.io.DefaultBHttpClientConnection.sendRequestEntity(DefaultBHttpClientConnection.java:153)
        at com.bmuschko.gradle.docker.shaded.org.apache.hc.core5.http.impl.io.HttpRequestExecutor.execute(HttpRequestExecutor.java:148)
        at com.github.dockerjava.httpclient5.HijackingHttpRequestExecutor.execute(HijackingHttpRequestExecutor.java:50)
        at com.bmuschko.gradle.docker.shaded.org.apache.hc.core5.http.impl.io.HttpRequestExecutor.execute(HttpRequestExecutor.java:225)
        at com.bmuschko.gradle.docker.shaded.org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager$InternalConnectionEndpoint.execute(PoolingHttpClientConnectionManager.java:596)
        at com.bmuschko.gradle.docker.shaded.org.apache.hc.client5.http.impl.classic.InternalExecRuntime.execute(InternalExecRuntime.java:220)
        at com.bmuschko.gradle.docker.shaded.org.apache.hc.client5.http.impl.classic.MainClientExec.execute(MainClientExec.java:107)
        at com.bmuschko.gradle.docker.shaded.org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
        at com.bmuschko.gradle.docker.shaded.org.apache.hc.client5.http.impl.classic.ExecChainElement$1.proceed(ExecChainElement.java:57)
        at com.bmuschko.gradle.docker.shaded.org.apache.hc.client5.http.impl.classic.ConnectExec.execute(ConnectExec.java:181)
        at com.bmuschko.gradle.docker.shaded.org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
        at com.bmuschko.gradle.docker.shaded.org.apache.hc.client5.http.impl.classic.ExecChainElement$1.proceed(ExecChainElement.java:57)
        at com.bmuschko.gradle.docker.shaded.org.apache.hc.client5.http.impl.classic.ProtocolExec.execute(ProtocolExec.java:165)
        at com.bmuschko.gradle.docker.shaded.org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
        at com.bmuschko.gradle.docker.shaded.org.apache.hc.client5.http.impl.classic.ExecChainElement$1.proceed(ExecChainElement.java:57)
        at com.bmuschko.gradle.docker.shaded.org.apache.hc.client5.http.impl.classic.HttpRequestRetryExec.execute(HttpRequestRetryExec.java:93)
        at com.bmuschko.gradle.docker.shaded.org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
        at com.bmuschko.gradle.docker.shaded.org.apache.hc.client5.http.impl.classic.ExecChainElement$1.proceed(ExecChainElement.java:57)
        at com.bmuschko.gradle.docker.shaded.org.apache.hc.client5.http.impl.classic.RedirectExec.execute(RedirectExec.java:116)
        at com.bmuschko.gradle.docker.shaded.org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
        at com.bmuschko.gradle.docker.shaded.org.apache.hc.client5.http.impl.classic.ExecChainElement$1.proceed(ExecChainElement.java:57)
        at com.bmuschko.gradle.docker.shaded.org.apache.hc.client5.http.impl.classic.ContentCompressionExec.execute(ContentCompressionExec.java:128)
        at com.bmuschko.gradle.docker.shaded.org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
        at com.bmuschko.gradle.docker.shaded.org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:178)
        at com.bmuschko.gradle.docker.shaded.org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:67)
        at com.github.dockerjava.httpclient5.ApacheDockerHttpClientImpl.execute(ApacheDockerHttpClientImpl.java:149)
        ... 3 more
        Suppressed: java.io.IOException: native write() failed : Broken pipe
                at com.github.dockerjava.httpclient5.UnixDomainSocket$UnixSocketOutputStream.write(UnixDomainSocket.java:319)
                at com.bmuschko.gradle.docker.shaded.org.apache.hc.core5.http.impl.io.SessionOutputBufferImpl.flushBuffer(SessionOutputBufferImpl.java:117)
                at com.bmuschko.gradle.docker.shaded.org.apache.hc.core5.http.impl.io.SessionOutputBufferImpl.flush(SessionOutputBufferImpl.java:126)
                at com.bmuschko.gradle.docker.shaded.org.apache.hc.core5.http.impl.io.ChunkedOutputStream.close(ChunkedOutputStream.java:228)
                at com.bmuschko.gradle.docker.shaded.org.apache.hc.core5.http.impl.io.DefaultBHttpClientConnection.sendRequestEntity(DefaultBHttpClientConnection.java:154)
                at com.bmuschko.gradle.docker.shaded.org.apache.hc.core5.http.impl.io.HttpRequestExecutor.execute(HttpRequestExecutor.java:148)
                at com.github.dockerjava.httpclient5.HijackingHttpRequestExecutor.execute(HijackingHttpRequestExecutor.java:50)
                at com.bmuschko.gradle.docker.shaded.org.apache.hc.core5.http.impl.io.HttpRequestExecutor.execute(HttpRequestExecutor.java:225)
                at com.bmuschko.gradle.docker.shaded.org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager$InternalConnectionEndpoint.execute(PoolingHttpClientConnectionManager.java:596)
                at com.bmuschko.gradle.docker.shaded.org.apache.hc.client5.http.impl.classic.InternalExecRuntime.execute(InternalExecRuntime.java:220)
                at com.bmuschko.gradle.docker.shaded.org.apache.hc.client5.http.impl.classic.MainClientExec.execute(MainClientExec.java:107)
                at com.bmuschko.gradle.docker.shaded.org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
                at com.bmuschko.gradle.docker.shaded.org.apache.hc.client5.http.impl.classic.ExecChainElement$1.proceed(ExecChainElement.java:57)
                at com.bmuschko.gradle.docker.shaded.org.apache.hc.client5.http.impl.classic.ConnectExec.execute(ConnectExec.java:181)
                at com.bmuschko.gradle.docker.shaded.org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
                at com.bmuschko.gradle.docker.shaded.org.apache.hc.client5.http.impl.classic.ExecChainElement$1.proceed(ExecChainElement.java:57)
                at com.bmuschko.gradle.docker.shaded.org.apache.hc.client5.http.impl.classic.ProtocolExec.execute(ProtocolExec.java:165)
                at com.bmuschko.gradle.docker.shaded.org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
                at com.bmuschko.gradle.docker.shaded.org.apache.hc.client5.http.impl.classic.ExecChainElement$1.proceed(ExecChainElement.java:57)
                at com.bmuschko.gradle.docker.shaded.org.apache.hc.client5.http.impl.classic.HttpRequestRetryExec.execute(HttpRequestRetryExec.java:93)
                at com.bmuschko.gradle.docker.shaded.org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
                at com.bmuschko.gradle.docker.shaded.org.apache.hc.client5.http.impl.classic.ExecChainElement$1.proceed(ExecChainElement.java:57)
                at com.bmuschko.gradle.docker.shaded.org.apache.hc.client5.http.impl.classic.RedirectExec.execute(RedirectExec.java:116)
                at com.bmuschko.gradle.docker.shaded.org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
                at com.bmuschko.gradle.docker.shaded.org.apache.hc.client5.http.impl.classic.ExecChainElement$1.proceed(ExecChainElement.java:57)
                at com.bmuschko.gradle.docker.shaded.org.apache.hc.client5.http.impl.classic.ContentCompressionExec.execute(ContentCompressionExec.java:128)
                at com.bmuschko.gradle.docker.shaded.org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
                at com.bmuschko.gradle.docker.shaded.org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:178)
                at com.bmuschko.gradle.docker.shaded.org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:67)
                at com.github.dockerjava.httpclient5.ApacheDockerHttpClientImpl.execute(ApacheDockerHttpClientImpl.java:149)
                at com.github.dockerjava.httpclient5.ApacheDockerHttpClient.execute(ApacheDockerHttpClient.java:8)
                at com.github.dockerjava.core.DefaultInvocationBuilder.execute(DefaultInvocationBuilder.java:228)
                at com.github.dockerjava.core.DefaultInvocationBuilder.lambda$executeAndStream$1(DefaultInvocationBuilder.java:269)
                at java.base/java.lang.Thread.run(Thread.java:834)

Other info:

$ docker info
Client:
 Debug Mode: false

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 27
 Server Version: 19.03.8
 Storage Driver: overlay2
  Backing Filesystem: <unknown>
  Supports d_type: true
  Native Overlay Diff: true
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 
 runc version: 
 init version: 
 Security Options:
  apparmor
  seccomp
   Profile: default
 Kernel Version: 5.4.0-7634-generic
 Operating System: Pop!_OS 20.04 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 12
 Total Memory: 31.21GiB
 Name: pop-os
 ID: QSAE:JS3V:PLGQ:3OB3:ZZLU:YEKT:H3NL:IPLP:JDFN:D5EU:FTHN:UR25
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Username: mosgjig
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: No swap limit support

$ docker version
Client:
 Version:           19.03.8
 API version:       1.40
 Go version:        go1.13.8
 Git commit:        afacb8b7f0
 Built:             Tue Jun 23 22:26:12 2020
 OS/Arch:           linux/amd64
 Experimental:      false

Server:
 Engine:
  Version:          19.03.8
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.13.8
  Git commit:       afacb8b7f0
  Built:            Thu Jun 18 08:26:54 2020
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.3.3-0ubuntu2
  GitCommit:        
 runc:
  Version:          spec: 1.0.1-dev
  GitCommit:        
 docker-init:
  Version:          0.18.0
  GitCommit:

@bmuschko
Copy link
Owner

You may want to open the issue against the Docker Java project. Based on the stack trace, it looks more like an issue on their end. Thanks!

@bmuschko
Copy link
Owner

Closing this issue here. I think it's a Docker Java issue.

@tlefevre
Copy link

I just ran into this issue and found this. My error was this:

Error during callback java.lang.RuntimeException: java.io.IOException: com.sun.jna.LastErrorException: [104] Connection reset by peer at com.github.dockerjava.httpclient5.ApacheDockerHttpClientImpl.execute(ApacheDockerHttpClientImpl.java:187) at com.github.dockerjava.httpclient5.ApacheDockerHttpClient.execute(ApacheDockerHttpClient.java:9)

I messed up the docker tag, which was the reason it broke like this.

Journactl had the following error for the docker.service: level=error msg="Handler for POST /build returned error: invalid reference format"

Which led me to go back and check the tag being produced. Just for future reference if anyone else runs into the same problem. Benjamin is correct that the problem lies with the docker-java client (or docker).

@MarkRx
Copy link
Contributor

MarkRx commented Dec 18, 2023

For anyone who stumbles across this in the future: Check the docker service or unix socket for a timeout setting. See #1209

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
question
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@tdjordan @bmuschko @mosgjig @tlefevre @MarkRx