Failing data connection with STARTTLS (Explicit TLS): tls session not reused #49

wackerm · 2020-01-16T09:59:43Z

When trying to connect to ftp servers with STARTTLS (Explicit TLS) the data connection will fail with common ftp server configurations, because the tls session is not reused on the client.

Error-Message on client:
425-Unable to build data connection: Operation not permitted

Error-Message on server:
client did not reuse SSL session, rejecting data connection

Btw. this is no firewall issue: a plain connection (without encryption) works fine.

Also the data connection works, if i change the ftp server configuration (here ProFTPD) and add
TLSOptions NoSessionReuseRequired.
But this does not solve my problem, since I want to use goftp to connect to remote servers and the above configuration flag is uncommon. So i think the ssl session resuage should be solved in goftp client code.

Client log:

goftp: 0.000 #1 opening control connection to [127.0.0.1]:21
goftp: 0.003 #1 sending command AUTH TLS
goftp: 0.004 #1 got 234-AUTH TLS successful
goftp: 0.004 #1 sending command USER username
goftp: 0.025 #1 got 331-Password required for username
goftp: 0.025 #1 sending command PASS ******
goftp: 0.026 #1 got 230-User username logged in
goftp: 0.026 #1 sending command PBSZ 0
goftp: 0.027 #1 got 200-PBSZ 0 successful
goftp: 0.027 #1 sending command PROT P
goftp: 0.027 #1 got 200-Protection set to Private
goftp: 0.027 #1 successfully upgraded to TLS
goftp: 0.027 #1 sending command FEAT
goftp: 0.027 #1 got 211-Features:
 SITE MKDIR
 PBSZ
 AUTH TLS
 MFF modify;UNIX.group;UNIX.mode;
 REST STREAM
 MLST modify*;perm*;size*;type*;unique*;UNIX.group*;UNIX.mode*;UNIX.owner*;
 UTF8
 EPRT
 SITE SYMLINK
 EPSV
 SITE UTIME
 MDTM
 SITE RMDIR
 SSCN
 TVFS
 SITE COPY
 MFMT
 SIZE
 PROT
 LANG de-DE.UTF-8*
 CCC
End
goftp: 0.027 #1 sending command EPSV
goftp: 0.028 #1 got 229-Entering Extended Passive Mode (|||4771|)
goftp: 0.028 #1 opening data connection to [127.0.0.1]:4771
goftp: 0.028 #1 upgrading data connection to TLS
goftp: 0.028 #1 sending command MLSD /
goftp: 0.028 #1 got 150-Opening ASCII mode data connection for MLSD
goftp: 0.035 #1 unexpected result: 425-Unable to build data connection: Operation not permitted

Server log (ProFTPD):

2020-01-16 10:27:53,153 mod_tls/2.6[7869]: TLS/TLS-C requested, starting TLS handshake
2020-01-16 10:27:53,173 mod_tls/2.6[7869]: client supports secure renegotiations
2020-01-16 10:27:53,173 mod_tls/2.6[7869]: TLSv1.2 connection accepted, using cipher ECDHE-RSA-AES128-GCM-SHA256 (128 bits)
2020-01-16 10:27:53,176 mod_tls/2.6[7869]: Protection set to Private
2020-01-16 10:27:53,177 mod_tls/2.6[7869]: starting TLS negotiation on data connection
2020-01-16 10:27:53,183 mod_tls/2.6[7869]: client did not reuse SSL session, rejecting data connection (see the NoSessionReuseRequired TLSOptions parameter)
2020-01-16 10:27:53,184 mod_tls/2.6[7869]: unable to open data connection: TLS negotiation failed

The text was updated successfully, but these errors were encountered:

BobCashStory · 2020-01-22T15:51:39Z

Did you have a look at : #30

wackerm · 2020-01-22T18:22:36Z

Yes i have seen #30 and even tried the recommended fix. But this does not help. I think this is a different issue. Here the EPSV-Mode is not a problem, the above mentioned error exists also with PASV-Mode.
I try to find more informations about the SSL-Session-Reusage. I will dig deeper into the function prepareDataConn()...

wackerm · 2020-01-23T07:25:49Z

I found some details on tls session reusage here https://security.stackexchange.com/a/183135:

There are two types of session resumption:

Session identifiers are the original technique for implementing session resumption. These identifiers are unique values a server gives to each client. The server will store the session information alongside the session identifier. When the client connects a second time, it presents the server with the session ID, and the server will resume the session.

Session tickets are an encrypted blob of data containing information about the session that the server gives to clients. The clients will cache this ticket and will send it to the server next time it connects. The server now only has to store the key to decrypt the ticket. This is similar to a session ID, but rather than the server storing each per-client session, the server offloads this storage to the client. This is the most common form of session resumption.

So RFC5077 seems to be applicable here (https://tools.ietf.org/html/rfc5077).

It would be great if anybody has some hints how we could implement this in go...

muirdm · 2020-01-28T03:08:50Z

Go already supports session tickets via the standard library crypto/tls package. It looks like you need to set ClientSessionCache in tls.Config to enable session caching. Can you try setting ClientSessionCache (to e.g. tls.NewLRUClientSessionCache(0))?

mickael-kerjean · 2020-02-11T01:08:36Z

Same issue here as well. I tried the ClientSessionCache trick with a size of 32 and 0 and the FTPS server still complain ReadDir unexpected response: 522-SSL connection failed; session reuse required: see require_ssl_reuse option in vsftpd.conf man page

It seems TLS has 2 ways to avoid renegociating SSL on every request:

a ticket based mechanism
a session based mechanism

The catch is Golang doesn't seem to support the session based mechanism: FiloSottile/go@2a38c62. I hope I an wrong somewhere and that my understanding of TLS simply isn't good enough 🤞

wackerm · 2020-02-11T09:21:48Z

@muirdm well, i was using this configuration ClientSessionCache in tls.config right from the start, but it did not help (same experience than @mickael-kerjean)...

The issue is, that the ssl session for the data channel should be the same as the established session for the control channel.
I am not sure, if it has to be a session id based caching as session resumption could also work based on session tickets. this leads to following questions, which i could not answer at the moment:

Do common FTP-Servers (ProFTPD, vsftpd, FileZilla) support session tickets for session resumption or is this independet from the server side (i guess not)?
Is the implementation of session tickets working with the current code of goftp?
Since crypto/tls does not support session id based caching yet and the corresponding proposing issue #25228 is on hold since may 2018, there might be no help from @FiloSottile or other core devs. So
do we need a custom implementation of session id based caching for goftp?

wackerm · 2020-02-11T10:44:44Z

To answer the first question, i just checked with the source code from FileZilla Server (see AsyncSslSocketLayer.cpp, around Line 1259) which uses OpenSSL under the hood. It seems to support both session cache variants. So i guess it should work indeed also with session tickets...
By the way ProFTPD uses OpenSSL under the hood aswell (mod_tls )...

wackerm · 2020-02-11T16:11:51Z

I can confirm, that session resumption with FileZilla Server is working, because as mentioned above, FileZilla Server supports session tickets, while ProFTPD (at least in Version 1.3.5b, default in Debian Stretch) does not...
This answers question 1 and 2.

luckcolors · 2020-09-17T16:46:11Z

Hello.
I'm trying to use the library to connect to a Filezilla FTP server with implicit, but i'm running in the same problem, the client is not reusing the tls session.

Current attempts i've tried as suggested:
ClientSessionCache(32).
and not defining a session cache (haven't tried using a cache with size 0 but i don't think it makes a difference).

The error i get is specifically when attempting to upload/download anything wich causes the creation of a data connection:

EOF

Have there been any updates on this? I really would like to avoid having to disable SSL session reuse on the server.

jwatson-gcu · 2020-09-17T20:43:58Z

I think setting the cache to size zero is the key.

tls.Config.ClientSessionCache = tls.NewLRUClientSessionCache(0)

LitixThomas · 2021-12-03T15:16:24Z

No issue,

golang tls lib handle SessionTickets automatically. The tls.Config.SessionTicketsDisabled must be set to false and the important step is define a tls.Config.ServerName. Only with a valid ServerName can tls lib handle Sessionstickets correctly.

tls.Config.SessionTicketsDisabled = false // is default
tls.Config.ServerName = "<your servername>"

Tested with Filezilla Server and go1.17.3. Working for me.

yakuter · 2022-03-24T09:04:54Z

Thank you LitixThomas We were dealing with this problem for two days When we set ServerName and ClientSessionCache, it just worked smoothly 👌 Thank you very much for this great advice

dropthemasquerade · 2022-04-14T16:20:19Z

I think setting the cache to size zero is the key.

tls.Config.ClientSessionCache = tls.NewLRUClientSessionCache(0)

tlsConfig := &tls.Config{
	ClientAuth:               tls.RequestClientCert,
	InsecureSkipVerify:       true,
	//MinVersion:               tls.VersionTLS12,
	//CurvePreferences:         []tls.CurveID{tls.CurveP521, tls.CurveP384, tls.CurveP256},
	//PreferServerCipherSuites: true,
	//CipherSuites: []uint16{
	//	tls.TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,
	//	tls.TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,
	//	tls.TLS_RSA_WITH_AES_256_GCM_SHA384,
	//	tls.TLS_RSA_WITH_AES_256_CBC_SHA,
	//},
	SessionTicketsDisabled: false,
	ServerName: ftpAddr,
	ClientSessionCache: tls.NewLRUClientSessionCache(0),
}

I was trying many times, finally, ClientSessionCache: tls.NewLRUClientSessionCache(0), is working, thank you ! @jwatson-gcu

FiloSottile mentioned this issue Feb 12, 2020

proposal: crypto/tls: implement Session IDs resumption golang/go#25228

Open

ivandeex mentioned this issue Jun 16, 2021

backend/ftp: support for TLS session reuse rclone/rclone#5412

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Failing data connection with STARTTLS (Explicit TLS): tls session not reused #49

Failing data connection with STARTTLS (Explicit TLS): tls session not reused #49

wackerm commented Jan 16, 2020

BobCashStory commented Jan 22, 2020

wackerm commented Jan 22, 2020 •

edited

Loading

wackerm commented Jan 23, 2020

muirdm commented Jan 28, 2020

mickael-kerjean commented Feb 11, 2020 •

edited

Loading

wackerm commented Feb 11, 2020 •

edited

Loading

wackerm commented Feb 11, 2020

wackerm commented Feb 11, 2020

luckcolors commented Sep 17, 2020 •

edited

Loading

jwatson-gcu commented Sep 17, 2020

LitixThomas commented Dec 3, 2021

yakuter commented Mar 24, 2022 •

edited

Loading

dropthemasquerade commented Apr 14, 2022

Failing data connection with STARTTLS (Explicit TLS): tls session not reused #49

Failing data connection with STARTTLS (Explicit TLS): tls session not reused #49

Comments

wackerm commented Jan 16, 2020

BobCashStory commented Jan 22, 2020

wackerm commented Jan 22, 2020 • edited Loading

wackerm commented Jan 23, 2020

muirdm commented Jan 28, 2020

mickael-kerjean commented Feb 11, 2020 • edited Loading

wackerm commented Feb 11, 2020 • edited Loading

wackerm commented Feb 11, 2020

wackerm commented Feb 11, 2020

luckcolors commented Sep 17, 2020 • edited Loading

jwatson-gcu commented Sep 17, 2020

LitixThomas commented Dec 3, 2021

yakuter commented Mar 24, 2022 • edited Loading

dropthemasquerade commented Apr 14, 2022

wackerm commented Jan 22, 2020 •

edited

Loading

mickael-kerjean commented Feb 11, 2020 •

edited

Loading

wackerm commented Feb 11, 2020 •

edited

Loading

luckcolors commented Sep 17, 2020 •

edited

Loading

yakuter commented Mar 24, 2022 •

edited

Loading