Pass contentLength to the worker

[mike76-dev]

There is an unused contentLength field in s3.UploadPart. On the other hand, when a file is uploaded from the renterd UI, the Content-Length header is always set.
Unfortunately, when I try to set the Content-Length header in the API request to the worker, it gets stripped out (probably because of the encoding). For this reason, I set X-Content-Length, which can also be checked by the handler.

[peterjan]

Hi @mike76-dev, thanks for your contribution, could you give a little more context to the PR, I don't really follow how or why it is useful and where we are using the Content-Length. If we do decide to add it I think we'd add it as an upload option rather than an extra parameter too by the way.

[mike76-dev]

I'm encrypting the data in renterd before it goes over to the satellite. The satellite can store the file metadata, the partial slabs, the files uploaded by the renter pending uploading to the network, or the parts of a multipart upload. The additional encryption is important, because it prevents a malicious satellite operator from accessing the filenames or the contents of the files. The data is decrypted back either in renterd or in the dashboard of the satellite web portal, and only the renter has the encryption key.
Now, when parts of a multipart upload are uploaded to the satellite, it has to put them together and upload to the network. If each part was encrypted separately, it's not possible to decrypt back the complete file, because the offset of an individual part is unknown during upload to the satellite.
I've found a workaround where I want to prepend each part with its length and then encrypt using ChaCha20. This works, but I need to know the length of the piece of data to encrypt. It's not a problem with conventional single-part uploads, because the Content-Length header is usually set correctly. But I'm missing that with the multipart uploads.
If you are OK with that, I can rewrite the code to have the content length as an upload option.

[ChrisSchinnerl]

Try setting req.ContentLength = contentLength instead. That way the Content-Length header will be set.
But contentLength should be part of opts and only be set if opts.ContentLength != 0.

It would also help us a lot if you did the same for PutObject in backend.go since that also seems to have a size that is currently not used.

[ChrisSchinnerl]

If you want to go the extra mile you can extend the client methods to try and cast r.(io.Seeker) iff opts.ContentLength = 0 and then seek to the end and back to the beginning to determine the content length of the input. That would make sure that calling the client methods with an os.File or bytes.Reader will set the content length implicitly.

[mike76-dev]

That would make sure that calling the client methods with an os.File or bytes.Reader will set the content length implicitly.

So, no need to set req.ContentLength = <offset> in this case?

[ChrisSchinnerl]

No, in that case we don't set it and just use the seeker.

[ChrisSchinnerl]

Oh sry I misunderstood your question. We still need to set the content length. Just not from the input but from the offset of the seeker.

Please also make sure to handle all errors

[ChrisSchinnerl]

Suggested change

	Size int64
	ContentLength int64

[ChrisSchinnerl]

Thanks for the contribution!