[Feature Request] Secret Chats in Telegram Desktop

[marcovelon]

This issue is a continuation of telegramdesktop#871 and telegramdesktop#16835 and is dedicated to the code bounty campaign related to lack of implementation of a Secret Chat feature (end-to-end encryption in private messages) in Telegram Desktop.

The objective is to implement the Secret Chats feature into this Telegram client: https://github.com/telegramdesktop/tdesktop

It is possible to do it with 3 different ways (but not limited to):

• Migrating tdesktop to TDLib / Including TDLib to Telegram Desktop and using its Secret Chat API partly comment from @lukejohnsonrp
• Extracting the Secret Chats code from TDLib or telegram-api (https://github.com/vysheng/tg / https://github.com/vysheng/tgl) and reimplementing it into tdesktop
• Reimplementing it from scratch based on the official public specification available here https://core.telegram.org/api/end-to-end and https://core.telegram.org/schema/end-to-end
• Only protocol handlers need to be implemented, because all the cryptographic functions are already present in the Telegram Desktop (source)

This code bounty doesn't limit the programmer to any specific way of implementation, as soon as the final result will make possible to use Secret Chats on the open source Telegram Desktop (tdesktop) client on Linux and Windows in the exactly same way as it's done in mobile clients.

UPDATE 04/2022 / Funds distribution scheme:

There will be 3 payouts based on completion of the following stages:

• [1/3] Estabilishing Secret Chats via DH key exchange (ability to send and accept Secret Chat requests) [COMPLETE] in sergiotarxz@41fcaa8
• [2/3] Ability to send and receive encrypted messages in estabilished Secret Chats [COMPLETE] in sergiotarxz@41fcaa8
• [3/3] Re-keying (PFS) and media sharing in Secret Chats

The developer(s) will receive 1 ETH for completing one of the stages from the list above, totalling 3 ETH for all stages. The source code should be available and compilable.

The adjacent functionality such as deleting and configuring Secret Chat options can be done during any phase of the work progress described in the list above, however it must be done before or with 3/3.

I have created a verified signature for my Ethereum address containing the funds for this issue: https://etherscan.io/verifySig/4431 (https://etherscan.io/address/0xd19ee4a49b9214c4c22694bb01f225baf35f6efc)

Any voluntary donations are welcome. You can send them to the address above.

My email for communication is [email protected]

CURRENT PROJECT FUNDING:

The overall funding now is 13000 USDC and 1.18 ETH (~3929 USD)

[marcovelon]

The campaign has been funded with 2.2 ETH which is equivalent to 0.15 BTC (turned out it's impossible to fund campaigns with BTC on Gitcoin). The proof of funds is available in the invoice attached to the campaign available at https://gitcoin.co/issue/marcovelon/tdesktop/2/100026398

UPDATE 04/2022: the campaign has been withdrawn from Gitcoin because their platform is totally broken. The funds distribution will be performed manually by me after completion of steps written in the first post. The current funding is 3 ETH and doesn't limit this only to 1 coder.

[OMTDesign]

I just want to thank you on your efforts to implement secret/encrypted chats into telegram desktop. This feature has been missing since 2015, and has been asked for regularly by the community since. Yet the developers express no interest and close any issue, referring back to a post from 2015.

I wish you and the community well and hope this functionality is implemented.

[basedcookie]

Update: currently working on adding in the logic for the secret chat, GUI is almost finished. Sometime in the next week or 2 I will be uploading my local commits to my account so the code will be available. Mods who see this, let me know if the inactivity issue has been solved, I forget we have to post regular WIP on here.

[arch-btw]

Thank you so much for working on this @basedcookie.

When it's done maybe you can post a crypto address to donate to (BTC, XMR or LTC for example), because I can't for the life of me figure out how gitcoin works.

[marcovelon]

Since the bounty has been abandoned by basedcookie (no updates for more than 2 months), it is now open for a new applicant.

If you are interested to code this feature, you can enroll here: https://gitcoin.co/issue/marcovelon/tdesktop/2/100026398

[amritkumarj]

I had started working on this project, and will soon be posting the updates.

[OMTDesign]

Sad to see them @basedcookie leave. But good to see there is still interest in this project and hopefully this feature will be implemented.

[amritkumarj]

@OMTDesign Yeah, as now I am working on this project it will very soon be completed, I have expertise in cryptographic algorithms and hence it will be not a very difficult task for me

[arch-btw]

@dhruvjain1122 which approach are you taking? Migration to TDLib?

[basedcookie]

@arch-btw can't speak for them, but back when I was working on this I was trying to implement the feature directly into the existing code, personally that proved to be a little challenging and time consuming. If I were to work on it again I would take the approach of migrating to TDLib

[marcovelon]

Alright, I think many of you will agree with the fact we have reached some point of absurd never seen before in FOSS community and it feels like Telegram purposely instructs their open source relations team to avoid this topic with an utmost effort.

I am not sure what exactly is causing a nearly zero interest in this issue, however my suspections are that it's either one of those:

• lack of marketing for this issue, yet I highly doubt it, because there is a big ask for this feature since tdesktop appearance. See Secret chat telegramdesktop/tdesktop#871 [Feature Request] Secret Chats telegramdesktop/tdesktop#68 [Feature Request] Secret chat telegramdesktop/tdesktop#118 Secret Chat  telegramdesktop/tdesktop#619 Telegram Secret Chat? telegramdesktop/tdesktop#8071 [Feature Request] Add "Start secret chat" in the desktop version telegramdesktop/tdesktop#17216 no secret chat! there is no button to start a new chat telegramdesktop/tdesktop#16938 Secret chats [Question]  telegramdesktop/tdesktop#16878 [Feature Request] Secret Chats on Desktop Client telegramdesktop/tdesktop#16772 [Question] Why Desktop hasn't secret chats if Unigram has? telegramdesktop/tdesktop#16409 [Feature Request] Secret Chat on Desktop (Like Signal) telegramdesktop/tdesktop#16388 Secret chat telegramdesktop/tdesktop#10162 and so on https://github.com/telegramdesktop/tdesktop/issues?q=is%3Aissue+secret+chat - the amount of requests is ridiculously high.

• insufficient funds announced by me, however, I expect more people to donate crypto to developers who will achieve that, so shouldn't be the case either.

• the Telegram dev community itself being influenced by Pavel Durov's dubious views on IT Security with his fallacious argumentation from here https://tsf.telegram.org/manuals/e2ee-simple#2-why-are-there-no-secret-chats-on-desktop-apps . I would even say that they practice sophistry, since all those statements are extremely misleading and here is why:

Desktops are less personal. We log in from many places on web and desktop. Many of our desktops are monitored by network administrators at work, or accessed by family members at home. We often leave our desktops and even laptops unattended. It's much easier to keep an eye on your phone. [Telegram team's argumentation]

No, they are not less personal; laptops and towers can be even more personal than mobile devices.
On desktop apps we log in with the same frequency as on mobile due to same session-preserving mechanisms. Most people don't let network administrators or family members to access their personal desktops, but in case they do, they should be aware of security risks and take measures to avoid personal data leak (why not writing about this instead of making misleasing statements?). We often leave our mobile unattended and many people don't even use proper security locking mechanisms on Android phones; desktop users are covered by automatic screen locking and login screen; many desktop users also use full disk encryption. It's also easy to keep an eye on your desktop in case it's a laptop, and tower desktops should be protected by full disk encryption. Additionally, the probability of having a mobile device stolen/lost is significantly higher than for desktop. [My counter-argumentation]

Open systems vs. Sandboxed systems. Desktop systems and browsers are much more open environments, as opposed to sandboxed environments like mobile OSs. To have Secret Chats, you need your app to store data on the device, and implementing this is slightly more complicated in open systems. The Web and Desktop apps currently get all data from the cloud on startup and discard it when you quit or log out. This would mean losing all your secret chats each time you shut down your computer. [Telegram team's argumentation]

The sandboxing on mobile devices is as insecure as desktops without sandboxing ("open systems"...); there is no storage sandboxing on mobile devices and all the apps have access to the same storage, since one app can access the files shared by/with another app located on the emulated storage; also, there are lots of efficient sandboxing for desktop, such as firejail or using VMs/containers, which is much more secure than mobile OS sandboxing solutions.
To have Secret Chats, you need your app to store data on the device, and implementing this is no way more complicated on desktops, because this is how most programs (or "apps") work since the hard drive technology was created; on desktop devices, especially running Linux with encrypted home partitions or full disk encryption, there is a thing called 'local storage' which is intended to be used for such cases; this is how browsers and other network apps store all the sensitive information as well, and nothing makes tdesktop more special than a regular browser to think or implement a different solution for such a simple task as this. The Web and Desktop apps currently get most data from the local storage on startup and don't discard it when you quit or log out, so there is no point to mislead users stating the otherwise; cloud clients synchronise the data with the local storage in most use cases. This means you shouldn't lose your secret chats at all because they should be stored on the local storage, in the same way as other desktop applications do. [My counter-argumentation]

A chaos of chats with the same person. If each chat only connects two devices, users get many different chats with the same person. Imagine two people that have a laptop, a desktop at work and a mobile device each. That’s 9 secret chats in total, and most of them are useless at any given moment. [Telegram team's argumentation]

This is not a problem at all because people who use Secret Chats on mobile devices have exactly same issues when changing their phones, so they are aware of that. Also people who use E2EE chats are mostly aware of how they work and should understand that encryption keys are always stored on the local storage. Again, this is how most other desktop apps worked for decades and making a "problem" of it seems to be a misleading and unprofessioal. [My counter-argumentation]

Search-related troubles. There’s an added twist when it comes to finding a certain message. Since no server-side search for messages is possible in the E2EE paradigm, the user would have to remember on their own where to look for a particular message (was it in this chat on this device, or in that chat on that device?). Telegram wouldn’t be able to help in any way. [Telegram team's argumentation]

It's possible to search messages in Secret Chats on mobile. E2EE paradigm use client-side search for messages and this is how search is supposed to work for Secret Chats. Users don't need Telegram's help for that, only the possibility to search messages locally, which is already implemented. [My counter-argumentation]

Temporary sessions. And if that wasn’t enough, desktop sessions tend to be a lot more transient. One of the important features of secret chats is that they are destroyed on your device when you log out. Most desktop users log out rather frequently. Telegram Web users do this even more often. [Telegram team's argumentation]

Desktop sessions work in a same way as mobile sessions. Secret chats are destroyed on your device when you log out and this is why they are secret, nothing wrong about it. Most desktop users don't log out frequently and there is a big lack of context in the original statement. Telegram Web users should know better why they use Web client, it's their own choice, however, all this is not a valid argument to not implement secret chats on Web client either, since web-browsers are even used to store cryptocurrency (Metamask and such), and there is no problem with that for people who are aware of security risks; there is no problem enlightning people on security risks associated with web-browsers usage, this is what Telegram team should do instead of writing lots of misleading reasons to not implement Secure Chats. [My counter-argumentation]

I understand that most regular users recklessly follow their misleading arguments because Durov positions himself as man of knowledge in programming, while he is not, and everything he has done so far was outsourcing programmers, not even mentioning how his brother Nikolai Durov coded everything for Pavel during startups and never continued to help him because of marketing oriented ideology of Pavel (I know this from Nikolay himself from the personal conversation with him some years ago).
But what's more disappointing, is that even young developers from Telegram open source team, such as anonymous guy john-preston do share this weird ideology, for some reason supporting everything written on their page explaining "Why are Secret Chats only available on their devices of origin".

Just look at this:

john-preston commented on Jun 23, 2020
@dm17 I say that PR with a secret chat option won't be merged because I can't imagine anyone to produce such PR in such quality that it could be merged. It is a very big task that will have changes in almost all of the application logic and a huge amount of UI changes that should be discussed with the designer of the current interface, so I just don't see any reason to discuss the PR of secret chats in tdesktop — no one will do it.

For example, multiple accounts support is a much easier feature and it has a much less UI integration and still no one PR-ed it (and no one will, but I hope to implement it some day).

See how many times this guy used the wording "no one", supposing that "no one will do it" and even if someone will, he won't merge it.
It seems that we are dealing with some amateur frontend coder who doesn't understand cryptography or has no experience with coding practical client/server applications with PKI and who is assuming that just because he doesn't know how it can be done, no one else does.
So, based on his logic, there are no programmers in the entire world who can code a simple client-side encryption.

I know that this guy has triggered skepticism from many experienced and knowledgeable people in this topic who instead of trying to prove him wrong, decided to not waste their time and just to leave Telegram. Some people even tried, but Durov often refute such stuff with well-marketed sophistry that results to work on his major part of the audience, since most of them don't have any knowledge in infosec and prefer believing what he says just because they trust him.

Another reason behind all this hesitance on this issue could be that they are afraid of letting too much users to have conversations out of their control, ensuring that they can provide users communications logs to the law enforcement when asked, since their normal "cloud" chats permit to do that easily. Actually, the increased amount of censorship in their platform lately indicates that it could be true.

Anyway, I am stopping to try finding only 1 coder for this task via non-efficient platforms such as Gitcoin (whose most important features stopped working completely and devs don't have any ETA to fix their stuff) because I think it's better if more people worked on this issue as supposed to be in the open source community.
I have asked to refund my funds from Gitcoin, which was done, and from now will watch the contributions made by particular devs to this issue. Once this task is complete, I will distribute my cryptofunds to each developer based on their amount of work, which will also allow others to donate to them. We could also try to launch the campaign on https://tallyco.in purposely for funding this issue.
My Ethereum address verification with the funds prepared for payout on this issue: https://etherscan.io/verifySig/4431

If you know how we can organize this better, please let's discuss it.

p.s.
@arch-btw he became inactive on this issue since the first week of his application

@basedcookie thanks for the feedback, and while I understand that this task may have looked challenging to you and this is normal even if your coding level is good - simply because encryption is rarely employed in C/C++ coding tasks, but I still believe it's a relatively easy task for programmers who have specific experience with coding encryption algorithms on C/C++. I also think that the lack of interest for this issue by people with experience in this field is caused by Telegram's shady policies on E2EE encryption, described by me earlier.

[dm17]

@marcovelon - good eye!
Yes, I just think it is too difficult for them to get a well hidden and functional backdoor into the secret chat functionality on FOSS desktop. In this age of information warfare, I think it is lazy not to assume such things. If a Russian company can re-domicile as Emirati and then censor RT, way ahead of a war that ends up using Telegram as a primary information source on the opposing side... This is going to get complicated to analyze.
Of course I love Telegram - the engineering marvel - but intentions factor in when we have to spend years guessing about their insistence to avoid E2EE on FOSS desktop while mobile can afford two full implementations from scratch (Telegram and Telegram X)... It begs the context of mobile being more secure. Yes, phones are more secure - with respect to their own root-of-trust! Desktops - despite these black box operating systems running transparently with respect to the host OS called "management engines" - have still not removed root-of-trust from each user's control as effectively as mobile solutions (where the user is sandboxed away from having full permission and even ownership of their own hardware). The context of mobile being more secure is: mobile is a more secure and foolproof eavesdropping platform.

[marcovelon]

There are still many variables that can make phones less secure, for example users themselves, the OS being used, proprietary hardware with same hidden surprises similar to ME/PSP and so on. For example, one laptop with soldered RAM running coreboot and Debian is already more secure than most mobile devices running stock Android or iOS vulnerable to Pegasus, while one mobile phone running security-hardened AOSP with F-Droid apps only can be more secure than most desktops running Windows 10, but in any case this is already a completely another subject.

The main point is, Telegram's team have invented all this really weird argumentation as an excuse for not implementing Secure Chats on tdesktop, and that makes this whole story comical. And note how they don't even mention mobile device vs desktop platforms security, they just say that 'X is less personal than Y', which is already very vague and pointless, whatever it means.

[dm17]

@dm17, good remark. I also understand your point behind mobile devices being more secure and while what you said is valid, there are still many variables that can make phones less secure, for example users themselves, the OS being used, proprietary hardware with same hidden surprises similar to ME/PSP and so on. For example, one laptop with soldered RAM running coreboot and Debian is already more secure than most mobile devices running stock Android or iOS vulnerable to Pegasus, while one mobile phone running security-hardened AOSP with F-Droid apps only can be more secure than most desktops running Windows 10, but in any case this is already a completely another subject.

The main point is, Telegram's team have invented all this really weird argumentation as an excuse for not implementing Secure Chats on tdesktop, and that makes this whole story comical. And note how they don't even mention mobile device vs desktop platforms security, they just say that 'X is less personal than Y', which is already very vague and pointless, whatever it means.

They'll never grant you that anything can be as secure as mobile - because as I described it cannot. This is because they have a different definition of security than you or I. Their definition is: how securely can eavesdropping be executed? Smartphone security systems are far more ideal for that.
I'm not implying that they all believe this or understand this, but the "thought leaders" that they must follow to have jobs all seem to enforce this view.

[marcovelon]

They are not in a position to "grant" that to anyone while their project still has some open source community support. They would have to go fully proprietary for enforcing such views you described. Currently, they already are facing reputational losses due to more and more incoming gag orders they have to attend, thus neglecting the open source community will make them uncompetitive. Honestly, I am quite happy with truly secure platforms like Matrix, xmpp with OMEMO or Briar. The only reason why I personally have to deal with Telegram is some significant amount of contacts that find it hard or impractical (for them) to use anything more secure, and I have to be glad that they at least don't force me to use Whatsapp or other proprietary chatting clients, so I can make encrypted chat with them using a phone. Well, I admit that reading some channels is nice too, but it's manageable to live without them by substituting them with a personal collection of quality RSS feeds.

Anyway, let's give it a last chance trying finding coders for this issue, otherwise it seems that secure chats on tdesktop are never going to happen.

[Tpimp]

@marcovelon
I'm interested
And I'm just about done with my current project.

I've looked at the effort to add TDLIB to tdesktop. A huge amount of the tdesktop codebase is making QWidgets look good. As a Qt developer, I've stopped burning time on that long ago. Instead of shoehorning TDLIB into tdesktop, I think a fresh application should be written around the TDLIB library. I looked at Cutegram and in many ways its just way too much Qml to run smoothly and error free. Qml should be used to describe the UX and everything else (business logic and lower) should be raw performant C++.

Both avenues of using mtproto or generating a qml api from the schema (cutegram) seem less likely for long term success than just using TDLIB ( a choice that might not have been available at the start of these other projects).

Anyhow if there is enough interest I'd be willing to start a new project for a modern telegram desktop client.
Benefits of new client

• We start with the design first and can base it on modern Telegram application styles and techniques.
• The UX is in Qml so things that took months to build in visuals for tdesktop, instead take days.
• The code base and architecture follow a strict rule of UX is in Qml, backend in C++. Making development by future developers easier.
• Support everything we can get out of TDLIB, including secret chats
• Clear packaging/deploy strategy to provide appImages, flatpaks, and even package manager (.deb/.rpm) packages

I think a solid 4-6 months to reach 1.0 (working full time). Maybe a crowdfunding campaign? Or we could start it and just let people show up naturally and contribute over the next couple months/years. Anyone wanting to chat more about funding or collaborating on a project can find me on Device Chain discord.

[dm17]

@marcovelon I'm interested And I'm just about done with my current project.

I've looked at the effort to add TDLIB to tdesktop. A huge amount of the tdesktop codebase is making QWidgets look good. As a Qt developer, I've stopped burning time on that long ago. Instead of shoehorning TDLIB into tdesktop, I think a fresh application should be written around the TDLIB library. I looked at Cutegram and in many ways its just way too much Qml to run smoothly and error free. Qml should be used to describe the UX and everything else (business logic and lower) should be raw performant C++.

Both avenues of using mtproto or generating a qml api from the schema (cutegram) seem less likely for long term success than just using TDLIB ( a choice that might not have been available at the start of these other projects).

Anyhow if there is enough interest I'd be willing to start a new project for a modern telegram desktop client. Benefits of new client

* We start with the design first and can base it on modern Telegram application styles and techniques.

* The UX is in Qml so things that took months to build in visuals for tdesktop, instead take days.

* The code base and architecture follow a strict rule of UX is in Qml, backend in C++. Making development by future developers easier.

* Support everything we can get out of TDLIB, including [secret chats](https://core.telegram.org/tdlib/docs/classtd_1_1td__api_1_1secret_chat.html)

* Clear packaging/deploy strategy to provide appImages, flatpaks, and even package manager (.deb/.rpm) packages

I think a solid 4-6 months to reach 1.0 (working full time). Maybe a crowdfunding campaign? Or we could start it and just let people show up naturally and contribute over the next couple months/years. Anyone wanting to chat more about funding or collaborating on a project can find me on Device Chain discord.

If you wrote it from scratch, then why make it telegram-specific anyway rather than a Pidgin-like app that supports Telegram? Anyway, I refuse to believe the easiest way to get secret chats working on tdesktop for linux is rewriting it from scratch.

[Tpimp]

I think scope is the big reason for not supporting every different messaging protocol, design something that does telegram really well first....

Maybe you're right and it is easier. But I just spent nearly 2 hours tracing through the tdesktop codebase and I'm dizzy. The code itself is fine but there is a lot of QWidget specific issues all throughout and I don't feel that's a good use of a developers time as QWidgets are always going to requires tons of work to build modern UX.

Also because from my understanding the project must simultaneously change its underlying logic to use TDLIB (not just for secret chat but to migrate the other Telegram features as well) you end up with a LOT of changes. Sometimes when changing that much code on a project this big and old, it is simply a nightmare to test regressions alone.

A new clean slate prevents many of those problems and replaces them with the requirement to do some things over. Id rather participate in a fresh effort with a reduced codebase size... That is just my two cents, I could be wrong.

[dm17]

Strange to think you can redo Telegram and do better than its creators, considering it is a world-class UX (yes I realize you're not redesigning the UX but even replicating a UX entirely is difficult, no?). Furthermore, Telegram X redid it from scratch with a much larger support/funding and still hasn't accomplished all of the features of the original - and hasn't gotten as many users. Doesn't that put the rewrite into perspective?
Why are all the complicated features easy to bring to tdesktop for Linux except end-to-end encryption, which actually is totally FOSS (because it is two clients connecting and clients are open source). Versus client-server, which the telegram server is closed source.

[Tpimp]

The design and look of the application isn't being debated... but I've rewritten entire QWidget projects in qml in a fraction of the time and with better results. Google around the many projects that have done this over the last 5 years. It's like I would have to explain the logic and advantages of higher level languages.

If the attitude is such that a new effort is discouraged and mentioning of it will make people triggered and cranky, that's fine. Im looking for fun projects to work on not to change people's attitudes about software. Best of luck!

[dm17]

Not cranky at all; just wondering how you would see my questions... Figured hard questions would be useful gates for starting such a large undertaking... Especially the last one about this merely being a client<->client feature we are missing (and clients are FOSS). Not only are you free to do as you please, but myself and others in this thread would like to see you attempt it.

[Tpimp]

It's a big undertaking. I can't imagine it happening with a single individual alone.

I would be interested in joining an effort or team. But I think many have moved on from telegram. Even the developer of cutegram seems to have moved on for "Oxygen". And I'm personally more interested in the technology of the matrix protocol myself.

The single reason I would consider doing a Telegram client is because as far as I can tell there is not a Qt6 TDLib client yet. I see someone asked about linking issues on the Qt forums late last year but it could have been someone from this thread attempting to link it into tdesktop.

In theory I could help build the first modern Qt6 Telegram desktop client using the "modern" Telegram TD library with all its bells and whistles. But I'd have to really be into Telegram to do that.
Would people use it? Bigger question all together. I shouldn't have suggested you're cranky. I get you just want what seems to be a low hanging fruit of a feature in an application you already use and someone suggests "I build new application..." - I'd be annoyed too. 😄

[marcovelon]

@Tpimp thank you for manifesting interest in this.

Your proposal on starting a new Telegram client sounds interesting, but since this issue here is centered only around one specific feature, as also noted by @dm17, it seems like the current funding and organization here won't be sufficient for a scale of your proposal.
Personally, I am not ready to fund a whole new project at this time, but would surely consider sponsoring some part of it if there was a crowdfunding campaign that accepted crypto (I suggest tallyco.in). I am convinced that with your great public portfolio and experience you may be able to bring sufficient funding on that, if you created a well-formed campaign exposing a full project plan.

Meanwhile, I am considering delisting Migrating tdesktop to TDLib from possible solutions present in this issue, because:

1. funding for migrating the whole client to TDLib here is probably insufficient due to time requirements for amount of that work
2. this brings nothing else but more confusion, because devs start looking for it in a first place assuming it's the easiest solution, ending with full understanding of how many rewrites will it require after reviewing the current telegram-desktop code.
3. since we are working in a scope of the fork aiming for making a PR with Secret Chats to telegramdesktop/tdesktop, there is no point to migrate it to TDLib while telegramdesktop/tdesktop is not using it.

To make it pragmatic, we just need to implement a few client-side cryptographic methods fully specified by the official documentation. This requires minimum UI coding and most of its work is implementing the core functionality.

What can be done with TDLib in the current situation is trying to include it in telegramdesktop/tdesktop and adding Secret Chats with calls to its API, essentially for DH key exchange and encrypting/decrypting text and media.

[lukejohnsonrp]

I have dedicated a few hours to dig into particular libraries that provide support for E2EE chats in Telegram to determine complexity of this implementation and concluded that its difficulty referred by the official devs is being exaggerated.

This report will provide a better perspective of what needs to be done here.

vysheng/tgl

Reviewing https://github.com/majn/telegram-purple code in order to discover how secret chats are implemented with tgl, because it still has secret chats working.

Creating secret chats

Requesting secret chats

tgl_do_create_encr_chat_request [tgl/queries-encrypted.c] => 
get_dh_config_on_answer [tgl/queries-encrypted.c] =>
bl_do_set_dh_params [tgl/binlog.c] =>
tgl_do_send_create_encr_chat [tgl/queries-encrypted.c] => 
[state = sc_waiting] =>
bl_do_encr_chat [tgl/binlog.c] =>
send_encr_request_on_answer [tgl/queries-encrypted.c] =>
tglf_fetch_alloc_encrypted_chat [tgl/structures.c] =>
[wait for peer to accept] =>
tglu_work_update [tgl/updates.c]  =>
tglf_fetch_alloc_encrypted_chat [tgl/structures.c]=> 
[state = sc_ok] =>
bl_do_encr_chat [tgl/binlog.c] =>
tgl_do_create_keys_end [tgl/queries-encrypted.c]

Accepting secret chats

tglu_work_update [tgl/updates.c] => 
tglf_fetch_alloc_encrypted_chat [tgl/structures.c] => 
[state = sc_request] =>
bl_do_encr_chat [tgl/binlog.c] =>
tgl_do_accept_encr_chat_request [tgl/queries-encrypted.c] => 
get_dh_config_on_answer [tgl/queries-encrypted.c] => 
bl_do_set_dh_params [tgl/binlog.c] =>
[we accepted it and send response] =>
tgl_do_send_accept_encr_chat [tgl/queries-encrypted.c] => 
[state = sc_ok] =>
bl_do_encr_chat [tgl/binlog.c] =>
send_encr_accept_on_answer [tgl/queries-encrypted.c] => 
tgl_do_send_encr_chat_layer [tgl/queries-encrypted.c]

Sending secret chat messages

tgl_do_send_message [tgl/queries.c] =>
bl_do_edit_message_encr [tgl/binlog.c] =>
tgl_do_send_msg [tgl/queries.c] => 
tgl_do_send_encr_msg [tgl/queries-encrypted.c]
encr_finish [tgl/queries-encrypted.c]
encrypt_decrypted_message [tgl/queries-encrypted.c] => 
TGLC_aes_ige_encrypt [tgl/crypto/aes_altern.c] => 
do_ige_encrypt [tgl/crypto/aes_altern.c] => 
gcry_cipher_encrypt [libgcrypt]

Receiving secret chat messages

tglu_work_update [tgl/updates.c] => 
tglf_fetch_encrypted_message [tgl/structures.c] => 
decrypt_encrypted_message [tgl/structures.c] => 
TGLC_aes_ige_encrypt [tgl/crypto/aes_altern.c] => 
do_ige_decrypt [tgl/crypto/aes_altern.c] => 
gcry_cipher_decrypt [libgcrypt]

The whole implementation is quite minimalistic and straightforward. Reading its code gives full understanding of how secret chats work.

For both encryption and decryption the TGLC_aes_ige_encrypt wrapper function is used, that either call do_ige_encrypt or do_ige_decrypt for invoking gcry_cipher_encrypt / gcry_cipher_decrypt functions from libgcrypt

Currently it may crash on incoming encrypted messages, but it's not because it can't decrypt them. It looks like something related to mtproto protocol upgrades, but reviewing it wasn't within a scope of my objectives. majn/telegram-purple#567

tdlib / C++

Reviewing tdlib-purple code in order to discover how secret chats are implemented with tdlib.

tdlib-purple makes full use of abstraction that tdlib offers. No events related to encryption and decryption happen in the plugin itself, and for that reason we should observe tdlib.

Creating secret chats

Requesting secret chats

td_api::createNewSecretChat =>
MessagesManager::create_new_secret_chat =>
SecretChatsManager::create_chat =>
SecretChatActor::create_chat => 
SecretChatActor::do_create_chat_impl =>
[state = State::SendRequest] =>
SecretChatActor::run_auth => 
DhHandshake::get_g_b / auth_state_.handshake.get_g_b =>
send_net_query / QueryType::EncryptedChat / telegram_api::messages_requestEncryption / auth_state_.handshake.get_g_b =>
[auth_state_.state = State::WaitRequestResponse] =>
SecretChatActor::on_update_chat / telegram_api::encryptedChat =>
DhHandshake::set_g_a / auth_state_.handshake.set_g_a =>
DhHandshake::gen_key / auth_state_.handshake.gen_key => 
SecretChatActor::calc_key_hash =>
[auth_state_.state = State::Ready]  => 
[state = SecretChatState::Active]

Accepting secret chats

UpdatesManager::on_pending_updates =>
UpdatesManager::on_update / telegram_api::updateEncryption =>
SecretChatsManager::on_update_chat =>
SecretChatsManager::flush_pending_chat_updates =>
SecretChatsManager::do_update_chat =>
SecretChatActor::check_status =>
SecretChatActor::on_update_chat / telegram_api::encryptedChatRequested =>
[auth_state_.state = State::SendAccept] => 
DhHandshake::set_g_a / auth_state_.handshake.set_g_a => 
SecretChatActor::run_auth =>
DhHandshake::gen_key / auth_state_.handshake.gen_key => 
SecretChatActor::calc_key_hash =>
send_net_query / QueryType::EncryptedChat / telegram_api::messages_acceptEncryption / auth_state_.handshake.get_g_b  =>
SecretChatActor::on_update_chat / telegram_api::encryptedChat =>
[state = SecretChatState::Active] 

State updates during key exchange

SecretChatActor::run_auth

SecretChatActor::send_update_secret_chat

SecretChatActor::on_outbound_action / secret_api::decryptedMessageActionNotifyLayer

SecretChatActor::on_inbound_action / secret_api::decryptedMessageActionNotifyLayer

Sending secret chat messages

MessagesManager::do_send_message =>
MessagesManager::on_text_message_ready_to_send =>
MessagesManager::send_secret_message =>
SecretChatsManager::send_message =>
SecretChatActor::send_message =>
SecretChatActor::send_message_impl =>
SecretChatActor::create_encrypted_message =>
Transport::write => 
Transport::write_e2e_crypto =>
Transport::write_crypto_impl => 
Transport::calc_message_ack_and_key / Transport::calc_message_key2
KDF / KDF2 [td/mtproto/KDF.cpp]
aes_ige_encrypt [tdutils/td/utils/crypto.cpp]

Notice how Transport::write_e2e_crypto for secret chat messages and Transport::write_crypto for cloud messages share exactly same cryptography, implemented in Transport::write_crypto_impl. For more details look up Transport::write

Receiving secret chat messages

UpdatesManager::process_qts_update / telegram_api::updateNewEncryptedMessage or UpdatesManager::process_get_difference_updates / telegram_api::EncryptedMessage =>
SecretChatsManager::on_new_message =>
SecretChatsManager::add_inbound_message => 
SecretChatActor::add_inbound_message =>
SecretChatActor::do_inbound_message_encrypted =>
SecretChatActor::decrypt  =>
Transport::read =>
Transport::read_e2e_crypto =>
Transport::read_crypto_impl =>
KDF / KDF2 [td/mtproto/KDF.cpp]
aes_ige_decrypt [tdutils/td/utils/crypto.cpp]
Transport::calc_message_ack_and_key / Transport::calc_message_key2

Transport::read reads the chat for both cloud and secret chats.
For cloud chats it calls Transport::read_crypto and sets info->type to PacketInfo::Common
For secret chats it calls Transport::read_e2e_crypto and sets info->type to PacketInfo::EndToEnd
Both Transport::read_crypto and Transport::read_e2e_crypto call Transport::read_crypto_impl to decrypt messages via aes_ige_decrypt.
This means cloud and secret chats share exactly same cryptographic functions, thus they should be already present in Telegram Desktop.

Telegram Desktop / C++

A quick review of the cryptographic functions already existing in Telegram Desktop demonstrates the presence of all the functions required for secret chats implementation.

Since cloud chats use exactly same cryptographic methods used for secret chats, the necessary cryptographic implementation is already present in Telegram Desktop and can be found in the following source code files:

• Telegram/SourceFiles/mtproto/details/mtproto_dc_key_creator.cpp
• Telegram/SourceFiles/mtproto/details/mtproto_dc_key_binder.cpp
• Telegram/SourceFiles/mtproto/details/mtproto_dcenter.cpp
• Telegram/SourceFiles/mtproto/details/mtproto_bound_key_creator.cpp
• Telegram/SourceFiles/mtproto/details/mtproto_received_ids_manager.cpp
• Telegram/SourceFiles/mtproto/mtproto_dh_utils.cpp
• Telegram/SourceFiles/mtproto/mtproto_dc_options.cpp
• Telegram/SourceFiles/mtproto/mtproto_auth_key.cpp
• Telegram/SourceFiles/mtproto/session_private.cpp
• Telegram/SourceFiles/core/core_cloud_password.cpp
• Telegram/lib_base/base/openssl_help.h

For example, these AES IGE functions wrap the original deprecated openssl AES_ige_encrypt function (openssl/crypto/aes/aes_ige.c):

• aesIgeEncrypt
• aesIgeDecrypt

They are absolute equivalents for wrappers aes_ige_decrypt and aes_ige_encrypt from TDLib.

All the functionality related to DH key exchange is also present in Telegram Desktop, because mtproto use it for transport layer security and cloud chats.

Hence, what should be done to implement secret chats in Telegram Desktop is adding protocol-specific headers, routines and event binds related to secret chats, using cryptographic functions already present in Telegram Desktop.

Speaking of UI implementation:

1. Add a green lock icon
2. Add context menu items related to secret chats

And of course there is absolutely no need to migrate Telegram Desktop to TDLib in order to implement secret chats.

Unfortunately not myself a C++ coder, otherwise it would have been done, because it's really just a week of work for an average C++ coder.

[marcovelon]

Thank you for confirming my and others point of view regarding this. I believe your research will help to facilitate the implementation.

[sergiotarxz]

Hi @marcovelon are you still there? When you give signals of life I will submit the code.

Shoot him a message here mailto:[email protected]

@spidy0x0 I did it, yesterday, I hope he is doing well.

[sergiotarxz]

I have halted working on tdesktop until @marcovelon tell us something, just to do not risk myself to continue working potentially for nothing. Sorry to you all.

[sergiotarxz]

Still no answer from @marcovelon

[marcovelon]

@sergiotarxz this bounty is supposed to be paid after the code is published, thus I will only be able to perform payouts after proper code publishing.

I am not checking this thread often as I got rid of Telegram nor more interested in it since I am a proud SimpleX Chat user by now (as well as all my peers), but the bounty is still there and will be paid to you once the code is published.

[sergiotarxz]

@marcovelon I will publish this night, I just wanted to confirm there is someone to pay me when I do.

[spidy0x0]

@marcovelon I will publish this night, I just wanted to confirm there is someone to pay me when I do.

For sure

[sergiotarxz]

I have submited the code to github, but I am testing the deployment to see if it is in a functional state with all the recursive submodules.

(This comes with only main UI functionality of sending and receiving secret messages, it lacks message persistence and advanced UI for secret chats including sending media, markdown and other things)

[sergiotarxz]

Update: Some issue happened with the lib_tl submodule, it should be fixed now, I am waiting for it to end compiling and I will share instructions with you to compile by yourself when I am sure everything is ok with the publishing.

[sergiotarxz]

I have found some warnings in the compilation, I will fix them after the compilation ends, I check everything is ok and I send you instructions to compile.

[sergiotarxz]

Everything was not ok, I am fixing it, sorry.

[sergiotarxz]

I think it is fixed, to compile you will need to do the following:

Make a clean recursive clone of my repo:

git clone --recursive https://github.com/sergiotarxz/tdesktop tdesktopWithUI

Change directory into that repository:

cd tdesktopWithUI
bash Telegram/build/prepare/linux.sh
Ensure you have docker installed and running and create a compilation script for easy usage later

cat <<'EOF' > compile.sh
docker run --rm -it \
    -v "$PWD:/usr/src/tdesktop" \
    tdesktop:centos_env \
    /usr/src/tdesktop/Telegram/build/docker/centos_env/build.sh \
    -D TDESKTOP_API_ID=[Replace with your key id get it in https://my.telegram.org/] \
    -D TDESKTOP_API_HASH=[Replace with your key hash get it in https://my.telegram.org/]
EOF

Run it with:

bash compile.sh # sudo bash compile.sh if you are not in the docker group

The possibility exist that your computer is not capable to compile telegram desktop with the default settings and it causes your computer to crash, in this case you can do a few things:

• Modify /usr/src/tdesktop/Telegram/build/docker/centos_env/build.sh to set -j1 instead of --parallel
• Attempt to make a Release build instead of a Debug one
• Make a big swapfile to be less limited by your memory constraints

The resulting build will be stored in one of these directories out/Release/, out/Debug or out/RelWithDebInfo my current settings currently output to Debug. (Although it shouldn't because of build.sh configuration, but I did not bother to investigate because of the long compilation time after solving it, probably there is some typo.)

[sergiotarxz]

Shoot any doubts, bugs or defects into me, thank you.

[sergiotarxz]

@marcovelon and everybody interested.

Have you already tried it?

[spidy0x0]

@marcovelon and everybody interested.

Have you already tried it?

[sergiotarxz]

Sorry, I forgot a step.
PD: I have put the missing step in the instructions.

[ScottRFrost]

I am a proud SimpleX Chat user

@marcovelon I'm not sure what your threat model is, but SimpleX chat uses the same likely compromised encryption used by Signal and Facebook Messenger. Signal was heavily funded by the US government during its inception and has ties to the US intelligence community. I'd consider anything you write there (as well as Signal or Facebook Messenger, of course) to be compromised by any state actors.

[spidy0x0]

I think it is fixed, to compile you will need to do the following:

Make a clean recursive clone of my repo:

git clone --recursive https://github.com/sergiotarxz/tdesktop tdesktopWithUI

Change directory into that repository:

cd tdesktopWithUI bash Telegram/build/prepare/linux.sh Ensure you have docker installed and running and create a compilation script for easy usage later

cat <<'EOF' > compile.sh
docker run --rm -it \
    -v "$PWD:/usr/src/tdesktop" \
    tdesktop:centos_env \
    /usr/src/tdesktop/Telegram/build/docker/centos_env/build.sh \
    -D TDESKTOP_API_ID=[Replace with your key id get it in https://my.telegram.org/] \
    -D TDESKTOP_API_HASH=[Replace with your key hash get it in https://my.telegram.org/]
EOF

Run it with:

bash compile.sh # sudo bash compile.sh if you are not in the docker group

The possibility exist that your computer is not capable to compile telegram desktop with the default settings and it causes your computer to crash, in this case you can do a few things:

• Modify /usr/src/tdesktop/Telegram/build/docker/centos_env/build.sh to set -j1 instead of --parallel
• Attempt to make a Release build instead of a Debug one
• Make a big swapfile to be less limited by your memory constraints

The resulting build will be stored in one of these directories out/Release/, out/Debug or out/RelWithDebInfo my current settings currently output to Debug. (Although it shouldn't because of build.sh configuration, but I did not bother to investigate because of the long compilation time after solving it, probably there is some typo.)

while preparing the linux environment bash Telegram/build/prepare/linux.sh , this happened

[sergiotarxz]

It happened to me too, it appears the freedesktop git is having issues, I achieved to do it just waiting and retrying. (Sorry for not having a better solution)

Do not worry for retrying since your current progress is saved by docker.

[sergiotarxz]

I can always send you my build docker image that is correctly created already.

[spidy0x0]

I can always send you my build docker image that is correctly created already.

yeah, that would be great

[sergiotarxz]

Run this to import my docker image:

curl -O -L http://sergiotarxz.me/centos_env.tar

docker load -i centos_env.tar

docker tag c50d6b1afc5b27227e05144e3cadb1ac111e554981c38df25849270d09953e8c tdesktop:centos_env

And to run compile.sh.
@spidy0x0

[spidy0x0]

Run this to import my docker image:

curl -O -L http://sergiotarxz.me/centos_env.tar

docker load -i centos_env.tar

docker tag c50d6b1afc5b27227e05144e3cadb1ac111e554981c38df25849270d09953e8c tdesktop:centos_env

And to run compile.sh. @spidy0x0

damn its 28GB ?
huge

[sergiotarxz]

I am trying to make a slim version without history.

[sergiotarxz]

@spidy0x0 Nope, that's the real weight, history did not play a role in that. :(

[sergiotarxz]

Still waiting @marcovelon answer before developing further.

[sergiotarxz]

Marco told me he was going to review the development during the last week, but finally he probably could not.

[marcovelon]

I was able to compile and verify latest Sergio's work.

I find the meta that would fit into a description of "functional UI implementation for secret chats" only half done without them being stored persistently. Having to initiate a new secret chat every app relaunch isn't very convenient. I don't think there is much to code to implement a configuration that would store metadata information for each chat, given that (a) tdesktop already has internal APIs for persistence configuration that can be perfectly used and (b) that we are on desktop platform meaning there is no need to focus too much on additional security (i.e. encrypted config strings with some persistent account-related key/hash/whatever might be more than enough, in case TG doesn't do it already for its internal config storage). Considering that, I expect Sergio to be able to make it briefly.

Given that we are at 13000$ of total funding for UI previously mentioned by me (without considering the bonus of ~1 ETH for the minor finishing like PFS), I think it's fair to send 8000$ to Sergio now and other 5000$ once we got some basic persistence of chats without losing them every relaunch. Please type your objections below, in case any.

[spidy0x0]

Congrats @sergiotarxz 🎉
Looking forward to the chat persistence.

And also easily installable builds/releases (.deb) etc

[sergiotarxz]

I do not know about possible internal already implemented ways to persist chats, I think there are not such means.

I was planning simply to implement a sqlite database with the needed data, there are things that make it more difficult such as the need to correctly restore the order in the list of conversations which I think is going to be hell on earth.

I have to inform you that there are still pending features such as implementing auto deletion, sending media, re keying and correctly reading and writing markdown which I think you may have forgotten, so next phase is not only implementing persistence. I can give priority to that so you can start playing with the development with something you can really use for the daily life, but there is more things to do.

Also I will have to merge latest developments from tdesktop before starting to work which is always painful, unrewarding and time consuming, last time almost made me to quit. But it is very important to do that because not doing so means my development has more probability to become obsolete if telegram makes breaking changes with old versions.

My recommendation for resolution order is:

• Merge upstream and fix compatibility breakages and merge conflicts
• Persistence
• Re keying (Old chats may become unusable without that so it really affects the use of the program)
• Media and markdown
• Secret chat-specific features such as auto deletion.
• Flathub release so everybody can use it in Linux and to increase visibility and increase the chances of community members wanting to maintain the development and/or help merging this into upstream. (?)

[marcovelon]

Sent 8000 USDC in https://etherscan.io/tx/0xaa4313996e7e94e0978b92a02bb589a4d0b42f8411f55b6c459023f257b3b392 to the address Sergio sent me via email.

My recommendation for resolution order is: ...

Sounds good! I think you could make a MR after all that in any case. There is a whole team of active contributors in upstream (both paid and non-paid) actively working on making sure Telegram Premium features work in the FOSS program, so I am pretty sure some of them could help to polish this code here later. After all, according to their main spokesmen in the past issue IDs referenced here, this feature was long-awaited by all of them and the only obstacle that prevented them from implementing it was Secret Chat cryptographic functionality and autonegotiation itself, which is now implemented and they have not to worry about it anymore.