Electron Bug Fixes and Miscellaneous Changes (#2441)

* migrate the downloadpage generation script to this repo

* tiny docs fix

* fix 2 electron issues

* support intel platform

* fix save preferences bug

docs/release-guide.md

@@ -1,41 +1,55 @@
 # Release Guide
 
 ## Creating release notes
+
 ### Pre-requisites
+
 - You need the correct upstream master. You should see
-```bash 
-git remote -v | grep upstream                                                                  
+
+```bash
+git remote -v | grep upstream
 upstream        [email protected]:cryptoadvance/specter-desktop.git (fetch)
 upstream        [email protected]:cryptoadvance/specter-desktop.git (push)
 ```
+
 - You need a GitHub token:
-If you don't have one, get one here https://github.com/settings/tokens and make sure to tick the boxes for repo and workflow as below:
+  If you don't have one, get one here https://github.com/settings/tokens and make sure to tick the boxes for repo and workflow as below:
 
 ![](./images/release-guide/github-token.png)
 
 Using the new token, run
- ```bash
- export GH_TOKEN=YOURTOKEN
- ```
+
+```bash
+export GH_TOKEN=YOURTOKEN
+```
+
 - You need Docker running
 - Checkout the master branch and ensure a clean workspace.
 
 Now, you can run
+
 ```bash
 ./utils/release.sh --release-notes
 ```
+
 Or, if you want to directly set the new version:
+
 ```bash
 ./utils/release.sh --new-version v1.13.1 --release-notes
 ```
+
 ## Creating a new tag
+
 Update your master branch after the release notes PR ([example](https://github.com/cryptoadvance/specter-desktop/commit/65ff6959d7fd85cba745e4d454b30031839f857f/)) has been merged and then run:
+
 ```bash
 git tag v1.13.1 && git push upstream v1.13.1
 ```
+
 ## GitLab - releasing stage
-Creating a tag triggers the release process of the GitLab runners. 
-There exists a mirror  of the GitHub repo on GitLab, but only when a tag is created on GitHub will the release part of the runners execute. You can check the status here: 
+
+Creating a tag triggers the release process of the GitLab runners.
+There exists a mirror of the GitHub repo on GitLab, but only when a tag is created on GitHub will the release part of the runners execute. You can check the status here:
 https://gitlab.com/cryptoadvance/specter-desktop/-/pipelines
 
 There are three stages:
@@ -50,14 +64,17 @@ The first relevant stage is "releasing". Here, the Windows, Linux and pip releas
 - specter_desktop-v1.13.1-x86_64-linux-gnu.tar.gz
 
 The three jobs in more detail:
+
 - release_binary_windows: is creating a binary for specterd and for Windows (Windows runner)
 - release_electron_linux_windows: Creates a specterd for Linux, an AppImage for Linux and an executable for Windows (Linux runner).
-- release_pip: Is releasing a pypi package on [pypi](https://pypi.org/project/cryptoadvance.specter/) and creates a tarball of the pip package for the GitHub release page (Linux runner). 
+- release_pip: Is releasing a pypi package on [pypi](https://pypi.org/project/cryptoadvance.specter/) and creates a tarball of the pip package for the GitHub release page (Linux runner).
 
 For details look at `.gitlab-ci.yml`
 
 ## MacOS
+
 Ideally, directly after the tag is created, start with the MacOS release. This has to be done manually, for now. There is a script for this:
+
 ```bash
 ./utils/build-osx.sh  --version v1.13.1 --appleid "Satoshi Nakamoto (appleid)" --mail "[email protected]" make-hash specterd electron sign upload
 ```
@@ -70,54 +87,37 @@ This script also runs `github.py upload `, so two more binares and the hash and
 - SHA256SUMS-macos.asc
 
 ## GitLab - post releasing
-Back to GitLab, the final stage is "post releasing". 
+
+Back to GitLab, the final stage is "post releasing".
 
 In this stage, the invididual SHA256-hashes and signatures are combined into two final files:
+
 - SHA256SUMS
 - SHA256SUMS.asc
 
 Everything, apart from the MacOS files, are pulled from the GitLab environment, the MacOS files from GitHub.
 Don't forget to delete the two MacOS files (`SHA256SUMS-macos` and `SHA256SUMS-macos.asc`) on the GitHub release page in the end.
 
 ## Trouble shooting
+
 If the MacOS signatures are missing, it can happen that the following Exception will be raised:
+
 ```bash
   File "/builds/cryptoadvance/specter-desktop/utils/github.py", line 295, in download_artifact
   raise Exception(
   Exception: Status-cod04 for url ... )
 ```
+
 In any case, if the macOS binaries arrive on GitHub too late, you have to manually delete the already created `SHA256SUMS` and `SHA256SUMS.asc`, otherwise the upload to GitHub will fail if you rerun the release signatures job on GitLab - for details see ([this PR](https://github.com/cryptoadvance/specter-desktop/pull/689)). The green arrow in the screenshot is where you rerun the release signatures job on GitLab:
 
 ![](./images/release-guide/rerun-release-signatures.png)
 
-## Editing the text on the GitHub release page
-We are running a script here to create a markdown file that can be used for copy and paste. The script uses the latest release on GitHub to create links to the artifacts in the release, so it only makes sense to run it once the previous step has been completed. 
-- Checkout this repo: `[email protected]:cryptoadvance/corp-notes.git`
-- `cd download-page`
-- Run `./build.sh`
-
-The result `gh_page.md` can be found in the build directory.
-Edit the release on GitHub and paste the md-file there along with the release notes from `release-notes.md`.
-
-## Website 
-The above script also produces html files for the website (in the same directory). The second item needs to have the release notes on GitHub, otherwise the content for the details button is missing. So, you need to re-run `./build.sh` after the release notes on GitHub are done.
-- `download-page_current.html`
-- `download-page_releases.html`
-
-Login into: https://specter.solutions/wp-login.php
-
-- Go to "Pages"
-- Hover over "Specter Desktop - Elementor" and choose `Edit with Elementor`
-- Click somewhere on area 1 (see screenshot), then somewhere on area 2, select all, delete, and paste: `download-page_current.html`
-
-![](./images/release-guide/website-1.png)\
-- Then click `update`
-Note: If you see jinja tags, you probably pasted some templates.
-
-Do the same for the "Specter Releases" part, just, in this case, replace with:\
-`download-page_releases.html`
-
-![](./images/release-guide/website-2.png)\
+## GitHub release page and download page
 
-You can then just exit the editing with Elementor.
+This is handled by the script `./utils/generate_downloadpage.sh`. As a prerequisite, you need to clone the `specter-static` repo which contains the specter website. Clone it on the same level than specter-desktop.
+Running that script will:
 
+- install the prerequisites (basically markdown, see pyproject.toml)
+- generate the GH-page and the download-page based on the `utils/templates`.
+- Asks whether it should replace/update/initialize the Github Release page for the latest version
+- copies over the new download-pages and asks whether it should commit/push those

docs/self-signed-certificates.md

@@ -1,22 +1,23 @@
 # SSL Certificate
+
 ## Why a certificate is important
 
 Browsers require secure communication with the server to use camera API. Without it we can't use QR code scanning.
 
-If you are running a VPS it's easy - you just [issue a new certificate](./reverse-proxy#adding-https) with Letsencrypt.
+If you are running a VPS it's easy - you just [issue a new certificate](./reverse-proxy.md#adding-https) with Letsencrypt.
 
 If you are only using the node at home and want to use it from your local network and via camera, you need to run it via SSL.
 
 ## Easy solution
 
 The easiest solution is to simply add `--ssl` to the serve-command and the certificate will get created automatically in the specter-home-folder.
+
 ```
 python3 -m cryptoadance.specter server --ssl
 ```
 
 ## Manual creation
 
-
 A second way, which provides more customization, is to run the [`gen-certificate.sh`](gen-certificate.sh) script in this folder with your node's IP address as an argument:
 
 ```sh
@@ -33,7 +34,7 @@ Provide these files to Specter as arguments:
 python -m cryptoadvance.specter server --cert=./cert.pem --key=./key.pem
 ```
 
-*Note:* Adding `--tor=your-tor-password` will create a tor hidden service with https.
+_Note:_ Adding `--tor=your-tor-password` will create a tor hidden service with https.
 
 ### Specter with Nginx
 
@@ -53,7 +54,7 @@ The config should look like this:
 server{
   listen 80 default_server;
   listen 443 ssl http2;
-  
+
   server_name your_domain_or_ip;
 
   ssl_certificate /etc/ssl/certs/cert.pem;

pyinstaller/electron/basic_auth.html

@@ -33,8 +33,9 @@ <h2>The server at '<span id="server">...</span>' is prompting for authorization<
             const fs = require('fs')
             const { ipcRenderer } = require('electron')
             const helpers = require('./helpers')
-            const getAppSettings = helpers.getAppSettings
-            const appSettingsPath = helpers.appSettingsPath
+            const config = require('./config')
+            const getAppSettings = config.getAppSettings
+            const appSettingsPath = config.appSettingsPath
 
             function cancel() {
                 window.close();

pyinstaller/electron/main.js

@@ -8,10 +8,11 @@ const contextMenu = require('electron-context-menu')
 const { appSettingsPath, specterdDirPath, appSettings, platformName, appNameLower } = require('./src/config.js')
 const { logger } = require('./src/logging.js')
 const downloadloc = require('./downloadloc')
-const { downloadSpecterd } = require('./src/download.js')
+const { downloadSpecterd, destroyProgressbar } = require('./src/download.js')
 const getDownloadLocation = downloadloc.getDownloadLocation
 const { startSpecterd, quitSpecterd } = require('./src/specterd.js')
-const { getFileHash, getAppSettings, versionData, isDev, devFolder, isMac } = require('./src/helpers.js')
+const { getFileHash, versionData, isDev, devFolder, isMac } = require('./src/helpers.js')
+const { getAppSettings } = require('./src/config.js')
 const { showError, updatingLoaderMsg, initMainWindow, loadUrl, initTray } = require('./src/uiHelpers.js')
 
 // Quit again if there is no version-data in dev
@@ -191,13 +192,7 @@ app.on('before-quit', (event) => {
     quitted = true
     quitSpecterd()
     if (mainWindow && !mainWindow.isDestroyed()) {
-      if (progressBar) {
-        // You can only destroy the progress bar if it hadn't been closed before
-        if (progressBar.browserWindow) {
-          progressBar.destroy()
-        }
-        progressBar = null
-      }
+      destroyProgressbar()
       mainWindow.destroy()
       mainWindow = null
       prefWindow = null

pyinstaller/electron/package.json

@@ -32,7 +32,15 @@
       "entitlements": "./build/entitlements.mac.plist",
       "hardenedRuntime": true,
       "icon": "./icons/icon.icns",
-      "provisioningProfile": "/Users/kim/Specter_Desktop_vanilla.provisionprofile"
+      "provisioningProfile": "/Users/kim/Specter_Desktop_vanilla.provisionprofile",
+      "target": [
+        {
+          "target": "dmg",
+          "arch": [
+            "universal"
+          ]
+        }
+      ]
     },
     "win": {
       "icon": "./icons/icon.ico"

pyinstaller/electron/settings.html

@@ -115,10 +115,11 @@ <h3 class="mt-8">Use a custom specterd file</h3>
             const fs = require("fs");
             const path = require("path");
             const { ipcRenderer } = require("electron");
-            const helpers = require("./helpers");
+            const helpers = require("./src/helpers");
+            const config = require("./src/config");
             const getFileHash = helpers.getFileHash;
-            const getAppSettings = helpers.getAppSettings;
-            const appSettingsPath = helpers.appSettingsPath;
+            const getAppSettings = config.getAppSettings;
+            const appSettingsPath = config.appSettingsPath;
             const specterdDirPath = helpers.specterdDirPath;
 
             function toggleAdvanced() {

pyinstaller/electron/src/config.js

@@ -104,6 +104,7 @@ module.exports = {
     specterAppLogPath,
     versionDataPath,
     versionData,
+    getAppSettings,
     isDev: isDev,
     devFolder,
 }

pyinstaller/electron/src/download.js

@@ -10,6 +10,8 @@ const ProgressBar = require('electron-progressbar')
 const { updateSpecterdStatus, updatingLoaderMsg, createProgressBar } = require('./uiHelpers.js')
 const { startSpecterd } = require('./specterd.js')
 
+let progressBar
+
 // The standard quit item cannot be replaced / modified and it is not triggering the
 // before-quit event on MacOS if a child window is open
 const dockMenuWithforceQuit = Menu.buildFromTemplate([
@@ -85,7 +87,7 @@ const download = (uri, filename, callback) => {
       let receivedBytes = 0
       const totalBytes = res.headers['content-length']
       logger.info(`Total size to download: ${totalBytes}`)
-      const progressBar = createProgressBar(totalBytes)
+      progressBar = createProgressBar(totalBytes)
       // Add Force Quit item during download for MacOS dock
       if (isMac) {
         app.dock.setMenu(dockMenuWithforceQuit)
@@ -139,7 +141,18 @@ const download = (uri, filename, callback) => {
   })
 }
 
+const destroyProgressbar = () => {
+  if (progressBar) {
+    // You can only destroy the progress bar if it hadn't been closed before
+    if (progressBar.browserWindow) {
+      progressBar.destroy()
+    }
+    progressBar = null
+  }
+}
+
 module.exports = {
   downloadSpecterd: downloadSpecterd,
   download: download,
+  destroyProgressbar,
 }

pyinstaller/electron/src/helpers.js

@@ -5,7 +5,7 @@ const crypto = require('crypto')
 const readLastLines = require('read-last-lines')
 const isMac = process.platform === 'darwin'
 
-const { versionData, appSettingsPath, specterAppLogPath } = require('./config.js')
+const { versionData, specterAppLogPath } = require('./config.js')
 const { logger } = require('./logging.js')
 
 // Use different version-data.jsons

pyinstaller/electron/src/uiHelpers.js

@@ -4,6 +4,7 @@ const { logger } = require('./logging')
 const { appSettings, isDev, platformName } = require('./config')
 const ProgressBar = require('electron-progressbar')
 const { isMac } = require('./helpers')
+const { shell } = require('electron')
 
 // Initialized with initMainWindow
 let mainWindow

pyproject.toml

@@ -85,4 +85,8 @@ test = [
     "python-gitlab==2.10.1",
     # requirements for stuff in ./utils
     "requests==2.26.0",
+]
+
+gendownloadpage = [
+    "markdown==3.3.7"
 ]