Cannot get yarnAdminServer to run with mysql2 updates

[toni-sharpe]

This is fixed (possibly).

I was using startAdminServer and should have been using startAdminDevServer but I usually rely on my history in ITerm, not my memory, so must have accidentally had that in my history for some reason.

Still, all seems to be working for me with this so I'll close. Details belowe in case anyone wants a look, but there is still a problem with startAdminServer for me locally ...

I leave this for someone with better knowledge than me to close, details below.

================

Description

I ususally run the TMux alongside yarn startAdminServer so that I can get at the :3030 port and develop locally.

However, I think this recent update has caused problems there. It uses mysql2 in Knex whereas from the error output below, it seems yarn startAdminServer requires mysql

➜  owid-grapher git:(master) yarn startAdminServer
node:internal/modules/cjs/loader:1078
  throw err;
  ^

Error: Cannot find module 'mysql'
Require stack:
- /owid-grapher/itsJustJavascript/db/DatabaseConnection.js
- /owid-grapher/itsJustJavascript/db/wpdb.js
- /owid-grapher/itsJustJavascript/adminSiteServer/app.js
    at Module._resolveFilename (node:internal/modules/cjs/loader:1075:15)
    at Module._load (node:internal/modules/cjs/loader:920:27)
    at Module.require (node:internal/modules/cjs/loader:1141:19)
    at require (node:internal/modules/cjs/helpers:110:18)
    at Object.<anonymous> (/owid-grapher/db/DatabaseConnection.ts:1:1)
    at Module._compile (node:internal/modules/cjs/loader:1254:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1308:10)
    at Module.load (node:internal/modules/cjs/loader:1117:32)
    at Module._load (node:internal/modules/cjs/loader:958:12)
    at Module.require (node:internal/modules/cjs/loader:1141:19) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
    '/owid-grapher/itsJustJavascript/db/DatabaseConnection.js',
    '/owid-grapher/itsJustJavascript/db/wpdb.js',
    '/owid-grapher/itsJustJavascript/adminSiteServer/app.js'
  ]
}

Node.js v18.16.0

Expected behaviour

It should work with mysql2 (or maybe, I am getting port :3030 in an odd fashion and there is a better way, but even so, it's still broken).

Steps to reproduce

I think this shouold just happen if you pull down the latest master, run to install then try yarn startAdminServer

Screenshots

N/A see text output

Environment

MacOS M1 Sonoma 14.5
ITerm

Notes

Everything seems fine: the DB is visible in DBeaver, TMux is running without any error panels. Docker Desktop is working fine too. It's just this one command that won't start and let the :3030 port work. My fork is synced, I am on the master branch and I have definitely pulled the code locally.

Full outputs

Full TMux Docker out

Last login: Thu Jul  4 06:30:54 on ttys006
➜  owid-grapher git:(master) make up
==> Checking your local environment has the necessary commands...
==> Validating your .env file for make up
.env file valid for make up
==> Checking port
==> Building grapher
yarn lerna run build
lerna notice cli v8.1.3

   ✔  @ourworldindata/types:build  [existing outputs match the cache, left as is]
   ✔  @ourworldindata/utils:build  [existing outputs match the cache, left as is]
   ✔  @ourworldindata/components:build  [existing outputs match the cache, left as is]
   ✔  @ourworldindata/core-table:build  [existing outputs match the cache, left as is]

                       | |
  __,  ,_    __,    _  | |     _   ,_
 /  | /  |  /  |  |/ \_|/ \   |/  /  |
 \_/|/   |_/\_/|_/|__/ |   |_/|__/   |_/
   /|            /|

[+] Running 8/8
 ✔ db 7 layers [⣿⣿⣿⣿⣿⣿⣿]      0B/0B      Pulled                                                                  2.8s
   ✔ 022417830bcf Already exists                                                                                 0.0s
   ✔ fe10b20dcdae Already exists                                                                                 0.0s
   ✔ 624f2f22548b Already exists                                                                                 0.0s
   ✔ 0eb07ea8ad2a Already exists                                                                                 0.0s
   ✔ 8b517dfb74ad Already exists                                                                                 0.0s
   ✔ ebef84d68e30 Already exists                                                                                 0.0s
   ✔ 604975d7252e Already exists                                                                                 0.0s
[+] Building 0.1s (6/6) FINISHED                                                                 docker:desktop-linux
 => [db-load-data internal] load build definition from Dockerfile                                                0.0s
 => => transferring dockerfile: 287B                                                                             0.0s
 => [db-load-data internal] load metadata for docker.io/mysql/mysql-server:latest                                0.0s
 => [db-load-data internal] load .dockerignore                                                                   0.0s
 => => transferring context: 2B                                                                                  0.0s
 => [db-load-data 1/2] FROM docker.io/mysql/mysql-server:latest                                                  0.0s
 => CACHED [db-load-data 2/2] RUN rpm --import https://repo.mysql.com/RPM-GPG-KEY-mysql-2023  && microdnf -y up  0.0s
 => [db-load-data] exporting to image                                                                            0.0s
 => => exporting layers                                                                                          0.0s
 => => writing image sha256:ed0a77ef9f76cf368844e714ae0d4855131ae47c0efbde8ef9c3f99141922fc9                     0.0s
 => => naming to docker.io/library/owid-grapher-db-load-data                                                     0.0s
[+] Running 4/4
 ✔ Network owid-grapher_default             Created                                                              0.1s
 ✔ Volume "owid-grapher_mysql_data_public"  Created                                                              0.0s
 ✔ Container owid-grapher-db-1              Created                                                              0.1s
 ✔ Container owid-grapher-db-load-data-1    Created                                                              0.1s
Attaching to db-1, db-load-data-1
db-1            | [Entrypoint] MySQL Docker Image 8.0.32-1.2.11-server
db-load-data-1  | [Entrypoint] MySQL Docker Image 8.0.32-1.2.11-server
db-load-data-1  | Waiting for DB to be online
db-1            | [Entrypoint] Initializing database
db-1            | 2024-07-04T04:32:38.040473Z 0 [Warning] [MY-011068] [Server] The syntax '--skip-host-cache' is deprecated and will be removed in a future release. Please use SET GLOBAL host_cache_size=0 instead.
db-1            | 2024-07-04T04:32:38.040527Z 0 [Warning] [MY-010918] [Server] 'default_authentication_plugin' is deprecated and will be removed in a future release. Please use authentication_policy instead.
db-1            | 2024-07-04T04:32:38.041084Z 0 [System] [MY-013169] [Server] /usr/sbin/mysqld (mysqld 8.0.32) initializing of server in progress as process 17
db-1            | 2024-07-04T04:32:38.062348Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
db-load-data-1  | mysqladmin: [Warning] Using a password on the command line interface can be insecure.
db-1            | 2024-07-04T04:32:38.421445Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
db-1            | 2024-07-04T04:32:39.501331Z 6 [Warning] [MY-010453] [Server] root@localhost is created with an empty password ! Please consider switching off the --initialize-insecure option.
db-1            | [Entrypoint] Database initialized
db-1            | 2024-07-04T04:32:43.649433Z 0 [Warning] [MY-011068] [Server] The syntax '--skip-host-cache' is deprecated and will be removed in a future release. Please use SET GLOBAL host_cache_size=0 instead.
db-1            | 2024-07-04T04:32:43.649908Z 0 [Warning] [MY-010918] [Server] 'default_authentication_plugin' is deprecated and will be removed in a future release. Please use authentication_policy instead.
db-1            | 2024-07-04T04:32:43.649919Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.32) starting as process 60
db-1            | 2024-07-04T04:32:43.660750Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
db-1            | 2024-07-04T04:32:43.726756Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
db-1            | 2024-07-04T04:32:43.816242Z 0 [Warning] [MY-010068] [Server] CA certificate ca.pem is self signed.
db-1            | 2024-07-04T04:32:43.816261Z 0 [System] [MY-013602] [Server] Channel mysql_main configured to support TLS. Encrypted connections are now supported for this channel.
db-1            | 2024-07-04T04:32:43.824559Z 0 [System] [MY-010931] [Server] /usr/sbin/mysqld: ready for connections. Version: '8.0.32'  socket: '/var/lib/mysql/mysql.sock'  port: 0  MySQL Community Server - GPL.
db-1            | 2024-07-04T04:32:43.824906Z 0 [System] [MY-011323] [Server] X Plugin ready for connections. Socket: /var/run/mysqld/mysqlx.sock
db-1            | Warning: Unable to load '/usr/share/zoneinfo/iso3166.tab' as time zone. Skipping it.
db-1            | Warning: Unable to load '/usr/share/zoneinfo/leapseconds' as time zone. Skipping it.
db-1            | Warning: Unable to load '/usr/share/zoneinfo/tzdata.zi' as time zone. Skipping it.
db-1            | Warning: Unable to load '/usr/share/zoneinfo/zone.tab' as time zone. Skipping it.
db-1            | Warning: Unable to load '/usr/share/zoneinfo/zone1970.tab' as time zone. Skipping it.
db-1            |
db-1            | [Entrypoint] ignoring /docker-entrypoint-initdb.d/*
db-1            |
db-1            | 2024-07-04T04:32:45.375296Z 11 [System] [MY-013172] [Server] Received SHUTDOWN from user root. Shutting down mysqld (Version: 8.0.32).
db-1            | 2024-07-04T04:32:46.400173Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.32)  MySQL Community Server - GPL.
db-1            | [Entrypoint] Server shut down
db-1            |
db-1            | [Entrypoint] MySQL init process done. Ready for start up.
db-1            |
db-1            | [Entrypoint] Starting MySQL 8.0.32-1.2.11-server
db-1            | 2024-07-04T04:32:47.546567Z 0 [Warning] [MY-011068] [Server] The syntax '--skip-host-cache' is deprecated and will be removed in a future release. Please use SET GLOBAL host_cache_size=0 instead.
db-1            | 2024-07-04T04:32:47.546863Z 0 [Warning] [MY-010918] [Server] 'default_authentication_plugin' is deprecated and will be removed in a future release. Please use authentication_policy instead.
db-1            | 2024-07-04T04:32:47.546875Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.32) starting as process 1
db-1            | 2024-07-04T04:32:47.550801Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
db-1            | 2024-07-04T04:32:47.599880Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
db-1            | 2024-07-04T04:32:47.714196Z 0 [Warning] [MY-010068] [Server] CA certificate ca.pem is self signed.
db-1            | 2024-07-04T04:32:47.714216Z 0 [System] [MY-013602] [Server] Channel mysql_main configured to support TLS. Encrypted connections are now supported for this channel.
db-1            | 2024-07-04T04:32:47.722064Z 0 [System] [MY-011323] [Server] X Plugin ready for connections. Bind-address: '::' port: 33060, socket: /var/run/mysqld/mysqlx.sock
db-1            | 2024-07-04T04:32:47.722096Z 0 [System] [MY-010931] [Server] /usr/sbin/mysqld: ready for connections. Version: '8.0.32'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MySQL Community Server - GPL.
db-load-data-1  | Waiting for MySQL server to answer.
db-load-data-1  | mysqld is alive
db-load-data-1  | mysql: [Warning] Using a password on the command line interface can be insecure.
db-load-data-1  | Checking if data dump has been downloaded
db-load-data-1  | found owid_metadata.sql.gz
db-load-data-1  | Creating user 'grapher'
db-load-data-1  | mysql: [Warning] Using a password on the command line interface can be insecure.
db-load-data-1  | ++ : grapher
db-load-data-1  | ++ : grapher
db-load-data-1  | ++ : grapher
db-load-data-1  | ++ : db
db-load-data-1  | ++ : 3306
db-load-data-1  | ++ : /tmp-downloads
db-load-data-1  | ==> Refreshing grapher database
db-load-data-1  | ++ fillGrapherDb
db-load-data-1  | ++ echo '==> Refreshing grapher database'
db-load-data-1  | ++ _mysql --database= -e 'DROP DATABASE IF EXISTS grapher;CREATE DATABASE grapher;ALTER DATABASE grapher CHARACTER SET utf8mb4 COLLATE utf8mb4_0900_as_cs;'
db-load-data-1  | ++ '[' -z grapher ']'
db-load-data-1  | ++ mysql --default-character-set=utf8mb4 -hdb -ugrapher -pgrapher -P 3306 --database= -e 'DROP DATABASE IF EXISTS grapher;CREATE DATABASE grapher;ALTER DATABASE grapher CHARACTER SET utf8mb4 COLLATE utf8mb4_0900_as_cs;'
db-load-data-1  | mysql: [Warning] Using a password on the command line interface can be insecure.
db-load-data-1  | Importing live Grapher metadata database (owid_metadata)
db-load-data-1  | ++ '[' -f /tmp-downloads/owid_metadata.sql.gz ']'
db-load-data-1  | ++ echo 'Importing live Grapher metadata database (owid_metadata)'
db-load-data-1  | ++ import_db /tmp-downloads/owid_metadata.sql.gz
db-load-data-1  | ++ cat /tmp-downloads/owid_metadata.sql.gz
db-load-data-1  | ++ gunzip
db-load-data-1  | ++ sed 's/.*DEFINER=`.*//'
db-load-data-1  | ++ _mysql grapher
db-load-data-1  | ++ '[' -z grapher ']'
db-load-data-1  | ++ mysql --default-character-set=utf8mb4 -hdb -ugrapher -pgrapher -P 3306 grapher
db-load-data-1  | mysql: [Warning] Using a password on the command line interface can be insecure.
db-1            | 2024-07-04T04:33:18.682837Z 0 [Warning] [MY-013865] [InnoDB] Redo log writer is waiting for a new redo log file. Consider increasing innodb_redo_log_capacity.
db-load-data-1  | ==> ✅ Grapher DB refresh complete
db-load-data-1  | ✅ All done, grapher DB is loaded ✅
db-load-data-1  | ++ echo '==> ✅ Grapher DB refresh complete'
db-load-data-1  | + return 0
db-load-data-1  | + echo '✅ All done, grapher DB is loaded ✅'
db-load-data-1 exited with code 0

[grapher] 0:docker* 1:admin  2:vite  3:lerna  4:welcome-                       "Tonis-MacBook-Air.loc" 04:35 04-Jul-24

Node modeules delete folder and with `--legacy-peer-deps` not good either

➜  owid-grapher git:(master) rm -rf node_modules
➜  owid-grapher git:(master) npm install
npm ERR! code ERESOLVE
npm ERR! ERESOLVE unable to resolve dependency tree
npm ERR!
npm ERR! While resolving: [email protected]
npm ERR! Found: [email protected]
npm ERR! node_modules/react
npm ERR!   react@"^17.0.2" from the root project
npm ERR!
npm ERR! Could not resolve dependency:
npm ERR! peer react@"^0.13.0 || ^0.14.0 || ^15.0.0 || ^16.0.0" from [email protected]
npm ERR! node_modules/mobx-react
npm ERR!   mobx-react@"5" from the root project
npm ERR!
npm ERR! Fix the upstream dependency conflict, or retry
npm ERR! this command with --force or --legacy-peer-deps
npm ERR! to accept an incorrect (and potentially broken) dependency resolution.
npm ERR!
npm ERR!
npm ERR! For a full report see:
npm ERR! /Users/brainstem/.npm/_logs/2024-07-04T04_40_05_653Z-eresolve-report.txt

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/brainstem/.npm/_logs/2024-07-04T04_40_05_653Z-debug-0.log
➜  owid-grapher git:(master) npm install --legacy-peer-deps
npm ERR! code EUNSUPPORTEDPROTOCOL
npm ERR! Unsupported URL Type "workspace:": workspace:^

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/brainstem/.npm/_logs/2024-07-04T04_40_56_527Z-debug-0.log
➜  owid-grapher git:(master)

[toni-sharpe]

Hey @marcelgerber I see this is closed, could I know why? If it's a quick explanation, I'd appreciate the knowledge :-) Naturally, I understand if there's not time to explain.

[marcelgerber]

Ah, I was under the assumption that it's good to close now that you've fixed it.

As for fixing the yarn startAdminServer command itself, I'm pretty sure you can resolve that by running yarn cleanTsc && yarn lerna run build && yarn buildTsc.

Us in the team pretty much all use make up, which takes care of all of this automatically.

[toni-sharpe]

@marcelgerber fair enough, my immediate problem was missing the dev bit calling startAdminServer directly, which it sounds like I would never do, because of make up. Understood, thanks :-)