v2024.3.2-prealpha

CHANGELOG.md

@@ -1,17 +1,27 @@
 
 <div class="release-notes">
 
-This is a detailed list of changes in each version.
+This page contains a detailed list of changes made in every build of PhotoStructure.
 
-- Major releases have posts summarizing bigger changes. See [the posts tagged with "release notes"](/tags/release-notes/).
+## PhotoStructure versions
 
-- New releases, starting in 2023, use "calendar versioning," or [CalVer](https://calver.org/), using scheme `YYYY.MM.BUILD`. `BUILD` starts at zero at the beginning of the month, and gets incremented for every prealpha, alpha, beta, or stable release. For non-stable releases, `-$channel` is appended to the version format.
+- Major versions may have posts summarizing more visible changes. Check out [the posts tagged with "release notes"](/tags/release-notes/).
 
-- "Pre-release" builds (marked with `-alpha` or `-beta`) have not been thoroughly tested, and may not even launch.
+- PhotoStructure uses "calendar versioning," or [CalVer](https://calver.org/), using the template `YYYY.MM.BUILD`. `BUILD` starts at zero at the beginning of the month, so `2024.3.3` is the _fourth_ build from March 2024.
 
-- Only run `alpha` or `beta` builds if you have [recent backups](/faq/how-do-i-safely-store-files/).
+### Experimental builds
 
-- Upgrades to new versions are automatic, but older versions of PhotoStructure may not be able to open libraries created by newer versions of PhotoStructure.
+- Experimental builds end with a `-prealpha`, `-alpha`, or `-beta` suffix.
+
+- **These experimental builds have not been thoroughly tested, and may not even launch.**
+
+- Only run these builds if you have [recent backups](/faq/how-do-i-safely-store-files/).
+
+- If you run one of these builds, please consider hopping into our [chat](/go/discord) to make sure we can squash any bugs you encounter.
+
+### Version upgrades
+
+- Library upgrades to new versions of PhotoStructure are automatic, but older versions of PhotoStructure cannot open libraries from newer versions of PhotoStructure.
 
 <!-- TODO: -->
 <!-- - 🐛 Sync doesn't seem to no-op properly for a completed directory -->
@@ -23,7 +33,44 @@ This is a detailed list of changes in each version.
 
 <!-- fix "tag context" for "next previous" context. I'd always done a search, clicked a thumb, and then clicked esc to go back to the search results. But...  if you click a thumb from a search,  and then click "next" or "previous", it ignores that you can from a search, and does the chronological next asset, which is very confusing/irritating. -->
 
+<!-- - 🐛 (todo): Thumbnails in tag samples are [chronologically ordered](https://discord.com/channels/818905168107012097/1148116413190520923) -->
+
+<!-- - 🐛 (need to verify): Ensure progress is updated during library rebuild (the prior build didn't properly update the `Progress` table, so your computer was busy but you didn't know why). -->
+
+<!--- 📦 (TODO) Add sync reports for AssetFile re-syncs during library rebuilds and video transcodes -->
+
+## v2024.3.2-prealpha
+
+**Released 2024-03-14 🥧**
+
+- 🐛 The warning message `Error: env(): failed to read .env file` caused [sync to fail to run](https://discord.com/channels/818905168107012097/818907922767544340/1217724819483791421]). This warning is now only emitted by the main service, and only if the file exists.
+
+- 🐛 Database migrations were edited to try to gracefully recover from some types of partially-applied migrations. This should remedy many issues like [this](https://discord.com/channels/818905168107012097/818907922767544340/1215748274401710191).
+
+- 🐛 Geolocation fields are now deleted if GPS is [(0,0)](https://en.wikipedia.org/wiki/Null_Island). Upgraded libraries will auto-resync all assets tagged with `Where|Ghana|Western|Takoradi` (the nearest city to [Null Island](https://en.wikipedia.org/wiki/Null_Island)).
+
+- 📦 The `/settings` page redirected to the health check page if settings took longer than a second to fetch(!!).
+
+- 📦 Added several more `/System/Volume` exclude globs to avoid macOS system subdirectories (thanks for the [assist, AlanH!](https://discord.com/channels/818905168107012097/818907922767544340/1216585399615492136)
+
+- 📦 Add `tagGeoSynonyms` setting:
+
+  > Due to EXIF and XMP specification drift, there are several ways for geolocation information to be encoded in files. When PhotoStructure applies the "tagGeoTemplate", we'll use these "synonyms" to build the geo tag (first synonym with a value wins). See https://exiftool.org/forum/index.php?topic=13811.msg74413#msg74413 for details.
+
+- 📦 Add `writeGeolocationTagsToLibraryCopies` setting:
+
+  > When enabled, inferred geolocation tags will backfill into Country/State/City tags in the library copy (by default, into an XMP sidecar).
+  >
+  > This defaults to false, as reverse-geo lookup results can change over time (and should be done on-demand, rather than stored statically and drift into inaccuracies).
+
+- 📦 Merged `commandTimeoutMs` and `statTimeoutMs` settings--they both defaulted to 30s, and given the presence of `taskTimeoutMs` (which defaults to 2m) having all three timeouts was confusing.
+
+- 📦 `syncCronTZ` now defaults to `TZ` if `TZ` is a valid IANA time zone (like "America/Los_Angeles").
+
+- 📦 `info` improvements: include captured-at raw EXIF values for files, and if `--load-library` is specified, include db library setup metadata (like `libraryDbFile`, `libraryDbBackupDir`, and `useReplica`).
+
 <a id="v2024.3.1-prealpha"></a>
+
 ## v2024.3.1-prealpha ["Zep"](https://discord.com/channels/818905168107012097/818905168690413611/1215774241979502612)
 
 **Released 2023-03-08**

VERSION.json

@@ -1,12 +1,12 @@
 {
-  "version": "2024.3.1-prealpha",
+  "version": "2024.3.2-prealpha",
   "versionMajor": 2024,
   "versionMinor": 3,
-  "versionPatch": 1,
+  "versionPatch": 2,
   "versionPrerelease": [
     "prealpha"
   ],
-  "release": "2024.3.1-prealpha+20240308150212",
-  "gitSha": "d92384e8659ff46fbae8e6f81fa5867114a4a843",
-  "gitDate": "2024-03-08T23:02:12.000Z"
+  "release": "2024.3.2-prealpha+20240314095249",
+  "gitSha": "afbc9f97b4200bd8ade975822680115d865447d3",
+  "gitDate": "2024-03-14T16:52:49.000Z"
 }

data/migrations/models/20181030_models.sql

@@ -1,15 +1,23 @@
-CREATE TABLE IF NOT EXISTS Example (
-  -- used for integration tests
+DROP INDEX IF EXISTS example_name_udx;
+
+DROP TABLE IF EXISTS Example;
+
+CREATE TABLE Example -- used for integration tests
+(
   id INTEGER NOT NULL PRIMARY KEY,
   name VARCHAR(255) NOT NULL,
   isExample INTEGER,
   createdAt BIGINT NOT NULL,
   updatedAt BIGINT NOT NULL
 );
 
-CREATE UNIQUE INDEX IF NOT EXISTS example_name_udx ON Example (name);
+CREATE UNIQUE INDEX example_name_udx ON Example (name);
+
+DROP INDEX IF EXISTS progress_uri_udx;
+
+DROP TABLE IF EXISTS Progress;
 
-CREATE TABLE IF NOT EXISTS Progress (
+CREATE TABLE Progress (
   -- records sync process state
   id INTEGER NOT NULL PRIMARY KEY,
   uri VARCHAR(255) NOT NULL,
@@ -24,10 +32,14 @@ CREATE TABLE IF NOT EXISTS Progress (
   updatedAt BIGINT NOT NULL
 );
 
-CREATE UNIQUE INDEX IF NOT EXISTS progress_uri_udx ON Progress (uri);
+CREATE UNIQUE INDEX progress_uri_udx ON Progress (uri);
 
-CREATE TABLE IF NOT EXISTS Tag (
-  -- Tags are associated to Assets and support hierarchies.
+DROP INDEX IF EXISTS tag_path_udx;
+
+DROP TABLE IF EXISTS Tag;
+
+CREATE TABLE Tag -- Tags are associated to Assets and support hierarchies.
+(
   id INTEGER NOT NULL PRIMARY KEY,
   parentId INTEGER,
   -- this is the full path of the Tag, and is an ASCII SEP-separated string.
@@ -38,9 +50,25 @@ CREATE TABLE IF NOT EXISTS Tag (
   FOREIGN KEY (parentId) REFERENCES Tag (id)
 );
 
-CREATE UNIQUE INDEX IF NOT EXISTS tag_path_udx ON Tag (_path);
+CREATE UNIQUE INDEX tag_path_udx ON Tag (_path);
+
+DROP INDEX IF EXISTS asset_timeline;
+
+DROP INDEX IF EXISTS assetfile_uri_udx;
 
-CREATE TABLE IF NOT EXISTS Asset -- Asset). -- heuristics (to make sure, for example, that RAW/JPEG pairs share the same -- AssetFiles. AssetFiles are coalesced with existing Assets based on Curator -- An Asset embodies an item in a library, backed by 1 or more "related"
+DROP INDEX IF EXISTS assetfile_recent_idx;
+
+DROP INDEX IF EXISTS assetfile_sha_idx;
+
+DROP INDEX IF EXISTS assetfile_exifuid_idx;
+
+DROP INDEX IF EXISTS assetfile_shown_udx;
+
+DROP TABLE IF EXISTS Asset;
+
+DROP TABLE IF EXISTS AssetFile;
+
+CREATE TABLE Asset -- An Asset embodies a photo or video in a library, backed by 1 or more AssetFile rows
 (
   id INTEGER NOT NULL PRIMARY KEY,
   -- true if the asset *can* be shown
@@ -59,12 +87,11 @@ WHERE
   shown = 1
   AND hidden = 0;
 
-CREATE TABLE IF NOT EXISTS AssetFile (
-  -- Asset, and is the file revision that will be shown for the Asset. -- exactly one Asset. There should be only one "shown" AssetFile for a given -- An AssetFile points to a file (via the URI) and is always associated to
+CREATE TABLE IF NOT EXISTS AssetFile -- An AssetFile is a file on disk that is associated with an Asset.
+(
   id INTEGER NOT NULL PRIMARY KEY,
   assetId INTEGER NOT NULL,
-  -- boolean(1 == true), true for the instance of an asset file that is shown
-  -- in the UI for the given assetId
+  -- 1 if the asset file is the primary file to be shown for the asset
   shown INTEGER NOT NULL DEFAULT 0,
   -- May be longer than 1 K as this is the full native path to files:
   uri VARCHAR(2048) NOT NULL,
@@ -109,8 +136,12 @@ CREATE UNIQUE INDEX IF NOT EXISTS assetfile_shown_udx ON AssetFile (assetId)
 WHERE
   shown = 1;
 
-CREATE TABLE IF NOT EXISTS AssetTag (
-  -- join table to associate Assets to Tags.
+DROP TABLE IF EXISTS AssetTag;
+
+DROP INDEX IF EXISTS assettag_fks;
+
+CREATE TABLE AssetTag -- join table to associate Assets to Tags.
+(
   assetId INTEGER NOT NULL,
   tagId INTEGER NOT NULL,
   FOREIGN KEY (assetId) REFERENCES Asset (id) ON DELETE CASCADE,

data/migrations/models/20190425_merged_assets.sql

@@ -1,4 +1,6 @@
-CREATE TABLE IF NOT EXISTS MergedAsset (
+DROP TABLE IF EXISTS MergedAsset;
+
+CREATE TABLE MergedAsset (
   -- When Exif UIDs are updated (like v5) and previously-separated assets are
   -- merged, this table stores those redirections. 
   -- id is the "merge victim", the asset id that doesn't exist anymore.

data/migrations/models/20190730_session.sql

@@ -1,4 +1,6 @@
-CREATE TABLE IF NOT EXISTS Session (
+DROP TABLE IF EXISTS SESSION;
+
+CREATE TABLE SESSION (
   sid VARCHAR(255) NOT NULL PRIMARY KEY,
   expired BIGINT NOT NULL,
   sess VARCHAR(2048) NOT NULL

data/migrations/models/20190923_asset_file_v2_nofk.sql

@@ -1,3 +1,5 @@
+DROP TABLE IF EXISTS new_AssetFile;
+
 CREATE TABLE new_AssetFile (
   id INTEGER NOT NULL PRIMARY KEY,
   assetId INTEGER NOT NULL,

data/migrations/models/20191013_asset_v2_nofk.sql

@@ -35,6 +35,8 @@ SET
 WHERE
   capturedAtLocal IS NULL;
 
+DROP TABLE IF EXISTS new_Asset;
+
 CREATE TABLE new_Asset (
   id INTEGER NOT NULL PRIMARY KEY,
   capturedAtLocal INTEGER NOT NULL,

data/migrations/models/20200116_heartbeat.sql

@@ -1,5 +1,5 @@
-CREATE TABLE IF NOT EXISTS Heartbeat (
-  -- used for db health checks
+CREATE TABLE IF NOT EXISTS Heartbeat -- used for db health checks
+(
   id INTEGER NOT NULL PRIMARY KEY,
   name VARCHAR(255) NOT NULL,
   createdAt BIGINT NOT NULL,

data/migrations/models/20200128_asset_file_v3_nofk.sql

@@ -1,4 +1,6 @@
-CREATE TABLE "new_AssetFile" (
+DROP TABLE IF EXISTS new_AssetFile;
+
+CREATE TABLE new_AssetFile (
   id INTEGER NOT NULL PRIMARY KEY,
   assetId INTEGER NOT NULL,
   shown INTEGER NOT NULL DEFAULT 0,

data/migrations/models/20200130_advisory_lock.sql

@@ -1,4 +1,10 @@
-CREATE TABLE IF NOT EXISTS AdvisoryLock (
+DROP TABLE IF EXISTS AdvisoryLock;
+
+DROP INDEX IF EXISTS lock_name_lockId_udx;
+
+DROP INDEX IF EXISTS lock_acquired_udx;
+
+CREATE TABLE AdvisoryLock (
   id INTEGER NOT NULL PRIMARY KEY,
   -- name of the advisory lock being requested:
   name VARCHAR(255) NOT NULL,
@@ -8,8 +14,8 @@ CREATE TABLE IF NOT EXISTS AdvisoryLock (
   createdAt BIGINT NOT NULL
 );
 
-CREATE UNIQUE INDEX IF NOT EXISTS lock_name_lockId_udx ON AdvisoryLock (name, lockId);
+CREATE UNIQUE INDEX lock_name_lockId_udx ON AdvisoryLock (name, lockId);
 
-CREATE UNIQUE INDEX IF NOT EXISTS lock_acquired_udx ON AdvisoryLock (name, acquired)
+CREATE UNIQUE INDEX lock_acquired_udx ON AdvisoryLock (name, acquired)
 WHERE
   acquired = 1;

data/migrations/models/20200512_asset_file_v4_nofk.sql

@@ -1,6 +1,8 @@
 -- remove the rightHash and searchHash (which didn't improve asset
 -- aggregations), and add FPS:
-CREATE TABLE "new_AssetFile" (
+DROP TABLE IF EXISTS new_AssetFile;
+
+CREATE TABLE new_AssetFile (
   id INTEGER NOT NULL PRIMARY KEY,
   assetId INTEGER NOT NULL,
   shown INTEGER NOT NULL DEFAULT 0,

data/migrations/models/20200827_tag_v2_nofk.sql

@@ -1,4 +1,6 @@
-CREATE TABLE IF NOT EXISTS new_Tag (
+DROP TABLE IF EXISTS new_Tag;
+
+CREATE TABLE new_Tag (
   -- Tags are associated to Assets and support hierarchies.
   id INTEGER NOT NULL PRIMARY KEY,
   parentId INTEGER,

data/migrations/models/20201006_progress_meta.sql

@@ -1,3 +1,7 @@
+DROP INDEX IF EXISTS progressmeta_fk_name_udx;
+
+DROP TABLE IF EXISTS ProgressMeta;
+
 CREATE TABLE ProgressMeta (
   -- Used to store state of a sync. Prevents future sync jobs from re-doing prior work.
   id INTEGER NOT NULL PRIMARY KEY,

data/migrations/models/20201007_operation.sql

@@ -1,3 +1,5 @@
+DROP TABLE IF EXISTS Operation;
+
 CREATE TABLE Operation (
   -- used to store user-requested operations, like "forced" syncs
   id INTEGER NOT NULL PRIMARY KEY,

data/migrations/models/20210322_sha_blocklist.sql

@@ -1,4 +1,6 @@
-CREATE TABLE "ShaBlocklist" (
+DROP TABLE IF EXISTS ShaBlocklist;
+
+CREATE TABLE ShaBlocklist (
   -- Files with any of these SHAs are not to be imported.
   sha VARCHAR(64) NOT NULL PRIMARY KEY
 );

data/migrations/models/20210604_changed_tag.sql

@@ -1,5 +1,7 @@
-CREATE TABLE ChangedTag (
-  -- used to store tags that have assets added or removed (and need their assets re-counted)
+DROP TABLE IF EXISTS ChangedTag;
+
+CREATE TABLE ChangedTag -- used to store tags that have assets added or removed (and need their assets re-counted)
+(
   tagId ID INTEGER NOT NULL PRIMARY KEY,
   FOREIGN KEY (tagId) REFERENCES Tag (id) ON DELETE CASCADE
 );

data/migrations/models/20210930_asset_revision.sql

@@ -1,6 +1,6 @@
-DROP TABLE IF EXISTS "AssetRevision";
+DROP TABLE IF EXISTS AssetRevision;
 
-CREATE TABLE "AssetRevision" (
+CREATE TABLE AssetRevision (
   -- Any change made to an Asset through the PhotoStructure UI creates an "AssetRevision" which can then be merged with any external changes made to files or sidecars.
   id INTEGER NOT NULL PRIMARY KEY,
   assetId INTEGER NOT NULL,