From ac59af19dd93f51b5d7dc99beb3931ff3c80dbee Mon Sep 17 00:00:00 2001 From: Jeremy Ruston Date: Wed, 27 Nov 2024 15:43:44 +0000 Subject: [PATCH] Add some information about SQLite --- .../multiwikidocs/tiddlers/HelloThere.tid | 6 +++-- .../multiwikidocs/tiddlers/Installation.tid | 5 ++-- .../tiddlers/MWS Architecture.tid | 2 ++ .../multiwikidocs/tiddlers/MWS and SQLite.tid | 25 +++++++++++++++++++ 4 files changed, 34 insertions(+), 4 deletions(-) create mode 100644 editions/multiwikidocs/tiddlers/MWS Architecture.tid create mode 100644 editions/multiwikidocs/tiddlers/MWS and SQLite.tid diff --git a/editions/multiwikidocs/tiddlers/HelloThere.tid b/editions/multiwikidocs/tiddlers/HelloThere.tid index 027a35b09de..0d74fd3023b 100644 --- a/editions/multiwikidocs/tiddlers/HelloThere.tid +++ b/editions/multiwikidocs/tiddlers/HelloThere.tid @@ -6,10 +6,12 @@ tags: TableOfContents [img width=200 [MWS Banner.png]] ~MultiWikiServer is a new development that drastically improves ~TiddlyWiki's capabilities when running as a server under Node.js. It brings ~TiddlyWiki up to par with common web-based tools like ~WordPress or ~MediaWiki by supporting multiple wikis and multiple users at the same time. -Planned features include: +Features under development include: * Hosting multiple wikis at once, using the [[Bags and Recipes]] mechanism for sharing data between them -* Robust authentication and authorisation options +* Based on [[SQLite|MWS and SQLite]] for performance and reliability +* Robust built-in synchronisation handlers for syncing data to the filesystem +* Flexible authentication and authorisation options * Improved handling of file uploads and attachments, allowing gigabyte video files to be uploaded and streamed * Instantaneous synchronisation of changes between the server and all connected clients * Workflow processing on the server, for example to automatically compress images, or to archive webpages diff --git a/editions/multiwikidocs/tiddlers/Installation.tid b/editions/multiwikidocs/tiddlers/Installation.tid index 735e884d396..bc436a3e928 100644 --- a/editions/multiwikidocs/tiddlers/Installation.tid +++ b/editions/multiwikidocs/tiddlers/Installation.tid @@ -8,10 +8,11 @@ These instructions require minimal knowledge of the terminal. There are also [[a # Download the code [[direct from GitHub|https://github.com/TiddlyWiki/TiddlyWiki5/archive/refs/pull/7915/head.zip]] # Open a terminal window and set the current directory to the root of the downloaded folder # Install the dependencies with the command: <<.copy-code-to-clipboard "npm install">> -# Start the server with the command: <<.copy-code-to-clipboard "npm start">> -# To use MWS, visit http://localhost:8080 in a browser on the same computer +# To verify that MWS is working correctly, start the server with the command: <<.copy-code-to-clipboard "npm start">> and then visit http://localhost:8080 in a browser on the same computer # When you have finished using MWS, stop the server with ctrl-C See [[Troubleshooting]] if you encounter any errors. + + To update your copy of MWS in the future with newer changes will require re-downloading the code, taking care not to lose any changes you might have made. diff --git a/editions/multiwikidocs/tiddlers/MWS Architecture.tid b/editions/multiwikidocs/tiddlers/MWS Architecture.tid new file mode 100644 index 00000000000..7eac6e44853 --- /dev/null +++ b/editions/multiwikidocs/tiddlers/MWS Architecture.tid @@ -0,0 +1,2 @@ +title: Architecture +tags: TableOfContents diff --git a/editions/multiwikidocs/tiddlers/MWS and SQLite.tid b/editions/multiwikidocs/tiddlers/MWS and SQLite.tid new file mode 100644 index 00000000000..202e7d8e8f5 --- /dev/null +++ b/editions/multiwikidocs/tiddlers/MWS and SQLite.tid @@ -0,0 +1,25 @@ +title: MWS and SQLite +tags: Architecture + +! Introduction + +SQLite is a very popular open source embedded SQL database with some [[unusual characteristics|https://www.sqlite.org/different.html]]. It has proved itself to be robust, fast and scalable, and has been widely adopted in a range of applications including web browsers, mobile devices, and embedded systems. + +The "embedded" part means that developers access SQLite as a library of C functions that run as part of a larger application. This contrasts with more familiar database applications like Microsoft's SQL Server or Oracle that are accessed as network services. + +MWS uses SQLite for the tiddler store and associated data. It brings many advantages: + +* ''Performance'': the optimising query engine inside SQLite makes it much faster and more efficient than could be achieved in plain JavaScript. In some cases, it is [[faster than writing directly to the file system||https://www.sqlite.org/fasterthanfs.html]] +* ''Reliability'': SQLite uses protocols that [[ensure data integrity and consistency|https://www.sqlite.org/hirely.html]], even when the application crashes +* ''Scalability'': SQLite can handle extremely [[large datasets and complex queries|https://www.sqlite.org/limits.html]] +* ''Portability'': SQLite databases are stored as [[a single file|https://www.sqlite.org/fileformat.html]] that can be easily copied and moved between systems + +! Misconceptions + +TiddlyWiki 5 has always incorporated a database. Until MWS, that database has always been a custom tiddler database written in JavaScript. Over the years it has been enhanced and optimised with indexes and other database features that have given us reasonably decent performance for a range of common operations. + +In terms of the traditional architecture of TiddlyWiki, MWS uses SQLite as the basis for an internal API that is equivalent to that of the `$tw.Wiki` object: basic CRUD operations on a database of tiddlers stored by their titles. + +In the context of MWS, SQLite is just a fast and efficient equivalent of TiddlyWiki's existing JavaScript database engine. It gives us the option of persisting the database in file storage, but we also retain the option to run the database entirely within memory and rely on a file synchronisation process to save changes as individual `.tid` files in the file system, just as we do today. + +One particular misconception to avoid is the idea that SQLite replaces the folders of `.tid` files that characterise the Node.js configuration of TiddlyWiki. These are separate components with a different purpose. The tiddler files are the result of syncing a database to the filesystem, and that database can be conceptually interchanged between our custom JavaScript database or the new SQLite implementation in MWS.