Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cordova sqlite express EXPERIMENTAL WIP #730

Open
wants to merge 2 commits into
base: dev
Choose a base branch
from

Conversation

brodycj
Copy link
Contributor

@brodycj brodycj commented Dec 10, 2017

NOW FIXED (see comment below):

This experimental WIP change shows that the new BUG #666 workaround solution causes selfTest to fail on built-in android.database implementation (Android-sqlite-connector not used).

This is a blocking issue for #687: new major release using built-in sqlite libraries on Android/iOS/macOS

A FUTURE TODO item is to cover this scenario in an open/close/delete test.

@brodycj
Copy link
Contributor Author

brodycj commented Dec 11, 2017

Here is some logcat output. The first few errors are expected in selfTest. The SQLITE_BUSY errors need more investigation.

W/System.err( 6140): remove failed: ENOENT (No such file or directory) : /data/data/com.phonegap.plugins.sqlite.tests/databases/___$$$___litehelpers___$$$___test___$$$___.db
W/System.err( 6140): remove failed: ENOENT (No such file or directory) : /data/data/com.phonegap.plugins.sqlite.tests/databases/___$$$___litehelpers___$$$___test___$$$___.db-journal
W/System.err( 6140): remove failed: ENOENT (No such file or directory) : /data/data/com.phonegap.plugins.sqlite.tests/databases/___$$$___litehelpers___$$$___test___$$$___.db-shm
W/System.err( 6140): remove failed: ENOENT (No such file or directory) : /data/data/com.phonegap.plugins.sqlite.tests/databases/___$$$___litehelpers___$$$___test___$$$___.db-wal
V/SQLitePlugin( 6140): Android db implementation: built-in android.database.sqlite package
V/info    ( 6140): Open sqlite db: /data/data/com.phonegap.plugins.sqlite.tests/databases/___$$$___litehelpers___$$$___test___$$$___.db
D/SystemWebChromeClient( 6140): file:///android_asset/www/plugins/cordova-sqlite-express-wip/www/SQLitePlugin.js: Line 175 : OPEN database: ___$$$___litehelpers___$$$___test___$$$___.db
I/chromium( 6140): [INFO:CONSOLE(175)] "OPEN database: ___$$$___litehelpers___$$$___test___$$$___.db", source: file:///android_asset/www/plugins/cordova-sqlite-express-wip/www/SQLitePlugin.js (175)
D/SystemWebChromeClient( 6140): file:///android_asset/www/plugins/cordova-sqlite-express-wip/www/SQLitePlugin.js: Line 179 : OPEN database: ___$$$___litehelpers___$$$___test___$$$___.db - OK
I/chromium( 6140): [INFO:CONSOLE(179)] "OPEN database: ___$$$___litehelpers___$$$___test___$$$___.db - OK", source: file:///android_asset/www/plugins/cordova-sqlite-express-wip/www/SQLitePlugin.js (179)
D/SystemWebChromeClient( 6140): file:///android_asset/www/plugins/cordova-sqlite-express-wip/www/SQLitePlugin.js: Line 136 : cannot start next transaction: database not open
I/chromium( 6140): [INFO:CONSOLE(136)] "cannot start next transaction: database not open", source: file:///android_asset/www/plugins/cordova-sqlite-express-wip/www/SQLitePlugin.js (136)
D/SystemWebChromeClient( 6140): file:///android_asset/www/plugins/cordova-sqlite-express-wip/www/SQLitePlugin.js: Line 175 : OPEN database: ___$$$___litehelpers___$$$___test___$$$___.db
I/chromium( 6140): [INFO:CONSOLE(175)] "OPEN database: ___$$$___litehelpers___$$$___test___$$$___.db", source: file:///android_asset/www/plugins/cordova-sqlite-express-wip/www/SQLitePlugin.js (175)
I/SQLiteConnectionPool( 6140): The connection pool for +data+data+com_phonegap_plugins_sqlite_tests+databases+___$$$___litehelpers___$$$___test___$$$____db has been closed but there are still 1 connections in use.  They will be closed as they are released back to the pool.
V/SQLitePlugin( 6140): Android db implementation: built-in android.database.sqlite package
V/info    ( 6140): Open sqlite db: /data/data/com.phonegap.plugins.sqlite.tests/databases/___$$$___litehelpers___$$$___test___$$$___.db
D/WCND    (  171): is_cp2_alive_ok: open polling interface: /proc/mdbg/loopcheck, fd = 14
D/WCND    (  171): is_cp2_alive_ok: loop: /proc/mdbg/loopcheck is OK
D/StatusBar.MSimNetworkController(  876): refreshViews connected={ wifi } phoneId = 0 level=4 mMSimcombinedSignalIconId=0x7f02067b/com.android.systemui:drawable/stat_sys_wifi_signal_4 mMSimcombinedActivityIconId=0x7f02062a mAirplaneMode=false mMSimDataActivity=0 mMSimPhoneSignalIconId=0x7f0205f4/com.android.systemui:drawable/stat_sys_signal_4_auto_rotate mMSimDataDirectionIconId=0x0 mMSimDataSignalIconId=0x7f0205f4 mMSimDataTypeIconId=0x0/(null) mNoMSimIconId=0x7f02017f/com.android.systemui:drawable/ic_qs_no_sim mMSimMobileActivityIconId=0x0/(null) mWifiIconId=0x7f02067b mBluetoothTetherIconId=0x7f020650 mRoamingIconId=0x 0/(null)
D/StatusBar.MSimNetworkController(  876): refreshSignalCluster : called
W/System.err(  746): remove failed: ENOENT (No such file or directory) : /data/system/recent_tasks/1789_task.xml.bak
I/WearableService( 1501): Wearable Services stopping
E/SQLiteLog( 6140): (5) database is locked
E/SQLiteDatabase( 6140): Failed to open database '/data/data/com.phonegap.plugins.sqlite.tests/databases/___$$$___litehelpers___$$$___test___$$$___.db'.
E/SQLiteDatabase( 6140): android.database.sqlite.SQLiteDatabaseLockedException: database is locked (code 5): , while compiling: PRAGMA journal_mode
E/SQLiteDatabase( 6140): #################################################################
E/SQLiteDatabase( 6140): Error Code : 5 (SQLITE_BUSY)
E/SQLiteDatabase( 6140): Caused By : The database file is locked.
E/SQLiteDatabase( 6140): 	(database is locked (code 5): , while compiling: PRAGMA journal_mode)
E/SQLiteDatabase( 6140): #################################################################
E/SQLiteDatabase( 6140): 	at android.database.sqlite.SQLiteConnection.nativePrepareStatement(Native Method)
E/SQLiteDatabase( 6140): 	at android.database.sqlite.SQLiteConnection.acquirePreparedStatement(SQLiteConnection.java:1067)
E/SQLiteDatabase( 6140): 	at android.database.sqlite.SQLiteConnection.executeForString(SQLiteConnection.java:778)
E/SQLiteDatabase( 6140): 	at android.database.sqlite.SQLiteConnection.setJournalMode(SQLiteConnection.java:464)
E/SQLiteDatabase( 6140): 	at android.database.sqlite.SQLiteConnection.setWalModeFromConfiguration(SQLiteConnection.java:438)
E/SQLiteDatabase( 6140): 	at android.database.sqlite.SQLiteConnection.open(SQLiteConnection.java:337)
E/SQLiteDatabase( 6140): 	at android.database.sqlite.SQLiteConnection.open(SQLiteConnection.java:220)
E/SQLiteDatabase( 6140): 	at android.database.sqlite.SQLiteConnectionPool.openConnectionLocked(SQLiteConnectionPool.java:512)
E/SQLiteDatabase( 6140): 	at android.database.sqlite.SQLiteConnectionPool.open(SQLiteConnectionPool.java:206)
E/SQLiteDatabase( 6140): 	at android.database.sqlite.SQLiteConnectionPool.open(SQLiteConnectionPool.java:178)
E/SQLiteDatabase( 6140): 	at android.database.sqlite.SQLiteDatabase.openInner(SQLiteDatabase.java:908)
E/SQLiteDatabase( 6140): 	at android.database.sqlite.SQLiteDatabase.open(SQLiteDatabase.java:878)
E/SQLiteDatabase( 6140): 	at android.database.sqlite.SQLiteDatabase.openDatabase(SQLiteDatabase.java:699)
E/SQLiteDatabase( 6140): 	at android.database.sqlite.SQLiteDatabase.openOrCreateDatabase(SQLiteDatabase.java:714)
E/SQLiteDatabase( 6140): 	at android.database.sqlite.SQLiteDatabase.openOrCreateDatabase(SQLiteDatabase.java:707)
E/SQLiteDatabase( 6140): 	at io.sqlc.SQLiteAndroidDatabase.open(SQLiteAndroidDatabase.java:70)
E/SQLiteDatabase( 6140): 	at io.sqlc.SQLitePlugin.openDatabase(SQLitePlugin.java:212)
E/SQLiteDatabase( 6140): 	at io.sqlc.SQLitePlugin.access$000(SQLitePlugin.java:32)
E/SQLiteDatabase( 6140): 	at io.sqlc.SQLitePlugin$DBRunner.run(SQLitePlugin.java:327)
E/SQLiteDatabase( 6140): 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1112)
E/SQLiteDatabase( 6140): 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:587)
E/SQLiteDatabase( 6140): 	at java.lang.Thread.run(Thread.java:818)
E/SQLitePlugin( 6140): unexpected error, stopping db thread
E/SQLitePlugin( 6140): android.database.sqlite.SQLiteDatabaseLockedException: database is locked (code 5): , while compiling: PRAGMA journal_mode
E/SQLitePlugin( 6140): #################################################################
E/SQLitePlugin( 6140): Error Code : 5 (SQLITE_BUSY)
E/SQLitePlugin( 6140): Caused By : The database file is locked.
E/SQLitePlugin( 6140): 	(database is locked (code 5): , while compiling: PRAGMA journal_mode)
E/SQLitePlugin( 6140): #################################################################
E/SQLitePlugin( 6140): 	at android.database.sqlite.SQLiteConnection.nativePrepareStatement(Native Method)
E/SQLitePlugin( 6140): 	at android.database.sqlite.SQLiteConnection.acquirePreparedStatement(SQLiteConnection.java:1067)
E/SQLitePlugin( 6140): 	at android.database.sqlite.SQLiteConnection.executeForString(SQLiteConnection.java:778)
E/SQLitePlugin( 6140): 	at android.database.sqlite.SQLiteConnection.setJournalMode(SQLiteConnection.java:464)
E/SQLitePlugin( 6140): 	at android.database.sqlite.SQLiteConnection.setWalModeFromConfiguration(SQLiteConnection.java:438)
E/SQLitePlugin( 6140): 	at android.database.sqlite.SQLiteConnection.open(SQLiteConnection.java:337)
E/SQLitePlugin( 6140): 	at android.database.sqlite.SQLiteConnection.open(SQLiteConnection.java:220)
E/SQLitePlugin( 6140): 	at android.database.sqlite.SQLiteConnectionPool.openConnectionLocked(SQLiteConnectionPool.java:512)
E/SQLitePlugin( 6140): 	at android.database.sqlite.SQLiteConnectionPool.open(SQLiteConnectionPool.java:206)
E/SQLitePlugin( 6140): 	at android.database.sqlite.SQLiteConnectionPool.open(SQLiteConnectionPool.java:178)
E/SQLitePlugin( 6140): 	at android.database.sqlite.SQLiteDatabase.openInner(SQLiteDatabase.java:908)
E/SQLitePlugin( 6140): 	at android.database.sqlite.SQLiteDatabase.open(SQLiteDatabase.java:878)
E/SQLitePlugin( 6140): 	at android.database.sqlite.SQLiteDatabase.openDatabase(SQLiteDatabase.java:699)
E/SQLitePlugin( 6140): 	at android.database.sqlite.SQLiteDatabase.openOrCreateDatabase(SQLiteDatabase.java:714)
E/SQLitePlugin( 6140): 	at android.database.sqlite.SQLiteDatabase.openOrCreateDatabase(SQLiteDatabase.java:707)
E/SQLitePlugin( 6140): 	at io.sqlc.SQLiteAndroidDatabase.open(SQLiteAndroidDatabase.java:70)
E/SQLitePlugin( 6140): 	at io.sqlc.SQLitePlugin.openDatabase(SQLitePlugin.java:212)
E/SQLitePlugin( 6140): 	at io.sqlc.SQLitePlugin.access$000(SQLitePlugin.java:32)
E/SQLitePlugin( 6140): 	at io.sqlc.SQLitePlugin$DBRunner.run(SQLitePlugin.java:327)
E/SQLitePlugin( 6140): 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1112)
E/SQLitePlugin( 6140): 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:587)
E/SQLitePlugin( 6140): 	at java.lang.Thread.run(Thread.java:818)
D/SystemWebChromeClient( 6140): file:///android_asset/www/plugins/cordova-sqlite-express-wip/www/SQLitePlugin.js: Line 197 : OPEN database: ___$$$___litehelpers___$$$___test___$$$___.db FAILED, aborting any pending transactions
I/chromium( 6140): [INFO:CONSOLE(197)] "OPEN database: ___$$$___litehelpers___$$$___test___$$$___.db FAILED, aborting any pending transactions", source: file:///android_asset/www/plugins/cordova-sqlite-express-wip/www/SQLitePlugin.js (197)
D/SystemWebChromeClient( 6140): file:///android_asset/www/plugins/cordova-sqlite-express-wip/www/SQLitePlugin.js: Line 879 : selfTest ERROR with message: Open database error: Error: Could not open database
I/chromium( 6140): [INFO:CONSOLE(879)] "selfTest ERROR with message: Open database error: Error: Could not open database", source: file:///android_asset/www/plugins/cordova-sqlite-express-wip/www/SQLitePlugin.js (879)
W/System.err( 6140): remove failed: ENOENT (No such file or directory) : /data/data/com.phonegap.plugins.sqlite.tests/databases/___$$$___litehelpers___$$$___test___$$$___.db-shm
W/System.err( 6140): remove failed: ENOENT (No such file or directory) : /data/data/com.phonegap.plugins.sqlite.tests/databases/___$$$___litehelpers___$$$___test___$$$___.db-wal

Christopher J. Brody added 2 commits December 11, 2017 20:05
needed for selfTest to pass on builtin android.database implementation
(no Android-sqlite-connector / Android-sqlite-native-driver libraries)

ref: storesafe#730
@brodycj brodycj force-pushed the cordova-sqlite-express-wip branch from 7269cba to d412eb5 Compare December 12, 2017 01:41
@brodycj brodycj changed the title Cordova sqlite express BROKEN EXPERIMENTAL WIP Cordova sqlite express EXPERIMENTAL WIP Dec 12, 2017
@brodycj
Copy link
Contributor Author

brodycj commented Dec 12, 2017

This experimental WIP is now FIXED to pass selfTest on Android by fixing the SQLiteAndroidDatabase class to end transaction if active before closing.

P.S. Another TODO item is to update some existing tests to pass on the builtin android.database implementation.

@brodycj
Copy link
Contributor Author

brodycj commented Dec 12, 2017

An additional issue I noticed is that the builtin android.database implementation seems to treat numeric parameter values (integer or floating point) as string values. Needs further investigation.

@brodycj
Copy link
Contributor Author

brodycj commented Dec 12, 2017

Another thing I noticed from the logcat output is that the builtin Android sqlite database implementation attempts to execute PRAGMA journal_mode to check the actual value when opening the database. This is what triggered the failure originally discussed here. But if I would check the result of PRAGMA journal_mode I do not get 100% same result on builtin android.database implementation vs Android-sqlite-connector, iOS, macOS ("osx"), Windows, or WP8.

The following test case shows that PRAGMA journal_mode returns journal_mode value of "persist" on builtin android.database implementation, "delete" on the other implementations Android-sqlite-connector/iOS/macOS/Windows/WP8:

        it(suiteName + 'Check journal mode (plugin ONLY)', function(done) {
          if (isWebSql) pending('SKIP: NOT SUPPORTED for (WebKit) Web SQL');

          var db = openDatabase("Check-sqlite-PRAGMA-encoding.db", "1.0", "Demo", DEFAULT_SIZE);

          expect(db).toBeDefined();

          db.executeSql('PRAGMA journal_mode', [], function(rs) {
            expect(rs).toBeDefined();
            expect(rs.rows).toBeDefined();
            expect(rs.rows.length).toBe(1);
            expect(rs.rows.item(0).journal_mode).toBe('??');

            // Close (plugin only) & finish:
            (isWebSql) ? done() : db.close(done, done);
          }, function(error) {
            // NOT EXPECTED:
            expect(false).toBe(true);
            expect(error.message).toBe('--');
            done();
          });
        }, MYTIMEOUT);

P.S. TBD it is not certain whether which journal_mode value of DELETE or PERSIST would be more optimal on mobile platforms, especially Android & iOS. Performance comparison should be measured using something such as https://github.com/brodybits/Cordova-sqlite-perftest. Assuming no major difference in performance I would favor using journal_mode=PERSIST to reduce the number of possible difference between the different platform implementations.
P.P.S. This test shows PRAGMA journal_mode set to delete by default in case of builtin android.database implementation (database opened with androidDatabaseImplementation: 2 setting) on Android 8.1 (emulator). TBD needs further investigation. I am now thinking it would probably be best to keep PRAGMA journal_mode set to delete in all cases. TBD not sure if this should be explicitly done in case of builtin android.database implementation. TBD should probably be set explicitly in all other cases.

@brodycj brodycj mentioned this pull request Dec 12, 2017
32 tasks
brodycj pushed a commit to brodycj/cordova-sqlite-legacy that referenced this pull request Dec 14, 2017
Needed for new BUG 666 workaround solution to pass selfTest
in case of builtin android.database implementation
(no Android-sqlite-connector / Android-sqlite-native-driver libraries).

Ref:
- storesafe/cordova-sqlite-storage#730
- storesafe/cordova-sqlite-storage#666
brodycj pushed a commit to brodycj/cordova-sqlite-legacy that referenced this pull request Dec 14, 2017
Close db before opening (ignore close error)

Now tested on builtin android.database implementation
(no Android-sqlite-connector / Android-sqlite-native-driver libs).

Ref:
- storesafe/cordova-sqlite-storage#666
- storesafe/cordova-sqlite-storage#730
@brodycj brodycj changed the base branch from storage-master to dev February 18, 2019 21:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant