Access SQLite database as Android asset (was: Request add user io readwrite)

[baidwwy]

Hi,
I want to open the database in memory
I want to open the database in android assets

I Googled and found that vfs is needed, but I don’t know how to implement it. Can you add related interfaces? thanks

Sorry, I don’t speak English, this is the content translated by Google

[utelle]

Request add user io readwrite

Sorry, I have no idea what this issue title is supposed to mean. Can you please describe in more detail, what you try to accomplish, how this is related to this project, and what you want me to do?

I want to open the database in memory I want to open the database in android assets

SQLite supports memory databases through the special database file name :memory:.

Currently this project does not include direct support for Android. However, based on the instructions for Android to be found on the SQLite website it should be possible to build Android support files.

I Googled and found that vfs is needed, but I don’t know how to implement it. Can you add related interfaces? thanks

Please mention the sources from which you get your information. Maybe that could help to clarify your issue. The VFS interface is an integrated part of SQLite itself. This project implements a VFS shim to support database encryption. So, which interface is missing for your use case?

Sorry, I don’t speak English, this is the content translated by Google

Unfortunately the quality of Google translate is limited.

[baidwwy]

thank you for your reply
Like this
https://github.com/mavstuff/sqlite_android
https://github.com/mavstuff/sqlite_android/blob/cda2845bc8e8f70046c030d798b21838a775a5b8/sqlite3/src/main/jni/sqlite/assetvfs.c#L106
add readwrite callback?

[utelle]

Like this https://github.com/mavstuff/sqlite_android

I took a quick look. This project implements a SQLite VFS that allows to access databases from Android assets. The project is based on the original unchanged SQLite source files (sqlite3.c and sqlite3.h). Does this project work for you? That is, can you build and use it successfully?

I guess you want in your own application a SQLite version with encryption and therefore would like to use SQLite3MultipleCiphers. In principle, this shouldn't be too difficult to accomplish. You have to replace the source files sqlite3.c and sqlite3.h by the amalgamation files of SQLite3MultipleCiphers (sqlite3mc_amalgamation.c and sqlite3mc_amalgamation.h), which are provided with a release. Additionally, you will have to adjust the build file Android.mk - you will have to add several compile time symbols, so that the encryption extension and other SQLite extensions, you may want to keep, will be compiled correctly. Furthermore, you will have to adjust the implementation of the function sqlite3_assetvfs(), so that it references the SQLite3MC VFS. Other adjustments may be necessary depending on the actual requirements of your appliation.

https://github.com/mavstuff/sqlite_android/blob/cda2845bc8e8f70046c030d798b21838a775a5b8/sqlite3/src/main/jni/sqlite/assetvfs.c#L106
add readwrite callback?

AAsset_read is a function from the Android NDK. What do you want to accomplish with a readwrite callback?

The main problem I see is that Android assets are read-only resources. Therefore the write method of the assetvfs is actually a no-op. That is, if you want an encrypted database as an Android asset, you will have to create it elsewhere, because assetvfs can't create it.

For me it is still unclear, what you really want to accomplish. Please describe your use case in more detail. Otherwise it is impossible to provide useful advice.

[baidwwy]

I want to replace the IO of sqlite3mc
like this

void myioopen {
    AAssetManager_open
}
void myioread {
    AAsset_read
}

sqlite3mc_ioopen = myioopen 
sqlite3mc_ioread = myioread

I tried to modify it myself, but failed.
This is too complicated for me, so please add the interface

Anyway, thank you for your time

[utelle]

I want to replace the IO of sqlite3mc like this

Actually, SQLite3MC does not perform IO on its own. As I mentioned in a previous post, SQLite3MC is a so-called VFS shim. This means that SQLite3MC sits between SQLite and the actual VFS (Virtual File System). Read and write request issued by the SQLite library are forwarded by SQLite3MC to the actual VFS, which performs the real IO. SQLite3MC just manipulates the read buffer, after it has been filled by the real VFS, by decrypting it, before returning to SQLite, and it manipulates the write buffer by encrypting it, before passing it to the real VFS.

void myioopen {
    AAssetManager_open
}
void myioread {
    AAsset_read
}

sqlite3mc_ioopen = myioopen 
sqlite3mc_ioread = myioread

I tried to modify it myself, but failed. This is too complicated for me, so please add the interface

Sorry, this is not going to happen. The SQLite3MC VFS handles encryption/decryption only, while the real VFS handles the actual IO. Your requested change would break that principle.

In my previous post I already layed out the general route you will have to follow: you will have to make adjustnents, so that the assetvfs VFS can co-operate with SQLite3MC:

[...] You have to replace the source files sqlite3.c and sqlite3.h by the amalgamation files of SQLite3MultipleCiphers (sqlite3mc_amalgamation.c and sqlite3mc_amalgamation.h), which are provided with a release. Additionally, you will have to adjust the build file Android.mk - you will have to add several compile time symbols, so that the encryption extension and other SQLite extensions, you may want to keep, will be compiled correctly. Furthermore, you will have to adjust the implementation of the function sqlite3_assetvfs(), so that it references the SQLite3MC VFS. Other adjustments may be necessary depending on the actual requirements of your application.

BTW, I intend to convert this issue into a discussion, because it is actually not a SQLite3MC issue. Nevertheless, it may be an interesting topic for other users.

[baidwwy]

In other words （pMethods->xRead）Is the IO of sqlite3 ,not of mc

SQLite3MultipleCiphers/src/sqlite3mc_vfs.c

Line 704 in 0074daa

int rc = REALFILE(pFile)->pMethods->xRead(REALFILE(pFile), buffer, count, offset);

I get it, thank you very much

[utelle]

In other words （pMethods->xRead）Is the IO of sqlite3 ,not of mc

Well, not exactly. This is where the actual IO request from the SQLite library is forwarded to the underlying real VFS.

Beware that this is not the only place in sqlite3mc_vfs.c where the xRead method of the underlying VFS is called.

I strongly recommend to not tamper with the implementation in sqlite3mc_vfs.c. Instead you should adjust the assetvfs implementation to co-operate with SQLite3MC.