Replaced OrientDB with ArcadeDB (latest version)

[lvca]

OrientDB is not supported anymore and has issues with Apple Silicon (see #709). ArcadeDB is a (conceptual) fork, with a new engine, much faster and lighter and no use of JNA/JNI, just 100% pure java and embeddable. Same license (Open Source - Apache 2) and with an active community.

The SQL layer is the same between ArcadeDB and OrientDB, so no need to rewrite the queries.

Run the test cases and it's all green.

[mhberger]

Cool! That is a lot less invasive.

Have tried kicking the wheels. My primary use case is to install the
distribution binary (either download or using something like sdkman).
Then test/check with simple blog (which has a custom type) and jbake.org.

In this case we run the build to generate the binaries and use those.

Doing the build

../gradlew clean test smokeTest build

Has one failing test.

  OvenTest > shouldBakeWithRelativeCustomPaths() FAILED

However, we want the zip file so the following does it.

../gradlew clean assemble

Installing and running to generate simple blog on iMac Pro Intel produces
“Baked 47 items in 3050ms” and on MacBook Pro M1 Max “Baked 47 items in 1777ms”.
Note to run from a known state, I have to

rm -rf cache

I wonder if there is an option to not use the cache or clean it out the
start of a run?

Installing and running to generate jbake.org generates the following error on both iMac
Pro Intel and MacbBook Pro M1 Max.

4:52:30.631 ERROR jbake - An unexpected error occurred: Record too big to be stored in bucket 'Documents_4' (5), size=65008 max=57342. Change the `arcadedb.bucketDefaultPageSize` database settings and try again

Where is a good place to change this setting? I tried jbake.config.

[lvca]

@mhberger I see one difference is that some tests used "memory:" storage type of OrientDB, but in Arcade that's not available: only persistent databases with ArcadeDB. Could it be the database is not deleted between tests from the file system?

I've seen the same test failing, but if I run it separately passes.

You can set this at startup (before the database is created/opened): GlobalConfiguration.BUCKET_DEFAULT_PAGE_SIZE.setValue(131072) or by defining a setting or env variable java ... -Darcadedb.bucketDefaultPageSize=131072

NOTE: The page size is a temporary limitation. In the future, if a record is larger than a page, it will be managed transparently without the need to set the page size.

[mhberger]

Thanks! Running with the following does the trick. And for the note that
it should not be required in the future.

rm -rf cache
export JAVA_OPTS=' -Darcadedb.bucketDefaultPageSize=131072'
jbake -b

Some rough timings.

Site        | Hardware            | Database   | Bake Info
----------- | ------------------- | ---------- | ----------
simple blog | iMac Pro Intel      | OrientDB   | Baked 47 items in 4044ms
simple blog | iMac Pro Intel      | SQLite     | Baked 47 items in 2839ms
simple blog | MacBook Pro M1 Max  | SQLite     | Baked 47 items in 1836ms
simple blog | MacBook Pro M1 Max  | ArcadiaDB  | Baked 47 items in 3019ms
            |                     |            |
jbake.org   | iMac Pro Intel      | OrientDB   | Baked 196 items in 10731ms
jbake.org   | iMac Pro Intel      | SQLIte     | Baked 196 items in 15461ms
jbake.org   | iMac Pro Intel      | ArcadiaDB  | Baked 196 items in 12882m
jbake.org   | MacBook Pro M1 Max  | SQLite     | Baked 196 items in 9611ms
jbake.org   | MacBook Pro M1 Max  | ArcadiaDB  | Baked 196 items in 7636ms

[lvca]

Not bad as the first implementation. We can do much better. By the way, I've set this in the constructor, so no need to set anything anymore:

// USE A 4X BIGGER PAGE THAN THE DEFAULT
GlobalConfiguration.BUCKET_DEFAULT_PAGE_SIZE.setValue(Bucket.DEF_PAGE_SIZE * 4);

Also, I see Document records have a type property. What types are supported? My idea is to use the ArcadeDB polymorphism and create a subtype of Document, like:

Post <--- Document

So you can avoid indexing the type and the queries would be faster. Instead of doing select * from Documents where type='%s' order by date desc you could do select * from %s where order by date desc.

Unless type is a dynamic property that can change in a Document or unless you can have arbitrary types created at run-time, using the polymorphism helps. And of course, if you do select from Document, all the records of Document and sub-types will be returned.

[lvca]

Hi guys, any plans to replace OrientDB with ArcadeDB?

[lprimak]

It's still failing one test :( I would love to run this on m new CI server (which is broken at the moment, since it's M1 :)

[lvca]

@lprimak what test is failing? I've got a MacBook Pro 16 Intel and everything passes:

BUILD SUCCESSFUL in 1m 1s
18 actionable tasks: 3 executed, 15 up-to-date

[lprimak]

Well it fails Travis CI that's set up for the project here.

[lvca]

I see, the failing test is:

@Test
public void shouldBakeWithRelativeCustomPaths() throws Exception {
  sourceFolder = TestUtils.getTestResourcesAsSourceFolder("/fixture-custom-relative");
  configuration = (DefaultJBakeConfiguration) new ConfigUtil().loadConfig(sourceFolder);
  File assetFolder = new File(configuration.getDestinationFolder(), "css");
  File aboutFile = new File(configuration.getDestinationFolder(), "about.html");
  File blogSubFolder = new File(configuration.getDestinationFolder(), "blog");


  final Oven oven = new Oven(configuration);
  oven.bake();

  assertThat(oven.getErrors()).isEmpty();
  assertThat(configuration.getDestinationFolder()).isNotEmptyDirectory();
  assertThat(assetFolder).isNotEmptyDirectory();
  assertThat(aboutFile).isFile();
  assertThat(aboutFile).isNotEmpty();
  assertThat(blogSubFolder).isNotEmptyDirectory();
}

It looks like this is false assertThat(aboutFile).isFile();. How this can be related to the underlying ArcadeDB?

[jonbullock]

Apologies for the delay in getting back to you on this, I need to think about how best to manage the transition from OrientDB to ArcadeDB for users before I merge this in.

[lvca]

By the way, we have an importer if needed: https://docs.arcadedb.com/#OrientDB-Importer. You can also call it via API:

File databaseDirectory = new File(DATABASE_PATH);
URL inputFile = OrientDBImporterIT.class.getClassLoader().getResource("orientdb-export-small.gz");
OrientDBImporter importer = new OrientDBImporter(("-i " + inputFile.getFile() + " -d " + DATABASE_PATH + " -s -o").split(" "));
importer.run().close();

[yeroc]

@jonbullock is this PR the best way forward for Apple Silicon (M1) users?

[jonbullock]

Thanks for the pointer about the import @lvca I haven't forgotten about this, I do want to explore this further but my time is extremely limited at present.

@yeroc PR #765 should buy the required time for me to explore this further.