Added CLI migration support

[domax]

No description provided.

[stuartgunter]

This is a pretty significant enhancement, so it will take some time for me to go through this. I do appreciate your contribution... but for large new features like this, it would've been nice to be involved in a conversation about it first. Other users of dropwizard-cassandra have asked about this before and would probably have requirements to feed in too.

To help my research, please could you give me a bit of background on the choice for cassandra-migration. Why was it chosen? What alternatives were considered? etc.

Thanks

[domax]

Hi. Yes, I understand, it sounds reasonable.

When I looked for core functionality for migration, I stuck myself with the following rules:

• it should mimic FlyWay approach (as most popular and simple);
• it should be Java for easy integration;
• it should have CI and be published in Maven Central.
• the more stars - the better.

After search and shallow review, I found 2 projects:

1. Contrast-Security-OSS/cassandra-migration
2. patka/cassandra-migration

1st project has a lot of implemented features - almost everything I wanted to have, plus lots of stars - but it has one big drawback: design is not ideal, it is impossible to use it as a library w/o changes.

2nd project has good design, but lack of useful features - like, e.g. Java migration scripts.

So, I put away both of them and continued search. Third library I found for research was builtamont-oss/cassandra-migration - this one was used finally.

It is a fork of 1st one, but fully re-written on Kotlin (it is not important in this case, b/c it's Java in the background). It fits ideally to my rules I mentioned above, it got rid of drawbacks of original library, well covered with tests, published in Maven, has Travis CI account - very close to requirements you follow in your project ;).
As for me it is very good implementation of Cassandra migration feature.

[stuartgunter]

Great summary! Thanks for giving your thought process.

Out of interest, did you look into https://github.com/sky-uk/cqlmigrate at all? Any thoughts? I'm just toying with the idea of parameterising the migration lib (in a similar way to how certain config options are typed). What do you think of that option? I just wonder if it would provide flexibility for those that have already invested in other solutions (even internal home-grown projects). But haven't thought it through at all yet.

Quick disclaimer: I know some of the people that created it at Sky, but I have never used it myself. Someone else asked about C* migration libraries and I pointed them towards this one purely because it was the only one I knew about.

[domax]

I didn't hear about sqlmigrate tool before. I didn't yet review code of this tool, but after learning its README, I have couple comments.

First of all I didn't find there support of any kind of programming migration scripts - only CQLs. I believe it's important. Sometimes CQL is not enough, some more sophisticated logic is needed.

2nd thing that caught my eye at once is that sqlmigrate requires to pre-populate not only keyspace/schema (that is completely logical, b/c of typical database security policies) but migration table as well. Actually it wants a dedicated keyspace with only one table there. I don't think it's good solution, b/c you may have e.g. 2 microservices based on dropwizard, both of them use Cassandra as persistent layer - so that you'll have 2 separate keyspaces for each microservice (well - in case if you follow microservice paradigm ;)). But this migration tool violates data isolation with it's own shared keyspace for both microservices. Plus, in case if you cannot parametrize a name of locks table, you cannot organize migration scripts for 2 different microservices - data about migrations of these microservices will be mixed there.

And, finally 3rd thing - if I understand it right, it supports only a migration action: no gathered info about current migration state (which scripts are applied, which are pending), no validation (are migration scripts and current database structure correspond each other), no possibility to make a baseline (start migration support from non-empty database).

Please correct me if I'm wrong somewhere.

[stuartgunter]

Thanks for looking into it. It feels like it would be safer (more forwards compatible) to provide options for other kinds of migration libraries in the future. Whether cqlmigrate is one of them, I'm not entirely sure (again, this is mostly because I haven't personally used it).

I do think the concept of CLI-based migration is a good one, and I'm quite happy to support the library you've chosen to implement in this PR.

I'm thinking of merging this PR, then making some very minor modifications to parameterise the migration library for forwards compatibility with future variants. This would essentially add a single value to the config to select the migration approach to use, e.g.:

migration:
  type: (cassandra-migration|cqlmigrate|whatever)

This would allow you to use the approach that works for you, while also allowing others that perhaps prefer to use separate CQL files (or have already invested in using another approach) to continue in their current operations without having to make large scale changes.

Any objections?

[domax]

No objections - I like this idea! )