refactor the Query class to be more modular

[sytone]

Following up from a discussion in a PR comment.

The Query class is getting bigger, and more filters are being added. How about we make a Query engine/factory and then register the filters with it based off a Filter class. Then we can pass in the query string and the factory will iterate through the registered filters to choose which to execute. This would probably also have a folder for all the filters to be in to make the codebase cleaner.

Taking this approach, the query engine could be used multiple times allowing grouping and by extension OR/AND based logic.

So, you would register a sub query filter which is a implementation of the engine which would return the set of tasks to pass on to the next filter to process based on this logic. There would be an AND filter and an OR filter in this case and then all data between the () would be passed in.

The refactor would mean the creation of Filters for each one we have in query already and the creation of the base class to inherit off.

[claremacrae]

I am very interested in this. Thank you for writing it up.

I'd like to suggest a similar take, but slicing the abstraction differently, by field/property type/data type (am unsure of the vocabulary).

Adding support for a new search type

For each piece of data that might be present in a task, there is the possibility to use it in:

• A filter on its value (e.g. starts (before|after|on) <date>)
• Its presence or absence ( e.g. no start date, has start date)
• Sorting (sort by start)
• Soon, group by for various fields, including eventually group by start

Currently, the code for that is spread around several places, and adding support for a new piece of data involves multiple edits in Query.ts, and also Sort.ts and soon Group.ts.

For example, the way that in in Query.ts adding a new sort type needs adding to SortingProperty and sortByRegexp, as well as adding to Sort.ts. I found it's easy to add support for a new field, and then forget to add it to one key location, and to have failed to add it altogether!

Incomplete implementations

Some recent improvements have gone like this:

• Add happens filter. Then user requests sort by happens
• Someone adds has <start|scheduled|due> date filter. And does not has happens date and has done date

Common implementation details

Also, there will sometimes be code in common, for example, sort by urgency knows that this must be a decreasing numeric sort.
And group by urgency ought to behave the same, which potentially means replicating code currently.

There is also a lot of duplicated code between the various date-based searches.

Suggestion

I'm imagining a file called something vaguely like DueDateProperty.ts that has methods vaguely like:

• value(Task task)
• filter(Task task)
• sort(Task task1, Task task2)
• groupLabel(Task task)

And maybe also things like:

• parseFilterInstruction(str line)
• parseSortInstruction(str line)
• parsePresenceAbsenceInstruction(str line)

These are just meant to be examples to try and convey the idea... I don't know for sure if the parse functions would go in here, for example.

And of course all the date-based ones would have very little code, and where the only thing that would differ would be which field value(Task) would return, with much of the rest of the implementations being common code between all of them - aside from odd niggles like has start date behaving differently...

Example Benefits

• Adding new search types would be pretty easy (as in the factory proposal above)
• Also, it would be likely that developers would easily add new sort and grouping options at the same time
• Nearly all the code for a new property would be written by adding a single new file, rather than edits spread across multiple files
• It could lead naturally to fewer and simpler tests, using individual Tasks (for filtering) or pairs of tasks (for sorting)

[sytone]

Really we need to make a tokenizer to take the query, convert to the tokens and then parse and execute. Give me a bit to think it through, the fields we have are static so they can just be tokens in the grammar used for the query. Ill try and make a grammar definition this evening based of the existing query language.

[claremacrae]

Here is what I mean by my explanation above, or at least, a step on the way:
#690 'Create date filter classes'