GroupedNot When used in the fluent API appears to still not work

[NikRimington]

Hey Shannon,

I've just updated to v1.0.5 using it in an Umbraco 8.6.4 site and I'm not sure that the changes you made to the GroupedNot method to fix #180, are being reflected when used in the fluent API.

I have the following code elements:

var query = string.IsNullOrEmpty(queryTerm) ? searcher.CreateQuery().NodeTypeAlias(Recording.ModelTypeAlias)

query in this case ends up as an IBooleanOperation, after this I have the following:

if (validFilteredItems.Count > 0)
                    query= query.And().GroupedNot(new[] { "id" }, featuredItems.Select(i => i.ToString()).ToArray());

I have to use the .And() because IBooleanOperation doesn't have .GroupedNot directly available to it.

This results in the following query:

+__NodeTypeAlias:recording +(id:2197 id:2204 id:2206)

Where as I think it should result in

+__NodeTypeAlias:recording -id:2197 -id:2204 -id:2206

Am I missing something in how I'm using the fluent API? Or is there still an issue around this type of query?

Cheers,

Nik

[Shazwazza]

Looks like a different issue all together, i'll have to add more tests for .And().GroupedNot

You might be able to try the AndNot method which accepts a sub query, tbh I'm unsure what that will produce, i'll have to add tests for that as well.

[Shazwazza]

The other issue is really the Fluent API with "Not" becomes a little bit misleading since Or().GroupedNot would never make sense since that won't filter out anything. The Not functionality in lucene is a filter and means AND_NOT so in theory And().GroupedNot sort of makes sense but Or().GroupedNot doesn't.

I suppose in a future major version the 'Not' filtering can be reviewed with how that might work better.

I just found the work around to your issue though which is used in tests, you can build up the query like this too:

var qry = internalSearcher.CreateQuery();
qry.NodeTypeAlias("test");
qry.GroupedNot(new[] { "id" }.ToList(), new[] { "1", "2", "3" });
... 

[NikRimington]

That's interesting because I can't actually do you work around as the code doesn't seem to allow:

When I create the query:

searcher.CreateQuery().NodeTypeAlias(Recording.ModelTypeAlias)

It's generates an IBooleanOperation which when I try and do .GroupedNot() I get the error IBooleanOperations does not contain a definition for .GroupedNot()

searcher is an instance of ISearcher gotten from the index.

My current work around is to manually add Not's using a loop, but it's not ideal.

[Shazwazza]

Ah, i see the work around is in tests because it casts to this type:

criteria = (LuceneSearchQuery)searcher.CreateQuery();

not ideal. I'll see what can be done correctly but won't be till next week sometime.

[bjarnef]

I noticed the following didn't work when I have registered the field as integer. Otherwise if not defining the field as integer it works.

index.FieldDefinitionCollection.TryAdd(new FieldDefinition("completedByCount", FieldDefinitionTypes.Integer));

var searcher = index.GetSearcher();
var query = searcher.CreateQuery()
    .GroupedOr(new[] { "__NodeTypeAlias" }, new[] {
        Training.ModelTypeAlias
    });

if (trainingId.HasValue)
{
    query = query.Not().Field("__NodeId", trainingId.Value.ToString());
}

if (brandId.HasValue)
{
    query = query.And().Field("searchPath", brandId.Value.ToString());
}

if (notCompleted.HasValue && notCompleted.Value == true)
{
    query = query.And().GroupedNot(new[] { "completedByCount" }, new[] { total });
}

var results = query.OrderBy(new SortableField("name", SortType.String)).Execute(pageSize * page);
var totalResults = results.TotalItemCount;
var pagedResults = results.Skip(pageSize * (page - 1));

The raw lucene query:
+(__NodeTypeAlias:training) +searchPath:1094 -completedByCount:2

It seems I can use the following instead, when field is defined as integer.

query = query.AndNot(x => x.RangeQuery<int>(new[] { "completedByCount" }, 0, total, maxInclusive: false));

[Shazwazza]

When fields are typed (like numbers, etc...) they must be searched on accordingly otherwise lucene doesn't know what you are searching on which is why .GroupedNot(new[] { "completedByCount" }, new[] { total }); will not work for a numerical field since it will generate a full text query and not a range query.

[Shazwazza]

Hi @NikRimington,

I've pushed tests here https://github.com/Shazwazza/Examine/blob/dev/src/Examine.Test/Search/FluentApiTests.cs#L680 and the result is correct. So either this issue is resolved in Examine, or I'm not setting up this test correctly.

Any chance you would be able to clone this repo and run that test to make sure I haven't missed anything?