Adding new entity (slow)

[neplan]

Is there a way to improve the adding of new entities on the client side?

It can happen in our application, that we need to create a lot of new entities on the client. Which then needs to be saved on the server.
Adding a lot of new entities in the context on the client takes forever.
At the moment we create the entity and we add it in the context.
Is there a suggestion on how to improve that?

Thanks,

Alessandro

[Daniel-Svensson]

You don't provide any context (version, runtime, number of entities ...), so there is not much to go on and it is difficult to say if it is slow or not.

You should have no problem adding a few thousand entities per second,
you can find some very old numbers from #111 which managed 5000 entities in around 680 ms on laptop class machine which was followed up by #126 to further speed up performance.

In general you should be able to expect the client side openria code to be faster than the actual saving on the server side (especially if you use entity framework 6).

A common problem is to do (individual) Add on collections bound to the UI, for example if you display the contents of an EntitySet.
Each add will trigger a collection change event, and an UI update.
Many UI component providers have their own way of handle "batch" updates to solve that scenario.

To se what and why you think it is slow the best way is to start profiling your application and see where the time is spent.

[neplan]

Dear Svensson

Sorry. Here some more information:
We are adding around 20000 new entities and it's taking around 324 seconds, which I think it's quite long. The list is not bound to any GUI control.
We are using .NET Framework 4.8 and the OpenRia 5.4.

Could it be that we should avoid adding new entities over the navigation property? I will try to add the new entities directly in the context to see if something will change.

I will do a performance analysis and I can put the analysis here.

Best regards,

Alessandro

[Daniel-Svensson]

It would be interesting if you could share your benchmark code as well as any profiling results (to see which methods take the most time) that are part of your analysis.

Could it be that we should avoid adding new entities over the navigation property? I will try to add the new entities directly in the context to see if something will change.

That might make a difference, in my previous test I did not have any navigation properties loaded (they can load to much more change notifications beeing triggered and slow downs) .

Some thoughs (mainly from my previous investigation several years ago)

• I did consider caching "GetIdentity" calls (would reduce allocations at least and some speedup), but it was still a small part of time for normal "add" without navigation properties
• The time to create entities and set their properties took a lot more time than expected (more than the Add)

also I did some small fixes which might speed up your scenario #451 #453

[Daniel-Svensson]

I hope you are well and I look forward to hear about the results from your investigation.

I just wanted to let you know that I have made a little investigation about raw add performance and have made some minor improvements for setting entity properties that might make it into a new release).

With the preliminary changes (#456) 500 000 simple entities can be created in around 4s (and adding them takes a few more seconds), but even without the changes you should be able to create the entities (and set 3 properties on each of the 500 000 instances) in around 6-8s. (AMD zen3)

The test did not include any navigation properties

[neplan]

Dear Daniel

Thank you for the effort. I hope I can do the performance analysis in the next few weeks and post the results here.

Best regards,

Alessandro

[neplan]

Dear Daniel

I have made a performance analysis, during our add procedure:

I hope it helps to see the problem.

Best regards,

Alessandro

[Daniel-Svensson]

It looks like it spends almost all the time in the AddAttachInferer.Visit*Ä

You might be able to significantly reduce the time spent if you make sure to set as few navigion properties properties (EntityRef, EntityCollection) as possible and try to only set foreign key properties instead. Based on code below it seems that they should be skipped if not initialised. EntityCollections are more expensive than EntityRefs so if you must set one, se if you can do it one-way only.

However there might be some room for further performance improvements, but you would need to provide some sample code (both generated entities and a loop which adds the entitites).

Maybe you can add some classes to the Benchmarks project (add a new namespace "NavigationProperties", "Complex" or something else. it is fine to remove properties which are not keys/foreign keys and change names it you need to ), or just provide a small console app with generated code + add logic

Ideas:

• AddAttachInferrer (at least when started from EntitySet.Add) should probably bee able to stop on entities which are already been Added previously (but need to check EntitySet.AddInternal to se if there are some validation which might be lost)
  • does this mean checking status == added ( && maybe CheckCrossContainer) and do we need to look at IsInferred ?
• Use a private array to cache MetaType.AssociationMembers (set it in constructor?) => 7% or ~~38s faster
• Try to use Entity.EntitySet (if not null in AddAttachInferrer.Visit) => 3-6% faster
• Advanced and probably complex: Add in memory index for foreign keys

[neplan]

Dear Daniel

We are not setting the entity collections manually. They are set automatically when the entities are loaded to the client.
For us It's not possible to reduce the navigation properties, since we have quite a complicated model behind, which cannot be changed.

I will try to create a project to replicate the problem.

Best regards,

Alessandro

Best regards,
Alessandro

[Daniel-Svensson]

Can you try the code from #487
It changes the time to add 20 000 items using an though an EntityCollection (navigion property) from around 35s to around 314ms
OpenRiaServices.Client.zip

[neplan]

Dear Daniel

I have tried the changed DLL and with the new one the adding of new entities (18539) was reduced from 91sec to 9sec. Nice improvement.

Best regards,

Alessandro

[Daniel-Svensson]

The performance fixes are part of 5.4.3 which is now released