persistence v3 mongo #228

JoTec2002 · 2024-03-21T18:34:34Z

Hello,
I know it’s a relatively large Pull Request, but I still hope that it can be used. I'm open for feedback/ improvements.
This pull request is in an some what beta state of the feature but I wanted to be able to hopefully get some feedback.

Description

The gool if this pull request is adding alternatively to the Entity Framework a complete MongoDB Database integration.
Current state:

Fully linked Asset CRUD operation with Database – API can be configured to either use the current implementation or the MongoDB Database interface
Limited file functionality
Files still need to be loaded currently on startup, to show up in Web UI.

Newly added dependencies:
MongoDB.Driver
MongoDB.Driver.GridFS

Motivation and Context

The implementation is created as part of a German University Project at DHBW Stuttgart.
If added an early Version of the University Document below.
Studienarbeit_Jonas_Graubner_V0.1.pdf

The motivation for adding an alternative to the Entity Framework is, that it seems impossible to fully map the interfaces to EF. The MongoDB Driver was able to achieve this.
An Interface was created, so that new Native DB Driver still cloud be added in the Future.
For ease of use a docker compose file was added to startup an MongoDB – File will be extended to also include the server itself.

Type of change

Please delete options that are not relevant.

New feature (non-breaking change which adds functionality)
This change requires a documentation update

How Has This Been Tested?

I used Apidog to run most of the queries against the original Server and afterwards my implementation and looked if the response was identical.
Test Environment:

For Original: Lasted Docker Container
For new: MongoDB in Docker Server in Visual Studio Debuger

Checklist:

My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
I have added tests that prove my fix is effective or that my feature works

@JoTec2002

Reimplemented Interfaces for DB Use: - IAasxFileServerInterfaceService - IAdminShellPackageEnvironmentService - IConceptDescriptionService - ISubmodelService AASX Server Blazor / Startup.cs Change to DB Class (@JoTec2002 change later to dynamic switch based on startup Parameter)

Implementation of Many File operations still Missing

-> Files can be loaded into MongoDB -> Server can be stateless

…ceV3-Mongo

Freezor

Some major issues found. Please resolve them first and finish the open TODO's in your code, before we can continue the review. On any questions, please feel free to contact me.

Freezor · 2024-06-13T07:50:58Z

content-for-demo/.gitignore

@@ -0,0 +1 @@
+/mongo


why do you need a new gitignore here. That should also be configurable in the existing gitignore

Freezor · 2024-06-13T07:51:41Z

src/AasxServerBlazor/Properties/launchSettings.json

@@ -10,7 +10,7 @@
    },
    "AasxServerBlazor": {
      "commandName": "Project",
-      "commandLineArgs": "--no-security --secret-string-api 1234 --aasx-in-memory 1000 --data-path \"C:\\Development\\Ronny\" --edit --external-blazor http://localhost:5001",
+      "commandLineArgs": "--no-security --aasx-in-memory 1000 --data-path \"C:\\GitClones\\aasx-server\\content-for-demo\\aasxs\" --edit --external-blazor http://localhost:5001 --with-mongodb \"mongodb://mongo:mongo@localhost:27017/\"",


please revert the changes for --data-path

Freezor · 2024-06-13T07:54:23Z

src/AasxServerStandardBib/Database.cs

+using System.IO;
+
+
+//Author: Jonas Graubner


Please don't use personalised author tags. I know that we do not have a template for that yet, but creating your own one will be harder to resolve in the end. All your commits can be viewed in the git history and if you want to add yourself as a contributor, you can add a new entry in the CONTRIBUTORS.md

Freezor · 2024-06-13T07:55:56Z

src/AasxServerStandardBib/AasxServerStandardBib.csproj

@@ -32,6 +32,8 @@
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
    </PackageReference>
    <PackageReference Include="Microsoft.IdentityModel.Tokens" Version="6.13.1" />
+    <PackageReference Include="MongoDB.Driver" Version="2.25.0" />


Is the license for these package checked, to be compatible with the open source guidelines?

Freezor · 2024-06-13T07:57:14Z

src/AasxServerStandardBib/Database.cs

+
+        public void Initialize(String connectionString)
+        {
+            //_client = new MongoClient("mongodb://AAS:[email protected]:27017/?authSource=AAS");


don't commit commented code. If you think this might be useful information as an example, please create a valid c# method summary

Freezor · 2024-06-13T08:13:52Z

src/AasxServerStandardBib/Services-DB/AssetAdministrationShellServiceDB.cs

+
+
+
+        public List<IAssetAdministrationShell> GetAllAssetAdministrationShells(List<SpecificAssetId> assetIds = null, string idShort = null)


Consider using LINQ for filtering to enhance readability and maintainability within this method.

Freezor · 2024-06-13T08:20:28Z

src/AasxServerStandardBib/Services-DB/AdminShellPackageEnvironmentServiceDB.cs

+
+        public void DeleteAssetInformationThumbnail(int packageIndex, IResource defaultThumbnail)
+        {
+            //TODO


Regarding your TODO comments, you have two options:

Please resolve the TODO before finalizing this PR.

If you cannot resolve it immediately, please add detailed information on why and create a GitHub issue to keep track of it.

Freezor · 2024-06-13T08:22:31Z

src/AasxServerStandardBib/Database.cs

+                stream.Close();
+            }else
+            {
+                //throw new ArgumentNullException(nameof(stream));


It's important to handle exceptions appropriately rather than commenting them out. In your WriteFile method, you've commented out the ArgumentNullException handling for the stream parameter. This approach can lead to unexpected behavior or silent failures if stream is null, as it will not be properly handled or logged.

Best practice dictates that exceptions should be handled or propagated correctly to maintain application reliability and provide meaningful error messages for debugging. Commenting out exceptions can obscure issues and make troubleshooting more difficult, which is considered a code smell.

To improve, consider either uncommenting the exception handling and adding appropriate logging or refactor the method to handle null stream scenarios more gracefully.

Freezor · 2024-06-13T08:23:47Z

src/AasxServerStandardBib/Database.cs

+            using (var cursor = _bucket.Find(filter, options))
+            {
+                var fileInfo = cursor.ToList().FirstOrDefault();
+                // fileInfo either has the matching file information or is null


This could rather be used as a method summary.

/// <summary> /// Retrieves the ObjectId of the latest file matching the specified filename in the GridFS bucket. /// </summary> /// <param name="filename">The filename of the file to retrieve.</param> /// <returns>The ObjectId of the matching file, or ObjectId.Empty if no matching file is found.</returns>

Freezor · 2024-06-13T08:24:28Z

src/AasxServerStandardBib/Database.cs

+            });
+        }
+
+        public void importAdminShellPackageEnv(AdminShellPackageEnv adminShellPackageEnv)


Use the standard C# naming for methods, which is PascalCase please

JoTec2002 added 19 commits February 21, 2024 17:42

Create docker-compose for MongoDB

9ec9d46

Add MongoDB Driver

7753a39

Insert AssetEnv into MongoDB on startup

f333ec0

AdminShellPackageEnvironmentServiceDB Finish AAS Region

ca400b4

AdminShellPackageEnvironmentServiceDB Finish Submodel Region

1f252ed

First Implementation of AdminShellPackageEnvironmentServiceDB

783839b

Implementation of Many File operations still Missing

Add Startup Argument for Mongo

2aaacb5

Rename Mongo Docker Compose

e98035d

Add MongoDB LFS

f379dd1

-> Files can be loaded into MongoDB -> Server can be stateless

Add Database LFS to AdminShellPackageEnvironmentServiceDB

433abaa

remove FileServerInterfaceDB

4517e31

No need for change in these Services for DB use

e172a33

Merge remote-tracking branch 'upstream/main' into jotec2002/persisten…

8544b84

…ceV3-Mongo

Merge remote-tracking branch 'upstream/main' into jotec2002/persisten…

7ac7d50

…ceV3-Mongo

Fix Merging Bugs

4d02201

Update Dependencies

0349a11

Add Thumbnail Endpoint for MongoDB Version

3496d02

Update Docker Deployment

1f7c178

Freezor requested changes Jun 13, 2024

View reviewed changes

Freezor added enhancement New feature or request database Relates to changes involving the database schema, migrations, queries, or overall database managemen labels Jun 13, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

persistence v3 mongo #228

persistence v3 mongo #228

JoTec2002 commented Mar 21, 2024

Freezor left a comment

Freezor Jun 13, 2024

Freezor Jun 13, 2024

Freezor Jun 13, 2024

Freezor Jun 13, 2024

Freezor Jun 13, 2024

Freezor Jun 13, 2024

Freezor Jun 13, 2024

Freezor Jun 13, 2024

Freezor Jun 13, 2024

Freezor Jun 13, 2024




		public List<IAssetAdministrationShell> GetAllAssetAdministrationShells(List<SpecificAssetId> assetIds = null, string idShort = null)

		@@ -0,0 +1 @@
		/mongo

persistence v3 mongo #228

Are you sure you want to change the base?

persistence v3 mongo #228

Conversation

JoTec2002 commented Mar 21, 2024

Description

Motivation and Context

Type of change

How Has This Been Tested?

Checklist:

Freezor left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment