resourceListCreator: Different order of input resources leads to different results

[RandomByte]

Expected Behavior

The order of resources as provided by workspace.byGlob should have no effect on the produced resources.json.

Current Behavior

Due to the minor randomization applied in byGlob (https://github.com/SAP/ui5-fs/blob/70d9cf42abc34faa73a86e2ccb75279a806c82d2/lib/AbstractReader.js#L45-L50), we spotted rare occurrences of differences in the generated resources.json in the current pomahawk voter.

This indicates that the correct generation of a resources.json currently depends on a certain order of resources. I'm still not sure what that order is.

Note that on my system, in >99% of all executions an equal resources.json has been produced. However, we do not know yet whether the content of that specific version is actually correct.

Example 1

--- local-pomahawk-build/maven-tooling (= majority of UI5 Tooling executions)
+++ local-pomahawk-build/ui5-tooling
@@ -3,8 +3,5 @@
             "module": "sap/ui/thirdparty/jqueryui/jquery-effects-shake.js",
             "size": 559,
             "format": "raw",
-            "isDebug": true,
-            "required": [
-                "sap/ui/thirdparty/jqueryui/jquery-effects-core.js"
-            ]
+            "isDebug": true
         },

Example 2

--- openui5/src/sap.ui.core/resources-0.json (= majority of executions)
+++ openui5/src/sap.ui.core/resources-44.json
@@ -1157,12 +1157,7 @@
            "module": "ui5loader.js",
            "size": 115623,
            "format": "raw",
-           "isDebug": true,
-           "required": [
-               "sap/ui/thirdparty/baseuri.js",
-               "sap/ui/thirdparty/es6-promise.js",
-               "sap/ui/thirdparty/es6-shim-nopromise.js"
-           ]
+           "isDebug": true
        },

Example 3

--- openui5/src/sap.ui.core/resources-0.json (= majority of executions)
+++ openui5/src/sap.ui.core/resources-68.json
@@ -7734,7 +7734,8 @@
            ],
            "condRequired": [
                "sap/ui/thirdparty/blanket.js"
-           ]
+           ],
+           "dynRequired": true
        },

Steps to reproduce the issue

Even though we already identified the mentioned pseudo-randomization as the cause (while not root cause), for future reference here's how we reproduced this locally. Basically we generated a lot of resources.json files and compared their sizes.

Note that it's probably easier to reproduce this now by reversing the order of resources in https://github.com/SAP/ui5-fs/blob/70d9cf42abc34faa73a86e2ccb75279a806c82d2/lib/AbstractReader.js#L45-L50 or by searching for -dbg resources and swapping them with their non--dbg counterparts.

1. Helper Scripts

start.sh

#!/bin/bash
set -e

mkdir -p dist/
mkdir -p logs/

trap "exit" INT TERM ERR
trap "kill 0" EXIT

./do-it.sh 0 &
./do-it.sh 20 &
./do-it.sh 40 &
./do-it.sh 60 &
./do-it.sh 80 &

wait

do-it.sh

#!/bin/bash
set -e

COUNTER=$1

while [ $COUNTER -ne $(($1 + 20)) ]
do
    echo " $COUNTER "
    UI5_CLI_NO_LOCAL=true ui5 build --dest "dist/dist-$COUNTER" --verbose --include-task generateResourcesJson > "logs/$COUNTER.log" 2>&1
    cp "dist/dist-$COUNTER/resources/sap/ui/core/resources.json" ./resources-$COUNTER.json
    COUNTER=$[$COUNTER +1]
done

2. Patch ui5-builder (optional)

If possible, patch the UI5 Builder at https://github.com/SAP/ui5-builder/blob/bc4a197090bfd9c2062d4dad92dc9b5b2eff147a/lib/builder/builder.js#L375 to only write out the resources.json

 								resource.setPath(resource.getPath().replace(
 									new RegExp(`^/resources/${project.metadata.namespace}`), ""));
 							}
+							if (!resource.getPath().endsWith("resources.json")) {
+								return;
+							}
 							return fsTarget.write(resource);
 						}));
 					});

3. Execute builds

Execute start.sh in a library of your choice. For example sap.ui.core.

4. Compare sizes of resulting JSON files

ls -lS resources-*

5. Cleanup once done analyzing

Cleanup

rm resources-*
rm -rf logs dist

Context

• UI5 Module Version (output of ui5 --version when using the CLI): v2.6.4
• Node.js Version: v14.15.0
• npm Version: 6.14.8
• OS/Platform: macOS

Affected components (if known)

• ui5-builder
• ui5-server
• ui5-cli
• ui5-fs
• ui5-project
• ui5-logger