TASK: Make E2E tests work for Neos 9.0 #3569

grebaldi · 2023-07-07T11:18:08Z

refs: #3546, #3406
~~includes: #3571~~
~~includes: #3618~~

This is the third and final in a series of PRs leading up to Neos 9.0. I want to make sure that our E2E tests can be executed locally on every maintained branch.

How to test this

You should of course be able to run the E2E tests locally via

make test-e2e-docker browser=chromium # or browser=chrome

The E2E tests use cweagans/composer-patches to include patches from the development collection. Documentation can be found here: https://github.com/cweagans/composer-patches/blob/1.x/README.md
We discussed to keep this dependency to make developing across repos easier ;)

Remaining TODOs

Make anything run at all
Fixture: 1Dimension
- Create event export from site export
- Fix all occurring E2E test failures
  - 1Dimension/createNewNodeInContainer.e2e.js Line 33: "AssertionError: expected false to be truthy"
    - Root cause is here: https://github.com/neos/neos-development-collection/blob/9.0/Neos.Neos/Resources/Private/Fusion/Prototypes/ContentCollection.fusion#L30
    - I suppose that both Neos.Node.inBackend and node.context.currentRenderingMode.edit can no longer be relied upon that easily
    - Known as: TASK: 9.0 migrate node.context.currentRenderingMode.edit rector#23
    - Most likely fixed by: FEATURE: Edit preview mode support for Neos 9 neos-development-collection#4067
  - 1Dimension/discarding.e2e.js Line 41: "AssertionError: After discarding we are back to the main page: expected 'user-admin__eyJsYW5ndWFnZSI6ImVuX1VTIn0=__d7315383-adb8-45a5-8124-ffabeee
    bc51a' to deeply equal '/sites/neos-test-site@user-admin;language=en_US'"
    - Find proof that /sites/neos-test-site@user-admin;language=en_US and user-admin__eyJsYW5ndWFnZSI6ImVuX1VTIn0=__d7315383-adb8-45a5-8124-ffabeeebc51a represent the same node
      - They don't. The expected node address should rather be user-admin__eyJsYW5ndWFnZSI6ImVuX1VTIn0=__f676459d-ca77-44bc-aeea-44114814c279. d7315383-adb8-45a5-8124-ffabeeebc51a presumably refers to the node that has just been discarded in this test. Redirect to the home page ought to happen, but fails.
    - There is also an error: TODO: SITE NOT FOUND; should not happen (for address NodeAddress[contentStream=9ce2581c-6e0a-467a-ad33-37c7133ce3a2, dimensionSpacePoint={"language":"en_US"}, nodeAggregateId=6b0b30db-933b-4c3d-b47b-0a7beed6c315, workspaceName=user-admin]
    - This happens because the guest frame is trying to render the discarded node
    - Nicer error message would need to be solved in https://github.com/neos/neos-development-collection/blob/9.0/Neos.Neos/Classes/Controller/Frontend/NodeController.php#L146 and https://github.com/neos/neos-development-collection/blob/9.0/Neos.Neos/Classes/Controller/Frontend/NodeController.php#L222
    - Known as: 9.0 BUG: Neos backend throws error on discarding newly created document node neos-development-collection#4024
    - May be fixed by: !!!FEATURE: (Neos.Neos) Track created nodes in PendingChangesProjection neos-development-collection#4393
  - 1Dimension/discarding.e2e.js Line 61: "AssertionError: Discarded node gone from the tree: expected true to be falsy"
    - Same "SITE NOT FOUND" error
  - 1Dimension/switchingDimensions.e2e.js Line 24: "AssertionError: Untranslated node gone from the tree: expected true to be falsy"
    - Will be fixed with: BUGFIX: Use $nodeAggregateId->value redirect params in NodesController neos-development-collection#4390
  - 1Dimension/switchingDimensions.e2e.js Line 23: "getReact cannot return DOM elements. Use Selector functions for this purpose."
    - After switching dimensions, the focused page tree "contextPath" ends up being a DOM node:
    - Regression through 9.0: Fix dimension switching #3173 - can be easily fixed
  - 1Dimension/treeMultiselect.e2e.js Line 29: "AssertionError: Node B's context path changed: expected 'user-admin__eyJsYW5ndWFnZSI6ImVuX1VTIn0=__5b0d6ac0-40ab-47e8-b79e-39de6c0700df' to deeply
    equal '/sites/neos-test-site/node-knm2pltb5454z/node-18qsaeidy6765/node-e8tw6sparbtp3@user-admin;language=en_US'"
    - Find proof that /sites/neos-test-site/node-knm2pltb5454z/node-18qsaeidy6765/node-e8tw6sparbtp3@user-admin;language=en_US and user-admin__eyJsYW5ndWFnZSI6ImVuX1VTIn0=__5b0d6ac0-40ab-47e8-b79e-39de6c0700df represent the same node
      - neos-ui/Tests/IntegrationTests/Fixtures/1Dimension/SitePackage/Resources/Private/Content/Sites.xml
        
        Line 472 in bc6569f
        
        <node identifier="5b0d6ac0-40ab-47e8-b79e-39de6c0700df" nodeName="node-e8tw6sparbtp3">
  - 1Dimension/treeMultiselect.e2e.js Line 46: "AssertionError: Node B's context path changed: expected 'user-admin__eyJsYW5ndWFnZSI6ImVuX1VTIn0=__5b0d6ac0-40ab-47e8-b79e-39de6c0700df' to deeply
    equal '/sites/neos-test-site/node-knm2pltb5454z/node-18qsaeidy6765/node-e8tw6sparbtp3@user-admin;language=en_US'"
    - Find proof that /sites/neos-test-site/node-knm2pltb5454z/node-18qsaeidy6765/node-e8tw6sparbtp3@user-admin;language=en_US and user-admin__eyJsYW5ndWFnZSI6ImVuX1VTIn0=__5b0d6ac0-40ab-47e8-b79e-39de6c0700df represent the same node
      - neos-ui/Tests/IntegrationTests/Fixtures/1Dimension/SitePackage/Resources/Private/Content/Sites.xml
        
        Line 472 in bc6569f
        
        <node identifier="5b0d6ac0-40ab-47e8-b79e-39de6c0700df" nodeName="node-e8tw6sparbtp3">
  - 1Dimension/treeMultiselect.e2e.js Line 63: "AssertionError: Node C's context path changed: expected 'user-admin__eyJsYW5ndWFnZSI6ImVuX1VTIn0=__84eb0340-ba34-4fdb-98b1-da503f967121' to deeply
    equal '/sites/neos-test-site/node-knm2pltb5454z/node-18qsaeidy6765/node-oml0cxaompt29@user-admin;language=en_US'"
    - Find proof that /sites/neos-test-site/node-knm2pltb5454z/node-18qsaeidy6765/node-oml0cxaompt29@user-admin;language=en_US and user-admin__eyJsYW5ndWFnZSI6ImVuX1VTIn0=__84eb0340-ba34-4fdb-98b1-da503f967121 represent the same node
      - neos-ui/Tests/IntegrationTests/Fixtures/1Dimension/SitePackage/Resources/Private/Content/Sites.xml
        
        Line 499 in bc6569f
        
        <node identifier="84eb0340-ba34-4fdb-98b1-da503f967121" nodeName="node-oml0cxaompt29">
Fixture: 2Dimension
- Create event export from site export
- Fix all occurring E2E test failures
  - 2Dimension/switchingDimensions.e2e.js Line 23: "A JavaScript error occurred on '...'."
    - Also, an error flash-message appears:
    - Solution lives here: https://docs.neos.io/guide/manual/routing/configuring-dimension-resolving
    - After adding config for dimension resolution, I get a weird thing:
    - Cause: Router route(): No route matched the route path "neos/service/content-dimensions/country.json"
    - Regression through 9.0: Support multiple Content Repository instances, decouple CR from Flow neos-development-collection#3848
    - Possibly fixed by: BUGFIX: (Neos.Neos) Restore show action of ContentDimensionsController neos-development-collection#4395
Merge both fixtures as described in Apply fixes from StyleCI neos-development-collection#3406
- Create unified test distribution with both, the 1Dimension and the 2Dimension case, as site packages with separate content repositories
  - This requires BUGFIX: Consistently use correct cr identifier across delegated cr:* commands neos-development-collection#4400 to work
- Add test for switching between sites
  - ~~We're running into Cannot execute the test when it is redirected to another application with auth token in the URL DevExpress/testcafe#7548 on this one~~
    - Nope, but are running into BUG: (9.0) User Workspace is null if first login to the site happens via "/neos/switch/to/..." neos-development-collection#4401, but only if the switch sites test is executed in isolation (because otherwise, switchDimensions runs first, which creates the missing user workspace)
  - ~~Update Testcafé to v3 and use proxyless mode to execute tests~~
    - I abandoned this, even though I actually did the update and it was quite easy. But since the Testcafé version wasn't the cause for the issue that I've seen for the "switching between sites" test, and the update breaks other tests, I've decided to postpone Testcafé v3 for now. However, this can easily be done in a separate PR :)
- Use better strategy for local docker setup, now that no more package switching needs to be done
Make E2E tests work on Circle CI
Restore PHP Unit Tests on Circle CI
~~Buckle up for collisions with BUGFIX: Reflect discard of node move changes correctly in the UI #3503~~ -> merge this first
Remove dangling patches after PRs have been merged

…e-tests-on-8-2' into task/e2e-tests-on-9-0

This includes the replacement of the 1Dimension site export with an equivalent ESCR event export, as well as the adjustment of the e2e-docker.sh script for bootstrapping the test environment.

…result This fixes a regression introduced in: #3173

Using the `PendingChangesProjection` from Neos.Neos, which after neos/neos-development-collection#4393 will track all nodes that have been newly created within the workspace that contains the change set that is being discarded.

….isEditMode(request) || Neos.Backend.isPreviewMode(request)`

This includes the replacement of the 2Dimension site export with an equivalent ESCR event export. Also, as a temporary fix, neos/neos-development-collection#4395 was added as a patch.

…e' into task/e2e-tests-on-9-0

As it turns out, using nodeAddresses rather than context paths, these feedbacks are no longer necessary.

With this change, the UI will default to the default inPlace edit mode, regardless of user-side selection. Actual full-blown support for EditPreviewModes will require deeper changes over the entire code base.

…tDistribution

…istribution

Tests/IntegrationTests/utils.js

skurfuerst

just looked over this - GREAT WORK <3 <3 I'll check that we get the dependent PRs merged soon, so that we can get this one in.

…395 from TestDistribution

mhsdesign · 2023-08-08T19:57:18Z

As there might be a collision as you pointed out, i will take care of reviewing and merging: #3503

Also it seems like neos/neos-development-collection#4067 is blocking this pr from moving forward so we should definitely focus there as well ;) But we discussed in the last escr meeting a plan for the preview modes its going forward - slowly ^^

mhsdesign · 2023-08-08T20:01:03Z

packages/neos-ui-backend-connector/src/Endpoints/index.ts

+                const nodeContextPathElement = d.querySelector('.node-context-path');
+                if (!nodeContextPathElement) {
                    throw new Error('.node-context-path is not found in the result');
                }

+                const nodeContextPath = nodeContextPathElement.innerHTML.trim();
+                if (!nodeContextPath) {
+                    throw new Error('.node-context-path is empty');
+                }


why did this work previously ??? 😂

I suppose .node-context-path was never empty before, but occurred to be empty during the tests, because of all the backend changes.

mhsdesign · 2023-08-08T20:02:47Z

packages/neos-ui-sagas/src/UI/ContentTree/index.js

@@ -142,7 +142,7 @@ export function * watchCurrentDocument({globalRegistry, configuration}) {
        }

        const state = yield select();
-        const childrenAreFullyLoaded = $get(['cr', 'nodes', 'byContextPath', contextPath, 'children'], state)
+        const childrenAreFullyLoaded = ($get(['cr', 'nodes', 'byContextPath', contextPath, 'children'], state) || [])


this fix could be backported as well no?

(would also take care of it just vote: 👍 👎 )

My plan was to leave the plow-js calls as they are for now and take care of replacing them in #3306, right after the E2E tests have been merged.

This reverts commit ce395ca.

mhsdesign · 2023-09-16T19:14:15Z

I did revert your hotfix for edit preview modes ce395ca

and merged this final implementation: #3618 into your branch

lets see what the tests say :D

…gistrySubjectProvider.php`

mhsdesign · 2023-09-21T14:41:27Z

I added a hack so the tests should now work in ci :D - will take care of a proper fix.

locally make test-e2e-docker doesnt seem to work as i think i have problems with http://onedimension.localhost:8081/neos and http://twodimensions.localhost:8081/neos

-> it seems one needs to modify the /etc/hosts ?

          echo 127.0.0.1 onedimension.localhost | sudo tee -a /etc/hosts
          echo 127.0.0.1 twodimensions.localhost | sudo tee -a /etc/hosts

so we need a tutorial for this for mac as well ;)

mhsdesign

I just ran the tests make test-e2e-docker locally on my mac and they work flawlessly!

As soon as the ci is green we can merge this :D

grebaldi added 2 commits July 7, 2023 12:28

Merge remote-tracking branch 'origin/bugfix/3301-part-2/fix-docker-e2…

65232ef

…e-tests-on-8-2' into task/e2e-tests-on-9-0

TASK: Initialize ESCR-driven E2E test setup for 1Dimension fixture

3b1f00c

This includes the replacement of the 1Dimension site export with an equivalent ESCR event export, as well as the adjustment of the e2e-docker.sh script for bootstrapping the test environment.

github-actions bot added Task 9.0 labels Jul 7, 2023

TASK: Replace context paths with node addresses in E2E assertions

6e8e0c9

grebaldi mentioned this pull request Jul 7, 2023

BUGFIX: Use $nodeAggregateId->value redirect params in NodesController neos/neos-development-collection#4390

Merged

grebaldi added 2 commits July 8, 2023 12:34

BUGFIX: Read .node-context-path innerHTML when parsing a node search …

eb3182f

…result This fixes a regression introduced in: #3173

This was referenced Jul 8, 2023

!!!FEATURE: (Neos.Neos) Track created nodes in PendingChangesProjection neos/neos-development-collection#4393

Merged

FEATURE: Edit preview mode support for Neos 9 neos/neos-development-collection#4067

Closed

mficzel and others added 6 commits July 9, 2023 16:12

TASK: Replace usage of Neos.Node.inBackend(node) with `Neos.Backend…

57a5bc1

….isEditMode(request) || Neos.Backend.isPreviewMode(request)`

TASK: Make E2E tests work for 2Dimension fixture

ad36bc7

This includes the replacement of the 2Dimension site export with an equivalent ESCR event export. Also, as a temporary fix, neos/neos-development-collection#4395 was added as a patch.

Merge remote-tracking branch 'upstream/90-replace-node-inbackend-usag…

a53af29

…e' into task/e2e-tests-on-9-0

TASK: Get rid of RemoveNode feedback in Move* changes

1375319

As it turns out, using nodeAddresses rather than context paths, these feedbacks are no longer necessary.

TASK: Add rudimentary support for EditPreviewModes in 9.0

ce395ca

With this change, the UI will default to the default inPlace edit mode, regardless of user-side selection. Actual full-blown support for EditPreviewModes will require deeper changes over the entire code base.

TASK: Update CircleCI php image to cimg/php:8.2-node

7b451a0

grebaldi force-pushed the task/e2e-tests-on-9-0 branch 2 times, most recently from 30cf764 to 58cfe98 Compare July 11, 2023 12:00

TASK: Adjust e2e.sh test script for CI to match local setup for Neos 9.0

e018877

grebaldi force-pushed the task/e2e-tests-on-9-0 branch from 58cfe98 to e018877 Compare July 11, 2023 12:15

TASK: Merge integration test site packages into one unified distribution

c43bb47

grebaldi force-pushed the task/e2e-tests-on-9-0 branch from e4356fe to 45c2cf6 Compare July 11, 2023 15:32

TASK: Adjust e2e.sh script to match new unified test setup

895ca5c

grebaldi force-pushed the task/e2e-tests-on-9-0 branch from 45c2cf6 to 895ca5c Compare July 11, 2023 15:40

grebaldi added 2 commits July 12, 2023 11:33

TASK: Remove patch for neos/neos-development-collection#4395 from Tes…

228f551

…tDistribution

TASK: Restore dependency to neos/buildessentials and phpunit in TestD…

cc06094

…istribution

mhsdesign reviewed Jul 12, 2023

View reviewed changes

Tests/IntegrationTests/utils.js Outdated Show resolved Hide resolved

grebaldi added 3 commits July 12, 2023 15:36

TASK: Add E2E test for switching between sites in a multi-site setup

b6f3970

Merge remote-tracking branch 'upstream/9.0' into task/e2e-tests-on-9-0

65a80c7

TASK: Fix StyleCI issues

c250be1

ahaeslich mentioned this pull request Jul 28, 2023

BUGFIX: (Neos.Neos) Restore show action of ContentDimensionsController neos/neos-development-collection#4395

Merged

TASK: Symlink composer.json in docker e2e setup (rather than cp)

3f2f886

skurfuerst approved these changes Aug 1, 2023

View reviewed changes

grebaldi added 2 commits August 2, 2023 10:49

Merge remote-tracking branch 'upstream/9.0' into task/e2e-tests-on-9-0

27c6bb4

TASK: Remove patches for neos/neos-development-collection#4393 and #4…

07b620a

…395 from TestDistribution

mhsdesign self-requested a review August 8, 2023 19:57

mhsdesign reviewed Aug 8, 2023

View reviewed changes

mhsdesign mentioned this pull request Sep 1, 2023

!!! TASK: 9.0 Set nodeName to null for regular created nodes #3515

Merged

mhsdesign added 9.0beta2 and removed 9.0beta2 labels Sep 1, 2023

mhsdesign added 2 commits September 16, 2023 21:10

Revert "TASK: Add rudimentary support for EditPreviewModes in 9.0"

d0c743c

This reverts commit ce395ca.

Merge branch 'bugfix/90editPreviewModes' into task/e2e-tests-on-9-0

3bf7dde

mhsdesign added 3 commits September 21, 2023 16:28

BUGFIX: E2E Error: cannot require ......./FlowContextTrait in `CRRe…

103afcf

…gistrySubjectProvider.php`

BUGFIX: make test-e2e-docker composer.json not found

084bd2e

Merge remote-tracking branch 'origin/9.0' into task/e2e-tests-on-9-0

353aa83

TASK: Document need for additional etc host entries for local tests

dcefe42

mhsdesign mentioned this pull request Sep 22, 2023

BUGFIX: Reflect discard of node move changes correctly in the UI #3503

Merged

mhsdesign approved these changes Sep 22, 2023

View reviewed changes

mhsdesign force-pushed the task/e2e-tests-on-9-0 branch 3 times, most recently from 3e0e179 to 50f1a4f Compare September 23, 2023 07:03

TASK: Move testing hack yaml to reduce impact on other projects

dd99278

mhsdesign force-pushed the task/e2e-tests-on-9-0 branch from 50f1a4f to dd99278 Compare September 23, 2023 07:04

mhsdesign merged commit d632b6a into 9.0 Sep 23, 2023
4 checks passed

mhsdesign deleted the task/e2e-tests-on-9-0 branch September 23, 2023 07:17

mhsdesign mentioned this pull request Sep 23, 2023

TASK: Adjust e2e pipeline for Neos 9.0 #3406

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

TASK: Make E2E tests work for Neos 9.0 #3569

TASK: Make E2E tests work for Neos 9.0 #3569

grebaldi commented Jul 7, 2023 •

edited by mhsdesign

Loading

skurfuerst left a comment

mhsdesign commented Aug 8, 2023

mhsdesign Aug 8, 2023

grebaldi Aug 17, 2023

mhsdesign Aug 8, 2023

grebaldi Aug 17, 2023

mhsdesign commented Sep 16, 2023

mhsdesign commented Sep 21, 2023 •

edited

Loading

mhsdesign left a comment

TASK: Make E2E tests work for Neos 9.0 #3569

TASK: Make E2E tests work for Neos 9.0 #3569

Conversation

grebaldi commented Jul 7, 2023 • edited by mhsdesign Loading

skurfuerst left a comment

Choose a reason for hiding this comment

mhsdesign commented Aug 8, 2023

mhsdesign Aug 8, 2023

Choose a reason for hiding this comment

grebaldi Aug 17, 2023

Choose a reason for hiding this comment

mhsdesign Aug 8, 2023

Choose a reason for hiding this comment

grebaldi Aug 17, 2023

Choose a reason for hiding this comment

mhsdesign commented Sep 16, 2023

mhsdesign commented Sep 21, 2023 • edited Loading

mhsdesign left a comment

Choose a reason for hiding this comment

grebaldi commented Jul 7, 2023 •

edited by mhsdesign

Loading

mhsdesign commented Sep 21, 2023 •

edited

Loading