Chrome for testing vs Chrome issues

[hagaygo]

Hi

Been using chrome automation for 8+ years. (using selenium + c#)

At the past , every now and then we had issues with newer chrome version , so we had to "pin" our agent chrome version to specific chrome version (prevent auto update and such) until we mitigate the issues.

About 1 year ago Selenium introduced an option to specify chrome version , so we happily embraced it since it allow us to specify chrome version in our code and we can test and evaluate our test suite before moving to newer chrome version. (and remove our regular chrome install from our agents)

Since the start there were some discrepancies between regular chrome version and CFT ones.

At the past normally we choose to stay one version back of stable , this way our developer never use the local chrome , for example if stable is 124 , our code specified 123 until 125 is out so we try 124.

Still there were few test that worked fine on regular chrome but required some tweaks to pass on CFT (still sometimes they pass sometime we had some inconsistent elements not find errors , stale elements and such)

Since version 127 we can't really upgrade to 128+.

CFT version 127 works quite o.k.

When moving to 128 , about 15-20% of our tests fails , some of them consistently and other sometimes fail and sometimes pass.
back then , we tested our test suite against regular chrome 128 , all tests passed.
When 129 was out , same behavior , regular Chrome all tests passed , CFT version 129 , about 15-20% fails.
Now again , same with 130.

If our test code was the issue/problem, regular chrome would not work perfectly fine.

Is there any suggestion how can we troubleshoot this ?

We tried toggling headless on/off , no change.

Our assumption is that CFT should be identical to regular chrome install , i guess there are few differences.

Thanks in advance.

[mathiasbynens]

Can you please report the specific bugs on https://crbug.com/new? Whenever you note an unexpected difference in CfT vs. regular Chrome it’s worth reporting.

[hagaygo]

@mathiasbynens Thanks for you reply , but the link seems general chromium and not CFT related ?

And our tests are really app specific i doubt i can reproduce them on a public website, thats why we are looking for troubleshoot/settings option when running CFT so we can share the logs/findings.

[mathiasbynens]

For CfT-specific issues you can use https://goo.gle/cftbug. It’s the same issue tracker just with the component prefilled. We can triage things from there.

[hagaygo]

I added also on https://issues.chromium.org/issues/374790917

@mathiasbynens Thanks for your help so far.