[BUG] Multiple issues with GSC

[Ahmed-E-86]

Describe the bug

I have 2 issues with Google Search Console:

The first issue:

Google Search Console marked some pages as not mobile friendly, and I went to more info, and I found these warnings:

The second issue:

• I migrated from an old domain to a new domain, and I wanted to keep some URLs as it used to be before, so I used url front matter to keep the old format.

An example:

---
url: /html/2021/1/name-of-the-file.html
--

But somehow Google Search Console, and Yandex Webmaster identify two pages for the same post, one with .html at the end, and one without .html at the end, and mark one of them as a duplicate.

I have no idea why I am having these issues, and how to fix them.

Build Environment

• OS: Ubuntu 21.10
• Theme version/commit [0.2.13, d4e7dd2]
• Hugo version [0.89.1]

Preview Environment

• OS: Ubuntu 21.10
• Browser: Vivaldi.

[HEIGE-PCloud]

For the first issue, I have unblocked all CSS and js file in the default robots.txt template. Now, the Google bot should be able to fetch them correctly.

fa0fb98

[Ahmed-E-86]

Still the same :/

[HEIGE-PCloud]

Sorry, forget about the Allow field

398a024

Please try again...

[Ahmed-E-86]

I did not receive any new errors. Thanks!

[HEIGE-PCloud]

For the second one, I am not so sure, but whether setting the url to

---
url: /html/2021/1/name-of-the-file
---

will solve it?

[HEIGE-PCloud]

Maybe https://gohugo.io/content-management/urls/#ugly-urls

[HEIGE-PCloud]

However, I believe url ends in .html will be automatically redirect to the url without .html

https://simpler-website.pages.dev/html/2020/12/proton-troubleshooting.html will be redirected to https://simpler-website.pages.dev/html/2020/12/proton-troubleshooting

So I think just keep it with no .html and everything will be fine (?

[Ahmed-E-86]

I checked one of the URLs end with .html, and I got this error:

I guess URLs end with .html redirected to URLs without .html, and search engines disprove these links, but that did not happen before, these issues started to happen lately.

Also, I use Waline which records counters for visitors, and I noticed a month ago, that there are new records for the same old .html pages but without .html at the end, and I removed them, but I found them again with higher numbers that keeps growing.

[HEIGE-PCloud]

Waline uses window.location.pathname to create record, so if the redirect happens successfully to your visitors, then a url without .html will be recorded by Waline.

[Ahmed-E-86]

Waline uses window.location.pathname to create record, so if the redirect happens successfully to your visitors, then a url without .html will be recorded by Waline.

But when I enter a URL ends with .html, Waline uses the record ends with .html not the new record.

Search engines recorded 2 URLs for the same posts, and if I remove the URL ended with .html, the redirected links will not work, so I have to keep it this way.

I feel that there is nothing to be done, because it will get worse whatever I do.

[Ahmed-E-86]

@HEIGE-PCloud Also, I found these SEO errors at Microsoft Bing Webmaster Tools: