[Question] Is there a way to customize the 404 logic?

[BrunoQuaresma]

I have an app built using NextJS, and usually, the 404 page of NextJS returns the 200 status code because it is a static page. So, I would like to check the page using its body, looking for the canonical meta tag. Is that possible?

[mre]

Hey Bruno,
interesting idea. This is not possible right now. I am thinking about an elegant way to make it happen, but it would require a user to work on the raw HTML response and feed the result back into lychee.

Anyone knows of a tool which supports that?

One could think of a scripting language like lua or a way to run custom shell scripts, but I don't think it will be quite elegant. The best solution I can think of is to create a custom checker based on lychee-lib as a library. It would require quite some Rust work I guess.

[BrunoQuaresma]

Hey Matthias, thanks for the reply.

I think a good UX could be something like:

lychee https://coder.com/docs/coder/blablabla --body "meta[canonical='https://coder.com/404']"

The user can pass a body option containing a CSS selector or even a REGEX if that is easier. Unfortunately, I have 0 skills in Rust 😅

[mre]

What if you checked multiple websites? Should the --body parameter be used for all of them?

[lebensterben]

@BrunoQuaresma

See this article at 35% mark: https://pretagteam.com/question/how-to-remove-404-error-in-nextjs-or-react-js

Next.js is able to return custom 404 page with 404 return code.

[lebensterben]

In all cases without exception, we should expect the server to comply to IETF RFC 7231.

Treating well-defined return code in non-standard way should not be considered.

[MichaIng]

Since currently with default -X get (and -X head anyway) only a small fraction of the response is actually downloaded, any flag/option which requires to download the whole response slows down the scan notably, which should be clear.

However, I guess the recursion feature has some basic function/method for this, probably only applied to text/html 200 responses, since it obviously as well requires to parse the whole response.

An I remember (cannot find currently) another request which required the whole response, so probably functions/methods can be reasonably shared as a basis for further such features.

[mre]

Closing this as there is a preferable solution provided by @lebensterben and for the reasons stated in the comments.
Your input was very welcome @BrunoQuaresma and thanks for using lychee. 😃