Improve portability

[matt335672]

Fixes #2908

Improves portability

• Use clearenv() if it exists
• Don't rely on <limits.h> being pulled in by <sys/param.h>

[abrossard]

Hi, I confirm that it compiles and runs fine on Solaris 11. Thanks for the very quick response.
There are compilation warnings, but not related, so this fix is fine from my point of view.
For references:

ssl_calls.c: In function ‘ssl_rc4_set_key’:
ssl_calls.c:224:5: warning: ‘RC4_set_key’ is deprecated [-Wdeprecated-declarations]
  224 |     RC4_set_key((RC4_KEY *)rc4_info, len, (tui8 *)key);
...
ssl_calls.c: In function ‘ssl_rc4_crypt’:
ssl_calls.c:251:5: warning: ‘RC4’ is deprecated [-Wdeprecated-declarations]
  251 |     RC4((RC4_KEY *)rc4_info, len, (tui8 *)data, (tui8 *)data);
...
and so on for MD5_init, MD5_Update, MD5_Final, 
...
verify_user_pam.c: In function ‘common_pam_login’:
verify_user_pam.c:257:15: warning: assignment to ‘int (*)(int,  struct pam_message **, struct pam_response **, void *)’ from incompatible pointer type ‘int (*)(int,  const struct pam_message **, struct pam_response **, void *)’ [-Wincompatible-pointer-types]
  257 |     pamc.conv = verify_pam_conv;

sesrun.c: In function ‘usage’:
sesrun.c:179:50: warning: format ‘%s’ expects argument of type ‘char *’, but argument 2 has type ‘int’ [-Wformat=]
  179 |     g_printf("    -t <type>             Default:%s\n", DEFAULT_TYPE);
      |                                                 ~^
      |                                                  |
      |                                                  char *
      |                                                 %d
sesrun.c:179:49: warning: ‘%s’ directive argument is null [-Wformat-overflow=]
  179 |     g_printf("    -t <type>             Default:%s\n", DEFAULT_TYPE);

[matt335672]

Thanks for the swift reply!

Before I merge this, let's look at your other issues.

• The openssl warning about RC4_set_key is interesting. We only try to call this function for OpenSSL versions under 3.0. What version of openSSL are you running on Solaris?
• The PAM one may be a concern. Are you using PAM, and does it seem to be working? There's a program called authtest which is built as part of the build but not distributed. See if that works. If not, I may have a fix for you.
• The sesrun error may cause xrdp-sesrun --help to segfault or display something odd for the -t default. This is happening as the macro DEFAULT_TYPE is being defined elsewhere in the Solaris includes. I've pushed another commit to rename this macro to DEFAULT_SESSION_TYPE.

[abrossard]

For openssl, it turns out that both versions are installed, 1.02.2 and 3.09. The latter is stored under /usr/openssl/3/bin and that wasn't in my path.
The login works and so does PAM, so I wouldn't worry about that one.

[matt335672]

Thanks.

I'll squash this and merge it.

[matt335672]

I'm not proposing to backport this to 0.9 as there seems to be little interest in Solaris on that branch.