rsyslogd stops logging?

[joshuabaird]

This very well could be an issue not related to this module, but, any advice would be appreciated.

On RHEL7 hosts, this module gets included as part of the 'base' profile. It does the following:

Jan 13 12:46:28 imqa-d1-cl05 puppet-agent[5222]: (/Stage[main]/Rsyslog/File[rsyslog_sysconfig]/content) content changed '{md5}2dd909cb96f91060d206c4936dac51ee' to '{md5}c17fe833a49d81d3220115f7972aaa7a'
Jan 13 12:46:28 imqa-d1-cl05 puppet-agent[5222]: (/Stage[main]/Rsyslog/File[rsyslog_logrotate_d_config]/content) content changed '{md5}79d4bbb4dc7a5b5ff0b1b1ebd06c2b0b' to '{md5}b4cab6e78c61ff2eee5c68774f834364'
Jan 13 12:46:28 imqa-d1-cl05 puppet-agent[5222]: (/Stage[main]/Rsyslog/File[rsyslog_config]/content) content changed '{md5}0dd94a0c285fb32f41fa5b226e83c26b' to '{md5}3921b2a8558b93e7f1417fb0fa24f5a3'
Jan 13 12:46:28 imqa-d1-cl05 rsyslogd: [origin software="rsyslogd" swVersion="7.4.7" x-pid="636" x-info="http://www.rsyslog.com"] exiting on signal 15.
Jan 13 12:46:28 imqa-d1-cl05 rsyslogd: [origin software="rsyslogd" swVersion="7.4.7" x-pid="13165" x-info="http://www.rsyslog.com"] start

After this, in /var/log/messages, I see what looks to be dmesg output:

Jan 13 12:46:28 imqa-d1-cl05 rsyslogd: [origin software="rsyslogd" swVersion="7.4.7" x-pid="636" x-info="http://www.rsyslog.com"] exiting on signal 15.
Jan 13 12:46:28 imqa-d1-cl05 kernel: [    0.000000] Initializing cgroup subsys cpuset
Jan 13 12:46:28 imqa-d1-cl05 kernel: [    0.000000] Initializing cgroup subsys cpu
Jan 13 12:46:28 imqa-d1-cl05 kernel: [    0.000000] Initializing cgroup subsys cpuacct
Jan 13 12:46:28 imqa-d1-cl05 kernel: [    0.000000] Linux version 3.10.0-327.el7.x86_64 ([email protected]) (gcc version 4.8.3 20140911 (Red Ha
t 4.8.3-9) (GCC) ) #1 SMP Thu Oct 29 17:29:29 EDT 2015

The output eventually ends, but so does ALL logging (to /var/log/messages). The only way to resume logging is to manually bounce the rsyslog service.

This does not happen on RHEL6.

Any ideas here?

[ghoneycutt]

I'm using this on EL7 and not seeing this behavior. It is odd that when Puppet restarts it, logging stops, then when you restart a second time, it works. When it is in the broken state, I would try running strace to see what's happening. Also curious how puppet restarts it. You could run with the -d for debug and see what commands are being used to restart the service. Perhaps it is doing it in such a way that is using bad sysconfig data or bad command line options and when you restart the service manually it is starting it differently.

[joshuabaird]

I'm curious if you are seeing the time stamped kernel logging in messages like I pasted in the example?

On Jan 13, 2016, at 3:36 PM, Garrett Honeycutt <[email protected]mailto:[email protected]> wrote:

I'm using this on EL7 and not seeing this behavior. It is odd that when Puppet restarts it, logging stops, then when you restart a second time, it works. When it is in the broken state, I would try running strace to see what's happening. Also curious how puppet restarts it. You could run with the -d for debug and see what commands are being used to restart the service. Perhaps it is doing it in such a way that is using bad sysconfig data or bad command line options and when you restart the service manually it is starting it differently.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/92#issuecomment-171424766.

[joshuabaird]

Ok - so the problem is that the module is not loading the required imjournal module for EL7/systemd. Adding the following line to rsyslog.conf fixes this issue:

$ModLoad imjournal

This is default configuration in EL7. It is also documented here:

https://access.redhat.com/solutions/2112651?tour=6

I guess we can add a conditional to rsyslog.conf.erb to load this module on EL7+? Thoughts?

[joshuabaird]

#93 opened.

EDIT: Looks like we are still missing a few default options in rsyslog.conf. Checking that now and will add to the PR if necessary.

[joshuabaird]

Ok - updated the PR and added a few new bits to the template and two new params - working_directory and journalstate_file.

[ghoneycutt]

PR is #93

[ghoneycutt]

#94