On Thu, Jun 19, 2014 at 8:49 AM, Winnie Lacesso
<[log in to unmask]> wrote:
> Greetings SL Users!
>
> Many apologies if this is not exactly SL-specific but all the servers run
> SL so am hoping it's okay to ask here. (& that someone has debugged this!)
>
> A cluster of WN echo their syslogs to 2 central log/mon hosts. When the WN
> changed from SL5 to SL6 (but the central log/mon hosts for various reasons
> must remain SL5), logwatch on the central log/mon hosts stopped reporting
> anything from them. (I read logwatch once a week on central log/mon to
> watch for disk or similar badness.)
I'd review the "rsyslog" packages on the SL 5 server, in particular
consider upgrading to the "rsyslog5" package. Be very careful doing
this: Runing "yum uninstall rsyslog" can clear a *lot* o fpackages!!!
And I'd review the release notes on the newer rsyslog.
> Does anyone have SL5 central log/mon hosts with SL6 "clients" syslogging
> to them, & have debugged what needs change/fix in the SL5 logwatch
> processing scripts to report about the SL6 clients (esp i.e. disk
> badness)?
>
> I've debugged it to
> /usr/share/logwatch/scripts/shared/onlyservice 'smartd'
> on the 2 SL5 log/mon hosts does not "pass thru" smart-logged entries from
> the SL6 WN. Near the end of ouptut of
>
> logwatch --debug 6 --detail 5 --service smartd --range Today --print
>
> is
>
> Processing Service: smartd
> ( cat /var/cache/logwatch/logwatch.AN03MJZX/messages | /usr/bin/perl
> /usr/share/logwatch/scripts/shared/onlyservice 'smartd' |/usr/bin/perl
> /usr/share/logwatch/scripts/shared/removeheaders '' |/usr/bin/perl
> /usr/share/logwatch/scripts/services/smartd) 2>&1
>
> On another pair of SL5 log/mon hosts with only SL5 clients logging to
> them, that finds exactly as expected.
>
> On the SL5 log/mon hosts with SL6 clients logging to them, onlyservice
> 'smartd' finds zero:
> root@smnat> grep -i smartd /var/log/messages | wc -l
> 1096
> root@smnat> grep -i smartd /var/log/messages | tail -2
> Jun 18 10:53:28 sm10.hadoop.cluster sm10 smartd[1811]: Device: /dev/sda [SAT], 37 Currently unreadable (pending) sectors
> Jun 18 11:01:06 sm05.hadoop.cluster sm05 smartd[1824]: Device: /dev/sda [SAT], 11 Offline uncorrectable sectors
> # yep, definitely there
> root@smnat> cat /var/log/messages |\
> /usr/bin/perl /usr/share/logwatch/scripts/shared/onlyservice 'smartd' > /tmp/m; wc /tmp/m
> 0 0 0 /tmp/m
>
> When the WN were SL5, it worked:
>
> root@sm00> cat /var/log/messages.8|/usr/bin/perl /usr/share/logwatch/scripts/shared/onlyservice 'smartd' > /tmp/m; wc /tmp/m
> 843 11879 105783 /tmp/m
> root@sm00> head -4 /tmp/m
> Apr 20 04:07:51 sm06.hadoop.cluster smartd[11331]: Device: /dev/sda, SMART Prefailure Attribute: 1 Raw_Read_Error_Rate changed from 111 to 112
> Apr 20 04:17:54 sm18.hadoop.cluster smartd[11547]: Device: /dev/sda, 1199 Currently unreadable (pending) sectors
> Apr 20 04:17:54 sm18.hadoop.cluster smartd[11547]: Device: /dev/sda, 1069 Offline uncorrectable sectors
> Apr 20 04:18:13 sm16.hadoop.cluster smartd[11470]: Device: /dev/sda, 1 Currently unreadable (pending) sectors
>
> Very grateful if someone has an SL6-client-compatile "onlyservice" part of
> logwatch for SL5 central log/mon host!
>
> Winnie Lacesso / Bristol University Particle Physics Computing Systems
> HH Wills Physics Laboratory, Tyndall Avenue, Bristol, BS8 1TL, UK
|
