Message from Syslogd

Question

During a ssh session running a long rsync-job the system crashed and I received the following output in my shell:

Message from syslogd@x123 at Aug 21 13:42:05 ...
 kernel:[1050642.629179] ------------[ cut here ]------------

Message from syslogd@x123 at Aug 21 13:42:05 ...
 kernel:[1050642.629670] invalid opcode: 0000 [#1] SMP

Message from syslogd@x123 at Aug 21 13:42:05 ...
 kernel:[1050642.629950] last sysfs file: /sys/devices/virtual/block/md0/md/mismatch_cnt

Message from syslogd@x123 at Aug 21 13:42:05 ...
 kernel:[1050642.636405] Stack:

Message from syslogd@x123 at Aug 21 13:42:05 ...
 kernel:[1050642.638071] Call Trace:

Message from syslogd@x123 at Aug 21 13:42:05 ...
 kernel:[1050642.640982] Code: 2e 48 8b 43 30 48 8b 53 28 48 8d 4b 28 48 89 42 08 48 89 10 49 8b 06 48 89 48 08 48 89 43 28 4c 89 73 30 49 89 0e e9 c6 00 00 00 <0f> 0b eb fe 49 8b 44 24 d8 48 b9 00 00 00 00 00 16 00 00 4c 8b

Since that occurrence I can't connect to my Debian Server. Even after reboots. However I can connect via ssh in Rescue Mode but I don't know what to do in order to get the system back to normal operation mode.

Any help is highly appreciated.

Sounds like your server is using software RAID and it's now suffering somehow. Do you still see all your disk mounts in rescue mode? — Janne Pikkarainen, Aug 24 '10 at 09:01
Hello Janne, yes I am using a software Raid. When I do ls /dev, I see sda, sda1, sda2, sdb, sdb1 and sdb2. But when I check ls /mnt, I only see cdrom/ dvd/ external/ external1/ floppy/ sda2/ sdb1/ sdb2/ test/ Does that mean that one hard disk is broken? — harp, Aug 24 '10 at 09:19
Do you have all the data accessible you should have? :) Does the "mount" command return all the same mount points that you have defined in /etc/fstab? — Janne Pikkarainen, Aug 24 '10 at 11:11
I fixed the problem with e2fsck. There seemed to be a problem with blocks. — harp, Aug 24 '10 at 11:35

harp · Answer 1 · 2010-08-24T13:04:35.827

2

I fixed the problem with e2fsck on my RAID-1 system. There seemed to be a problem with files and blocks. Solution is as follows:

Log in in Rescue Mode and execute the following:

# umount -a

# e2fsck -f -y /dev/sda2

# e2fsck -f -y /dev/sdb2

# shutdown -r -n now

edited Aug 24 '10 at 13:04

answered Aug 24 '10 at 11:39

harp

221
4
10

Fixing individual disks within a software RAID sounds scary to me. Wouldn't it be better to run e2fsck against the software RAID device (/dev/md0)? – Janne Pikkarainen Aug 24 '10 at 12:24
Well, I guess you are right. However, In my case it worked just fine. – harp Aug 24 '10 at 12:47
RAID-1, I presume? Then it might work. – Janne Pikkarainen Aug 24 '10 at 12:57
exactly, RAID-1 it is. – harp Aug 24 '10 at 13:03
fsck on individual drives and the RAID stack are different. Do individual first, then the stack. For not a RAID doing touch ./forcefsck and rebooting is much simpler, it will fsck the root before it mounts it. – Alan Corey Nov 18 '20 at 22:01

score 0 · Answer 2 · edited Aug 14 '13 at 14:29

0

Perform the following command to get a list of processes running syslog:

ps -ef |grep syslog

Which will in turn print out something like so:

root      3947     1 13 Aug14 ?        00:58:05 /sbin/rsyslogd -i /var/run/syslogd.pid -c 5
root     28092 25977  0 05:36 pts/0    00:00:00 grep syslog

Identify the process you want to terminate (in this case the one with id 3947) and kill it like so:

kill -9 3947

This should solve your problem.

edited Aug 14 '13 at 14:29

James Mertz

26,224
41
111
163

answered Aug 14 '13 at 13:43

user245612

1

1

Can you please provide a little more detail as to how this should be run and what exactly it does? – Taegost Aug 14 '13 at 14:03
I've edited your answer to give it more clarity, but you really need to explain why syslog is the culprit, and how to identify it. – James Mertz Aug 14 '13 at 14:32

Message from Syslogd

2 Answers2