MySQL error 28
Posted: Sat Sep 15, 2012 6:18 pm
So I have Zoneminder 1.24.4 running on CentOS 6 on an old Dell Supermicro rack server at work. We have 16 BNC cameras connected. At first, we would fill up the 80 GB hard drive incredibly quickly, so I mapped events over to a 2TB drive so we could keep a few months worth of events stored before they started to get purged. Everything has been going fine until yesterday, when the MySQL database crashed due to /dev/mapper/vg_localhost-lv_root not being able to write a pid file. I have cleared this directory out, but it only cleared out 30MB or so of space, and I don't want to try repairing the crashed zoneminder databases until I have MUCH more room free on the drive. What sort of data is being written that would fill the drive up so fast, and how can I set the machine up so this doesn't happen again? Here is some recent output from /var/log/mysqd.log:
Any help would be greatly appreciated.
Code: Select all
120914 16:29:56 mysqld_safe Number of processes running now: 0
120914 16:29:56 mysqld_safe mysqld restarted
120914 16:29:57 InnoDB: Initializing buffer pool, size = 8.0M
120914 16:29:57 InnoDB: Completed initialization of buffer pool
120914 16:29:57 InnoDB: Started; log sequence number 0 44243
120914 16:29:57 [ERROR] /usr/libexec/mysqld: Error writing file '/var/run/mysqld/mysqld.pid' (Errcode: 28)
120914 16:29:57 [ERROR] Can't start server: can't create PID file: No space left on device
120914 16:29:57 mysqld_safe Number of processes running now: 0
120914 16:29:57 mysqld_safe mysqld restarted
120914 16:29:57 InnoDB: Initializing buffer pool, size = 8.0M
120914 16:29:57 InnoDB: Completed initialization of buffer pool
120914 16:29:57 InnoDB: Started; log sequence number 0 44243
120914 16:29:57 [ERROR] /usr/libexec/mysqld: Error writing file '/var/run/mysqld/mysqld.pid' (Errcode: 28)
120914 16:29:57 [ERROR] Can't start server: can't create PID file: No space left on device
120914 16:29:57 mysqld_safe Number of processes running now: 0
120914 16:29:57 mysqld_safe mysqld restarted
120914 16:29:58 InnoDB: Initializing buffer pool, size = 8.0M
120914 16:29:58 InnoDB: Completed initialization of buffer pool
120914 16:29:58 InnoDB: Started; log sequence number 0 44243
120914 16:29:58 [ERROR] /usr/libexec/mysqld: Error writing file '/var/run/mysqld/mysqld.pid' (Errcode: 28)
120914 16:29:58 [ERROR] Can't start server: can't create PID file: No space left on device
120914 16:29:58 mysqld_safe Number of processes running now: 0
120914 16:29:58 mysqld_safe mysqld restarted
120914 16:29:58 Inn120914 16:29:58 mysqld_safe Number of processes running now: 0
120914 16:29:58 mysqld_safe mysqld restarted
120914 16:29:59 InnoDB: Initializing buffer pool, size = 8.0M
120914 16:29:59 InnoDB: Completed initialization of buffer pool
120914 16:29:59 InnoDB: Started; log sequence number 0 44243
120914 16:29:59 [ERROR] /usr/libexec/mysqld: Error writing file '/var/run/mysqld/mysqld.pid' (Errcode: 28)
120914 16:29:59 [ERROR] Can't start server: can't create PID file: No space left on device
120914 16:29:59 mysqld_safe Number of processes running now: 0
120914 16:29:59 mysqld_safe mysqld restarted
120914 16:29:59 InnoDB: Initializing buffer pool, size = 8.0M
120914 16:29:59 InnoDB: Completed initialization of buffer pool
120914 16:29:59 InnoDB: Started; log sequence number 0 44243
120914 16:29:59 [ERROR] /usr/libexec/mysqld: Error writing file '/var/run/mysqld/mysqld.pid' (Errcode: 28)
120914 16:29:59 [ERROR] Can't start server: can't create PID file: No space left on device
120914 16:29:59 mysqld_safe Number of processes running now: 0
120914 16:29:59 mysqld_safe mysqld restarted
120914 16:30:00 InnoDB: Initializing buffer pool, size = 8.0M
120914 16:30:00 InnoDB: Completed initialization of buffer pool
120914 16:30:00 InnoDB: Started; log sequence number 0 44243
120914 16:30:00 [ERROR] /usr/libexec/mysqld: Error writing file '/var/run/mysqld/mysqld.pid' (Errcode: 28)
120914 16:30:00 [ERROR] Can't start server: can't create PID file: No space left on device
120914 16:30:00 mysqld_safe Number of processes running now: 0
120914 16:30:00 mysqld_safe mysqld restarted
120914 16:30:00 InnoDB: Initializing buffer pool, size = 8.0M
120914 16:30:00 InnoDB: Completed initialization of buffer pool
120914 16:30:00 InnoDB: Started; log sequence number 0 44243
120914 16:30:00 [ERROR] /usr/libexec/mysqld: Error writing file '/var/run/mysqld/mysqld.pid' (Errcode: 28)
120914 16:30:00 [ERROR] Can't start server: can't create PID file: No space left on device
120914 16:30:00 mysqld_safe Number of processes running now: 0
120914 16:30:00 mysqld_safe mysqld restarted
120914 16:30:01 InnoDB: Initializing buffer pool, size = 8.0M
120914 16:30:01 InnoDB: Completed initialization of buffer pool
120914 16:30:01 InnoDB: Started; log sequence number 0 44243
120914 16:30:01 [ERROR] /usr/libexec/mysqld: Error writing file '/var/run/mysqld/mysqld.pid' (Errcode: 28)
120914 16:30:01 [ERROR] Can't start server: can't create PID file: No space left on device
120914 16:30:01 mysqld_safe Number of processes running now: 0
120914 16:30:01 mysqld_safe mysqld restarted
120914 16:30:01 InnoDB: Initializing buffer pool, size = 8.0M
120914 16:30:01 InnoDB: Completed initialization of buffer pool
120914 16:30:01 InnoDB: Started; log sequence number 0 44243
120914 16:30:01 [ERROR] /usr/libexec/mysqld: Error writing file '/var/run/mysqld/mysqld.pid' (Errcode: 28)
120914 16:30:01 [ERROR] Can't start server: can't create PID file: No space left on device
120914 16:30:01 mysqld_safe Number of processes running now: 0
120914 16:30:01 mysqld_safe mysqld restarted
120914 16:30:02 InnoDB: Initializing buffer pool, size = 8.0M
120914 16:30:02 InnoDB: Completed initialization of buffer pool
120914 16:30:02 InnoDB: Started; log sequence number 0 44243
120914 16:30:02 [ERROR] /usr/libexec/mysqld: Error writing file '/var/run/mysqld/mysqld.pid' (Errcode: 28)
120914 16:30:02 [ERROR] Can't start server: can't create PID file: No space left on device
120914 16:30:02 mysqld_safe Number of processes running now: 0
120914 16:30:02 mysqld_safe mysqld restarted
120914 16:30:02 InnoDB: Initializing buffer pool, size = 8.0M
120914 16:30:02 InnoDB: Completed initialization of buffer pool
120914 16:30:02 InnoDB: Started; log sequence number 0 44243
120914 16:30:02 [ERROR] /usr/libexec/mysqld: Error writing file '/var/run/mysqld/mysqld.pid' (Errcode: 28)
120914 16:30:02 [ERROR] Can't start server: can't create PID file: No space left on device
120914 16:30:02 mysqld_safe Number of processes running now: 0
120914 16:30:02 mysqld_safe mysqld restarted
120914 16:30:03 Inn120914 16:30:03 mysqld_safe Number of processes running now: 0
120914 16:30:03 mysqld_safe mysqld restarted
120914 16:30:03 InnoDB: Initializing buffer pool, size = 8.0M
120914 16:30:03 InnoDB: Completed initialization of buffer pool
120914 16:30:03 InnoDB: Started; log sequence number 0 44243
120914 16:30:03 [ERROR] /usr/libexec/mysqld: Error writing file '/var/run/mysqld/mysqld.pid' (Errcode: 28)
120914 16:30:03 [ERROR] Can't start server: can't create PID file: No space left on device
120914 16:30:03 mysqld_safe Number of processes running now: 0
120914 16:30:03 mysqld_safe mysqld restarted
120914 16:30:04 InnoDB: Initializing buffer pool, size = 8.0M
120914 16:30:04 InnoDB: Completed initialization of buffer pool
120914 16:30:04 InnoDB: Started; log sequence number 0 44243
120914 16:30:04 [ERROR] /usr/libexec/mysqld: Error writing file '/var/run/mysqld/mysqld.pid' (Errcode: 28)
120914 16:30:04 [ERROR] Can't start server: can't create PID file: No space left on device
120914 16:30:04 mysqld_safe Number of processes running now: 0
120914 16:30:04 mysqld_safe mysqld restarted
120914 16:30:04 InnoDB: Initializing buffer pool, size = 8.0M
120914 16:30:04 InnoDB: Completed initialization of buffer pool
120914 16:30:04 InnoDB: Started; log sequence number 0 44243
120914 16:30:04 [ERROR] /usr/libexec/mysqld: Error writing file '/var/run/mysqld/mysqld.pid' (Errcode: 28)
120914 16:30:04 [ERROR] Can't start server: can't create PID file: No space left on device
120914 16:30:04 mysqld_safe Number of processes running now: 0
120914 16:30:04 mysqld_safe mysqld restarted
120914 16:30:04 InnoDB: Initializing buffer pool, size = 8.0M
120914 16:30:04 InnoDB: Completed initialization of buffer pool
120914 16:30:05 InnoDB: Started; log sequence number 0 44243
120914 16:30:05 [ERROR] /usr/libexec/mysqld: Error writing file '/var/run/mysqld/mysqld.pid' (Errcode: 28)
120914 16:30:05 [ERROR] Can't start server: can't create PID file: No space left on device
120914 16:30:05 mysqld_safe Number of processes running now: 0
120914 16:30:05 mysqld_safe mysqld restarted
120914 16:30:05 InnoDB: Initializing buffer pool, size = 8.0M
120914 16:30:05 InnoDB: Completed initialization of buffer pool
120914 16:30:05 InnoDB: Started; log sequence number 0 44243
120914 16:30:05 [ERROR] /usr/libexec/mysqld: Error writing file '/var/run/mysqld/mysqld.pid' (Errcode: 28)
120914 16:30:05 [ERROR] Can't start server: can't create PID file: No space left on device
120914 16:30:05 mysqld_safe Number of processes running now: 0
120914 16:30:05 mysqld_safe mysqld restarted
120914 16:30:05 InnoDB: Initializing buffer pool, size = 8.0M
120914 16:30:05 InnoDB: Completed initialization of buffer pool
120914 16:30:06 InnoDB: Started; log sequence number 0 44243
120914 16:30:06 [ERROR] /usr/libexec/mysqld: Error writing file '/var/run/mysqld/mysqld.pid' (Errcode: 28)
120914 16:30:06 [ERROR] Can't start server: can't create PID file: No space left on device
120914 16:30:06 mysqld_safe Number of processes running now: 0
120914 16:30:06 mysqld_safe mysqld restarted
120914 16:30:06 InnoDB: Initializing buffer pool, size = 8.0M
120914 16:30:06 InnoDB: Completed initialization of buffer pool
120914 16:30:06 InnoDB: Started; log sequence number 0 44243
120914 16:30:06 [ERROR] /usr/libexec/mysqld: Error writing file '/var/run/mysqld/mysqld.pid' (Errcode: 28)
120914 16:30:06 [ERROR] Can't start server: can't create PID file: No space left on device
120914 16:30:06 mysqld_safe Number of processes running now: 0
120914 16:30:06 mysqld_safe mysqld restarted
120914 16:30:06 InnoDB: Initializing buffer pool, size = 8.0M
120914 16:30:06 InnoDB: Completed initialization of buffer pool
120914 16:30:06 InnoDB: Started; log sequence number 0 44243
120914 16:30:06 [ERROR] /usr/libexec/mysqld: Error writing file '/var/run/mysqld/mysqld.pid' (Errcode: 28)
120914 16:30:06 [ERROR] Can't start server: can't create PID file: No space left on device
120914 16:30:06 mysqld_safe Number of processes running now: 0
120914 16:30:06 mysqld_safe mysqld restarted
120914 16:30:07 Inn120914 16:30:07 mysqld_safe Number of processes running now: 0
120914 16:30:07 mysqld_safe mysqld restarted
120914 16:30:08 InnoDB: Initializing buffer pool, size = 8.0M
120914 16:30:08 InnoDB: Completed initialization of buffer pool
120914 16:30:08 InnoDB: Started; log sequence number 0 44243
120914 16:30:08 [ERROR] /usr/libexec/mysqld: Error writing file '/var/run/mysqld/mysqld.pid' (Errcode: 28)
120914 16:30:08 [ERROR] Can't start server: can't create PID file: No space left on device
120914 16:30:08 mysqld_safe Number of processes running now: 0
120914 16:30:08 mysqld_safe mysqld restarted
120914 16:30:08 InnoDB: Initializing buffer pool, size = 8.0M
120914 16:30:08 InnoDB: Completed initialization of buffer pool
120914 16:30:09 InnoDB: Started; log sequence number 0 44243
120914 16:30:09 [Note] Event Scheduler: Loaded 0 events
120914 16:30:09 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.1.61' socket: '/var/lib/mysql/mysql.sock' port: 3306 Source distribution
120914 16:47:22 [ERROR] /usr/libexec/mysqld: Table './zm/Config' is marked as crashed and should be repaired
120914 16:47:22 [ERROR] /usr/libexec/mysqld: Table './zm/Config' is marked as crashed and should be repaired
120914 16:47:46 [Note] /usr/libexec/mysqld: Normal shutdown
120914 16:47:46 [Note] Event Scheduler: Purging the queue. 0 events
120914 16:47:46 InnoDB: Starting shutdown...
120914 16:47:50 InnoDB: Shutdown completed; log sequence number 0 44243
120914 16:47:50 [Note] /usr/libexec/mysqld: Shutdown complete
120914 16:47:50 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
120915 11:23:07 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
120915 11:23:07 InnoDB: Initializing buffer pool, size = 8.0M
120915 11:23:07 InnoDB: Completed initialization of buffer pool
120915 11:23:07 InnoDB: Started; log sequence number 0 44243
120915 11:23:07 [Note] Event Scheduler: Loaded 0 events
120915 11:23:07 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.1.61' socket: '/var/lib/mysql/mysql.sock' port: 3306 Source distribution