MySQL error 28

Cucuy13 · Post by **Cucuy13** » Sat Sep 15, 2012 6:18 pm

So I have Zoneminder 1.24.4 running on CentOS 6 on an old Dell Supermicro rack server at work. We have 16 BNC cameras connected. At first, we would fill up the 80 GB hard drive incredibly quickly, so I mapped events over to a 2TB drive so we could keep a few months worth of events stored before they started to get purged. Everything has been going fine until yesterday, when the MySQL database crashed due to /dev/mapper/vg_localhost-lv_root not being able to write a pid file. I have cleared this directory out, but it only cleared out 30MB or so of space, and I don't want to try repairing the crashed zoneminder databases until I have MUCH more room free on the drive. What sort of data is being written that would fill the drive up so fast, and how can I set the machine up so this doesn't happen again? Here is some recent output from /var/log/mysqd.log:

Code: Select all

120914 16:29:56 mysqld_safe Number of processes running now: 0
120914 16:29:56 mysqld_safe mysqld restarted
120914 16:29:57  InnoDB: Initializing buffer pool, size = 8.0M
120914 16:29:57  InnoDB: Completed initialization of buffer pool
120914 16:29:57  InnoDB: Started; log sequence number 0 44243
120914 16:29:57 [ERROR] /usr/libexec/mysqld: Error writing file '/var/run/mysqld/mysqld.pid' (Errcode: 28)
120914 16:29:57 [ERROR] Can't start server: can't create PID file: No space left on device
120914 16:29:57 mysqld_safe Number of processes running now: 0
120914 16:29:57 mysqld_safe mysqld restarted
120914 16:29:57  InnoDB: Initializing buffer pool, size = 8.0M
120914 16:29:57  InnoDB: Completed initialization of buffer pool
120914 16:29:57  InnoDB: Started; log sequence number 0 44243
120914 16:29:57 [ERROR] /usr/libexec/mysqld: Error writing file '/var/run/mysqld/mysqld.pid' (Errcode: 28)
120914 16:29:57 [ERROR] Can't start server: can't create PID file: No space left on device
120914 16:29:57 mysqld_safe Number of processes running now: 0
120914 16:29:57 mysqld_safe mysqld restarted
120914 16:29:58  InnoDB: Initializing buffer pool, size = 8.0M
120914 16:29:58  InnoDB: Completed initialization of buffer pool
120914 16:29:58  InnoDB: Started; log sequence number 0 44243
120914 16:29:58 [ERROR] /usr/libexec/mysqld: Error writing file '/var/run/mysqld/mysqld.pid' (Errcode: 28)
120914 16:29:58 [ERROR] Can't start server: can't create PID file: No space left on device
120914 16:29:58 mysqld_safe Number of processes running now: 0
120914 16:29:58 mysqld_safe mysqld restarted
120914 16:29:58  Inn120914 16:29:58 mysqld_safe Number of processes running now: 0
120914 16:29:58 mysqld_safe mysqld restarted
120914 16:29:59  InnoDB: Initializing buffer pool, size = 8.0M
120914 16:29:59  InnoDB: Completed initialization of buffer pool
120914 16:29:59  InnoDB: Started; log sequence number 0 44243
120914 16:29:59 [ERROR] /usr/libexec/mysqld: Error writing file '/var/run/mysqld/mysqld.pid' (Errcode: 28)
120914 16:29:59 [ERROR] Can't start server: can't create PID file: No space left on device
120914 16:29:59 mysqld_safe Number of processes running now: 0
120914 16:29:59 mysqld_safe mysqld restarted
120914 16:29:59  InnoDB: Initializing buffer pool, size = 8.0M
120914 16:29:59  InnoDB: Completed initialization of buffer pool
120914 16:29:59  InnoDB: Started; log sequence number 0 44243
120914 16:29:59 [ERROR] /usr/libexec/mysqld: Error writing file '/var/run/mysqld/mysqld.pid' (Errcode: 28)
120914 16:29:59 [ERROR] Can't start server: can't create PID file: No space left on device
120914 16:29:59 mysqld_safe Number of processes running now: 0
120914 16:29:59 mysqld_safe mysqld restarted
120914 16:30:00  InnoDB: Initializing buffer pool, size = 8.0M
120914 16:30:00  InnoDB: Completed initialization of buffer pool
120914 16:30:00  InnoDB: Started; log sequence number 0 44243
120914 16:30:00 [ERROR] /usr/libexec/mysqld: Error writing file '/var/run/mysqld/mysqld.pid' (Errcode: 28)
120914 16:30:00 [ERROR] Can't start server: can't create PID file: No space left on device
120914 16:30:00 mysqld_safe Number of processes running now: 0
120914 16:30:00 mysqld_safe mysqld restarted
120914 16:30:00  InnoDB: Initializing buffer pool, size = 8.0M
120914 16:30:00  InnoDB: Completed initialization of buffer pool
120914 16:30:00  InnoDB: Started; log sequence number 0 44243
120914 16:30:00 [ERROR] /usr/libexec/mysqld: Error writing file '/var/run/mysqld/mysqld.pid' (Errcode: 28)
120914 16:30:00 [ERROR] Can't start server: can't create PID file: No space left on device
120914 16:30:00 mysqld_safe Number of processes running now: 0
120914 16:30:00 mysqld_safe mysqld restarted
120914 16:30:01  InnoDB: Initializing buffer pool, size = 8.0M
120914 16:30:01  InnoDB: Completed initialization of buffer pool
120914 16:30:01  InnoDB: Started; log sequence number 0 44243
120914 16:30:01 [ERROR] /usr/libexec/mysqld: Error writing file '/var/run/mysqld/mysqld.pid' (Errcode: 28)
120914 16:30:01 [ERROR] Can't start server: can't create PID file: No space left on device
120914 16:30:01 mysqld_safe Number of processes running now: 0
120914 16:30:01 mysqld_safe mysqld restarted
120914 16:30:01  InnoDB: Initializing buffer pool, size = 8.0M
120914 16:30:01  InnoDB: Completed initialization of buffer pool
120914 16:30:01  InnoDB: Started; log sequence number 0 44243
120914 16:30:01 [ERROR] /usr/libexec/mysqld: Error writing file '/var/run/mysqld/mysqld.pid' (Errcode: 28)
120914 16:30:01 [ERROR] Can't start server: can't create PID file: No space left on device
120914 16:30:01 mysqld_safe Number of processes running now: 0
120914 16:30:01 mysqld_safe mysqld restarted
120914 16:30:02  InnoDB: Initializing buffer pool, size = 8.0M
120914 16:30:02  InnoDB: Completed initialization of buffer pool
120914 16:30:02  InnoDB: Started; log sequence number 0 44243
120914 16:30:02 [ERROR] /usr/libexec/mysqld: Error writing file '/var/run/mysqld/mysqld.pid' (Errcode: 28)
120914 16:30:02 [ERROR] Can't start server: can't create PID file: No space left on device
120914 16:30:02 mysqld_safe Number of processes running now: 0
120914 16:30:02 mysqld_safe mysqld restarted
120914 16:30:02  InnoDB: Initializing buffer pool, size = 8.0M
120914 16:30:02  InnoDB: Completed initialization of buffer pool
120914 16:30:02  InnoDB: Started; log sequence number 0 44243
120914 16:30:02 [ERROR] /usr/libexec/mysqld: Error writing file '/var/run/mysqld/mysqld.pid' (Errcode: 28)
120914 16:30:02 [ERROR] Can't start server: can't create PID file: No space left on device
120914 16:30:02 mysqld_safe Number of processes running now: 0
120914 16:30:02 mysqld_safe mysqld restarted
120914 16:30:03  Inn120914 16:30:03 mysqld_safe Number of processes running now: 0
120914 16:30:03 mysqld_safe mysqld restarted
120914 16:30:03  InnoDB: Initializing buffer pool, size = 8.0M
120914 16:30:03  InnoDB: Completed initialization of buffer pool
120914 16:30:03  InnoDB: Started; log sequence number 0 44243
120914 16:30:03 [ERROR] /usr/libexec/mysqld: Error writing file '/var/run/mysqld/mysqld.pid' (Errcode: 28)
120914 16:30:03 [ERROR] Can't start server: can't create PID file: No space left on device
120914 16:30:03 mysqld_safe Number of processes running now: 0
120914 16:30:03 mysqld_safe mysqld restarted
120914 16:30:04  InnoDB: Initializing buffer pool, size = 8.0M
120914 16:30:04  InnoDB: Completed initialization of buffer pool
120914 16:30:04  InnoDB: Started; log sequence number 0 44243
120914 16:30:04 [ERROR] /usr/libexec/mysqld: Error writing file '/var/run/mysqld/mysqld.pid' (Errcode: 28)
120914 16:30:04 [ERROR] Can't start server: can't create PID file: No space left on device
120914 16:30:04 mysqld_safe Number of processes running now: 0
120914 16:30:04 mysqld_safe mysqld restarted
120914 16:30:04  InnoDB: Initializing buffer pool, size = 8.0M
120914 16:30:04  InnoDB: Completed initialization of buffer pool
120914 16:30:04  InnoDB: Started; log sequence number 0 44243
120914 16:30:04 [ERROR] /usr/libexec/mysqld: Error writing file '/var/run/mysqld/mysqld.pid' (Errcode: 28)
120914 16:30:04 [ERROR] Can't start server: can't create PID file: No space left on device
120914 16:30:04 mysqld_safe Number of processes running now: 0
120914 16:30:04 mysqld_safe mysqld restarted
120914 16:30:04  InnoDB: Initializing buffer pool, size = 8.0M
120914 16:30:04  InnoDB: Completed initialization of buffer pool
120914 16:30:05  InnoDB: Started; log sequence number 0 44243
120914 16:30:05 [ERROR] /usr/libexec/mysqld: Error writing file '/var/run/mysqld/mysqld.pid' (Errcode: 28)
120914 16:30:05 [ERROR] Can't start server: can't create PID file: No space left on device
120914 16:30:05 mysqld_safe Number of processes running now: 0
120914 16:30:05 mysqld_safe mysqld restarted
120914 16:30:05  InnoDB: Initializing buffer pool, size = 8.0M
120914 16:30:05  InnoDB: Completed initialization of buffer pool
120914 16:30:05  InnoDB: Started; log sequence number 0 44243
120914 16:30:05 [ERROR] /usr/libexec/mysqld: Error writing file '/var/run/mysqld/mysqld.pid' (Errcode: 28)
120914 16:30:05 [ERROR] Can't start server: can't create PID file: No space left on device
120914 16:30:05 mysqld_safe Number of processes running now: 0
120914 16:30:05 mysqld_safe mysqld restarted
120914 16:30:05  InnoDB: Initializing buffer pool, size = 8.0M
120914 16:30:05  InnoDB: Completed initialization of buffer pool
120914 16:30:06  InnoDB: Started; log sequence number 0 44243
120914 16:30:06 [ERROR] /usr/libexec/mysqld: Error writing file '/var/run/mysqld/mysqld.pid' (Errcode: 28)
120914 16:30:06 [ERROR] Can't start server: can't create PID file: No space left on device
120914 16:30:06 mysqld_safe Number of processes running now: 0
120914 16:30:06 mysqld_safe mysqld restarted
120914 16:30:06  InnoDB: Initializing buffer pool, size = 8.0M
120914 16:30:06  InnoDB: Completed initialization of buffer pool
120914 16:30:06  InnoDB: Started; log sequence number 0 44243
120914 16:30:06 [ERROR] /usr/libexec/mysqld: Error writing file '/var/run/mysqld/mysqld.pid' (Errcode: 28)
120914 16:30:06 [ERROR] Can't start server: can't create PID file: No space left on device
120914 16:30:06 mysqld_safe Number of processes running now: 0
120914 16:30:06 mysqld_safe mysqld restarted
120914 16:30:06  InnoDB: Initializing buffer pool, size = 8.0M
120914 16:30:06  InnoDB: Completed initialization of buffer pool
120914 16:30:06  InnoDB: Started; log sequence number 0 44243
120914 16:30:06 [ERROR] /usr/libexec/mysqld: Error writing file '/var/run/mysqld/mysqld.pid' (Errcode: 28)
120914 16:30:06 [ERROR] Can't start server: can't create PID file: No space left on device
120914 16:30:06 mysqld_safe Number of processes running now: 0
120914 16:30:06 mysqld_safe mysqld restarted
120914 16:30:07  Inn120914 16:30:07 mysqld_safe Number of processes running now: 0
120914 16:30:07 mysqld_safe mysqld restarted
120914 16:30:08  InnoDB: Initializing buffer pool, size = 8.0M
120914 16:30:08  InnoDB: Completed initialization of buffer pool
120914 16:30:08  InnoDB: Started; log sequence number 0 44243
120914 16:30:08 [ERROR] /usr/libexec/mysqld: Error writing file '/var/run/mysqld/mysqld.pid' (Errcode: 28)
120914 16:30:08 [ERROR] Can't start server: can't create PID file: No space left on device
120914 16:30:08 mysqld_safe Number of processes running now: 0
120914 16:30:08 mysqld_safe mysqld restarted
120914 16:30:08  InnoDB: Initializing buffer pool, size = 8.0M
120914 16:30:08  InnoDB: Completed initialization of buffer pool
120914 16:30:09  InnoDB: Started; log sequence number 0 44243
120914 16:30:09 [Note] Event Scheduler: Loaded 0 events
120914 16:30:09 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.1.61'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  Source distribution
120914 16:47:22 [ERROR] /usr/libexec/mysqld: Table './zm/Config' is marked as crashed and should be repaired
120914 16:47:22 [ERROR] /usr/libexec/mysqld: Table './zm/Config' is marked as crashed and should be repaired
120914 16:47:46 [Note] /usr/libexec/mysqld: Normal shutdown

120914 16:47:46 [Note] Event Scheduler: Purging the queue. 0 events
120914 16:47:46  InnoDB: Starting shutdown...
120914 16:47:50  InnoDB: Shutdown completed; log sequence number 0 44243
120914 16:47:50 [Note] /usr/libexec/mysqld: Shutdown complete

120914 16:47:50 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
120915 11:23:07 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
120915 11:23:07  InnoDB: Initializing buffer pool, size = 8.0M
120915 11:23:07  InnoDB: Completed initialization of buffer pool
120915 11:23:07  InnoDB: Started; log sequence number 0 44243
120915 11:23:07 [Note] Event Scheduler: Loaded 0 events
120915 11:23:07 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.1.61'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  Source distribution

Any help would be greatly appreciated.