Backups

[mikb]

I ended up putting this script
in etc/cron.daily. I then ran the script in this directory....and I believe it deleted all my daily.crons... I thought my directory had about a dozen things in it...but after I believed it deleted all because older than 30 days..

"${FIND} ${BKUP_PATH} -maxdepth 1 -type f -mtime +30 -delete"

I suggest you got very lucky there. ONLY losing the contents of cron.daily

I suspect you didn't set "BKUP_PATH" correctly in the script, so "find" ran against the current directory (/etc/cron/daily) and as you surmise, blew away everything older than 30 days. Possibly "BKUP_PATH" was blank (not set).

I don't see a problem putting this script in cron.daily -- I too put general scripts I've added in /usr/local/bin (or /usr/local/sbinfor system stuff) -- but I don't think the location of the script is the problem. You may need to post your actual script, in full, for someone to comment on the exact tiny-but-significant error!

[dazed]

I got it added to my crontab and is making backups
....but its like only 600 bytes...see above....is thats what it supposed to look like? (see in my last post)

Right now I have over 1Tb data and about 5700 events?
I understand its just the mysql, but user zmjm15 above said his file is like 9mb...

[dazed]

mikb I did set the backup path to my cron.daily folder (whereis was used for my path)
....i sinced moved it deeper...but the contents of the cron.daily folder had been erased.

this is the only thing I changed in the script besides mysql user and pass

Code: Select all

# Specify where the databases will be saved to
BKUP_PATH="/path/to/your/backup/folder"

I suggest you got very lucky there. ONLY losing the contents of cron.daily

Yes but what is gone?...thats is the real question...how long will my server limp along without the jobs that where in there?
All i had installed was ZM 1.28.1 on 12.04lts headless....wondering if I upgrade(not fresh) to 14.04lts if it will put lost jobs back.

[dazed]

My script set to run nightly....thanks for looking.

Do I need to adjust this line....write now the compressed file are landing in the path I specified for last 3 days.
in fact thats what there named with the time stamp so Im guessing thats ok.

${BKUP_PATH}/Nightly_dB_Backup_${timestamp}.sql.gz

Code: Select all

#!/bin/bash

# Path to the tools this scripts needs
MYSQLDUMP="/usr/bin/mysqldump"
GZIP="/bin/gzip"
FIND="/usr/bin/find"

# Check to see if this script has access to all the commands it needs
for CMD in $MYSQLDUMP $GZIP $FIND; do
  type $CMD &> /dev/null

  if [ $? -ne 0 ]; then
    echo
    echo "ERROR: The script cannot find the required command \"${CMD}\"."
    echo
    exit 1
  fi
done

# Enter SQL database login credentials
USER=""
PASS=""

# Specify which databases to backup
DB="--all-databases"

# Specify where the databases will be saved to
BKUP_PATH="/etc/cron.daily/mysql-database-backup"

# Format for the timestamp
timestamp="$(date +%Y-%m-%d_%a)"

# Run the backup and compress it
error=0
${MYSQLDUMP} -u${USER} -p${PASS} --events --single-transaction --quick ${DB} | ${GZIP} -9 > ${BKUP_PATH}/Nightly_dB_Backup_${timestamp}.sql.gz

if [ "$?" -ne 0 ]; then
   echo
   echo "WARNING: Mysqldump returned an error!"
   echo
   error=1
fi

# Now find & delete old backups we no longer need
${FIND} ${BKUP_PATH} -maxdepth 1 -type f -mtime +30 -delete

if [ "$?" -ne 0 ]; then
   echo
   echo "WARNING: Find returned an error!"
   echo
   error=1
fi

if [ "$error" -eq 0 ]; then
   echo "Mysql backup script completed successfully!"
fi

[mikb]

mikb I did set the backup path to my cron.daily folder

OK so that was a bad move. It says "path to your backups". Your backups are not -- or really really should not be -- stored in /etc/cron.daily. I guess you've realised that now

Anything in /etc/cron.daily is considered a script/binary to run, once a day, probably late at night.

Yes but what is gone?...thats is the real question...how long will my server limp along without the jobs that where in there?

No idea. If you have a system backup, go and look at it and put back what's missing. If you don't have a backup -- a reinstall seems heavy handed but obvious solution.

On my systems: "logrotate", "slocate", "certwatch" scripts live in there to rotate system logs/compress old logs, update the "locate" db, and to watch for expired certificates. But I'm running Slackware, so your system may have more or less than this.

[dazed]

So Its ok to have the script in cron.daily...just not a place fort the backups to reside
(as of right now I have the backups 1 deeper below the cron.daily).

So looking at the backup I posted last week...is that what is supposed to look like...why are others 8mb while mine is like 500 bytes?

here is todays backup...is this the actual mysql database backup? that I can use to restore?

Code: Select all

-- MySQL dump 10.13  Distrib 5.5.47, for debian-linux-gnu (x86_64)
--
-- Host: localhost    Database: 
-- ------------------------------------------------------
-- Server version	5.5.47-0ubuntu0.12.04.1

/*!40101 SET @OLD_CHARACTER_SET_CLIENT=@@CHARACTER_SET_CLIENT */;
/*!40101 SET @OLD_CHARACTER_SET_RESULTS=@@CHARACTER_SET_RESULTS */;
/*!40101 SET @OLD_COLLATION_CONNECTION=@@COLLATION_CONNECTION */;
/*!40101 SET NAMES utf8 */;
/*!40103 SET @OLD_TIME_ZONE=@@TIME_ZONE */;
/*!40103 SET TIME_ZONE='+00:00' */;
/*!40014 SET @OLD_UNIQUE_CHECKS=@@UNIQUE_CHECKS, UNIQUE_CHECKS=0 */;
/*!40014 SET @OLD_FOREIGN_KEY_CHECKS=@@FOREIGN_KEY_CHECKS, FOREIGN_KEY_CHECKS=0 */;
/*!40101 SET @OLD_SQL_MODE=@@SQL_MODE, SQL_MODE='NO_AUTO_VALUE_ON_ZERO' */;
/*!40111 SET @OLD_SQL_NOTES=@@SQL_NOTES, SQL_NOTES=0 */;

--
-- Current Database: `test`
--

CREATE DATABASE /*!32312 IF NOT EXISTS*/ `test` /*!40100 DEFAULT CHARACTER SET latin1 */;

USE `test`;

--
-- Dumping events for database 'test'
--
/*!40103 SET TIME_ZONE=@OLD_TIME_ZONE */;

/*!40101 SET SQL_MODE=@OLD_SQL_MODE */;
/*!40014 SET FOREIGN_KEY_CHECKS=@OLD_FOREIGN_KEY_CHECKS */;
/*!40014 SET UNIQUE_CHECKS=@OLD_UNIQUE_CHECKS */;
/*!40101 SET CHARACTER_SET_CLIENT=@OLD_CHARACTER_SET_CLIENT */;
/*!40101 SET CHARACTER_SET_RESULTS=@OLD_CHARACTER_SET_RESULTS */;
/*!40101 SET COLLATION_CONNECTION=@OLD_COLLATION_CONNECTION */;
/*!40111 SET SQL_NOTES=@OLD_SQL_NOTES */;

-- Dump completed on 2016-03-11  3:00:01

[dazed]

Got backup working...now to test it

[mikb]

So Its ok to have the script in cron.daily...just not a place fort the backups to reside
(as of right now I have the backups 1 deeper below the cron.daily).

With *nix systems, you are free to ignore all conventions and do whatever you like, as long as you a) understand the consequences, and b) take full responsibility for any mistakes. Or you can follow some conventions, and there are many to choose from

With that in mind, and not wanting to lecture you For the love of all that is holy, do not start using subdirectories under /etc/cron.daily for storing your backups. Keep your data somewhere sane. Your CCTV backups have nothing to do with cron and its daily doings. It's fine having the script in there, but nothing more.

Pick a location somewhere else. Where? Depends on what disk space you have, and how big your backups grow. Maybe under /usr/local/[somewhere], or /home/zoneminder/ [somewhere] or maybe a /mnt/mybigfatdatadisk/ that you created and use for storing ... "stuff". Or even an external NAS drive if you have one.

It might be better to be on a physical disk that is NOT the one zoneminder is on -- otherwise loss of the disk will mean loss of Zoneminder and the backup. That would be bad.

[dazed]

Thanx Mikb, I gonna work on that now....its just so confusing so many directories to choose from...some (to me) seem redundant....I have to

figure out I guess what bin, etc, usr, home(i think i know that one), plus many more mean. Im guessing /home is probably best, since I believe

that is the directory you backup and continue to use moving forward with new distro's(I could be totaly wrong also ).



Right now running off 3tb HDD...gonna have to think about another as you suggest as backup...once I can figure that out...much more

challenging thru terminal since I have no GUI...I was thinking about tackling webmin to see if that would simplify things for me...but from

what I gather I have to make sure Its locked down good...at least I figured out SSH with keyfile thru terminal and ftp so far...and this, Thanks!

[mikb]

Good place to start is here :-

https://en.wikipedia.org/wiki/Filesyste ... y_Standard

... although every linux distribution will vary, here and there ...

[dazed]

Thanx...I had a thought about the script deleting all the files in my cron.daily....If I dont see any other jobs in crontab -l does that
mean that I wont be needing those files that the script deleted in my cron.daily folder?...Which means that I wasnt using them anyway, or
do other programs call those jobs in cron.daily?

Thanks...it appears the script is backing up consitantly now for about 5 days, I did make a backup folder in my /home and thats were im keeping
them for now until I get another drive hooked up. Thanks for the link...also found some books to read as well.

[mikb]

You may need to look up some details on how cron works too Welcome to Linux ... an endless maze of stuff to learn!

Quick guide -- again this is Slackware, but ... "crond" (the bit that does the work) reads the instructions in /var/spool/cron/crontab/[user] and fires off various jobs accordingly. Different job lists for different users.

What you are seeing with "crontab -l" will depend on which user you are. I don't know who you are logged in as when you saw no jobs, but if you are ROOT, it will show some stuff, for certain.

Now, in the list of jobs for root is this sort of thing :-

Code: Select all

# Run hourly cron jobs at 47 minutes after the hour:
47 * * * * /usr/bin/run-parts /etc/cron.hourly 1> /dev/null
#
# Run daily cron jobs at 4:40 every day:
40 4 * * * /usr/bin/run-parts /etc/cron.daily 1> /dev/null
#
# Run weekly cron jobs at 4:30 on the first day of the week:
30 4 * * 0 /usr/bin/run-parts /etc/cron.weekly 1> /dev/null
#
# Run monthly cron jobs at 4:20 on the first day of the month:
20 4 1 * * /usr/bin/run-parts /etc/cron.monthly 1> /dev/null

The second line of that will go through everything in /etc/cron.daily and execute what it finds there in turn.

If you put your backup script in there, it will get run daily, at 4:40, by root. So be very careful not to have an accident, as root will allow you to shoot yourself in the foot so hard ... far more than you already have! As in "whoops, I deleted everything from / downward ..." "including my network mounted backups!" (!)

Your other content in /etc/cron.daily that you (may have) deleted is unknown to me. If you have backups, look at them to see what was wiped. otherwise you may need specific advice from someone who has the same o/s version as you. If you have another machine with this o/s on, look there, it's likely to be the same.

[mikb]

Looking back, you say your OS is Ubuntu 12.04LTS headless -- anyone care to peer into /etc/cron.daily and say what's lurking there?

[dazed]

Looking back, you say your OS is Ubuntu 12.04LTS headless -- anyone care to peer into /etc/cron.daily and say what's lurking there?

Well I finally upgraded thru OS to 14.04 today.

I can tell you whats in there now, since it offered replace my current one...

Well I still have my cronjob and backups since i moved the job and saved files to under /home directory.

Took several hours but I made it to 14.04LTS and 1.30.0 with much help from google and the content providers.

Code: Select all

ls -a cron.daily
.  ..  apache2  apport  apt  bsdmainutils  man-db  mlocate  ntpdate  upstart

[mikb]

Upgrading your OS will have, hopefully, repopulated /etc/cron.daily with the important stuff. You've still lost any tweaks and personalisations you did in there, if any, if you can't get them from backups. But if all seems to now be working, then that's ok -- and hope not to blow away important stuff in future