Google spiders are collecting all my images ....

Support and queries relating to all previous versions of ZoneMinder
Locked
occhiostanco
Posts: 48
Joined: Tue Mar 21, 2006 1:37 pm

Google spiders are collecting all my images ....

Post by occhiostanco »

... and maybe your events too.

I started seeing heavy load of my flat adsl router for hours!
I investigated the apache log files and found that google bots were scanning all my zm events and indexing all pictures!

Since zm needs they be in the web space (or linked from the web space)
this may be a relevant side effect.

Zm is a great sowftware and I think it could be better configured to leave events be stored outside the web space.

As a workaround I enabled (and suggest to) apache authentication (.htaccess files) for /var/www/zm

Thank You
Roberto
User avatar
lazyleopard
Posts: 403
Joined: Tue Mar 02, 2004 6:12 pm
Location: Gloucestershire, UK

Post by lazyleopard »

Another option is to add a robots.txt file to your web root directory. A simple "go away" one looks like this:

Code: Select all

User-agent: *
Disallow: /
Rick Hewett
jameswilson
Posts: 5111
Joined: Wed Jun 08, 2005 8:07 pm
Location: Midlands UK

Post by jameswilson »

but there must have been a link to you images folder to get it to start spidering or not?
James Wilson

Disclaimer: The above is pure theory and may work on a good day with the wind behind it. etc etc.
http://www.securitywarehouse.co.uk
User avatar
cordel
Posts: 5210
Joined: Fri Mar 05, 2004 4:47 pm
Location: /USA/Washington/Seattle

Post by cordel »

Options to prevent this but only with search engines that obey the rules.
First is the robots file like lazyleopard mentioned which is the easiest.
Also you should set apache to not index your folders, this will help even more.
Here is an example, this is what I have my rpms set by default when they are installed. If you compare this to your apache config you will see that following options "index" which means to index your folders to allow browsing.

Code: Select all

Alias /zm "/usr/lib/zm/html/"
<Directory "/usr/lib/zm/html">
    Options MultiViews FollowSymLinks
    AllowOverride All
    Order allow,deny
    Allow from all
</Directory>
Just remove the index from your apache config or you can add the entry above if you adjust the paths accordingly. Make sure you make a backup of your original config before making changes.
occhiostanco
Posts: 48
Joined: Tue Mar 21, 2006 1:37 pm

Post by occhiostanco »

> but there must have been a link to you images folder to get it to start
> spidering or not?[/quote]
Yes, zm needs a link in the web space to an external (to the web space)
event directory.
jameswilson
Posts: 5111
Joined: Wed Jun 08, 2005 8:07 pm
Location: Midlands UK

Post by jameswilson »

do you have auth off?
James Wilson

Disclaimer: The above is pure theory and may work on a good day with the wind behind it. etc etc.
http://www.securitywarehouse.co.uk
Locked