Page 1 of 1

Google spiders are collecting all my images ....

Posted: Thu Feb 15, 2007 1:59 pm
by occhiostanco
... and maybe your events too.

I started seeing heavy load of my flat adsl router for hours!
I investigated the apache log files and found that google bots were scanning all my zm events and indexing all pictures!

Since zm needs they be in the web space (or linked from the web space)
this may be a relevant side effect.

Zm is a great sowftware and I think it could be better configured to leave events be stored outside the web space.

As a workaround I enabled (and suggest to) apache authentication (.htaccess files) for /var/www/zm

Thank You
Roberto

Posted: Thu Feb 15, 2007 2:09 pm
by lazyleopard
Another option is to add a robots.txt file to your web root directory. A simple "go away" one looks like this:

Code: Select all

User-agent: *
Disallow: /

Posted: Thu Feb 15, 2007 2:16 pm
by jameswilson
but there must have been a link to you images folder to get it to start spidering or not?

Posted: Thu Feb 15, 2007 8:11 pm
by cordel
Options to prevent this but only with search engines that obey the rules.
First is the robots file like lazyleopard mentioned which is the easiest.
Also you should set apache to not index your folders, this will help even more.
Here is an example, this is what I have my rpms set by default when they are installed. If you compare this to your apache config you will see that following options "index" which means to index your folders to allow browsing.

Code: Select all

Alias /zm "/usr/lib/zm/html/"
<Directory "/usr/lib/zm/html">
    Options MultiViews FollowSymLinks
    AllowOverride All
    Order allow,deny
    Allow from all
</Directory>
Just remove the index from your apache config or you can add the entry above if you adjust the paths accordingly. Make sure you make a backup of your original config before making changes.

Posted: Fri Feb 16, 2007 1:07 pm
by occhiostanco
> but there must have been a link to you images folder to get it to start
> spidering or not?[/quote]
Yes, zm needs a link in the web space to an external (to the web space)
event directory.

Posted: Fri Feb 16, 2007 1:15 pm
by jameswilson
do you have auth off?