robots.txt as part of install

Nigel_Aves

Joined: 2005-07-11
Posts: 61
Posted: Sun, 2009-09-20 17:20

Simple request to add a robots.txt file to the install.

I've just created one that looks like this.

User-agent: *
Disallow: /application/
Disallow: /installer/
Disallow: /lib/
Disallow: /modules/
Disallow: /system/
Disallow: /themes/
Disallow: /var/logs/
Disallow: /var/modules/
Disallow: /var/resizes/
Disallow: /var/thumbs/
Disallow: /var/tmp/
Disallow: /var/uploads/
 
floridave
floridave's picture

Joined: 2003-12-22
Posts: 22890
Posted: Sun, 2009-09-20 18:07

Please add a feature request for this.
https://sourceforge.net/apps/trac/gallery/newticket

Dave
_____________________________________________
Blog & G2 || floridave - Gallery Team
 
nivekiam
nivekiam's picture

Joined: 2002-12-10
Posts: 16503
Posted: Sun, 2009-09-20 18:57

Just piping in with my opinion.

I hope it doesn't become part of the install. Let webmasters manage their sites themselves :)

Well I guess it wouldn't matter since the vast majority of people put Gallery into a sub-directory and robots.txt is only valid at the root of a website.
____________________________________________
Like Gallery? Like the support? Donate now!!! See G2 live here
 
Nigel_Aves

Joined: 2005-07-11
Posts: 61
Posted: Sun, 2009-09-20 19:08

I'm going to be running it at top level, I know that's strange! But I've also seen some "full service providers" use packages like (and including) Gallery2 as a top level package and people who use sites like that are not really your webmaster type :)

It was only a thought as it's a typical install item with packages like Joomla / Wordpress ....
 
nivekiam
nivekiam's picture

Joined: 2002-12-10
Posts: 16503
Posted: Sun, 2009-09-20 19:28

I've never installed Joomla, but I've never seen this in other programs, including WP, Drupal, and others. Also, those are full blown CMS packages where it could be generally assumed that your entire site is running under that. I've seen people install Gallery at the root of their site, it works fine, you're just in the minority that's all :)

All robots.txt does is keep a few good search engines like Google, Bing/Yahoo from crawling those sections of your site. Honestly, I don't see any need for doing this anyway. There's nothing under any of those directories except for installer that has any content they could hit. All php files except for index.php under G3 (at least everyone I've seen) have <?php defined("SYSPATH") or die("No direct script access."); right at the top. There's nothing to see there from the web, period.
____________________________________________
Like Gallery? Like the support? Donate now!!! See G2 live here