Search engine crawling errors

sreerag

Joined: 2010-01-17
Posts: 11
Posted: Sun, 2012-08-19 15:02

Google search engine crawling unwanted url in my site and creating large number of server error and 404 error examples of such url's are
http://cinespot.net/gallery/main.php?g2_view=rss.SimpleRender&g2_itemId=11058
http://cinespot.net/gallery/main.php?g2_view=panorama.Panorama&g2_itemId=86568
http://cinespot.net/gallery/main.php?g2_view=webdav.WebDavMount&g2_itemId=359185
http://cinespot.net/gallery/main.php?g2_view=panorama.Panorama&g2_itemId=49562

How to block ?g2_view=rss.SimpleRender , ?g2_view=panorama.Panorama&g2_itemId=,?g2_view=webdav.WebDavMount&g2_itemId=,?g2_view=panorama.Panorama&g2_itemId= crawling by search engine ???
 
Dayo

Joined: 2005-11-04
Posts: 1642
Posted: Sun, 2012-08-19 15:10

Put them into your robots.txt.

Here is a selection from mine

User-Agent: *
Disallow: *g2_view=ecard.SendEcard*
Disallow: *g2_view=rss.SimpleRender*
Disallow: *g2_view=core.UserAdmin*
Disallow: *g2_view=comment*
Disallow: /gallery2/popular*
Disallow: *slideshow.html*
Disallow: *g2_controller*
Disallow: *admin*
Disallow: *key/*

--
dakanji.com
 
sreerag

Joined: 2010-01-17
Posts: 11
Posted: Sun, 2012-08-19 15:19

Thanks a lot sir , Which robots txt file i need to update ??

I have three robots.txt file , one is in the root directory ((home) and rest 2 are public_html

http://cinespot.net/robots.txt
http://cinespot.net/gallery/robots.txt
 
Dayo

Joined: 2005-11-04
Posts: 1642
Posted: Sun, 2012-08-19 15:54

public_html
--
dakanji.com
 
smartblogger

Joined: 2013-08-07
Posts: 1
Posted: Wed, 2013-08-07 13:08

Is there any way instead of blocking from robots.txt