# See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file # # To ban all spiders from the entire site uncomment the next two lines: Sitemap: http://www.akhbarak.net/sitemaps/tags/sitemap_index.xml.gz Sitemap: http://www.akhbarak.net/sitemaps/sections/sitemap_index.xml.gz Sitemap: http://www.akhbarak.net/sitemaps/sources/sitemap_index.xml.gz Sitemap: http://www.akhbarak.net/sitemaps/videos/sitemap_index.xml.gz Sitemap: http://www.akhbarak.net/sitemaps/articles/sitemap_index.xml.gz Sitemap: http://www.akhbarak.net/sitemaps/static/sitemap_index.xml.gz User-Agent: Google Disallow: /admin/ User-agent: Googlebot Disallow: /admin/ User-agent: Googlebot-Image Disallow: /admin/ User-agent: Mediapartners-Google Disallow: /admin/ User-agent: * Disallow: /admin/ Disallow: /articles/limited/ Disallow: /topics/limited/ Disallow: /tags/limited/ Disallow: /clusters/limited/ #User-Agent: * #Disallow: /