Saturday , January 20 2018
Home / blogger / How to set custom robots txt in bloggeer

How to set custom robots txt in bloggeer

How to set custom robots txt in bloggeer. Hello friends today you are going to read a tutorial on what is robots txt and How to set custom robots txt in bloggeer. Robots txt is very important thing for any site.

What is robots txt

Web site owners use the robots txt file to give instructions about their site to web robots; this is called The Robots Exclusion Protocol.
How it works custom robots txt?
It works like a robot wants to vists a Web site URL  say http:// Before it does so, it firsts checks for, and finds

User-agent: *

Disallow: /

The User-agent: * means this section applies to all robots. The Disallow: /
tells the robot that it should not visit any pages on the site.

There are two important considerations when using /robots.txt:

robots can ignore your /robots.txt. Especially malware robots that scan
the web for security vulnerabilities,
and email address harvesters used by spammers will pay no attention.
the /robots.txt file is a publicly available file.
Anyone can see what sections of your server you don’t want robots to use.

if you want To allow all robots to complete access

then you can use

User-agent: *


or just create an empty robots.txt file

To exclude all robots from the entire server

User-agent: *

Disallow: /

To exclude a single robot.

use this robots.txt

User-agent: BadBot

Disallow: /

if want to allow just one robot to access your site
use below txt in this robots.txt i allow google to access my site

User-agent: Google


User-agent: *

Disallow: /

i hope now you understand about robots.txt if not then don’t worry
i am here to teach you.
you can ask me in comments.

How to set custom robots txt in blogger?

Login to Click on blog title in which blog you want to set
custom robots txt now click on settings.

Change Blogger URL

Search petferences

custom robots txt

Click edit custom robots txt like bellow  screen shoot.

custom robots txt
custom robots txt

check yes to  enable robots.txt

when you check yes you will see text box just past your robots.txt here

and click save changes.

custom robots txt
custom robots txt

here is robots.txt that you can use as your robots.txt

User-agent: Mediapartners-Google


User-agent: *

Disallow: /search

Allow: /


Check Also

Auto Post

Blogger Post Auto Post On Your Google puls

Today i will tell you how to Blogger Post Auto Post On Your Google puls. …

Leave a Reply

Your email address will not be published. Required fields are marked *