Alternative Of Robots Meta Directives What are robot meta labels?

Robots meta mandates (some of the time called “meta labels”) are bits of code that give crawlers directions to how to slither or file site page content. Though robots.txt record orders give bots ideas for how to slither a site’s pages, robot meta orders give all the more firm guidelines on the best way to creep and list a page’s substance. robots.txt generator.

There are two sorts of robot meta mandates: those that are important for the HTML page (like the meta robots tag) and those that the web worker sends as HTTP headers, (for example, x-robots-tag). Similar boundaries (i.e., the creeping or ordering guidelines a meta tag gives, for example, “no index” and “no follow” in the model above) can be utilized with both meta robots and the x-robots-tag; what contrasts is the manner by which those boundaries are imparted to crawlers.

Meta mandates give crawlers directions about how to slither and file data they find on a particular site page. In the event that these mandates are found by bots, their boundaries fill in as solid ideas for crawler indexation conduct. Yet, as with robots.txt documents, crawlers don’t need to follow your meta orders, so it’s almost certain that some malignant web robots will disregard your mandates.

The following are the boundaries that internet searcher crawlers comprehend and follow when they’re utilized in robots meta orders. The boundaries are not case-delicate, yet note that it is conceivable some web search tools may just follow a subset of these boundaries or may treat a few orders somewhat in an unexpected way.

Indexation-controlling boundaries:

Noindex: Tells an internet searcher not to file a page.

File: Tells an internet searcher to list a page. Note that you don’t have to add this meta tag; it’s the default.

Follow: Even if the page isn’t ordered, the crawler ought to follow every one of the connections on a page and pass value to the connected pages.

Nofollow: Tells a crawler not to follow any connections on a page or pass along any connection value.

Noimageindex: Tells a crawler not to list any pictures on a page.

Note: Equivalent to utilizing both the index and nofollow labels at the same time.

No archive: Search motors ought not to show a reserved connection to this page on a SERP.

Nocache: Same as noarchive, however just utilized by Internet Explorer and Firefox.

Nosnippet: Tells a web index not to show a scrap of this page (for example meta depiction) of this page on a SERP.

Nobody/noydir [OBSOLETE]: Prevents web search tools from utilizing a page’s DMOZ portrayal as the SERP scrap for this page. In any case, DMOZ was resigned in mid-2017, making this tag out of date.

Unavailable_after: Search motors should presently don’t list this page after a specific date.

Sorts of robots meta mandates

There are two primary sorts of robot meta mandates: the meta robots tag and the x-robots-tag. Any boundary that can be utilized in a meta robots tag can likewise be determined in an x-robots-tag.

We’ll discuss both the meta robots and x-robots label mandates underneath. robots.txt disallow all..

Meta robots tag

The meta robots tag, normally known as “meta robots” or conversationally as a “robots tag,” is essential for a website page’s HTML code and shows up as code components inside a page’s <head> area:


Code test:

<pre><meta name=”robots” content=”[PARAMETER]”></pre>

While the general <meta name=”robots” content=”[PARAMETER]”> tag is standard, you can likewise give mandates to explicit crawlers by supplanting the “robots” with the name of a particular client specialist. For instance, to focus on an order explicitly to Googlebot, you’d utilize the accompanying code:

<meta name=”googlebot” content=”[DIRECTIVE]”>

Need to utilize more than one order on a page? However long they’re focused to the equivalent “robot” (client specialist), different orders can be remembered for one meta mandate – simply separate them by commas. Here’s a model:

<meta name=”robots” content=”noimageindex, nofollow, nosnippet”>

This tag would advise robots not to record any of the pictures on a page, follow any of the connections, or show a scrap of the page when it shows up on a SERP.

In case you’re utilizing distinctive meta robots label mandates for various hunt client specialists, you’ll need to utilize separate labels for every bot.


While the meta robots tag permits you to control ordering conduct at the page level, the x-robots-tag can be incorporated as a component of the HTTP header to control the ordering of a page all in all, just as unmistakable components of a page. Meta robots.

While you can utilize the x-robots-tag to execute the entirety of similar indexation mandates as meta robots, the x-robots-label order offers essentially greater adaptability and usefulness than the meta robots tag doesn’t. In particular, the x-robots allow the utilization of normal articulations, executing slither mandates on non-HTML documents, and applying boundaries at a worldwide level.

To utilize the x-robots-tag, you’ll need to approach either your site’s header .php, .htaccess, or worker access record. From that point, add your particular worker design’s x-robots-label markup, including any boundaries. This article gives some extraordinary instances of what x-robots-label markup resembles in case you’re utilizing any of these three arrangements.

Here are a couple of utilization cases for why you may utilize the x-robots-tag:

Controlling the indexation of substance not written in HTML (like blaze or video)

Hindering indexation of a specific component of a page (like a picture or video), yet not of the whole page itself

Controlling indexation in the event that you don’t approach a page’s HTML (explicitly, to the <head> segment) or if your site utilizes a worldwide header that can’t be changed

Adding rules to whether a page ought to be recorded (ex. On the off chance that a client has remarked more than 20 times, file their profile page)

Search engine optimization best practices with robots meta mandates

All meta mandates (robots or something else) are found when a URL creeps. This implies that if a robots.txt record refuses the URL from creeping, any meta mandate on a page (either in the HTML or the HTTP header) won’t be seen and will, viably, be overlooked.

Much of the time, utilizing a meta robots tag with boundaries “no index, follow” ought to be utilized as an approach to confine creeping or indexation as opposed to utilizing robots.txt document refuses.

It is imperative to take note of that vindictive crawlers are probably going to totally overlook meta mandates and accordingly, this convention doesn’t make a decent security instrument. In the event that you have private data that you would prefer not to make openly accessible, pick a safer methodology, like secret key insurance, to hold guests back from survey classified pages.

You don’t have to utilize both meta robots and the x-robots-tag in total agreement – doing so would be repetitive.