{"id":84,"date":"2018-11-28T14:03:12","date_gmt":"2018-11-28T14:03:12","guid":{"rendered":"http:\/\/rosie.fourfrontdev.us\/tasks\/?p=84"},"modified":"2018-11-28T14:03:12","modified_gmt":"2018-11-28T14:03:12","slug":"robots-txt","status":"publish","type":"post","link":"http:\/\/rosie.fourfrontdev.us\/tasks\/robots-txt\/","title":{"rendered":"Robots txt"},"content":{"rendered":"<h2>The Robots.txt<\/h2>\n<pre class=\" language-text\">User-agent: *\r\nDisallow: \/<\/pre>\n<p>The above directive prevents the search engines from indexing any pages or files on the website. Say, however, that you simply want to keep search engines out of the folder that contains your administrative control panel. You&#8217;d code:<\/p>\n<pre class=\" language-text\">User-agent: *\r\nDisallow: \/administration\/<\/pre>\n<p>Or if you wanted to allow in all spiders except Google&#8217;s GoogleBot, you&#8217;d code:<\/p>\n<pre class=\" language-text\">User-Agent: googlebot\r\nDisallow: \/<\/pre>\n","protected":false},"excerpt":{"rendered":"<p>The Robots.txt User-agent: * Disallow: \/ The above directive prevents the search engines from indexing any pages or files on the website. Say, however, that you simply want to keep search engines out of the folder that contains your administrative control panel. You&#8217;d code: User-agent: * Disallow: \/administration\/ Or if you wanted to allow in &hellip; <a href=\"http:\/\/rosie.fourfrontdev.us\/tasks\/robots-txt\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Robots txt&#8221;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[21],"tags":[],"class_list":["post-84","post","type-post","status-publish","format-standard","hentry","category-robots-txt"],"_links":{"self":[{"href":"http:\/\/rosie.fourfrontdev.us\/tasks\/wp-json\/wp\/v2\/posts\/84","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/rosie.fourfrontdev.us\/tasks\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/rosie.fourfrontdev.us\/tasks\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/rosie.fourfrontdev.us\/tasks\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/rosie.fourfrontdev.us\/tasks\/wp-json\/wp\/v2\/comments?post=84"}],"version-history":[{"count":1,"href":"http:\/\/rosie.fourfrontdev.us\/tasks\/wp-json\/wp\/v2\/posts\/84\/revisions"}],"predecessor-version":[{"id":85,"href":"http:\/\/rosie.fourfrontdev.us\/tasks\/wp-json\/wp\/v2\/posts\/84\/revisions\/85"}],"wp:attachment":[{"href":"http:\/\/rosie.fourfrontdev.us\/tasks\/wp-json\/wp\/v2\/media?parent=84"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/rosie.fourfrontdev.us\/tasks\/wp-json\/wp\/v2\/categories?post=84"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/rosie.fourfrontdev.us\/tasks\/wp-json\/wp\/v2\/tags?post=84"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}