Earlier today, a client sent me an email.
“Google my name. The result is all screwed up. What gives?”
So, I did! And, just like he said… the title was totally wrong and the following text showed instead of my custom description:
A description for this result is not available because of this site's robots.txt – learn more.
Here's what I did:
- Tried uploading a new robots.txt; refreshed my browser; it still showed the wrong content
- Called my web host's tech support line and learned it was a WordPress issue
- Messed around with my .htaccess file
- Googled up, down and all around
What actually fixed it:
- Logged into my wp-admin
- Went to Settings, then Privacy
- Saw that “Ask search engines not to index this site.” was set to ON.
- OOPSIE! “Allow search engines to index this site.” Click and Save.
I re-Googled my client's name a few days later, and *whew!* everything was as it should be.
Lovable nerd dedicated to improving peoples' lives. Originally from Canada. Current home base: Hengelo, Netherlands. Visited 30 countries since 2013. [
I am also getting ‘A description for this result is not available because of this site’s robots.txt’ notice when I search my website (www.restoatlas.com) in Google Search. It was already unchecked ‘Discourage search engines from indexing this site’ for Search Engine Visibility in my WordPress Settings. how can i solve this issue?
Looks like it’s fixed now. 🙂
As mentioned, it can take a couple days for the changes to show up on Google.
I am too getting ‘A description for this result is not available because of this site’s robots.txt’ notice when I search my website (http://www.abacoot.com) in Google Search. I unchecked ‘Discourage search engines from indexing this site’ for Search Engine Visibility in my WordPress Settings. I assume all I need to do now is wait?
Yep! It should clear up in a few days. 🙂
Thanks man! This has been bugging me for ages 🙂
Cool beans!
I had a look at your site. Looks great. The headlines are catchy. It’s everything my “travel blog” (not really) should be… 😉
http://www.mrfreedom.ca/
Hi,
Recently i uploaded a robots.txt file in my server. Now my home page and another sub pages are indexed with an error. i.e. A description for this result is not available because of this site’s robots.txt – learn more. When will it clear and when will all pages and sub links Index? Google how much time taking to clear this error..?
Hi X Dude
I have the same issue with my clients site, I now uploaded the robots.txt file again. Meta description also has been added in the header.php . Also installed Yoast Plugin . is there something wrong . This was 2-3 days back . If you can have a look and let me know what can be the problem I would be rally greatful. the website name is http://www.volmopl.com
Looks fine now!
my site is http://www.chat92.com i am new on wordpress i am using word press on my server i got this error A description for this result is not available because of this site’s robots.txt
i open my wordpress 4.3 admin panel in settings no have page ( privacy ) where i need to solve this error i not know coding .
Looks like you fixed it. 🙂
My site have also same Problem. I made my site by WordPress. In my root No robot.txt file. but when I visit http://mysite-url/robot.txt then showing This file. I delete robotext file from cpanel & upload again but Problem was not sloved, But why ?
I have also same Problem. I made my site by WordPress. In my root No robot.txt file. but when I visit http://www.dahagram.com/robot.txt then showing This file. But why ?
http://www.dahagram.com/robots.txt may be generated on your site by a plugin.
Hi Dude, Just i want help from you..Actually my website shows like this in google search engine “A description for this result is not available because of this site’s robots.txt – learn more.” This is Php Website “www.rummywings.com” just check it & reply me back …what the problem is?
Sorry for the delay – it looks like you sorted it out. 🙂
Holy smoke did I learn a lesson on this one. Robots.txt was blocking some images and banner ads that were used on the site through disallow accidentally. This had been working for years with no problem. Changed Robots.txt and pages are coming back.
Question is why now… Google is coming down on sites that have malware and porn that is hidden. Coincidentally there were some hacked pages on another site that were using our code to link back to us after showing their junk. One on these pages had our Adsense links so Google thought that it was”our site”. I think Google saw this and linked it to blocked images and simply said no more blocked images for you.
My theory at this time…
Great! How to solve this issue for blogger? For my blogspot, some of the links show proper search result in Google, but some show this line, “A description for this result is not available because of this site’s robots.txt – learn more.”
Not sure! I’ve never used that system myself.
Is there a “discourage search engines” checkbox somewhere in the admin area? How about for individual posts?
I tried everything listed above. No help, sadly. I went to Settings, but there is NO Privacy option. Suggestions?
Yeah! They changed things in a recent version update.
The “Search Engine Visibility” option now appears near the bottom of the “Reading” tab.
I can’t believe this was happening. I thought my site was banned from Google search or something. But any ways, the new settings location for this option is:
Settings>Reading>Searching Engine Visibility>Search Engine Visibility
Click off the box, and Save.
I was getting an error when I google searched my site. But now hopefully this should fix it. I’ll give it a week. How this setting was set is my question. I would never want to click on this setting, so something else did it.
Most of the time, if someone is taking time to build a site – they want to block search engines from finding it until they’re ready to reveal it. Often times, they forget to un-check that box once they’ve published their site.
Hey, thanks for posting this. A client of mine was showing the same error message in the SERPs. Sure enough, I checked their settings and the “discourage search engines from indexing this site” option was ticked. I think the option is under Settings > Reading now if you’ve updated to WordPress 3.8.1
Hi,
I’m having the same problem, My site was ok, but from when I added file robots.txt to make google index faster, then google blocked my site. I read their advice and deleted my robots.txt file from my host system. Then I didn’t know how long I have to wait to let google index my site, I waited 12 hours later and search my site on google, google still say that “A description for this result is not available because of this site’s robots.txt – learn more.” So I dont know what to do!
Just like you said, maybe the problem isnt my robots.txt, I also check my setting from my wordpress dashboard, I cannot find the privacy tab, I went to settings and check this box: “Search Engine Visibility: Discourage search engines from indexing this site”
Am I did right? What I need to do now? How long I need to wait until google index my site again?
I maybe thought that I should download all my website file from my host, unstall and install again, and then upload all the file I downloaded, should I do that?
I’m sorry, my comment pretty long but I just want you to know the details!
Please help me!
Thank you!
Sorry for the delay! I was moving.
Has it all sorted itself out? Google usually takes a couple days to catch up.
hello, i have the same problem, i deleted and uploaded a new robots text yesterday. now still have the same question. I hope some days later will it will be well as yours.
Did this issue resolve relatively quickly? We are having similar issue
Yep! Their Google listing showed properly four days later.