Yahoo Answers is shutting down on May 4th, 2021 (Eastern Time) and beginning April 20th, 2021 (Eastern Time) the Yahoo Answers website will be in read-only mode. There will be no changes to other Yahoo properties or services, or your Yahoo account. You can find more information about the Yahoo Answers shutdown and how to download your data on this help page.
Trending News
How do coursework.info allow Google to spider all of their content?
Currently there are two ways that I know of to allow Google to spider password protected content.
1) It can be done using 'first click free'
2) A cloaking solution - simple script that says if you are Googlebot (or another search engine), or you're logged in with a subscription, you can see everything. If not, you can't. Very simple. But also cloaking by Google's definition.
Coursework.info allow Google to spider everything somehow, but how are they doing it? I don't think they are using either of the above solutions.
Do you know of any other ways?
2 Answers
- JakeLv 710 years agoFavorite Answer
SEOmoz commented on the topic, suggesting you provide a meta description snippet for the search engine preview, their comment regarding allowing spider access:
It is possible to provide the full content to spiders (based on the user agent), but there are two risk: (1) It can be seen as cloaking and result in penalties, and (2) As soon as they spiders can reach the content, they'll index the links and make them publicly available - if you make the content available to spiders, you may end up bypassing the login structure. If the login is dynamically driven, it might work (you could just grant the spiders access), but then visitors may get frustrated when they go straight from search to a registration/login form. In that case, I'd also recommend some kind of snippet, just to let them know they're on the right track.
- Anonymous10 years ago
what makes you think they aren’t using first click free?
