Yahoo Answers is shutting down on May 4th, 2021 (Eastern Time) and beginning April 20th, 2021 (Eastern Time) the Yahoo Answers website will be in read-only mode. There will be no changes to other Yahoo properties or services, or your Yahoo account. You can find more information about the Yahoo Answers shutdown and how to download your data on this help page.

How to stop the bots from trying?

I run a couple of specialist crossword solving websites. They're both custom PHP throughout, but that doesn't stop nefarious bots from trying to access what would normally be Wordpress or Joomla pages - looking for ways to infect blogs and forums.

I get lots of hits like this: ...../word/index.php?option=com_registration&task=register

I had a little difficulty in dealing with this because the part of the url after /word, is used as parameter to run a query, so in theory anything is valid. For example, if a visitor wanted to know about the word "friend" they'd be directed to /word/friend.

I have changed my php so that if a search term ends in .php, .html, .aspx etc I return a 404 error but the dumb bots just keep trying anyway.

They aren't using up a massive amount of bandwidth, or affecting delivery of my services but they do appear all of the time in my analytics which I use to react to searches and make sure I'm carrying the correct answer.

Is there any more I can do?

Update 2:

Thanks Sandeep.

Blocking IPs doesn't work. They change every time, and each time an IP is blocked I stand more chance of blocking a legitimate user. I just wish the bots were a bit more intelligent so they could realise that my site isn't WP or Joomla and doesn't have registration pages or user comments.

1 Answer

Relevance
  • 8 years ago
    Favorite Answer

    Get IP address of those notorious visitors (bots) and block them via .htaccess on your root. Use deny from. You can do that by using IP deny manager on your control panel. Alternatively you can use .htaccess redirect to send them to a different site, or a different page (maybe homepage) when they access that url.

Still have questions? Get your answers by asking now.