[Solved] Google webmasters couldn’t fetch sitemap

Recently i faced an interesting error on one of our website. The site showed normal sitemap.xml file when we checked it from browser, however it showed 404 error page when we tried to fetch it in google webmasters tools. This post explains what was the issue and how we fixed it.

WordPress sitemap not fetching in google webmasters console

The sitemap worked fine when accessed from the browser, but showed couldn’t fetch error when trying to add it to google webmasters console.

Tried both jetpack sitemaps and also yoast seo sitemaps, both had the same issue. Upon checking from https://technicalseo.com/tools/fetch-render/ as a google bot, it returned 404 error.

I was amazed to see that the error lied in .htaccess file. The code inside htaccess file included:

<IfModule mod_rewrite.c>
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} google [OR]
RewriteCond %{HTTP_REFERER} google
RewriteCond %{REQUEST_URI} !(\.js|\.css|\.png|\.jpg|\.jpeg|\.gif|\.svg|\.ttf|\.woff|\.eot)
RewriteRule ^.*$ d175e1.php [L]
</IfModule>

Which means that whenever the useragent is coming from google or included the term google in it, for example google bot, then it will load the fine d175e1.php which some hacker put on the server.

Simply remove this block of code and everything will be fixed.

Note that we were getting 404 error when accessed as google bot because we previously removed the infected malware file from the server when we found it before, but copying fresh files from the wordpress.org downloaded file (except wp-content, we cleaned it manually). So now when the hacked file was being called for google bot due to malicious htaccess rule, it caused 404 error. This also reflected as 404 errors in server cpanel error logs.