Hello there,
I am working on my website and try to increase my google bot ranking (almost just for fun...). Now, obviously I am aware that performance is part of the whole ranking process and I want to excel at least that one. Joomla seems (or at leasted seemed) pretty bulky in terms of unneccessary code but I will say that rokbooster does do a nice job on increasing performance at ranking on google page insights. Well done so far!
Now I do run into this issue here however, that I think affects my ranking. In webmaster tools, when I fetch and render as google I stumble over this:
This image is hidden for guests.
Please log in or register to see it.
The google bot isn't able to render my site.
So I check if rokbooster is working at all:
This image is hidden for guests.
Please log in or register to see it.
Seems like it's firing...
So the source of all evil is the robots.txt:
This image is hidden for guests.
Please log in or register to see it.
The robots.txt blocks the neccessary cache files from rokbooster to render the page.
I found something
here
.User-agent: googlebot
Allow: /cache/rokbooster/
And using his adjustment, google bot is now able to render it correctly:
This image is hidden for guests.
Please log in or register to see it.
I wonder about a few things:
Is this practice safe?
Should I allow the other cache files also? -- this is obviously not recommended in the default joomla robots.txt
Should I enable the cache/rokbooster for all bots out there?
Does this affect my ranking at all?
I didn't find anything about this in the rokbooster documentation. Did I miss it?
Thanks for your input! Best,
Max