0
Welcome Guest! Login
0 items Join Now

SOLVED RT Resources still blocked after robot.txt fix

  • SOLVED RT Resources still blocked after robot.txt fix

    Posted 5 years 11 months ago
    • hello, i am running 3.6 joomla and php7

      the site has the myriad template and many RT resources (includind rokbuster) that are beeing blocket by googleboot.

      in my txt text a made this changes, but template, modules css, js are still being blocked as you can see the the attached image. well, i am not a developer and i ask for your help.

      this is my robots.txt

      User-agent: Googlebot
      Allow: www.pet-academy.com.pt/sitemap.xml
      Allow: *.css
      Allow: *.js
      Disallow: /administrator/
      Disallow: /cache/
      Disallow: /cli/
      Disallow: /components/
      Allow: /components/com_roksprocket/
      Allow: /components/com_rokgallery/
      Allow: /images/
      Disallow: /includes/
      Disallow: /installation/
      Disallow: /language/
      Disallow: /libraries/
      Disallow: /logs/
      Disallow: /media/
      Disallow: /modules/
      Allow: /modules/mod_facebook_slide_likebox/
      Disallow: /plugins/
      Allow: /plugins/system/rokbox/
      Disallow: /templates/
      Allow: /templates/rt_myriad/
      Disallow: /tmp/

      i aprecciate your attention

      Ana
    • MrT's Avatar
    • MrT
    • Preeminent Rocketeer
    • Posts: 101084
    • Thanks: 13481
    • Web Designer/Developer

    Re: SOLVED RT Resources still blocked after robot.txt fix

    Posted 5 years 11 months ago
    • This is because google is not "seeing" your site as end user would - if you look at the image of the site they show you you will see it doesn't look right - this is because googlebot has been prevented from accessing certain directories because of the content in your robots.txt file in your joomla root directory. You will need to comment out the lines that currently specify "disallow" on cache,components,libraries,modules and plugins (add a # at the start of the line). You could also make this change specifically for the googlebots too.

      This is what I use:
      User-agent: *
      Disallow: /administrator/
      Disallow: /bin/
      #Disallow: /cache/
      Disallow: /cli/
      #Disallow: /components/
      Disallow: /includes/
      Disallow: /installation/
      Disallow: /language/
      Disallow: /layouts/
      #Disallow: /libraries/
      Disallow: /logs/
      #Disallow: /modules/
      #Disallow: /plugins/
      #Disallow: /templates/
      Disallow: /tmp/

      Regards, Mark.
    • Please search forums before posting. Please make sure your post includes the version of the CMS you are using and a link to the problem. Annotations on screenshots can also be helpful to explain problems/goals. Please use the "secure" tab for confidential information.
  • Re: SOLVED RT Resources still blocked after robot.txt fix

    Posted 5 years 11 months ago
    • Mark, thank you for your answer.

      i have applied your suggestion. Still, the both cs and js in template other RT modules and plugins (and even gantry) are still being blocked

      example

      www.sitept/components/com_roksprocket/la.../assets/js/strips.js

      should i write allow do every single url that is being blocked?

      like
      allow:/templates/rt_myriad/css....

      Honestly i have tried to do it in many ways. Can it be the .htacess?
      well, if you have any other suggestion i apreciate your help.

      Thanks!
      Ana
    • MrT's Avatar
    • MrT
    • Preeminent Rocketeer
    • Posts: 101084
    • Thanks: 13481
    • Web Designer/Developer

    Re: SOLVED RT Resources still blocked after robot.txt fix

    Posted 5 years 11 months ago
    • In the robots.txt I gave you I commented out the disallow of /components therefore it should be allowed - so why you are blocking that still is puzzling.

      Please would you post your URL, superuser id and pswd in the secure tab of your post and I'll have a look for you.


      Please post your FTP logon, password and FTP URL in the secure area of your post.


      You can't do any of this in .htaccess that's a different thing - it's robots.txt you need to be changing.


      Regards, Mark.
    • Please search forums before posting. Please make sure your post includes the version of the CMS you are using and a link to the problem. Annotations on screenshots can also be helpful to explain problems/goals. Please use the "secure" tab for confidential information.
  • Re: SOLVED RT Resources still blocked after robot.txt fix

    Posted 5 years 11 months ago
    • This message contains only secure information that is visible to MrT, moderators and administrators
    • MrT's Avatar
    • MrT
    • Preeminent Rocketeer
    • Posts: 101084
    • Thanks: 13481
    • Web Designer/Developer

    Re: SOLVED RT Resources still blocked after robot.txt fix

    Posted 5 years 11 months ago
    • MrT wrote:
      Please post your FTP logon, password and FTP URL in the secure area of your post.

      You didn't provide ftp login as I requested?

      Regards, Mark.
    • Please search forums before posting. Please make sure your post includes the version of the CMS you are using and a link to the problem. Annotations on screenshots can also be helpful to explain problems/goals. Please use the "secure" tab for confidential information.
  • Re: SOLVED RT Resources still blocked after robot.txt fix

    Posted 5 years 11 months ago
    • This message contains only secure information that is visible to MrT, moderators and administrators
    • MrT's Avatar
    • MrT
    • Preeminent Rocketeer
    • Posts: 101084
    • Thanks: 13481
    • Web Designer/Developer

    Re: SOLVED RT Resources still blocked after robot.txt fix

    Posted 5 years 11 months ago
    • Ok something else is going on here because I changed the robots.txt to allow everything and still resources are being blocked.

      Are you using a CDN? if so, turn it off and clear it's cache. Possibly "cloudflare"?

      After that you really need to ask your host about this as something else is blocking access to those resource by googlebot (most likely a server setting higher up in your hosting).

      Regards, Mark.
    • Please search forums before posting. Please make sure your post includes the version of the CMS you are using and a link to the problem. Annotations on screenshots can also be helpful to explain problems/goals. Please use the "secure" tab for confidential information.
  • Re: SOLVED RT Resources still blocked after robot.txt fix

    Posted 5 years 11 months ago
    • thank you Mark for you tip

      All the best,

      Ana

Time to create page: 0.061 seconds