Cloudflare Protected Link returning 503 code

1. 78

   electronicianposted 9 years ago

   I have a link on one of my hubs which goes to a page protected by Cloudflare DDoS protection. Because this service performs a browser check before redirecting the user to the actual link it is being wrongly marked as a broken link by Hubpages' automated system.
   
   Initially I noticed the broken link warning but ignored it after checking that the link worked, but today I visited my hub outside of the edit mode and noticed that the link is crossed out and a message box pops up to tell people its a broken link before they click through - which really detracts from the page. Isn't this overkill and is there any way to request a review for an excpetion or something? This link is integral to the page which provides information about, and comparison of, various services and then links to the service provider, and it would substantially reduce the quality of the hub if I have to remove it.
   
   For reference this is the link: https://coinkite.com/faq/terminal

   replyreport
   1. STAFF

      pauldeedsposted 9 years agoin reply to this

      When fetched that URL returns a 503 Service Temporarily Unavailable HTTP Header, which indicates to us that the link is broken.  You can see that using curl or an online link checker.
      
      curl --head https://coinkite.com/faq/terminal
      HTTP/1.1 503 Service Temporarily Unavailable
      Server: cloudflare-nginx
      
      According to cloudflare's FAQ, you may be hitting your connection limit in a Cloudfare datacenter.  The broken link checker bases it's assessment on the HTTP header.  It can not follow non-standard client side redirects that are done in javascript.

      replyreport
      1. 78

         electronicianposted 9 years agoin reply to this

         When you click that link manually, however, you will see that it actually isn't broken - so I don't think the connection limit has been reached.
         
         There are two possibilities that I can think of:
         
         1) It is giving your system a 503 because it is only allowing browsers to access the page, not bots. I know 503 is the wrong error for that, but it could still be the case.
         
         2) There is a temporary 503 while a check is being completed, with results returned after a couple of seconds.
         
         Can you not suggest any solution? Couldn't you contact Cloudflare to request your bots be allowed through? Or perhaps try telling your bot to wait a bit longer to see if it gets results returned a couple of seconds after the 503?

         replyreport
         1. STAFF

            pauldeedsposted 9 years agoin reply to this

            I saw the 503 error using a fresh incognito window with Chrome.  So it's not just crawlers that are getting it.  There are millions of links on HubPages, it's not really practical for us to make individual exceptions for servers that aren't following w3c standards.  The fix here is for cloudflare to either return a 200 or do a 301 or 302 redirect to a page that does.
            
            That said, I believe if you "ignore" the broken link from the broken links page in my account, it will temporarily solve the problem (until the next time that link is checked).

            replyreport
            1. 78

               electronicianposted 9 years agoin reply to this

               Well, an imperfect solution is better than no solution. Thanks.

               replyreport