Access Problem!

[couttsj]

For the past several days I am having a lot of trouble reaching
a number of web sites, including my provider (Telus) and my bank CIBC). Telus is using a reverse proxy provided by Cloudflare (172.66.0.98), and CIBC is using Telus/Akamai (207.194.199.74). A reverse proxy sits in front of an origin server and ensures that no client ever communicates directly with that origin server.

Some time ago, I had a similar problem with "VBForums.com", and I was able to work around that issue. This problem however is different in that if I wait long enough, it eventually loads (most of the time). As I am preparing this message, now VBForums also has this same problem as I try to post this message.

What is going on?

J.A. Coutts

Addendum:
Browser is Firefox 151.0.2

[dday9]

Do you ever get any non-successful HTTP responses, specifically of the 5xx variety?

[couttsj]

Do you ever get any non-successful HTTP responses, specifically of the 5xx variety?

This problem exists only on port 443 to certain web sites. I have no problem on other ports and other sites. There seems to be an issue with my browser (Firefox 151.0.3) and reverse proxy supplied by Cloudflare. It may have something to do with the NoScript addon that I am using, which I am not willing to give up. It has protected me from malicious scripts for years.

J.A. Coutts

[dday9]

NoScript is incredibly invaluable. In fact, my philosophy has changed dramatically where I prefer to write my website with a No-JS first design.

The problem is that I think too many developers take for granted that nearly everyone enables JavaScript by default. It leads to sloppy development, e.g. why use the built-in HTML5 input validations when you could rewrite it using JavaScript.

I tend to think that Cloudflare might be overly relying on JS being enabled in order to work properly. In fact, a quick search engine search shows that Cloudflare is using this: https://developers.cloudflare.com/turnstile and there's this quote:

Turnstile adapts the challenge outcome to the individual visitor or browser. First, we run a series of small non-interactive JavaScript challenges to gather signals about the visitor or browser environment.

[couttsj]

NoScript is incredibly invaluable. In fact, my philosophy has changed dramatically where I prefer to write my website with a No-JS first design.

The problem is that I think too many developers take for granted that nearly everyone enables JavaScript by default. It leads to sloppy development, e.g. why use the built-in HTML5 input validations when you could rewrite it using JavaScript.

I tend to think that Cloudflare might be overly relying on JS being enabled in order to work properly. In fact, a quick search engine search shows that Cloudflare is using this: https://developers.cloudflare.com/turnstile and there's this quote:

Turnstile.workers.dev is already enabled as it is for any other allowable script, but unfortunately it is enabled for the page that it is on. Cloudflare intervention occurs before the handshake is complete and the connection is passed on to the real server. Encryption/Decription occurs on the proxy server.

J.a. Coutts

[couttsj]

Disabling IPv6 made no difference. Disabling NoScript also made no difference. Connecting directly with pop.gmail.com works like a charm, but connecting with mail.telus.com takes forever. Connecting with fastmail.com also works like a charm. VBForums runs after a while when I run "https://browser-compat.turnstile.workers.dev/" in another window.

J.A. Coutts

[couttsj]

I finally found the source of the problem. It was in the Firefox settings that deals with the items that are cleared when Firefox closes.
- Browsing & download history
- Cookies and site data (12.4 MB)
- Temporary cached files and pages (257 MB)
- Saved form info
- Site settings

By default the first 4 items have always been checked as far as I know. When I unchecked "Cookies and site data", things speed up with the exception of Gmail. The initial window loads fine, but then it is still painfully slow loading the rest of the data. It flashes between "Transferring data from mail.google.com" and "Transferring data from apis.google.com". It appears to be repeatedly trying to download certain data that it cannot get access to, and I don't have a clue what the problem is. If I wait long enough it eventually finishes loading.

J.A. Coutts

[couttsj]

More info! I tracked down the cache location used by Firefox:
C:\Users\Name\AppData\Local\Mozilla\Firefox\Profiles\326gc9lj.default-release\cache2\entries>

When Firefox is closed, this directory is empty.
After loading browser
33 File(s) 625,221 bytes
After loading Fastmail
63 File(s) 2,512,786 bytes
After loading Gmail
172 File(s) 6,690,755 bytes
This may explain why Fastmail is so much faster than Gmail. Is there any way to reduce the number of cached files required by Gmail? I am already using the compact format.

J.A. Coutts

[Steve R Jones]

Course, the theory behind cached files is that when they are there - they don't have to be written and thus speed things up.