[RESOLVED] Problem downloading JSon as string

I have an app, that subscribes to an API in the form of a JSon file. Recently the format of this JSon file has changed, and now i can't even download it...

Code:

---

Dim client As New WebClient
Dim rawResponseString As String = client.DownloadString("https://www.gov.uk/bank-holidays.json")

---

Can anyone tell me what the problem is?

Thanks

User agent requirements?

Quote:

---

Originally Posted by peterst

User agent requirements?

---

I don’t see that being the problem. It’s a public api. I have permission to use it, but since the format change, nothing but problems

It works with browser. It works with cURL. Just try adding user agent in header. Many services were updated last months with that requirement.

I don’t specify a header. How would I do that with WebClient.DownloadString?

Code:

---

Dim client As New WebClient
client.Headers.Add("User agent requirements");
Dim rawResponseString As String = client.DownloadString("https://www.gov.uk/bank-holidays.json")

---

I haven’t tried it yet, but I’m fairly sure this isn’t how to do it…

I am on phone so can't test, but you can read here about it: https://developer.mozilla.org/en-US/...ers/User-Agent

Use the proper strings. No need to copy Mozilla identifier, as you can use anything you want.

Quote:

---

Originally Posted by peterst

I am on phone so can't test, but you can read here about it: https://developer.mozilla.org/en-US/...ers/User-Agent

Use the proper strings. No need to copy Mozilla identifier, as you can use anything you want.

---

I googled it. Think I’ve found what I need… Thanks

Ok. I tried...

Code:

---

Dim client As New WebClient
client.Headers.Add(HttpRequestHeader.UserAgent, "ASP_UK_Holidays.")
Dim rawResponseString As String = client.DownloadString("https://www.gov.uk/bank-holidays.json")

---

Getting error...

Quote:

---

System.Net.WebException: 'The underlying connection was closed: An unexpected error occurred on a send.'

IOException: Received an unexpected EOF or 0 bytes from the transport stream.

---

This is the proper way to set the user agent string:

VB.NET Code:

Dim http = New WebClient()
http.Headers.Add("User-Agent", "ASP_UK_Holidays.")
Dim json As String = http.DownloadString("https://www.gov.uk/bank-holidays.json")
Console.WriteLine(json)

And the json is returned without errors.

Quote:

---

Originally Posted by peterst

This is the proper way to set the user agent string:

VB.NET Code:

Dim http = New WebClient()
http.Headers.Add("User-Agent", "ASP_UK_Holidays.")
Dim json As String = http.DownloadString("https://www.gov.uk/bank-holidays.json")
Console.WriteLine(json)

And the json is returned without errors.

---

It still breaks on the DownloadString line. Same error

It seems you are banned for some reason. Change your IP and try again.

Turns out i wasn't banned. I changed my app. framework to 4.5.2 and established a secure connection...

Code:

---

ServicePointManager.Expect100Continue = True
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12

---

Well, going this way, I think it is better to target .NET Framework 4.8 as latest, updated and most compatible with latest networking trends. Also it is supported on Win 7+ (well, 4.5.2 works on Vista+).

Or if it is a new project - directly target .NET 6 (where WebClient is obsolete as there is HttpClient...).

I got it working on an alternative workstation running Win7 pro, running VS2017 (hence 4.7.2). It needs Tls13. That’s the all important part that got it working…

There was another thread about GitHub API, where minimum TLS version is required, setting of User-Agent and accepted content type (Accept header).

Many public services were changed to stop endless outdated bots - first by closing the connection via TLS requirement. Then if the user agent is missing (outdated bots or lazy authors) - reject. Some other specific header values missing? Reject. This requires devs to check regularly for API changes and apply to their tools to be compliant.