[RESOLVED] Net.HttpWebRequest.GetResponse

[dbasnett]

I was programmatically trying to access some pages on our web site and a funny thing happened on the way to the home page, it throws an error. If you use a WebBrowser control accessing the page works fine. I did get some help from Karen Payne, but it makes me more confused as to why.

Give this a try and you'll see what I mean.

    Private Sub SendWebPG()
        Const webReqTMO As Integer = 1000
        'this works
        ' Const Hpage As String = "https://revisor.mo.gov/main/OneSection.aspx?section=1.025&bid=44"
        'this throws Net.WebException but shouldn't
        Const Hpage As String = "https://revisor.mo.gov/main/Home.aspx" 'FWIW this can be control clicked in the IDE
        Dim WEBreqst As Net.HttpWebRequest = Nothing
        Dim WEBResp As Net.HttpWebResponse = Nothing
        Dim siteUri As New Uri(Hpage)

        WEBreqst = CType(Net.HttpWebRequest.Create(siteUri), Net.HttpWebRequest) 'build request
        WEBreqst.Timeout = webReqTMO
        WEBreqst.KeepAlive = False

        Try
            WEBResp = CType(WEBreqst.GetResponse(), Net.HttpWebResponse) 'get the page
            Using reader As IO.StreamReader = New IO.StreamReader(WEBResp.GetResponseStream)
                Dim s As String = reader.ReadToEnd
                If s.Length > 100 Then
                    Debug.WriteLine(s.Substring(0, 100))
                Else
                    Debug.WriteLine(s)
                End If
            End Using
            Stop
        Catch we As Net.WebException
            'BUT this works, thanks Karen Payne
            Using reader As New IO.StreamReader(we.Response.GetResponseStream())
                Dim s As String = reader.ReadToEnd
                If s.Length > 100 Then
                    Debug.WriteLine(s.Substring(0, 100))
                Else
                    Debug.WriteLine(s)
                End If
            End Using
            Stop
        Catch ex As Exception
            Stop
        Finally
            Try
                If WEBResp IsNot Nothing Then
                    WEBResp.Close()
                    WEBResp.Dispose()
                    WEBResp = Nothing
                End If
            Catch nex As Exception
            End Try
        End Try
    End Sub

[dbasnett]

Adding this before getting the response

WEBreqst.UserAgent = ".NET Framework Test Client"

seems to have fixed the issue.

[bmwpete]

Using the VB browser it returns me {"The remote server returned an error: (404) Not Found."}, and as you say using IE9, on Win7 it opens the page.

Yes that worked, but what does it mean? Does not seem to effect any other pages I load.

[dbasnett]

Using the VB browser it returns me {"The remote server returned an error: (404) Not Found."}, and as you say using IE9, on Win7 it opens the page.

Yes that worked, but what does it mean? Does not seem to effect any other pages I load.

My guess is that it is a setting I have in IIS, but damned if I can find it.

[bmwpete]

It is specific to your site from what I have read.

[dbasnett]

It is specific to your site from what I have read.

No. The solution I found was shown when accessing other sites.

[bmwpete]

Well it seems to me that I can set .UserAgent to anything and it will open that page, so the site must be only checking for it not being null/empty.

That was worded badly, the site must reject any request to this page where the .UserAgent is null/empty.

[dbasnett]

Well it seems to me that I can set .UserAgent to anything and it will open that page, so the site must be only checking for it not being null/empty

And only on the home page. If there is a setting in IIS I can't find it..

The incidence of this error has gone up on the site since I posted this question. Imagine that.