Downloading Web Pages With Cookies?

[rjbudz]

I wrote a Windows app that locate all the reports on a certain web site. The site uses an integer for the report numbers, so it was just a matter of creating a loop to query for each report on the server, and downloading the HTML, if the reports exists.

This did the trick nicely:

     Private Sub GetNewReports()

        Dim x As Integer
        Dim wc As New Net.WebClient
        Dim html As String

        For x = 1 To 23004 'HIGH_REPORT_NUMBER
            CheckForCount = x
            If Not InDatabase(x) Then
                html = wc.DownloadString(URL_PREFIX & x.ToString & URL_SUFFIX)
                If InStr(html, NO_REPORT_INDICATOR) = 0 Then
                    Dim writer As StreamWriter = _
                     New StreamWriter(EXISTING_FILES_PATH & "\" & FILE_NAME_PREFIX & x.ToString & FILE_NAME_SUFFIX)
                    writer.Write(html)
                    writer.Close()
                    Console.WriteLine(" << Report " & x.ToString & " <<")
                    NewReportCount = NewReportCount + 1
                    ReportList.Add(x.ToString)
                Else
                    Console.WriteLine("<> Report " & x.ToString)
                End If
            Else
                Console.WriteLine(">< Report " & x.ToString)
                ExistingReportCount = ExistingReportCount + 1
                Continue For
            End If

        Next
    End Sub

Now, I want to reuse the code elsewhere, but now there is a catch. When I download the web pages, what I get is a page that tells me cookies have to be enabled, along with the usual shpeel about how to enable them.

Is there a way to get past this? How do I give it the data it needs as a cookie?

[nolim8ts]

I'm not sure if I'm on the right track but looking at your code, you haven't declared a cookie container to hold any cookies that the website might issue. To do so, place the below code where your declarations are. You might also need to Imports System.Net at the very top of your form.

Imports System.Net

Dim URL As String = "www.google.com"

Dim objRequest As HttpWebRequest = CType(WebRequest.Create(URL), HttpWebRequest)

Dim cookies As CookieContainer = New CookieContainer

The when you request another page which requires cookies, call

objRequest.CookieContainer = cookies