[RESOLVED] WebClient PostData Error

[JayJayson]

Hi I am trying to automate a few searches on http://www.192.com/businesses/advanced/ using the WebClient. I can get it to work using a WebBrowser which I would prefer to avoid for loading times. I got it to work using a WebRequest but would like to see if theres any reason why the WebClient code below keeps giving me the error: "The remote server returned an error: (500) Internal Server Error".

This is the code using the WebClient:

Dim WClient As New WebClient
 
' // I got the headers from IE Developer Tools
WClient.Headers.Add(HttpRequestHeader.UserAgent, "Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; WOW64; Trident/6.0)")
WClient.Headers.Add(HttpRequestHeader.ContentType, "application/x-www-form-urlencoded")
' // This header is used when trying to compress the postdata using the commented code below
' WClient.Headers.Add(HttpRequestHeader.AcceptEncoding, "gzip,deflate")
 
' // This is the postdata that is sent to the page as detected by IE Deleveloper Tools. I just add in the relevant search text i.e. company=company+name - with spaces replaced by +'s
Dim Post As String = "company=&description=&subbuild=&buildno=&buildname=&street=&locality=&town=&county=&postcode=&btnSearch=Submit"
Dim PostBytes As Byte() = Encoding.UTF8.GetBytes(Post)
 
' // This part is commented out but just to show that I have tried using the ICSharpCode.SharpZipLib dll to compress the PostData before sending to the page
' Dim PostCompressedBytes As Byte() = Nothing
' Using PostDataStream As New MemoryStream
'     Using PostCompress As New GZipOutputStream(PostDataStream)
'         'PostCompress.IsStreamOwner = False
'         PostCompress.Write(PostBytes, 0, PostBytes.Length)
'     End Using
'     PostCompressedBytes = PostDataStream.ToArray
' End Using
 
'Now when I try to write the postdata to the page and read the response I get the error message
Dim DocumentBytes As Byte() = WClient.UploadData("http://www.192.com/businesses/advanced/", PostBytes)

I have also tried using a NameValueCollection to compile the postdata which also fails.

I can get it to work using a WebRequest as below:

Dim postData As String = "company=&description=&subbuild=&buildno=&buildname=&street=&locality=&town=&county=&postcode=&btnSearch=Submit"
Dim postBytes As Byte() = Encoding.UTF8.GetBytes(postData)
Dim request = CType(WebRequest.Create("http://www.192.com/businesses/search/advanced/"), HttpWebRequest)
request.UserAgent = "Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; WOW64; Trident/6.0)"
request.Headers.Add(HttpRequestHeader.AcceptEncoding, "gzip,deflate")
request.Method = "POST"
request.KeepAlive = True
request.ContentType = "application/x-www-form-urlencoded"
request.Referer = "http://www.192.com/businesses/search/advanced/"
request.ContentLength = postBytes.Length
request.AutomaticDecompression = DecompressionMethods.GZip Or DecompressionMethods.Deflate
 
Using postStream As Stream = request.GetRequestStream
    postStream.Write(postBytes, 0, postBytes.Length)
End Using
 
Using response = request.GetResponse()
    Using responseStream = response.GetResponseStream()
        Using sr = New StreamReader(responseStream)
            MsgBox(sr.ReadToEnd())
        End Using
    End Using
End Using

Can anyone see if I have missed something in the WebClient approach or even let me know if it works for you so I know whether it is actually code related.

Thanks
Jay

[Chris001]

I think your problem is the url.

In your WebClient code you navigate to :http://www.192.com/businesses/advanced/

In your WebRequest code you navigate to: http://www.192.com/businesses/search/advanced/

[JayJayson]

Chris your absolutely right!

It's a happy coincidence that I used that url at all in the WebRequest because that link does not contain all the input boxes to perform an advanced search. I guess it's a fluke that I did use it in the WebRequest and that the postdata remains the same.

Thanks alot for your help. I wouldn't have tried that url without it being pointed out to me.