[RESOLVED] How to replicate this request?

**bretddog** · Apr 8th, 2010, 03:14 PM

Hi,

I have a program that accepts a connection by typing the following address in a web browser:

http://localhost:<port>/quotes/<parameter>

It produces a simple text-file displayed in the browser. Now.. I want to replicate this request from my VB program, and get the same text back as a string. Actually I'm able to do that, by the following code:

Code:

Private _client as New TcpClient
_client.Connect(localhost, port)

Then I try to send a query:

SendData("GET /quotes/<parameter>")

Code:

  
Public Sub SendData(ByVal str As String)
        Dim writer As New IO.StreamWriter(_client.GetStream)
        writer.Write(str & vbCr)
        writer.Flush()
    End Sub

So... I get actually the response.. but it takes like 45 seconds.. While in the browser it's instant. Is this perhaps because I connect and send the response separately? How do I send a connection request with the message in the same request? I guess also this connection needs to be closed after each request? At least I'm only able to get one response with my program, then I must restart it.

Btw.. the response I get looks like this,:

Code:

HTTP/1.0 200 OK
Date: Thu, 08 Apr 2010 20:43:41 GMT
Server: NfTinyHTTP 0.1
Expires: Mon, 26 Jul 1997 05:00:00 GMT
Last-Modified: Thu, 08 Apr 2010 20:43:41 GMT
Cache-Control: no-cache, must-revalidate
Pragma: no-cache
Content-type: text/html

<body><html><table>
<tr><td>SEQUENCE</td> <td>EXCHANGE</td> <td>BOARD</td> <td>TIME</td> <td>PAPER</td> <td>BID</td> <td>BID-DEPTH</td> <td>BID-DEPTH-TOTAL</td> <td>BID-NUMBER</td> <td>OFFER</td> <td>OFFER-DEPTH</td> <td>OFFER-DEPTH-TOTAL</td> <td>OFFER-N

And of course... If possible, i don't want the response coded as html...
.

**bretddog** · Apr 8th, 2010, 05:08 PM

Looks like I found an alternative way to connect here, by the WebRequest class:
http://stackoverflow.com/questions/6...uest-in-vb-net

Still.. I'm not sure this is such a good solution. I need to send perhaps several request per second.. So I really need the fastest way.. And I guess using the webRequest it will connect/disconnect on each request (?) so... Is it appropriate to use this code, or can I make some more intelligent requests?

Code:

  Dim request As WebRequest = WebRequest.Create(<Web Request link>)
        ' Get the response.
        Dim response As WebResponse = CType(request.GetResponse(), WebResponse)
        ' Get the stream containing content returned by the server.
        Dim dataStream As Stream = response.GetResponseStream()
        ' Open the stream using a StreamReader for easy access.
        Dim reader As New StreamReader(dataStream)
        ' Read the content.
        Dim responseFromServer As String = reader.ReadToEnd()

**visualAd** · Apr 8th, 2010, 06:05 PM

The reason your request is taking a long time is because you are not sending the HTTP request in the correct format. To signify the end of the request you need to send two carriage return line feed sequences.

You also need to ensure that specify the version of HTTP you are using, if you do not the web server may reject the request or assume the request is HTTP 1.1. HTTP 1.1 responses are often returned as chunked responses which means extra coding for you.

Code:

GET /path/to/page.html HTTP/1.0<CRLF>
Host: www.example.com<CRLF>
<CRLF>

Where <CRLF> is ControlChars.CRLF.

The majority of web pages are written in HTML; to interpret HTML you need a web browser such as Internet Explorer or Firefox. You can also use a Web Browser control to display the page as it would be shown in Internet Explorer.

**bretddog** · Apr 8th, 2010, 06:44 PM

Thanks a lot! one step closer..

With the two CrLf's it now updates instantly. My connection string looks like this:

"GET /quotes/STL.OSE HTTP/1.1" & vbCrLf & vbCrLf

Still the server response says "HTTP/1.0 200 OK". Does this indicate that only a single request can be made per connection? I'm still only able to make one request, then I get nothing, or an error. So in this case I could probably just as well use the WebRequest class I guess.

The data is just text/numbers in a table, and I need to populate the values into business layer objects, but I guess the feed is already html formatted from the program I'm connecting to (not web), so I need to strip it in any case. It's like a web output feed from a program, designed to use for Excel web-queries. So probably this is why it seems to only accept one request, as probably (?) that's how Excel does it..

**visualAd** · Apr 9th, 2010, 01:53 AM

Well from the source code you posted, it is an HTML formatted table which you need to display in a web browser. If you wish to extract the data from the HTML, I suggest you use (XMLDocument - if the output is well formed XHTML) or HTMLDocument if not. From the code above it looks like you will need the later.

http://msdn.microsoft.com/en-us/libr...ldocument.aspx

You can then use the getElementsByTagName() function to find all (<tr> - table row elements):

Some pseudo code to get you started:

Code:

tables = html.getElementsByTagName("table")

For Each table in tables
(
    rows = table.getElementsByTagName("tr")

    For Each row in rows
    (
        cells = row.getElementsByTagName("td")
 
        For each cell in cells
        (
                print cell.nodeValue + \t
        )

        print \r\n <-- new line
    )
)

Sorry, that does mean more coding. Give it a try and post back if you get any errors. Like the request, the start of the response body is indicated by two CRLF after the response headers.

The reason you are getting an HTTP/1.0 response back is because you are not using the Host header as shown in my original post. HTTP 1.1 requests must have a host header. I'd recommend you use HTTP/1.0 at first though or the WebRequest class (which will automatically deal with a chunked response). HTTP is stateless; in general, unless you send a keep-alive header, the connection will be closed at the end of the response and you will need to open a new one to send a new request. Even when you do send a keep-alive header, while the connection remains open, any further HTTP requests are run as if you sent them with a new connection.

**bretddog** · Apr 10th, 2010, 03:40 AM

Thanks for the tip and example on the HtmlDocument class.. I wouldn't have guessed that

Looks good, I should have what I need then. Btw, I tried to send also with the host header as you had specified, like this:

str = "GET /quotes/STL.OSE HTTP/1.1" & vbCrLf & "Host: http://localhost:1234" & vbCrLf & vbCrLf

..however it still gives the "HTTP/1.0 200 OK" as response. So I believe this is the only option.

Thread: [RESOLVED] How to replicate this request?

Thread Tools

Display

[RESOLVED] How to replicate this request?

Re: How to replicate this request?

Re: How to replicate this request?

Re: How to replicate this request?

Re: How to replicate this request?

Re: How to replicate this request?

Posting Permissions