-
Oct 8th, 2020, 03:37 PM
#1
Thread Starter
Addicted Member
Vb6: Winsock data arrival multiple packets
I am receiving a large request from a website, and data arrives in multiple parts. like on 2 parts
how can i get data to textbox in one part?
Private Sub winsock_DataArrival(ByVal bytesTotal As Long)
Dim strResponse As String\
winsock.GetData strResponse, vbString, bytesTotal
strResponse = FormatLineEndings(strResponse)
' we append this to the response box becuase data arrives
' in multiple packets
txtResponse.Text = txtResponse.Text & strResponse
End sub
-
Oct 8th, 2020, 04:00 PM
#2
Re: Vb6: Winsock data arrival multiple packets
You have to use a buffer, append the data to the buffer as it arrives then read the data from your buffer when the time is right. You can not get the data in one go when all of the data is not there yet, you have to buffer and wait which is basically what is being done above by appending it to the textbox but normally this would be done with a variable rather than a display control.
-
Oct 8th, 2020, 04:09 PM
#3
Thread Starter
Addicted Member
Re: Vb6: Winsock data arrival multiple packets
Originally Posted by DataMiser
You have to use a buffer, append the data to the buffer as it arrives then read the data from your buffer when the time is right. You can not get the data in one go when all of the data is not there yet, you have to buffer and wait which is basically what is being done above by appending it to the textbox but normally this would be done with a variable rather than a display control.
please give me code to do it, i am a newbie
thank you
-
Oct 8th, 2020, 10:13 PM
#4
Re: Vb6: Winsock data arrival multiple packets
Try it, it is pretty simple and that is how you learn.
-
Oct 9th, 2020, 07:19 AM
#5
Thread Starter
Addicted Member
Re: Vb6: Winsock data arrival multiple packets
Originally Posted by DataMiser
Try it, it is pretty simple and that is how you learn.
i told you i am a newbie i searched everywhere in goolge and couldnt find code
how can write code when i am a newbie ?
just say you dont want to help
-
Oct 9th, 2020, 07:28 AM
#6
Re: Vb6: Winsock data arrival multiple packets
What is not working in your 1st post?
Code:
Private Sub winsock_DataArrival(ByVal bytesTotal As Long)
Dim strResponse As String
winsock.GetData strResponse, vbString, bytesTotal
strResponse = FormatLineEndings(strResponse)
' we append this to the response box becuase data arrives
' in multiple packets
txtResponse.Text = txtResponse.Text & strResponse
End sub
-
Oct 9th, 2020, 07:29 AM
#7
Re: Vb6: Winsock data arrival multiple packets
This line
txtResponse.Text = txtResponse.Text & strResponse
. . . already appends to your buffer.
> how can i get data to textbox in one part?
So you want a single txtResponse.Text = m_sBuffer then you have to change above buffering code to
m_sBuffer = m_sBuffer & strResponse
. . . so that it appends to a member string variable m_sBuffer instead.
The real problem is how to figure out when is the full response complete in the buffer? When are there no more chunks expected to arrive from the server?
This depends on the network protocol used e.g. for http there is "Content-Length" header and its value has to be parsed when headers are received and then checked on each m_sBuffer = m_sBuffer & strResponse if buffer is full enough according "Content-Length" value.
So for proper http protocol parsing you have to implement a simple state machine. First state is "receiving headers". Once there are two consequtive new-lines in m_sBuffer then parse "Content-Length" header value and change state to "receiving body". On data receive if Len(m_sBuffer) >= lContentLength + lHeadersLength then all is complete by assigning txtResponse.Text = m_sBuffer which is your original wish.
Not very simple, not terribly complicated. Just interesting assignment for a junior dev.
cheers,
</wqw>
Last edited by wqweto; Oct 11th, 2020 at 02:19 AM.
-
Oct 9th, 2020, 08:39 AM
#8
Thread Starter
Addicted Member
Re: Vb6: Winsock data arrival multiple packets
Originally Posted by wqweto
This line
txtResponse.Text = txtResponse.Text & strResponse
. . . already appends to your buffer.
> how can i get data to textbox in one part?
So you want a single txtResponse.Text = m_sBuffer then you have to change above buffering code to
m_sBuffer = m_sBuffer & strResponse
. . . so that it sppends to a member string variable m_sBuffer instead.
The real problem is how to figure out when is the full response complete in the buffer? When are there no more chunks expected to arrive from the server?
This depends on the network protocol used e.g. for http there is "Content-Length" header and its value has to be parsed when headers are received and then checked on each m_sBuffer = m_sBuffer & strResponse if buffer is full enough according "Content-Length" value.
So for proper http protocol parsing you have to implement a simple state machine. First state is "receiving headers". Once there are two consequtive new-lines in m_sBuffer then parse "Content-Length" header value and change state to "receiving body". On data receive if Len(m_sBuffer) >= lContentLength + lHeadersLength then all is complete by assigning txtResponse.Text = m_sBuffer which is your original wish.
Not very simple, not terribly complicated. Just interesting assignment for a junior dev.
cheers,
</wqw>
i already thought about content lenght and i cant use it, because the website give different content each time, so content length is not fixed
could you please help with the code? i have looked everywhere online and found nothing
-
Oct 9th, 2020, 09:01 AM
#9
Hyperactive Member
Re: Vb6: Winsock data arrival multiple packets
The answer is that you cannot unless you are able to understand the protocol. TCP is a stream that is sent to you in chunks as the network is able to transmit it. Unless you can come to an arrangement with the owner of the website to add some kind of length field at the beginning of the request there is no way for you to know when its done unless it closes the connection when its finished. If that is the case you could just buffer until you get a close event fires.
The other complication to this is you will most likely receive a different amount of "messages" depending on the network (LAN/WAN/Other)
In addition if the traffic is HTTPS you are screwed unless you can decode it and also have some length field at the beginning.
-
Oct 9th, 2020, 09:26 AM
#10
Thread Starter
Addicted Member
Re: Vb6: Winsock data arrival multiple packets
Originally Posted by vbwins
The answer is that you cannot unless you are able to understand the protocol. TCP is a stream that is sent to you in chunks as the network is able to transmit it. Unless you can come to an arrangement with the owner of the website to add some kind of length field at the beginning of the request there is no way for you to know when its done unless it closes the connection when its finished. If that is the case you could just buffer until you get a close event fires.
The other complication to this is you will most likely receive a different amount of "messages" depending on the network (LAN/WAN/Other)
In addition if the traffic is HTTPS you are screwed unless you can decode it and also have some length field at the beginning.
im not convinced with what you telling me, i know the solution but im a newbie i dont know what code i should write
i have to get all buffer to memory first and only then copy it to textbox, i tried this code but it showed error
Private Sub tcpClient_DataArrival(ByVal bytesTotal As Long)
' determine the data type and process it
' get the buffer contents
ReDim gbIOBuff(bytesTotal)
tcpClient.GetData gbIOBuff, vbArray + vbByte, bytesTotal
If copyPtr = 0 Then ' we are at the beginning of a new reply
' save size & type of reply
CopyMemory recLen, gbIOBuff(0), 4
recLen = ntohl(recLen)
CopyMemory codeCheck, gbIOBuff(52), 2
codeCheck = ntohs(codeCheck)
ReDim gbReplyBuff(bytesTotal) ' clear old data record & start new one
Else
ReDim Preserve gbReplyBuff(copyPtr + bytesTotal) ' retain data & bump the size
End If
' copy the received data to the work buffer at location referenced by copyptr
CopyMemory gbReplyBuff(copyPtr), gbIOBuff(0), bytesTotal
If UBound(gbReplyBuff) = recLen Then
' we got the entire record
If recLen = gilocLen Or recLen = gijobLen Or recLen = giSockRtn Then
If codeCheck = glLocRequest Then
Process_Locations ' 1 time only at Form_Load
Else
If codeCheck = glJobRequest Then
Process_JobStats
Else
Beep
sbStatus.Panels(2).Text = "Error: " & codeCheck & ", No text returned"
intTics = 0 ' reset display timer
End If
End If
End If
copyPtr = 0 ' reset the flag\pointer
Else
' save the current record length\next copy location
copyPtr = UBound(gbReplyBuff)
End If
End Sub
-
Oct 9th, 2020, 11:45 AM
#11
Re: Vb6: Winsock data arrival multiple packets
Why all of this nonsense anyway?
Ignoring the SSL/TLS issue (which can be hacked around with a ton of effort as some have done) almost everything you are struggling with is handled by existing HTTP classes like WinHttpRequest, or XMLHTTP, or even the crusty old ITC ("WinInet control").
-
Oct 9th, 2020, 01:37 PM
#12
Thread Starter
Addicted Member
Re: Vb6: Winsock data arrival multiple packets
Originally Posted by dilettante
Why all of this nonsense anyway?
Ignoring the SSL/TLS issue (which can be hacked around with a ton of effort as some have done) almost everything you are struggling with is handled by existing HTTP classes like WinHttpRequest, or XMLHTTP, or even the crusty old ITC ("WinInet control").
winhttp + xmlhttp + inet i tried all of them and they cant grab cookies
only choice i have left is winsock bcoz only with winsock i could grab cookies
-
Oct 9th, 2020, 01:39 PM
#13
Re: Vb6: Winsock data arrival multiple packets
Originally Posted by dilettante
Ignoring the SSL/TLS issue (which can be hacked around with a ton of effort as some have done)
Btw, there is no TLS 1.3 client built-in on latest Win10 and there is no TLS 1.2 client built-in on XP (and 2000/NT) which is why I would have to use a custom impl if need be.
cheers,
</wqw>
-
Oct 9th, 2020, 01:50 PM
#14
Thread Starter
Addicted Member
Re: Vb6: Winsock data arrival multiple packets
Originally Posted by dilettante
Why all of this nonsense anyway?
Ignoring the SSL/TLS issue (which can be hacked around with a ton of effort as some have done) almost everything you are struggling with is handled by existing HTTP classes like WinHttpRequest, or XMLHTTP, or even the crusty old ITC ("WinInet control").
winhttp + xmlhttp + inet i tried all of them and they cant grab cookies
only choice i have left is winsock bcoz only with winsock i could grab cookies
-
Oct 9th, 2020, 02:05 PM
#15
Re: Vb6: Winsock data arrival multiple packets
Originally Posted by geekmaro
i told you i am a newbie i searched everywhere in goolge and couldnt find code
how can write code when i am a newbie ?
just say you dont want to help
I did help, more than once already. I showed you how to append the text, I told you that normally you would use a buffer variable.
I am not going to write the code for you so you can just paste it in, you learn nothing that way.
As for how you write code as a newbie, you start by typing in some code, modifying examples, test and repeat until it works. This is how you learn to be a coder. If you do not want to put in some effort to learn to do it then you should forget about being a coder and look for something better suited to you.
-
Oct 9th, 2020, 02:09 PM
#16
Re: Vb6: Winsock data arrival multiple packets
Originally Posted by geekmaro
i already thought about content lenght and i cant use it, because the website give different content each time, so content length is not fixed
could you please help with the code? i have looked everywhere online and found nothing
That is why you read the content length from the header then you know what to expect. So it is not a matter of you can't use it it is a matter of you can't hard code it to some preset value you have to actually write a few lines of code to see what it is.
-
Oct 9th, 2020, 02:13 PM
#17
Re: Vb6: Winsock data arrival multiple packets
Originally Posted by wqweto
Btw, there is no TLS 1.3 client built-in on latest Win10
Really?
Well currently it is still considered "experimental" but you can still turn the option on in SChannel via "Internet Options."
From all reports this should leave experimental status and be on by default in the next Win10 update (20H2) coming soon.
-
Oct 10th, 2020, 10:36 AM
#18
Thread Starter
Addicted Member
Vb6: Winsock get data fix ( i pay $200 to who can fix this)
i want to grab all page source using winsock from this site: http://apache.org
but problem is on data arrival event i get data on 2 parts
i need to get all data at once into textbox
i offer $200, can pay with any payment method you prefer; paypal, bitcoin etc..
thank u all
-
Oct 10th, 2020, 10:43 AM
#19
Re: Vb6: Winsock get data fix ( i pay $200 to who can fix this)
the API URLDownloadToFile is quite handy when grabbing files.
what u need is the directory list of the page, if the http server is allowing browsing.
if the page is generated or html, u can create a parser for that purpose. usually its all about looking at the html and check for "similarities" of the files you want to grab.
-
Oct 10th, 2020, 10:46 AM
#20
Thread Starter
Addicted Member
Re: Vb6: Winsock get data fix ( i pay $200 to who can fix this)
Originally Posted by baka
the API URLDownloadToFile is quite handy when grabbing files.
what u need is the directory list of the page, if the http server is allowing browsing.
if the page is generated or html, u can create a parser for that purpose. usually its all about looking at the html and check for "similarities" of the files you want to grab.
already tried that didnt work
need solution that use buffer to get all data first then paste it to textbox
-
Oct 10th, 2020, 10:51 AM
#21
Re: Vb6: Winsock get data fix ( i pay $200 to who can fix this)
didnt work? I have a site-grabber program and I use this API.
u want the page source of http://apache.org/? like index.html?
or is it something else you want? maybe its not apache.org you want but a generic page downloader or a page that uses php?
-
Oct 10th, 2020, 10:56 AM
#22
Thread Starter
Addicted Member
Re: Vb6: Winsock get data fix ( i pay $200 to who can fix this)
Originally Posted by baka
didnt work? I have a site-grabber program and I use this API.
u want the page source of http://apache.org/? like index.html?
or is it something else you want? maybe its not apache.org you want but a generic page downloader or a page that uses php?
i must use only winsock, no other control
i dont need downloader, i only need to grab page source code to winsock using data arrival event
-
Oct 10th, 2020, 11:04 AM
#23
Re: Vb6: Winsock get data fix ( i pay $200 to who can fix this)
that is very basic, you should show the code you are using and we could help you out what you did wrong.
the importance is the handshake/header when you send the GET command so that the server will send you the whole page back.
using URLDownloadToFile you will not need anything like that, just 1 line to grab the index.html and a vb6 open as input to get the data into the textbox.
-
Oct 10th, 2020, 11:19 AM
#24
Re: Vb6: Winsock get data fix ( i pay $200 to who can fix this)
example:
Code:
Private Declare Function URLDownloadToFileA Lib "urlmon" (ByVal pCaller As Long, ByVal szURL As String, ByVal szFileName As String, ByVal dwReserved As Long, ByVal lpfnCB As Long) As Long
Private Sub Form_Load()
Dim ff%, sLocalFile$, Data() As Byte
sLocalFile = App.Path & "\tmp.txt"
Call URLDownloadToFileA(0&, "http://apache.org/index.html", sLocalFile, &H10, 0&)
ff = FreeFile
ReDim Data(0 To FileLen(sLocalFile) - 1)
Open sLocalFile For Binary As #ff
Get #ff, , Data
Close #ff
Text1.Text = StrConv(Data, vbUnicode)
End Sub
this will download the page source and put it in text1.text, remember to change text1 to MultiLine
also textbox is limited how much it can show. so its not broken.
use RichTextBox instead.
Last edited by baka; Oct 10th, 2020 at 11:25 AM.
-
Oct 10th, 2020, 01:04 PM
#25
Thread Starter
Addicted Member
Re: Vb6: Winsock get data fix ( i pay $200 to who can fix this)
Originally Posted by baka
example:
Code:
Private Declare Function URLDownloadToFileA Lib "urlmon" (ByVal pCaller As Long, ByVal szURL As String, ByVal szFileName As String, ByVal dwReserved As Long, ByVal lpfnCB As Long) As Long
Private Sub Form_Load()
Dim ff%, sLocalFile$, Data() As Byte
sLocalFile = App.Path & "\tmp.txt"
Call URLDownloadToFileA(0&, "http://apache.org/index.html", sLocalFile, &H10, 0&)
ff = FreeFile
ReDim Data(0 To FileLen(sLocalFile) - 1)
Open sLocalFile For Binary As #ff
Get #ff, , Data
Close #ff
Text1.Text = StrConv(Data, vbUnicode)
End Sub
this will download the page source and put it in text1.text, remember to change text1 to MultiLine
also textbox is limited how much it can show. so its not broken.
use RichTextBox instead.
i told you i cannot use any other control than winsock
my project require strictly winsock, because i must login first and then obtain cookies and only then grab page source
so please only winsock with data arrival event
not any other control
-
Oct 10th, 2020, 01:04 PM
#26
Thread Starter
Addicted Member
Re: Vb6: Winsock get data fix ( i pay $200 to who can fix this)
still searching for someone who can fix this..
-
Oct 10th, 2020, 01:26 PM
#27
Re: Vb6: Winsock get data fix ( i pay $200 to who can fix this)
You keep throwing in these oddball requirements. Why can't you use any other controls? Every thread of yours is very odd.
Also, we all know you aren't scraping www.apache.org. Why are you hiding you true intentions? What are you afraid of?
-
Oct 10th, 2020, 01:28 PM
#28
Re: Vb6: Winsock data arrival multiple packets
Originally Posted by geekmaro
winhttp + xmlhttp + inet i tried all of them and they cant grab cookies
only choice i have left is winsock bcoz only with winsock i could grab cookies
WinHttpRequest manages cookies automatically if used properly. Just keep re-using the same instance per "session."
Why would you need to "grab" cookies? Sounds like something nefarious is going on here.
If they wanted to permit bots and non-web clients to access their site they would provide a web service. Use that. It is hard to imagine any responsible financial web site that would tolerate web scraping.
-
Oct 10th, 2020, 01:42 PM
#29
Thread Starter
Addicted Member
Re: Vb6: Winsock get data fix ( i pay $200 to who can fix this)
Originally Posted by OptionBase1
You keep throwing in these oddball requirements. Why can't you use any other controls? Every thread of yours is very odd.
Also, we all know you aren't scraping www.apache.org. Why are you hiding you true intentions? What are you afraid of?
why should i show you my url? who are you ?
either you give me code and get paid, or please stay away from my topic
thank you
-
Oct 10th, 2020, 02:00 PM
#30
Re: Vb6: Winsock get data fix ( i pay $200 to who can fix this)
-
Oct 10th, 2020, 02:01 PM
#31
Thread Starter
Addicted Member
Re: Vb6: Winsock get data fix ( i pay $200 to who can fix this)
Originally Posted by OptionBase1
No thanks.
better.
-
Oct 10th, 2020, 02:31 PM
#32
Re: Vb6: Winsock data arrival multiple packets
Originally Posted by dilettante
From all reports this should leave experimental status and be on by default in the next Win10 update (20H2) coming soon.
I have it switched on here on my 2004 and SSPI/Schannel is not working against TLS 1.3 only servers at all.
Code:
Connecting to tls13.1d.pw
Error: The message received was unexpected or badly formatted. Protocol version.
Originally Posted by dilettante
Sounds like something nefarious is going on here.
He just wants to donate $200 to some charity, no?
cheers,
</wqw>
-
Oct 10th, 2020, 05:06 PM
#33
Hyperactive Member
Re: Vb6: Winsock get data fix ( i pay $200 to who can fix this)
Try that, I haven't used it for a long time, I don't know if it will work
Code:
Private Sub Command1_Click()
With Winsock
.RemoteHost = "apache.org"
.RemotePort = 80
.Close
.Connect
End With
End Sub
Private Sub Winsock_Connect()
Winsock.SendData "GET www.apache.org HTTP/1.0" & vbCrLf _
& "Host: apache.org" & vbCrLf _
& "Accept: */*" & vbCrLf & vbCrLf
End Sub
Private Sub Winsock_DataArrival(ByVal bytesTotal As Long)
Dim X As String
Winsock.GetData X
Text1.Text = X
End Sub
Private Sub Winsock_Error(ByVal Number As Integer, Description As String, ByVal Scode As Long, ByVal Source As String, ByVal HelpFile As String, ByVal HelpContext As Long, CancelDisplay As Boolean)
Text1.Text = "Error " & Description
End Sub
-
Oct 10th, 2020, 05:41 PM
#34
Re: Vb6: Winsock get data fix ( i pay $200 to who can fix this)
Originally Posted by geekmaro
why should i show you my url? who are you ?
either you give me code and get paid, or please stay away from my topic
thank you
It's a valid question, and a good one. The site you are actually targeting doesn't prohibit what you are doing, so being clearly disingenuous about the site and everything else just has a bad smell to it. The site you are working with has an even worse smell about it, but it's your money, and you are free to make whatever mistake you want. However, lying about your intentions is clearly wrong, and nobody who has seen these threads believe you are being honest about this. Therefore, the thread is closed.
My usual boring signature: Nothing
-
Oct 10th, 2020, 07:46 PM
#35
Thread Starter
Addicted Member
Re: Vb6: Winsock data arrival multiple packets
Originally Posted by wqweto
I have it switched on here on my 2004 and SSPI/Schannel is not working against TLS 1.3 only servers at all.
Code:
Connecting to tls13.1d.pw
Error: The message received was unexpected or badly formatted. Protocol version.
He just wants to donate $200 to some charity, no?
cheers,
</wqw>
is is strange i cant find a single serious coder to write me code for my project
i been searching for over 1 month now
and not a single one of you guys could give me code
all what you do is just blablablabla
-
Oct 11th, 2020, 02:58 AM
#36
Re: Vb6: Winsock data arrival multiple packets
Here is a generic downloader that implements the state machine for the http protocol:
Code:
Private Enum UcsStateEnum
ucsIdle
ucsWaitRecvHeaders
ucsWaitRecvBody
ucsWaitSendBody
ucsWaitBoundary
End Enum
It does not currently support cookies or any headers customization though and is based on cAsyncSocket class, not Winsock control.
Also note that http://bbhyip.ru site momentarily redirects to https secure version so you need http over TLS to access anything there.
cheers,
</wqw>
p.s. Full sample here.
-
Oct 11th, 2020, 04:52 AM
#37
Thread Starter
Addicted Member
Re: Vb6: Winsock data arrival multiple packets
Originally Posted by wqweto
Here is a generic downloader that implements the state machine for the http protocol:
Code:
Private Enum UcsStateEnum
ucsIdle
ucsWaitRecvHeaders
ucsWaitRecvBody
ucsWaitSendBody
ucsWaitBoundary
End Enum
It does not currently support cookies or any headers customization though and is based on cAsyncSocket class, not Winsock control.
Also note that http://bbhyip.ru site momentarily redirects to https secure version so you need http over TLS to access anything there.
cheers,
</wqw>
p.s. Full sample here.
i dont need downloader, because i have to login into site and browse it, i get source code only to detect if specific keyword detected then move to another page
that is why i use winsock control with data arrival event
-
Oct 11th, 2020, 05:06 AM
#38
Re: Vb6: Winsock data arrival multiple packets
Originally Posted by geekmaro
is is strange i cant find a single serious coder to write me code for my project
i been searching for over 1 month now
and not a single one of you guys could give me code
all what you do is just blablablabla
why not ask the Mod's to close this Thread if your not Happy with the response you get.
your response ....all what you do is just blablablabla..... to the
People that are trying to help is just insulting
to hunt a species to extinction is not logical !
since 2010 the number of Tigers are rising again in 2016 - 3900 were counted. with Baby Callas it's 3901, my wife and I had 2-3 months the privilege of raising a Baby Tiger.
-
Oct 11th, 2020, 05:25 AM
#39
Thread Starter
Addicted Member
Re: Vb6: Winsock data arrival multiple packets
Originally Posted by ChrisE
why not ask the Mod's to close this Thread if your not Happy with the response you get.
your response ....all what you do is just blablablabla..... to the
People that are trying to help is just insulting
you just arrived you dont know what happened before
i asked for help for a whole month to give me code to fix issue
i even offered $200 to anyone who can write me code
but people just kept talking and no one solved my problem
that is why i said that
-
Oct 11th, 2020, 08:11 AM
#40
Re: Vb6: Winsock data arrival multiple packets
this is not a "pay for code" site, but a discussion, sharing, educational, helping, teaching forum.
I will never take a job, and most of the other members will not as well.
in the other thread you created, I pointed out what to do to fix your issue, but the response was:
"friend im tired
just give me code please"
instead of thanks I will look into it, and next you try to figure it out yourself, if you fail after many attempts, you post another question, with a new code that you dont understand or want help with.
all the energy you put to post here and insult people or ask the same question over and over could be used to actually fix your program.
Posting Permissions
- You may not post new threads
- You may not post replies
- You may not post attachments
- You may not edit your posts
-
Forum Rules
|
Click Here to Expand Forum to Full Width
|