Download simultaneously several parts of one file

[Dragokas]

... with support for resuming, like download managers do.

I need that my program will download a large file ~ 1,5 GB. (on client side by user wish).
Source link for downloading is usual 7z-file on http-protocol.
I also think to move to WebDAV to use Yandex API.

Are there any ready-to-use solutions?
Any help on list of preferred objects / APIs / other technology usefull for this case will be appreciated.

Thank you.

Regards,
Alex.

[Elroy]

Hi Dragokas,

My clients often download a file that's in the 104Mbyte range. Not quite as big as you're wanting, but I think my procedures will get it done. It's all in a little-known "secret" that custom user controls have a built-in ability to do this. However, you've got to sometimes "hammer" on large files to keep them going. That's what I've worked out. My procedures keep hammering until the file is downloaded.

Also, I should say that these types of downloads are a bit tricky, and they seem to be a bit dependent on the actual website you're downloading them from. I've attached a little demo that illustrates how I get it done. I've also shown how to do it with a small file that's sitting out on my website.

You'll just have to try it and see if it'll work for the file you want to download.

Also, you'll see a frmWorking form commented out. When integrated into my system, that's a small form that indicates the progress. Let me know and I'll go ahead and work that into the demo. But first, try it and see if this is going to work for you.

Regards,
Elroy

[Elroy]

You know? The more I stare at this code and think about it, I'm not sure this is going to work for you. The problem is that the entire file is read into memory before it's written out. You might be able to write parts of the file out as it comes in inside of the UserControl_AsyncReadProgress event, but that's not something I've done.

I'll be curious as to how you work this out.

Good Luck,
Elroy

[fafalone]

Do you mean having multiple downloads running on the same exact file, just at different start points? The way to go about that would be adding range request HTTP headers,
Range: bytes=start-end

you can omit the ending number to get the rest of a file, e.g. to start at 12,345 bytes in Range: bytes=12345-

Some servers don't support this; if not I don't think there's any other way since it would send the whole file in each request.

[baka]

as fafalone says, the server need to support multiple downloads from same client and resume.
i would use winsock, as i can control the http-protocol and select the amount of bytes each packege is sent and then save it to the harddrive when the package is completed. using multiple sockets that works simultaneously to achieve the "multi" download and a data array to keep track of what segment is done of the big file. not that hard to do, but still a bit of work.

[LeandroA]

I'm not sure if it's what I'm looking for but maybe I can help you, it's not ssl

http://leandroascierto.com/blog/desc...multiconexion/

(The code is old so the links are down)

[Dragokas]

Hi, Leandro Ascierto !
I looked at the picture of the program.
This is really what I looking for. Is that a WinAPI HttpAddRequestHeaders based implementation? Can you share, please, a download link to your sources? Looks like, your site is currently not working (I read it via Google cache).

Thank you, fafalone, you are right. Now I know it can be done using headers. I'll try it.

baka, yea, thanks, I tested via program like FlashGet. Server supports starting/resuming download from different start points of one file.

Elroy, I saw this technique with UserControl.AsyncRead before. I liked it because async. mode. I think it will be helpfull in cases with files up to 100 MB. max,
where it is not very hard for user to re-download file in case it will be damaged.

to @all:
1. Are there a way to verify integrity of each part of the file before concatenating?
2. Is there any thought why InternetReadFile is not safe to call from DllMain (look in MSDN Remarks)? (I mean async. mode)

Regards,
Alex.

[LeandroA]

Hi, it's a more native method using Socket

This is the link to Download

  Dim strCommand As String
    
    strCommand = "GET " + GetFileFromURL(m_strURL) + " HTTP/1.1" + vbCrLf
    strCommand = strCommand + "Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/vnd.ms-powerpoint, application/vnd.ms-excel, application/msword, application/x-shockwave-flash, */*" + vbCrLf
    strCommand = strCommand + "Referer: " + GetHostFromURL(m_strURL) + vbCrLf
    strCommand = strCommand + "User-Agent: Mozilla/4.0 (compatible; MSIE 5.5; Windows 98; Win 9x 4.90)" + vbCrLf
    strCommand = strCommand + "Host: " + GetHostFromURL(m_strURL) + vbCrLf
    If m_BitesRangeStart > 0 Then strCommand = strCommand & "Range: bytes=" & m_BitesRangeStart & "-" & IIf(m_BitesRangeEnd > 0, m_BitesRangeEnd, "") & vbCrLf
    strCommand = strCommand + vbCrLf
    
    cmSocket.SendData strCommand

You could surely adapt it to HttpAddRequestHeaders

[Dragokas]

Thank you, Leandro!
Excellent huge work of several developers. It works as planned. Tested on XP/7/8.1/10 on VMWare with network adapter connection.
However, it cannot create a connection on my host machine (Win10) that is using PPP connection (3g modem):
window receives SOCKET_MESSAGE with Error # 10051 : Network is unreachable.

Any idea, how to localize a problem?