Download Files From Web With Progressbar

[kleinma]

Here is a sample project I did up that uses a class I wrote (WebFileDownloader)

The WebFileDownloader class provides methods for downloading a file from a URL and firing events to update progress in the GUI on a progress bar or whatever you may like.

The standard WebClient class in the .NET framework has a method for downloading a file, but there is no indication of its progress as it downloads, which is why I wrote this up.

Known limitations:
progress is returned as a long, and a progress bar takes an integer, so in the sample I convert the long to integer, which could error in BIG downloaded files.. a better solution would be to use the filesize being downloaded and calculate a good maximum to set the progress bar to. However this is GUI related, and not related to the WebFileDownloader class itself. (Just wanted to point it out)

Also make sure the URL is a full URL (ie http://www.mysite.com/file.zip and not www.mysite.com/file.zip)

Please post any corrections you may find. I use this code all the time, but that doesn't mean there isn't a bug or 2 lurking somewhere

[RingOfFire]

Thank you for the sample project.

I have a question: When the user clicks the close button (or exits application) during a download, download procedure keeps going on.

I'd like to know how to add the following feature:

When the application is stopped somehow
(by user, by system or other failure)

1. Stop the download process
2. Delete the file that was downloaded partially

[dannyf]

I noticed the buttons and progress bar in this project have the "XP" look.

How did you do that?

I can't find an option anywhere in Visual Studio?

Edit: looking through the zip file I can't find any use of manifest files or anything either?

[kleinma]

I enable visual styles in the Sub Main routine.

This code here

'SUB MAIN WHERE WE ENABLE VISUAL STYLES, AND RUN MAIN FORM
    Public Shared Sub Main()
        Application.EnableVisualStyles()
        Application.DoEvents()
        Application.Run(New frmMain)
        Application.Exit()
    End Sub

It also requires you set the buttons flatstyle property to system instead of standard. (any control that has a flatstyle property should be set to system to render it in XP theme).

.NET 2005 supports XP theme right from the get go, without needing to call this.

[shery]

how can i add download time estimation to this code ? i'm a beginner please help me

[cpatzer]

Kleinma,

Thanks again, I remember you helping me with this problem on an earlier date. I do have a quesition though, of course. How does this handle proxy servers or does it? If not how should I go about implementing that functionality?

Thanks,

Christian

[kleinma]

It currently doesn't and I don't have a proxy server to test any code against.

However I would imagine it would probably be as simple as setting the proxy property of the WebRequest object in the DownloadFileWithProgress routine.

After the line

wRemote = WebRequest.Create(URL)

try adding

wRemote.Proxy = new WebProxy()

but specify the needed information in one of the WebProxy overloaded constructors.

That should be all there is to it.

Let me know how you make out.

[cpatzer]

Will do thanks.

[bluehairman]

Hi, I tried adding this to program to auto update and noticed that WebFileDownloader doesn't seem to work. I use VS 2005 and it says:

'Type 'WebFileDownloader' is not defined'

[kleinma]

did you actually add the WebFileDownloader.vb class file to your project?

[bluehairman]

O.O No I didn't. I never thought of that. Sorry, I'm new to .NET and don't understand some things yet.

[kleinma]

No problem, but that is what you need to do. Once you add that class file to your project, you can access that class.

[jsloan2]

Thank you for the sample project.

I have a question: When the user clicks the close button (or exits application) during a download, download procedure keeps going on.

I'd like to know how to add the following feature:

When the application is stopped somehow
(by user, by system or other failure)

1. Stop the download process
2. Delete the file that was downloaded partially

Excellent class, I just used it for a small updater program, however i did notice one thing, the downloader does not stop when the form is closed. The application remains present in the taskmanager until the download completes.

I was wondering is anyone had any insight on how to make the downloader stop. I've never really used any web/http classes before they aren't really my forte. Maybe some kind of cancel event or something could be raised? I'm not really sure where this would be coded though. And i guess we'd have to clean up the partial stream. I'm kinda tired right now as I'm posting this so my brains not working 100%.

Maybe just adding a cancel event that changes a member variable blnCancel = True and modifying some of the loops to check that variable will work. Then raising the event when the form closes would work, might not be the cleanest but I'll try it after I wake up. If it works I'll repost modified code if that's ok.

Another thing I noticed is that download speed seem to dramatically increase after the downloaded stream starts reporting its size in MB rather than KB on my cable modem. I would say on average of 3-5x the speed it takes to go from 0 to 1MB. Not sure why this might be.

Anyway again, thanks for the class, saved me some time!

[kleinma]

well generally you would not want to allow the form to be closed while downloading. However if you want to offer a cancel option, you could simply set a boolean variable in the downloader class to indicate you want to cancel the download. Then in the DownloadFileWithProgress routine, check this boolean value in the do loop, and drop out of it if infact the cancel var is set to true. You may also want to delete the semi downloaded file when this happens, and maybe even add a new event like "DownloadCancelled". I did try to add a pause feature in recently, but so far it is still a little buggy.

Also I am not sure about what you say regarding the download speeds when the file is over 1MB downloaded. I don't notice any significant speed changes between 0-1 MB and then larger... How big are the files you are downloading?

[sdk1985]

well generally you would not want to allow the form to be closed while downloading. However if you want to offer a cancel option, you could simply set a boolean variable in the downloader class to indicate you want to cancel the download. Then in the DownloadFileWithProgress routine, check this boolean value in the do loop, and drop out of it if infact the cancel var is set to true. You may also want to delete the semi downloaded file when this happens, and maybe even add a new event like "DownloadCancelled". I did try to add a pause feature in recently, but so far it is still a little buggy.

Also I am not sure about what you say regarding the download speeds when the file is over 1MB downloaded. I don't notice any significant speed changes between 0-1 MB and then larger... How big are the files you are downloading?

I am also noticing this, just reported it in another topic. It goes from 60 kbps to 700 kbps after it switches to MB. Maybe there is a way to keep it at MB from the beginning? (file=3mb).

[kleinma]

are you sure it isn't just that it appears to be slower?

Remember the progress will look a lot faster when its just showing kilobytes downloaded versus megabytes...

The reason I say this, is because when progress of the download gets reported, its only in bytes, not in KB or MB.

The number of bytes is passed to the shared FormatFileSize method, which returns the correct display string (ie kb, mb, etc..)

so why don't you try NOT using the FormatFileSize, and see if you notice slowdown, I sure don't.

If you are using my exact sample project, in the _Downloader.AmountDownloadedChanged event, comment the line

lblProgress.Text = WebFileDownloader.FormatFileSize(iNewProgress) & " of " & WebFileDownloader.FormatFileSize(ProgBar.Maximum) & " downloaded"

and add the line

lblProgress.Text = iNewProgress.ToString & " of " & ProgBar.Maximum.ToString & " downloaded"

this will make the label only report the bytes download, and not format them into MB, KB, etc.. for display purposes. Again I think you are just thinking it looks faster before it has DLed a MB of the file.

[sdk1985]

Yes I am sure because I use an external program to measure download speeds (DU meter 3.07). Thats how I noticed the increase from 60 KB/s to 700 KB/s.

I'll try the code, do I need to replace the first line you gave?

Also there are some ! according to VB with the default project. (I am now using an exact copy of webfiledownloader.vb in my program)

Warning 1 Variable 'FS' is used before it has been assigned a value. A null reference exception could result at runtime. I:\Documents and Settings\Sebas\Mijn documenten\Visual Studio 2005\Projects\FileTransfer\FileTransfer\WebFileDownloader.vb 70 21 FileTransfer

Warning 2 Function 'FormatFileSize' doesn't return a value on all code paths. A null reference exception could occur at run time when the result is used. I:\Documents and Settings\Sebas\Mijn documenten\Visual Studio 2005\Projects\FileTransfer\FileTransfer\WebFileDownloader.vb 99 5 FileTransfer

[sdk1985]

Ok I replaced the line, now it's slow all the way. I think updating for every byte might be a little to much for the app. Now I reached 61-117 KBps, my maximum speed (of the same file) is 1058 KB/s. Do you know the line to format in MB instead of autoformat?
edit: when I remove the line entirely I get speeds from 640 - 890. Then the problem is solved... so the byte reporting is causing it to slow down.

edit2:

This also works (1 MB/s):

lblProgress.Text = "Bestandsgrootte: " & WebFileDownloader.FormatFileSize(Progbar.Maximum)

[kleinma]

well that is very common no matter what...

any UI updating ALWAYS slows an app down...

if you run a tight loop that scanned every file on your computer, it would run a bunch faster than if the same loop was also updating a label with the current file it was scanning.

If you still want to show UI progress, but speed it up, you could use some sort of if statement to determine if you should update the GUI with progress or not. Since the default is to update after even chunk of the download is completed, perhaps only have it update 1 out of every 4 times the event fires or something.

[sdk1985]

well that is very common no matter what...

any UI updating ALWAYS slows an app down...

if you run a tight loop that scanned every file on your computer, it would run a bunch faster than if the same loop was also updating a label with the current file it was scanning.

If you still want to show UI progress, but speed it up, you could use some sort of if statement to determine if you should update the GUI with progress or not. Since the default is to update after even chunk of the download is completed, perhaps only have it update 1 out of every 4 times the event fires or something.

Ok but it is still weird that if it switches from bytes/kb to MB then after that it's fine... Maybe because the mb figures have to change less often?

I'm new to vb 2005, thought the format was included in vb 2005. I see that you created it yourself.
So I started editing, just doing this:

Else
                Select Case Size / KB
                    'Case Is < 1000
                    '    Return (Size / KB).ToString("N") & "KB"
                    Case Is < 1000000
                        Return (Size / MB).ToString("N") & "MB"
                    Case Is < 10000000
                        Return (Size / MB / KB).ToString("N") & "GB"
                End Select

Solves the slowdown for me. (I commented the kb case).

[RingOfFire]

well generally you would not want to allow the form to be closed while downloading. However if you want to offer a cancel option, you could simply set a boolean variable in the downloader class to indicate you want to cancel the download. Then in the DownloadFileWithProgress routine, check this boolean value in the do loop, and drop out of it if infact the cancel var is set to true. You may also want to delete the semi downloaded file when this happens, and maybe even add a new event like "DownloadCancelled". I did try to add a pause feature in recently, but so far it is still a little buggy.

(...)

As I'm a new coder: can you help me a bit more to include this code to my project? Maybe you can give some more clues, so that I can learn at the same time.

Steps (according to what you've mentioned):
1. Setting a boolean class for cancelling download
2. Include this class within the DownloadFileWithProgress routine (inside do loop)
3. Add DownloadCancelled event for erasing partially downloaded file or other stuff on "post partial-download event".

Thanks a lot!

[Zoroxeus]

Hi Kleinma,
thanks for the great code,
I tried to adapt it to my case however I ran into an issue.
My links are asp type of links and they open a doc file in MS Word.
So they have the "?" charachter in the URL.
When I enter them on the app it's telling me that there are illegal charchters.
Any idea how this can be fixed ?
thanks

[sameer spitfire]

I need to download the file from the site which ask for userid and password

i use the following code to download the file( Without userid and password)

Dim wClient As New System.Net.WebClient
        Dim add As String = "http://203.199.49.90/XMLXX/CCnn/15KT/Pnn_001.zip"
        Try

download:
            wClient.DownloadFile(add, "n:\genbas\Pnn_011.zip")

But how i can pass the userid and passwords To the above string
Plase help

[cpatzer]

A simple way would be this:

http://203.199.49.90/XMLXX/CCnn/15KT...serid=sam&pwd=[encrypted password here]

[kleinma]

I need to download the file from the site which ask for userid and password

i use the following code to download the file( Without userid and password)

Dim wClient As New System.Net.WebClient
        Dim add As String = "http://203.199.49.90/XMLXX/CCnn/15KT/Pnn_001.zip"
        Try

download:
            wClient.DownloadFile(add, "n:\genbas\Pnn_011.zip")

But how i can pass the userid and passwords To the above string
Plase help

Specify the user id/password in the credentials property of the webclient before making the download call.

        Dim wClient As New System.Net.WebClient
        Dim add As String = "http://203.199.49.90/XMLXX/CCnn/15KT/Pnn_001.zip"
        Try
            wClient.Credentials = New Net.NetworkCredential("userid", "password")
            wClient.DownloadFile(add, "n:\genbas\Pnn_011.zip")

[xSmurf]

Nice job but how can i make it auto-update when start and how can I let it automaticly extract the rar

[kleinma]

Those questions are beyond the scope of what this example does. This example code is specifically just for downloading files with progress.

You should really ask your questions in the main VB.NET forum.

[xSmurf]

deleted

[michalss]

Nice sample but how can i add authentical for rapidshare.com. Thx

[kleinma]

Nice sample but how can i add authentical for rapidshare.com. Thx

In the downloader class, in the routine called DownloadFileWithProgress you could add the following line of code:

wRemote.Credentials = New Net.NetworkCredential("USERNAME", "PASSWORD")

this should go right after

wRemote = WebRequest.Create(URL)

and before

Dim myWebResponse As WebResponse = wRemote.GetResponse

[michalss]

In the downloader class, in the routine called DownloadFileWithProgress you could add the following line of code:

wRemote.Credentials = New Net.NetworkCredential("USERNAME", "PASSWORD")

this should go right after

wRemote = WebRequest.Create(URL)

and before

Dim myWebResponse As WebResponse = wRemote.GetResponse

Im sorry for my question but can you give me more info or simple exaple appl. Thx i will be more then greatfull...

EDIT : I Have Try your exaple and i was add this syntaxe but it did not work...

[kleinma]

what more do you need?

1) download and open up the my example code
2) go into the class file called WebFileDownloader
3) find the routine called DownloadFileWithProgress
4) find the line of code mentioned above
wRemote = WebRequest.Create(URL)
5) insert this code after it
wRemote.Credentials = New Net.NetworkCredential("USERNAME", "PASSWORD")
6) change USERNAME and PASSWORD to actual valid values
7) test code against site that needs authentication

[michalss]

what more do you need?

1) download and open up the my example code
2) go into the class file called WebFileDownloader
3) find the routine called DownloadFileWithProgress
4) find the line of code mentioned above
wRemote = WebRequest.Create(URL)
5) insert this code after it
wRemote.Credentials = New Net.NetworkCredential("USERNAME", "PASSWORD")
6) change USERNAME and PASSWORD to actual valid values
7) test code against site that needs authentication

Its not working i have try it just now

  FS = New FileStream(Location, FileMode.Create, FileAccess.Write)
            wRemote = WebRequest.Create(URL)
            wRemote.Credentials = New Net.NetworkCredential("myname", "mypass")
            Dim myWebResponse As WebResponse = wRemote.GetResponse
            RaiseEvent FileDownloadSizeObtained(myWebResponse.ContentLength)
            Dim sChunks As Stream = myWebResponse.GetResponseStream
            Do
                iBytesRead = sChunks.Read(bBuffer, 0, 256)
                FS.Write(bBuffer, 0, iBytesRead)
                iTotalBytesRead += iBytesRead
                If myWebResponse.ContentLength < iTotalBytesRead Then
                    RaiseEvent AmountDownloadedChanged(myWebResponse.ContentLength)
                Else

Can you check it for me pls?

EDIT : Its download only 5 Kb from 100 MB....

[kleinma]

they might be doing some sort of redirect on the server, so you may only be getting the first response.

I dont know how rapidshare servers work and if they just serve out the file when you navigate to it, or do some sort of redirect. Chances are if you are getting 5KB, its HTML.

[michalss]

they might be doing some sort of redirect on the server, so you may only be getting the first response.

I dont know how rapidshare servers work and if they just serve out the file when you navigate to it, or do some sort of redirect. Chances are if you are getting 5KB, its HTML.

Thx for quick response : I have try download http://rs201.rapidshare.com/files/82...9.0.0e-DVT.rar

How ever is any chance to get close have a look on this problem pls? Im really desperate to get it work. Im looking for solution almoust 2 months. I can also pay for your time if is it reuquire.... Thx

[kleinma]

do you have a premium acct for rapidshare?

[michalss]

do you have a premium acct for rapidshare?

Yes i have 2 if you need ...

[kleinma]

well just by looking at the site, they defenitly do some funky redirects (at least when u use the free downloading service). They might even do direct response writing of the files to prevent people from direct linking.

If you want to PM me one of your acct id/passwords, I will test trying to make it work, as it may be easier when you have an actual paid acct to work with. I can't promise I will be able to get back to you right away, but if you PM me with that info, I will look at it when I have a few free minutes.

[mjcm]

Here is a link to an article from Karl Moore, that can determine if a connection to the internet is available.

http://www.developerfusion.co.uk/show/3903/

[kleinma]

That code from Karl will actually error.. it should be written like this:

    Public Function IsConnectionAvailable() As Boolean
        ' Returns True if connection is available 
        ' Replace www.yoursite.com with a site that
        ' is guaranteed to be online - perhaps your 
        ' corporate site, or microsoft.com
        Dim objUrl As New System.Uri("http://www.yoursite.com/")
        ' Setup WebRequest
        Dim objWebReq As System.Net.WebRequest = Nothing
        objWebReq = System.Net.WebRequest.Create(objUrl)
        Dim objResp As System.Net.WebResponse = Nothing
        Try
            ' Attempt to get response and return True
            objResp = objWebReq.GetResponse
            objResp.Close()
            objWebReq = Nothing
            Return True
        Catch ex As Exception
            ' Error, exit and return False
            If objResp IsNot Nothing Then
                objResp.Close()
                objWebReq = Nothing
                Return False
            End If
        End Try
    End Function