-
Dec 29th, 2017, 05:41 AM
#1
Thread Starter
Junior Member
[RESOLVED] WebClient downloads only 1 GB, then fails & Round number to 1 decimal place
Hi! I wanna download 2 GB file with WebClient, but after it downloads 1 GB it stops without any error.
Thanks for any help!
Code:
Sub download(ByVal file As String, ByVal uri As String, ByVal ffile As String)
la_dl.Text = file
SW = Stopwatch.StartNew
Dim client As WebClient = New WebClient
AddHandler client.DownloadProgressChanged, AddressOf download_change
AddHandler client.DownloadFileCompleted, AddressOf client_DownloadCompleted
If My.Computer.FileSystem.DirectoryExists(Application.StartupPath & "/dl") = False Then
My.Computer.FileSystem.CreateDirectory(Application.StartupPath & "/dl")
End If
client.DownloadFileAsync(New Uri(uri), Application.StartupPath & "/dl/" & ffile)
End Sub
Last edited by henryolik; Dec 29th, 2017 at 06:01 AM.
Reason: Deleted second issue
-
Dec 29th, 2017, 05:44 AM
#2
Re: WebClient downloads only 1 GB, then fails & Round number to 1 decimal place
Originally Posted by henryolik
And the second thing...
Belongs in a thread of its own. One thread per topic and one topic per thread.
-
Dec 29th, 2017, 06:05 AM
#3
Thread Starter
Junior Member
Re: WebClient downloads only 1 GB, then fails & Round number to 1 decimal place
Okay, thanks for info!
-
Dec 29th, 2017, 09:33 AM
#4
Re: WebClient downloads only 1 GB, then fails & Round number to 1 decimal place
It probably doesn't stop without error. You maybe didn't choose to do anything to get the error if an error happens. I can't tell you because you didn't show your DownloadCompleted error handler. Are you checking the AsyncCompletedEventArgs.Error property?
A slight possibility is that in the time it takes the download to complete, the .NET Garbage Collector runs and notices your WebClient isn't being referenced by anything. Then it decides to "help" by destroying it for you. To solve this, you should store the WebClient in a field that has a lifetime at least as long as the download.
The pattern for downloading a file with this clunky event-based technique should look something like this:
Code:
Private _activeClient As WebClient
Public Sub DoDownload()
_activeClient = New WebClient()
AddHandler client.DownloadFileCompleted, AddressOf WhenFileDownloadCompletes
_activeClient.DownloadFileAsync(...)
End Sub
Private Sub WhenFileDownloadCompletes(...)
If e.Error Then
MessageBox.Show("There was an error downloading the file.")
Else
MessageBox.Show("The file was downloaded.")
End If
_activeClient.Dispose()
_activeClient = Nothing
End Sub
Try using something that looks more like that and see if you either get an error or if it starts finishing the job.
This answer is wrong. You should be using TableAdapter and Dictionaries instead.
-
Dec 29th, 2017, 10:04 AM
#5
Thread Starter
Junior Member
Re: WebClient downloads only 1 GB, then fails & Round number to 1 decimal place
I tried it, I'm now getting "An exception occurred during a WebClient request." error right on 1 GB downloaded. Here is the rest of code (download change + download completed):
Code:
Private Sub download_change(ByVal sender As Object, ByVal e As DownloadProgressChangedEventArgs)
Dim mbytesIn As Double = Double.Parse(e.BytesReceived.ToString()) / 1000000
Dim totalmBytes As Double = Double.Parse(e.TotalBytesToReceive.ToString()) / 1000000
Dim percentage As Double = mbytesIn / totalmBytes * 100
pb_load.Value = Int32.Parse(Math.Truncate(percentage).ToString())
percentage = Math.Round(percentage, 0)
la_perc.Text = percentage & " %"
Dim unita As String
If totalmBytes < 1 Then
totalmBytes = totalmBytes * 1000
unita = " kB"
If totalmBytes < 1 Then
totalmBytes = totalmBytes * 1000
unita = " B"
End If
ElseIf totalmBytes > 1000 Then
totalmBytes = totalmBytes / 1000
unita = " GB"
Else
unita = " MB"
End If
Dim unitb As String
If mbytesIn < 1 Then
mbytesIn = mbytesIn * 1000
unitb = " kB"
If mbytesIn < 1 Then
mbytesIn = mbytesIn * 1000
unitb = " B"
End If
ElseIf mbytesIn > 1000 Then
mbytesIn = mbytesIn / 1000
unitb = " GB"
Else
unitb = " MB"
End If
mbytesIn = Math.Round(mbytesIn, 1)
totalmBytes = Math.Round(totalmBytes, 2)
la_size.Text = mbytesIn & unitb & " / " & totalmBytes & unita
Dim speed As Double = e.BytesReceived / SW.ElapsedMilliseconds / 1000
Dim unit As String
If speed < 1 Then
unit = " kB/s"
speed = speed * 1000
If speed < 1 Then
unit = " B/s"
speed = speed * 1000
End If
Else
unit = " MB/s"
End If
speed = Math.Round(speed, 1)
la_speed.Text = "Download speed: " & speed & unit
End Sub
Private Sub client_DownloadCompleted(ByVal sender As Object, ByVal e As System.ComponentModel.AsyncCompletedEventArgs)
If Not e.Error Is Nothing Then
MessageBox.Show("There was an error downloading the file. Error: " & e.Error.Message)
SW.Stop()
predl()
Else
dl = True
SW.Stop()
predl()
End If
End Sub
-
Dec 29th, 2017, 10:45 AM
#6
Re: WebClient downloads only 1 GB, then fails & Round number to 1 decimal place
Was there any other information included with that exception? As it stands, that's a pretty minimalist error message, and MS usually gives you more than that....though not always USEFUL information.
My usual boring signature: Nothing
-
Dec 29th, 2017, 11:03 AM
#7
Thread Starter
Junior Member
Re: WebClient downloads only 1 GB, then fails & Round number to 1 decimal place
In English - "An exception occurred during a WebClient request".
No further information :/.
-
Dec 29th, 2017, 07:36 PM
#8
Re: WebClient downloads only 1 GB, then fails & Round number to 1 decimal place
The feeling I get from some search results is WebClient makes the curious choice to store the downloaded data in a MemoryStream until it completes, then tries to write that whole MemoryStream. That goes fine for (relatively) small files, but can get nasty for large files. MemoryStream is backed by an array, and if data exceeds capacity it re-allocates a new array at twice the size. For files that exceed 1GB, that might involve trying to allocate a greater-than-2GB array and for various reasons that can be a problem in .NET unless you write very special code to handle it. Not to mention on many machines "please allocate 3GB of total memory" is a tall order.
So you should probably just try using WebRequest to download bytes in "blocks" and write each "block" to the file as it comes in. C# will soon have a nice feature to help with this but it's not on the VB roadmap so I'm not sure why I mentioned it. This is a C# example of the technique.
This answer is wrong. You should be using TableAdapter and Dictionaries instead.
-
Dec 29th, 2017, 09:51 PM
#9
Re: WebClient downloads only 1 GB, then fails & Round number to 1 decimal place
Originally Posted by Sitten Spynne
I see no reason why that can't be implemented in VB.
-
Dec 31st, 2017, 09:30 AM
#10
Thread Starter
Junior Member
Re: WebClient downloads only 1 GB, then fails & Round number to 1 decimal place
I don't think WebClient downloads it to memory first. When downloading it writes every single bit straight to destination file on disk. So I don't think that's the reason. I'll probably split that file into 3 parts, download it and join it together afterwards
-
Jan 2nd, 2018, 09:16 AM
#11
Thread Starter
Junior Member
Re: WebClient downloads only 1 GB, then fails & Round number to 1 decimal place
Splitted, again, at 1 GB total downloaded it throws error. I'm really desperate now...
-
Jan 3rd, 2018, 05:48 AM
#12
Lively Member
Re: WebClient downloads only 1 GB, then fails & Round number to 1 decimal place
A few observations regarding your initial post:
1. You are using DownloadFileAsync without awaiting it. You should look up async and await.
2. You are using WebClient. A better alternative would be HttpClient.
-
Jan 3rd, 2018, 07:53 AM
#13
Re: WebClient downloads only 1 GB, then fails & Round number to 1 decimal place
Originally Posted by BlackRiver1987
1. You are using DownloadFileAsync without awaiting it. You should look up async and await.
The WebClient.DownloadFileAsync predates async/await and is not awaitable. It follows an older async pattern where it is paired with the DownloadFileCompleted event, much as the RunWorkerAsync method and RunWorkerCompleted event of the BackgroundWorker class are paired.
Originally Posted by BlackRiver1987
2. You are using WebClient. A better alternative would be HttpClient.
Why so? A WebClient can handle downloading a file just fine in general. What would an HttpClient add in this case? There may be something that I'm not aware of but I suspect that this issue would not be solved by using an HttpClient.
-
Jan 3rd, 2018, 07:59 AM
#14
Lively Member
Re: WebClient downloads only 1 GB, then fails & Round number to 1 decimal place
Originally Posted by jmcilhinney
The WebClient.DownloadFileAsync predates async/await and is not awaitable. It follows an older async pattern where it is paired with the DownloadFileCompleted event, much as the RunWorkerAsync method and RunWorkerCompleted event of the BackgroundWorker class are paired.
Thank you for that info. Did not know that.
Originally Posted by jmcilhinney
Why so? A WebClient can handle downloading a file just fine in general. What would an HttpClient add in this case? There may be something that I'm not aware of but I suspect that this issue would not be solved by using an HttpClient.
Just a personal preference, since it's easier to use when going full async.
-
Jan 3rd, 2018, 08:04 AM
#15
Re: WebClient downloads only 1 GB, then fails & Round number to 1 decimal place
Originally Posted by BlackRiver1987
Just a personal preference, since it's easier to use when going full async.
I'd tend to agree if you were using async/await throughout your code but that is probably not the case here, which you probably didn't realise.
-
Jan 3rd, 2018, 08:26 AM
#16
Re: WebClient downloads only 1 GB, then fails & Round number to 1 decimal place
Actually, HttpClient is recommended going forward.
WebClient is older and potentially based on a different HTTP stack. HttpClient supports the newer async patterns and is what MS supports in terms of UWP and newer .NET code.
But the main reason I'd suggest trying it is that "potentially different" aspect. Obviously we've found some mysterious problem with WebClient. I'd love to get to the bottom of it, but if we try HttpClient and it works that solves the problem. If it doesn't, we can revert back to WebClient and start looking for more answers.
If we can't (It's possible HttpClient is newer than VS 2010 and might not work at all), then my next question is, "Is there an InnerException in that exception that gets thrown? Usually if a .NET Exception says "An exception was thrown..." it puts that exception in the InnerException property."
This answer is wrong. You should be using TableAdapter and Dictionaries instead.
-
Jan 3rd, 2018, 09:07 AM
#17
Re: WebClient downloads only 1 GB, then fails & Round number to 1 decimal place
Originally Posted by Sitten Spynne
It's possible HttpClient is newer than VS 2010 and might not work at all
According to the documentation, HttpClient was introduced in .NET 4.5. VS 2010 was released with .NET 3.5, if I remember correctly. I'm not sure whether it can target .NET 4.5 or not but, even if it can, there's every chance that the project was created against an earlier version.
-
Jan 3rd, 2018, 10:35 AM
#18
Re: WebClient downloads only 1 GB, then fails & Round number to 1 decimal place
Yeah, given the context of this thread I'm thinking HttpClient won't really be available anyway.
Given that, taking the code in #5, I'd make this change:
Code:
Private Sub client_DownloadCompleted(...)
If Not e.Error Is Nothing Then
Dim actualException = e.Error
While actualException.InnerException IsNot Nothing
actualException = actualException.InnerException
End While
MessageBox.Show("There was an error downloading the file. Error: " & actualException.Message)
SW.Stop()
...
This will make sure it reports the lowest-level exception. If there's an InnerException at all, it's more likely to have useful information.
The next steps if that isn't illuminating are to try WebRequest/WebResponse and see if when you take lower-level control of the process you bypass the problem. Writing lower-level code is tougher, but usually gives you more insight into specific problems.
This answer is wrong. You should be using TableAdapter and Dictionaries instead.
-
Jan 4th, 2018, 06:16 AM
#19
Re: WebClient downloads only 1 GB, then fails & Round number to 1 decimal place
Just to satisfy my own curiosity have you tried not downloading the file to Temporary Projects, which I assume you are if running inside the IDE.
Code:
Private _activeClient As WebClient
Private Sub DoDownload()
Dim url As String = "http://...."
Dim directory As String = Environment.GetFolderPath(Environment.SpecialFolder.Desktop)
Dim fileName As String = IO.Path.GetFileName(url)
Dim path As String = IO.Path.Combine(directory, "dl", fileName)
IO.Directory.CreateDirectory(...
_activeClient = New WebClient()
AddHandler _activeClient.DownloadProgressChanged, ...
AddHandler _activeClient.DownloadFileCompleted, ...
_activeClient.DownloadFileAsync(.....
End Sub
Last edited by ident; Jan 4th, 2018 at 12:51 PM.
-
Jan 4th, 2018, 12:50 PM
#20
Re: WebClient downloads only 1 GB, then fails & Round number to 1 decimal place
Just Once again for my own curiosity. I admit I am grasping...
Code:
Imports System.Net
Public Class Form1
Private _activeClient As WebClient
Private Sub DoDownload()
.....
_activeClient = New WebClient()
_activeClient.Headers.Add(
"user-agent", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; .NET CLR 1.0.3705;)")
End Sub
End Class
-
Jan 27th, 2018, 07:47 AM
#21
Thread Starter
Junior Member
Re: WebClient downloads only 1 GB, then fails & Round number to 1 decimal place
Originally Posted by Sitten Spynne
Code:
Private Sub client_DownloadCompleted(...)
If Not e.Error Is Nothing Then
Dim actualException = e.Error
While actualException.InnerException IsNot Nothing
actualException = actualException.InnerException
End While
MessageBox.Show("There was an error downloading the file. Error: " & actualException.Message)
SW.Stop()
...
I added this to my code and now it's working! Maybe that error doesn't want to be revealed and rather works Otherwise, thanks for everybody's help!
-
Jan 27th, 2018, 02:13 PM
#22
Re: WebClient downloads only 1 GB, then fails & Round number to 1 decimal place
Originally Posted by Niya
I see no reason why that can't be implemented in VB.
I missed this when you posted it but you're right, the link was too close to the sentence about the new C# feature. I meant it to mean "Here's someone that did what I'm talking about in C#, a translation to VB should work"!
This answer is wrong. You should be using TableAdapter and Dictionaries instead.
-
Jan 28th, 2018, 06:34 AM
#23
Re: WebClient downloads only 1 GB, then fails & Round number to 1 decimal place
Originally Posted by Sitten Spynne
I missed this when you posted it but you're right, the link was too close to the sentence about the new C# feature. I meant it to mean "Here's someone that did what I'm talking about in C#, a translation to VB should work"!
Ah....Fair enough.
Posting Permissions
- You may not post new threads
- You may not post replies
- You may not post attachments
- You may not edit your posts
-
Forum Rules
|
Click Here to Expand Forum to Full Width
|