Printable View
I'm using both streamreader and writer in attempt to read in a specially formatted log file that must the be reformatted to a much larger size file(factor 200X) and stored back to disk. The problem is the amount of time the file takes to write which is about 200 bytes for every byte I read in and each log file is around 4Mbs text. I need a quicker way to write all the data out to disk.
You could make the output file's buffer larger, that would do fewer write operations but make each one larger, this speeds things up a lot. The argument for setting the buffer size is in the FileStream constructor.
For writing large files I tend to use buffers of more than 256kb ( 256*1024 bytes ).
And it also depends how many bytes you are reading in at a time. If you are reading in a single byte at a time then you will be badly slow. Read a load of bytes in and then process them all in memory in one go instead of moving one byte at a time.
I've tried changing the buffersize in the constructor which seems to knock off a couple of seconds for every 100k written. Here is my current code snip with your my take on your suggestion. Is this what you meant?
Dim File As Boolean
Dim cRaw(102400) As Char
Dim cFormat(102400) as Char
Dim iValue As Integer
File = True
Try
Dim fsr As FileStream = New FileStream(filename, FileMode.Open, FileAccess.Read, FileShare.None, 102400)
Dim sr As StreamReader = New StreamReader(fsr)
Dim fsw As FileStream = New FileStream("cough.txt", FileMode.Create, FileAccess.Write, FileShare.None, 524288)
Dim sw As StreamWriter = New StreamWriter(fsw)
Do
iValue = sr.Read(cRaw, 0, 102400)
sw.Write(cRaw)
If iValue = -1 Then
iValue = 0
End If
Loop While iValue
sr.Close()
sw.Close()
MsgBox("Conversion Complete")
Catch ex As Exception
File = False
End Try