-
Reading large files fast
Hello everyone,
I'm trying to get the the Hashcode of a file to look whether it corrupted or not. The problem is, that the file reading method that i use is atrociously slow (especially with files over 50mb) :(
Code:
StreamReader DataFile = new StreamReader(
(System.IO.Stream)File.OpenRead("somefile.any"),System.Text.Encoding.ASCII);
string Checksum = DataFile.ReadToEnd().GetHashCode().ToString();
could anyone here please suggest me something that would be faster? Would a loop maybe faster? I know it can be faster, but how?
Tank you very much in advance!
-
Check out the bufferstream() class. This class is optimized to provide the most efficient file access.
-
Ok, I got that, but somehow the checksum does only change if i change the file length :/.
Could you please show me an example of how to read the whole file into a buffer and get the HashCode? Thanx! :p
-
How about this?
Code:
using System;
using System.IO;
public class FileAccessExample
{
public static void Main()
{
string path = @"c:\test.txt";
BufferedStream buff = new BufferedStream (File.Open(path, FileMode.Open));
Console.WriteLine(buff.GetHashCode());
}
}
-
Code:
StreamReader sRetr = File.OpenText(PATH);
while (sRetr.Peek() > -1)
{
sData = sData + sRetr.ReadLine();
}
Try reading it line by line as above. I don't know if this will help but I suppose its worth a try!
-toto
-
First let me thank you for your help,
Basically it would work to read each line. But, if the file is VERY large, then it will lterally never work :(.
The problem with the buffered string was, that it does not compute the HashValue of the whole file. I need every byte in the calculation. Well now I came to the conclusion I could take a Hash from each line, and not adding every line into one variable.
Then I could add the checksums of every line, which schould be far smaller and create a checksum of the single ones :).
This is a pretty difficult thing i guess.
Maybe it would be possible to use
MD5CryptoServiceProvider().ComputeHash(ASCIIEncoding.ASCII.GetBytes(args));
with a filename? *g*
I got the idiea for the program from one called RetroSFV, it can create checksums for 1gig files within 30secs. I guess it's written in c++.
I would be happy for further support, but you've probably already heated your heads too much with this
:D