Parsing large textfiles?

Hello, I'm trying to do something very basic, though the fact that the text file I'm using is around 152mb in size makes it a bit harder.

I'm trying to parse out HTML tags from the file. My Regex matching function tells me there are around 1100 HTML tags to take out (which isn't that many considering how big the file is), and the parsing function itself is multithreaded, though the program still freezes. I've even tried to compile it under x64, with no luck.

So, how can I effectively parse huge files like that without having my program freeze and not work? The problem is not with my threading or my parsing, it's with the size of the file.

How are you parsing the file?

Quote:

---

Originally Posted by billboy

How are you parsing the file?

---

Before I was just opening the file into a string then parsing the string and outputting it again. After some research I figured that I should try the same thing but line-by-line instead. It went a lot faster and didn't freeze, just trying to see if it worked properly or not.

Use stream reader, line by line should not freeze

This works for me, and I am reading text files that are hundreds of megabytes too.

vb.net Code:

Dim myFileReader As New StreamReader(ReadFilePath)
Dim myLine As String = ""
myLine = myFileReader.ReadLine()