Faster Code Needed

[Donkeyxote]

I have a simple sub within a project that is executed about 2 or 3 x 10^6 times during processing of data stored in txt files. It basically does something like the following.

StrArray = Split(Str, ",") - Line results in StrArray with a length of about 12
RetVal1 = CINT(StrArray(0))
RetVal2 = CDBL(StrArray(1))
RetVal3 = DateTime.ParseExact(StrArray(2), "FormatStr", CulturalInfo)
...and a few more lines like the last three. I'm going by memory, but there isn't much more than that really.

The values Ret1, Ret2(...) are ByRef variables that are not reinitiated within the sub every time it is called.

That's about it. The result is a couple minutes of processing time, which is beyond my patience reserves. The Split() function seems to take the most amount of time.

I can't think of any way of speeding this up other than creating a non-managed DLL using C++. Would that be substantially quicker? I haven't used C++ much (the last version I ever used was Borland 2.0) but I imagine I could figure out some simple code like this. Would it be much of an improvement? Is there a better/easier way to speed this up?

Thanks.

[.paul.]

you could probably speed it up with LINQ, working with the entire file would be quicker

[techgnome]

hhmmmmm you're splitting it up on a comma.... I'm guessing it's a CSV text file of some kind? How are you reading the file?
You might be able to read the file into a DataTable instead.... should be faster than doing all that string manipulation.

-tg

[Shaggy Hiker]

I think you can attach to that like a database and suck the whole thing into a datatable in one step. I forget whether I have tried it, but I have seen something about that. I am doing something in one project where I read in CSV files, but I forget how I was doing it. I suspect that it wasn't all that fast, as the files were kind of unusual. If you are going for speed, I would definitely be looking at querying the file.

[techgnome]

yup pretty much that's the way it works.... get the connection string right and it'll suck it right in. Done it before, works great. They have the connectionstring over at connectionstrings.com ... I'm pretty sure that's where I got it. The down side is that the fields aren't typed, I think they stay as strings. Never tried it with a typed DT though... not sure if that would work.

-tg

[Shaggy Hiker]

Actually, the fields should be type Object, which is all you get for the untyped Datatable object.