[RESOLVED] multiple range checking algorithm vb6

[flyguille]

It sure *is* faster than your Lookup-Routine (which I've written according to your "instructions").

If you think you can do better than that, I'd surely want to see some code from *you*.
What we have currently, regarding your approach is only a "paper-tiger", there's no Copy&Pastable
demo - or even better - a zipped Demo-Project from you anywhere.

I for my part would surely like to see, how you fill up that huge Lookup-Array of yours properly.
To make it somewhat easier for you, I've now extracted the 88314 US-RangeRecords from the
DataSet my larger Demo is using, and one can download it here:
http://vbRichClient.com/Downloads/USRangesDetailed.zip

It contains a normal CSV-File (USRangesDetailed.csv, with IPfrom,IPto per Line) -
but also a Raw-CurrencyArray-File, called USRangesDetailed.bin, which one can load this way:

Private USRanges() As Currency

Sub InitRangeArray(FileName As String)
  ReDim USRanges(0 To FileLen(FileName) \ 8 - 1)
  Dim FNr As Long: FNr = FreeFile
  Open FileName For Binary Access Read As FNr
    Get FNr, , USRanges
  Close FNr
End Sub

After the Init-Routine above was called, USRanges has an Ubound of 88314*2-1 and contains the sorted entries of the
88314 US-RangeRecords as numeric values, the IPFrom-entries sitting at the even Indexes, the IPTo-entries at the odd IndexPositions.
(mainly thought for those who want to try themselves at a Binary-Search-based algorithm against that Array).

Would surely like to see some code from you, which got the Lookup-Arrays filled with that detailed set of US-Ranges
(your choice, whether you read the data from the *.csv or the *.bin-file).


I've mentioned that because my code is faster and takes less memory *despite* offering full country-resolution...

If I reduce the Size of my DataSet to "US-Ranges only", then I'd expect results far better than 1 µsec per IPCheck-call.


Nothing holds you back to reformat it to your liking, it's not that much code after all... <shrug>

As for understanding it though, ...from how you post and interact with other developers (ignoring proof delivered by code) -
I'd think you are still quite young and inexperienced (comparably), so don't expect "immediate results" just by looking at it...

Olaf

nope, but have better things to do... calm down, it is just an ideas exchanges!.

EDIT: which have better things to do, is why I didn't bother in creating a zip with a vb project inside. Or timing it.

I don't know about you, but I am 41yo, with commercial software published and commercializing it, supporting it, with sells and marketing ppl, oh and a graphics designer just hired a month ago....

Anyway, programmer since 10yo..., doing video games in z80 and MSX BASIC since then...., spanish guy, so my english is not perfect.

But all your asumptions, about young, and unexperienced was wrong..., maybe tomorrow as holiday, I can pick or reformat your code to make it more readable and then try to understand why you claim it be fast include it having LOOPS, and FOR/NEXT , to do the IPCHECK, because I see those loops, don't understand how your code can be faster.

But sure, if country flag as return, and/ better, currency flag, sure it is better, more featureful. I am just not sure about faster.

[Schmidt]

But all your asumptions, about young, and unexperienced was wrong...,

Nope, not really (and I wrote "comparably", if you care to look that up again).
I'm developing in C/C++ and VB6 for quite a bit longer than you (I'm in my fifties, but many Forum-Users here top that easily),
so you are (comparably) a "youngster" still.

maybe tomorrow as holiday, I can pick or reformat your code to make it more readable and then try to understand why you claim it be fast...

I dont "claim" it to be fast - it *is* fast and can be tested by anybody who cares to download, compile and run the Zip I've posted.
I can only encourage you, to post your own approach in a fully working code-example, so that *other* developers can look,
test and compare for themselves - currently it is only you who "claims" something (no real demo anywhere, just talking).

But as it seems currently, you will chicken out, not posting anything (which is a common pattern with challenged - ermm... "youngsters").

... it having LOOPS, and FOR/NEXT , to do the IPCHECK, because I see those loops, don't understand how your code can be faster.

That sentence alone shows, how inexperienced you are with performance-related things in High-Level-Languages.
Here it is often the longer routines which are faster than the "conveniently written High-Level-OneLiners".

I am just not sure about faster.

Well, I am - because I *timed* it already (and I usually try very hard, not to give wrong info in a Public Forum).

Olaf

[flyguille]

Nope, not really (and I wrote "comparably", if you care to look that up again).
I'm developing in C/C++ and VB6 for quite a bit longer than you (I'm in my fifties, but many Forum-Users here top that easily),
so you are (comparably) a "youngster" still.


I dont "claim" it to be fast - it *is* fast and can be tested by anybody who cares to download, compile and run the Zip I've posted.
I can only encourage you, to post your own approach in a fully working code-example, so that *other* developers can look,
test and compare for themselves - currently it is only you who "claims" something (no real demo anywhere, just talking).

But as it seems currently, you will chicken out, not posting anything (which is a common pattern with challenged - ermm... "youngsters").


That sentence alone shows, how inexperienced you are with performance-related things in High-Level-Languages.
Here it is often the longer routines which are faster than the "conveniently written High-Level-OneLiners".


Well, I am - because I *timed* it already (and I usually try very hard, not to give wrong info in a Public Forum).

Olaf

the only thing I can thinks, about why my methods, according to you is slower than yours, is that as it is 512MB lookup table, and tested with IP ramdonly generated, maybe MAYBE it is being penalty by the lookup table not being in the CPU's L1/L2 CACHÉ. Other way, how a simple array lookup will be slower than LOOPEd functions in a binary search.

You told that mine approach is 3us per IP check?

that is like a lot for modern CPUs, like 333K of runs per second. 333000, .hmmm there is a penalty somewhere.

I will give yours and mine a try... btw, don't forget it is just an idea exchange, nothing personal. So, take it easy!.

[Schmidt]

...the only thing I can thinks, about why my methods, according to you is slower than yours, is that as it is 512MB lookup table,

No, although such a huge "en-bloc" memory-allocation is generally a bad idea,
it is not the array-access which is slow.

Other way, how a simple array lookup will be slower than LOOPEd functions in a binary search.

As pointed out to you already several times - it is not the final array-acces which is slow -
it is all the operations which come *before* the array-access (preparing the correct Indexes) which make it slow.

You told that mine approach is 3us per IP check?

Yes - why didn't you test this already?
I've posted code for that.

To write it out more clearly again:
When you have a 4-dimensional LookupArray you will have to provide it with 4 Indexes in the end (to perform the array-read)
(let's call these Indexes a,b,c,d - and assume that they are explicit Integer-Variables at Function-level)...

So, before you do your final (and fast) Lookup as: FunctionResult = LookupArr(a, b, c, d)
you have to calculate the values of a, b, c and d (starting with everything you do on the Input-Param, since entering the Function).
That's where the 3µsec are caused in your case.
- Split (as a super-high-level-function) is of course "slower" than
- a few Instr/Mid$ calls (which are themselves higher-level-functions of the vbRuntime),
- which again are slower than a direct looping over a ByteArray (which avoids any calls into the vbRuntime)

That this is "surprising" to you baffles me a bit...
(giving your background in ASM, one would think that you know how compilers work -
and how they interact with a given runtime-library which provides certain language-features).

...btw, don't forget it is just an idea exchange, nothing personal. So, take it easy!.

Well, if you'd put yourself into my shoes for a moment, you'd see that my "getting impatient" is explainable
(given the fact, that I'm basically repeating myself for a few postings already).

As for testing (and learning) - here's another example I've put together,
which does a direct Binary-Search on a Currency-Array for only the US-Range
http://vbRichClient.com/Downloads/IP...USDetailed.zip (not requiring a vbRichClient5-reference in your project).

Due to the reduced range of records (only ~88000 for the US), it is of course faster than the "all countries version" I've posted earlier -
now needing only about 0.12 µsec per IPCheck-FunctionCall (when native compiled).

All of this "performance-discussion" is (whilst hopefully being educational) of course largely academic for the case of the OP.
Any approach which delivers the result after 100 µsec or so, would be entirely sufficient I'd say.

Olaf

[flyguille]

No, although such a huge "en-bloc" memory-allocation is generally a bad idea,
it is not the array-access which is slow.


As pointed out to you already several times - it is not the final array-acces which is slow -
it is all the operations which come *before* the array-access (preparing the correct Indexes) which make it slow.


Yes - why didn't you test this already?
I've posted code for that.

To write it out more clearly again:
When you have a 4-dimensional LookupArray you will have to provide it with 4 Indexes in the end (to perform the array-read)
(let's call these Indexes a,b,c,d - and assume that they are explicit Integer-Variables at Function-level)...

So, before you do your final (and fast) Lookup as: FunctionResult = LookupArr(a, b, c, d)
you have to calculate the values of a, b, c and d (starting with everything you do on the Input-Param, since entering the Function).
That's where the 3µsec are caused in your case.
- Split (as a super-high-level-function) is of course "slower" than
- a few Instr/Mid$ calls (which are themselves higher-level-functions of the vbRuntime),
- which again are slower than a direct looping over a ByteArray (which avoids any calls into the vbRuntime)

That this is "surprising" to you baffles me a bit...
(giving your background in ASM, one would think that you know how compilers work -
and how they interact with a given runtime-library which provides certain language-features).



Well, if you'd put yourself into my shoes for a moment, you'd see that my "getting impatient" is explainable
(given the fact, that I'm basically repeating myself for a few postings already).

As for testing (and learning) - here's another example I've put together,
which does a direct Binary-Search on a Currency-Array for only the US-Range
http://vbRichClient.com/Downloads/IP...USDetailed.zip (not requiring a vbRichClient5-reference in your project).

Due to the reduced range of records (only ~88000 for the US), it is of course faster than the "all countries version" I've posted earlier -
now needing only about 0.12 µsec per IPCheck-FunctionCall (when native compiled).

All of this "performance-discussion" is (whilst hopefully being educational) of course largely academic for the case of the OP.
Any approach which delivers the result after 100 µsec or so, would be entirely sufficient I'd say.

Olaf

I understand you, just I didn't know split() & VAL() was so slow. Include with in-stack parameter parsing.

Just asking about VB6.

Is there a penalty about calling a VBA functions? than a call to a function within module , or a class module in the VB project? I saw you are using a class aswell, because my knowledge is that both are __STDCALL. Unless things like NEAR / FAR CALL still exists in VB6/modern O.S.,.... but IIRC it not more exists in 64bits/32bits non paged memory mapping, right?.

Again, I thinks, IF TESTED with ramdomly IP CHECKS, 512MB don't fits in L1 or L2 cache, and every time the CPU must read the DIIMMs. Which is a penalty, as well as at LOADING TIME, to allocate 512MB you are right, is a bit overkill for loading time. Those two things are the CONS.

Now, may I will use your method, it comes handy, but right now, not time to check.

Again I didn't tested, because having a huge inbox stack right now, and I don't have time, just chat around this VBforum for rest a bit. I has an ETA of 2 months.