
Originally Posted by
axisdj
just wanted to let Olaf know I figured it out mostly.
Thanks again for the guidance.
Sorry for posting that late (shifted it "down my stack", because I didn't find the CSV-parsing-code anymore,
which produced the Binary for the LZMA-Blob, guess I've deleted it "for good" whilst "cleaning up" the original example).
Here again a fresh version (into a Class, named e.g. cUpdateFromCSV )
Code:
Option Explicit
Implements ICSVCallback
Private CSV As cCSV, IPs() As Byte, Countries() As Byte
Public Sub ConvertToBinaryFile(CSVFileName As String, BinFileName As String)
ReDim IPs(4 * 10 ^ 6) 'Space for ~1Mio records (currently used will be only about half as many)
ReDim Countries(2 * 10 ^ 6) 'same here (though the country-info will be only 2bytes per record)
Set CSV = New_c.CSV
CSV.ParseFile CSVFileName, Me
ReDim Preserve IPs(4 * CSV.RowsParsed - 1)
ReDim Preserve Countries(2 * CSV.RowsParsed - 1)
With New_c.FSO.CreateFileStream(BinFileName)
.WriteFromByteArr IPs 'write the IP-Bytes first into the stream (for better compressability,...
.WriteFromByteArr Countries '... they are followed by all the country-bytes in a separate block)
End With
End Sub
Private Function ICSVCallback_NewValue(ByVal RowNr As Long, ByVal ColNr As Long, B() As Byte, ByVal BValStartPos As Long, ByVal BValLen As Long) As Long
Select Case ColNr
Case 0 'Start-IP
Static S As String, SArr() As String
S = CSV.GetStringValue(B, BValStartPos, BValLen)
If InStr(S, ":") Then ICSVCallback_NewValue = 1: Exit Function 'stop parsing, when we reach the IPV6-section
SArr = Split(S, ".")
IPs(4 * RowNr + 3) = SArr(0): IPs(4 * RowNr + 2) = SArr(1): IPs(4 * RowNr + 1) = SArr(2): IPs(4 * RowNr + 0) = SArr(3)
Case 2 'Country-Code (we can simply copy the two Bytes over directly from the Input-Stream, no string-conversion needed)
Countries(2 * RowNr + 0) = B(BValStartPos)
Countries(2 * RowNr + 1) = B(BValStartPos + 1)
End Select
End Function
Usage then (after downloading the monthly updated CSV-Files from: https://db-ip.com/db/download/country
Code:
With New cUpdateFromCSV
.ConvertToBinaryFile App.Path & "\dbip-country-2017-09.csv", App.Path & "\IPRangesByCountry.bin"
End With
The Inital-Reading-Routine could be adapted to support both (LZMA-compressed and uncompressed formats) over the FileEnding like this:
Code:
Public Sub InitRangeDictionaryFrom(FileName As String)
Set IPRanges = New_c.SortedDictionary
Dim BSrc() As Byte, B() As Byte, C As Long, i As Long, K As Currency
If LCase$(Right$(FileName, 4)) = "lzma" Then 'decompress the ByteContent from LZMA first
BSrc = New_c.FSO.ReadByteContent(FileName)
New_c.Crypt.LZMADeComp BSrc, B
Else 'we assume uncompressed, raw binary format and read that into B directly
B = New_c.FSO.ReadByteContent(FileName)
End If
C = (UBound(B) + 1) \ 3
For i = 0 To C - 1 Step 2
K = B(i + i) + 256& * B(i + i + 1) + 65536 * B(i + i + 2) + 16777216@ * B(i + i + 3)
IPRanges.Add K, B(C + C + i) + 256& * B(C + C + i + 1) 'add the Country-Item under Key K
Next
End Sub
HTH (and yes, all your other assumptions were quite correct).
Olaf