What is the equivalent of "HasFieldsEnclosedInQuotes" in cCSV

[tmighty2]

The following works for me (VB.NET):

        Dim sPath As String = "d:\test.csv"

        Dim iCount As Integer = 0

        Using afile As FileIO.TextFieldParser = New FileIO.TextFieldParser(sPath, System.Text.Encoding.GetEncoding(1252))

            Dim CurrentRecord As String() ' this array will hold each line of data
            afile.TextFieldType = FileIO.FieldType.Delimited
            afile.Delimiters = New String() {","}
            afile.HasFieldsEnclosedInQuotes = False

            ' parse the actual file
            Do While Not afile.EndOfData

                iCount += 1

                If iCount = 1684 Then
                    Stop
                End If

                CurrentRecord = afile.ReadFields


                Dim s As String = ""
                s = CurrentRecord(0)

            Loop

The following does not:

    Set mCsv = New_c.CSV
    mCsv.ExpectedEncoding = CP_UTF8
    mCsv.ReplaceDoubledQuotChars = False
    mCsv.ColSepChar = ","
    mCsv.ParseFile sPath, Me

There seems to be a problem with quotes, but I don't see how to make RC6's cCSV behave like the other parser.

I have attached a simple VB6 sample project. Please add RC6 as a reference to test it.

Thank you!


ps: The problematic string is this (I have anonymized it by replacing letters, but apart from that, that is the original string:

20230210131552m3fhrarar8r844522hr43fa579fahr0p0mm,marmLm_PmmH,aaa hrpplmwhrrlm,aaa hrpplmwhrrlm,Frm Fmr 10 13:15:52 maw 2023,aaa mmaa,aa amlmvmry whrm mhram amm aa wmmaammr prmfmrmawmm apa ama.,"{""raaywaaamaa"":""mhr hrmzaa Phrkm mma Pmxx Pa-212 am, mmaahrmahramr maa mm ma,... wmram zmgmmamlla."",""rmaaaawaaamaa"":""Hier klicken"",""amm


And here is the "too much" column contents:

"rmaaaawaaamaa"":""Hier klicken"

[Zvoni]

The following works for me (VB.NET):

The following does not:

    Set mCsv = New_c.CSV
    mCsv.ExpectedEncoding = CP_UTF8
    mCsv.ReplaceDoubledQuotChars = False
    mCsv.ColSepChar = ","
    mCsv.ParseFile sPath, Me

You did see, that the Property is called REPLACEDoubledQuotChars?

And you set it to False, so "do not replace double quoted chars..."

hmm.... what could that mean.....

[tmighty2]

That was just the current try of settings that I had.
And I only did that because nothing else worked. I have tried all combinations available, I think.

[Zvoni]

Then we might have to wait for Olaf, since, apparently, you are using RC5/RC6

The only difference i see is CP1252 vs UTF8

[Schmidt]

You have to ensure valid CSV-format (regarding text-quoted columns)...

The error in the CSV is in Line #2 (the line which ends up with: ...730p0mm.mml)
where you forgot to put a single closing dblquote for that (text)-Column-Value: ...730p0mm.mml"

If you fix that, the cCSV-parser should do just fine.

Why the .NET-parser was choosing, to interpret this erroneus CSV as:
just 1 Record with just one "huuge" last (8.)column (instead of the correct 2 Records this file contains), I don't know...

Aside from that - I wouldn't place such huge JSON-Strings in CSV-cells directly,
because there's a lot of CSV-parsers out there, which would choke on such content.
Better to serialize such JSON-strings into an UTF8-ByteArray, which you then Base64-encode (via New_c.Crypt).

Anyways, here's a little Demo, where I tried to stress the cCSV-Class with (smaller) JSON-content a bit -
and everything worked as it should... just don't mess up the column-quoting (along with internal Doubling of already existing quote-chars).

Option Explicit

Implements ICSVCallback
Private mCsv As cCSV
 
Private Sub Form_Load()
  Dim S(0 To 1) As String
      S(0) = "'a', 24 lbs, Monitor 17', '[''arrval1'', null, 123, ''arrval2'']'"
      S(1) = "'b', 42 lbs, Monitor 21', '{''key1'':''value1\''...'',''key2'':''value2''}'"

  Debug.Print Replace(Join(S, vbLf), "'", ChrW$(34))

  Set mCsv = New_c.CSV
      mCsv.ExpectedEncoding = CP_UTF8
      mCsv.ParseBytes New_c.Crypt.VBStringToUTF8(Replace(Join(S, vbLf), "'", ChrW$(34))), Me
End Sub

Private Function ICSVCallback_NewValue(ByVal RowNr As Long, ByVal ColNr As Long, b() As Byte, ByVal BValStartPos As Long, ByVal BValLen As Long) As Long
  Dim S As String
      S = mCsv.GetStringValue(b, BValStartPos, BValLen)
  Debug.Print RowNr, ColNr, S

  If ColNr = 3 Then Debug.Print New_c.JSONDecodeToCollection(S).ItemByIndex(0)
End Function

Olaf

[tmighty2]

Hello Olaf,

thank you.

I will pass your remark on to the big online company who provided it to me.

Thank you!