Results 1 to 12 of 12

Thread: max size of binary files to split

  1. #1

    Thread Starter
    New Member
    Join Date
    Jan 2018
    Location
    Germany - Westerwald
    Posts
    15

    Question max size of binary files to split

    Hello,
    I use this code to merge and seperate my binary files on windows 10 computers.
    For years the filessize was up to 5 MB and it work stable. Since I use files larger then 100 MB there are problems on different computers.
    In the ide I see in GetBinFiles after sBuffer = Space$(nLen): sBuffer = <not enought memory> and Get #F, nStart, sBuffer run into error 7 'not enought memory'.
    Is there max file-size or limits in windows or hardware to do this?

    Code:
    Sub extractFile5()
    ' sF: source Filename
    ' sZipMiPF: destinaton Filename
    ' lngStart: recnumber to start
    ' nLen5: Size of the file to be extracted
      Call SaveBinFile(sZipMiPF, GetBinFile(sF, lngStart + 1, nLen5), 1)
    End Sub
    Sub matchFile5()
    ' sF: destinaton Filename
    ' sZipMiPF: source Filename
    ' lngStart: recnumber to start
    ' nLen5: Size of the file to be extracted
      Call SaveBinFile(sF, GetBinFile(sZipMiPF), lngStart)
    End Sub
    
    Function GetBinFile(ByVal sFilename As String, Optional ByVal nStart As Long = 1, Optional ByVal nLen As Long = 0)
    Dim F As Long, sBuffer As String
    On Error Resume Next
      If nLen > 0 Then
        sBuffer = Space$(nLen)
      Else
        sBuffer = Space$(FileLen(sFilename))
      End If
      If Err.Number = 0 Then
        F = FreeFile
        Open sFilename For Binary As #F
        Get #F, nStart, sBuffer
        Close #F
      End If
    On Error GoTo 0
      GetBinFile = sBuffer
    End Function
    
    Sub SaveBinFile(ByVal sFilename As String, ByVal sBuffer As String, ByVal nStart As Long)
    Dim F As Long
    On Error Resume Next
      F = FreeFile
      Open sFilename For Binary As #F
      Put #F, nStart, sBuffer
      Close #F
    End Sub

  2. #2
    PowerPoster Arnoutdv's Avatar
    Join Date
    Oct 2013
    Posts
    5,097

    Re: max size of binary files to split

    If the files are really binary files then don't use strings to store them, but use byte arrays.
    A string is a unicode representation of data, 2 bytes per 'character' . A byte array has just a single byte per element.
    This will reduce the memory needed by 2.

  3. #3
    Frenzied Member
    Join Date
    Dec 2014
    Posts
    1,799

    Re: max size of binary files to split

    Code:
    Dim Data() as Byte
    Open Filename For Binary Access Read As #FF
    TSize = FileLen(Filename) - 1
    ReDim data(TSize)
    Get #FF, , data
    Close #FF
    Code:
    Open Filename  For Binary Access Write As #FF
    Put #FF, , data
    Close #FF

  4. #4

    Thread Starter
    New Member
    Join Date
    Jan 2018
    Location
    Germany - Westerwald
    Posts
    15

    Re: max size of binary files to split

    Thanks Arnoutdv and baka.
    With your help, it's working fine now.
    @baka: A question of understanding: why the -1 at TSize?

  5. #5
    Frenzied Member
    Join Date
    Dec 2014
    Posts
    1,799

    Re: max size of binary files to split

    because you are Redim data(0 to size) and 0 is also used.
    if you would redim data(1 to size) u dont need to -1

  6. #6

    Thread Starter
    New Member
    Join Date
    Jan 2018
    Location
    Germany - Westerwald
    Posts
    15

    Resolved Re: max size of binary files to split

    Thanks for your patience

  7. #7

    Thread Starter
    New Member
    Join Date
    Jan 2018
    Location
    Germany - Westerwald
    Posts
    15

    Question Re: max size of binary files to split

    Hi,
    I need to come back to this topic.
    The zip files need to be stored in an Access database. Currently there is a DAO field type memo to save them.
    Memo (data type): A field data type. Such fields can contain up to 1.2 GB of text data.
    But VB crashes at files > 50MB in the line !BinMPic = StrConv(Data(), vbUnicode).
    How can I add this data or store it in the database?

    Code:
          Open sFileName For Binary Access Read As #F
          ReDim Data(1 To nLen)
          Get #F, nStart, Data
          Close #F
          If IsArray(Data) Then !BinMPic = StrConv(Data(), vbUnicode)

  8. #8
    PowerPoster Arnoutdv's Avatar
    Join Date
    Oct 2013
    Posts
    5,097

    Re: max size of binary files to split

    Then again the StrConv creates a string double the size of the byte array.
    And storing >100MB strings in a memo field, why??

  9. #9
    PowerPoster Zvoni's Avatar
    Join Date
    Sep 2012
    Location
    To the moon and then left
    Posts
    3,066

    Re: max size of binary files to split

    Why not use an ADO-Stream-Object (Put DAO where it belongs: The garbage-heap)?
    And use LONGBINARY instead of Memo --> http://allenbrowne.com/ser-49.html
    Last edited by Zvoni; Jan 18th, 2022 at 03:38 AM.
    One System to rule them all, One Code to find them,
    One IDE to bring them all, and to the Framework bind them,
    in the Land of Redmond, where the Windows lie
    ---------------------------------------------------------------------------------
    People call me crazy because i'm jumping out of perfectly fine airplanes.
    ---------------------------------------------------------------------------------
    Code is like a joke: If you have to explain it, it's bad

  10. #10

    Thread Starter
    New Member
    Join Date
    Jan 2018
    Location
    Germany - Westerwald
    Posts
    15

    Resolved Re: max size of binary files to split

    @Arnoutdv: Sorry (ashamed), with my programming knowledge only fish very close to the surface. It is always only small extensions where I work around.

    Thanks for your suggestions

    Have a good week

  11. #11
    PowerPoster Arnoutdv's Avatar
    Join Date
    Oct 2013
    Posts
    5,097

    Re: max size of binary files to split

    Just out of curiosity, what kind of data of 50MB do you store in your database?

  12. #12
    Super Moderator Shaggy Hiker's Avatar
    Join Date
    Aug 2002
    Location
    Idaho
    Posts
    37,332

    Re: max size of binary files to split

    It might be worth considering storing the files in one place, and storing the path to the file in the database.
    My usual boring signature: Nothing

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  



Click Here to Expand Forum to Full Width