Memory error when zipping a file

[InvisibleDuncan]

I’m getting an intermittent memory error when trying to compress a text file into a zip file. I use the standard .Net packaging component:

Imports System.IO.Packaging

    Private Sub ZipFile(ByVal zipFrom As String, ByVal zipTo As String)

        Try
            '# Open the zip file if it exists, otherwise create a new one.
            Dim Zip As Package = ZipPackage.Open(zipTo, IO.FileMode.OpenOrCreate, IO.FileAccess.ReadWrite)
            '# Add as many files as you like:
            AddToArchive(Zip, zipFrom)
            '# Close the zip file.
            Zip.Close()
            Zip = Nothing

        Catch ex As Exception
            Log.WriteLog(ex)
            Throw New ApplicationConfig.HandledException("Already Handled", ex)
        End Try

    End Sub

    Private Sub AddToArchive(ByVal zip As Package, ByVal fileToAdd As String)

        '# A Uri always starts with a forward slash "/" 
        Dim ZipUri As String = String.Concat("/", Path.GetFileName(fileToAdd))

        Dim PartUri As New Uri(ZipUri, UriKind.Relative)
        Dim ContentType As String = Net.Mime.MediaTypeNames.Application.Zip

        '# The PackagePart contains the information: 
        '# Where to extract the file when it's extracted (partUri) 
        '# The type of content stream (MIME type):  (contentType) 
        '# The type of compression:  (CompressionOption.Normal)   
        Dim PkgPart As PackagePart = zip.CreatePart(PartUri, ContentType, CompressionOption.Normal)

        '# Read all of the bytes from the file to add to the zip file.
        Dim FileContents As Byte() = File.ReadAllBytes(fileToAdd)

        '# Compress and write the bytes to the zip file.
        PkgPart.GetStream().Write(FileContents, 0, FileContents.Length)

        '# Clear the variables from memory.
        PkgPart = Nothing
        FileContents = Nothing

    End Sub

The error is happening somewhere within the AddToArchive routine, and is a System.OutOfMemoryException error.

This particular example happened while trying to zip a file of just over 300mb. However, I've had other occasions where it has zipped files larger than this, and others where it has fallen over on smaller (although still very large) files. I can also always zip the files manually on the server that runs this process.

Does anyone have any suggestions of what I can do to prevent this error from happening? I did try a slightly different approach, but I got the errors more frequently using this:

Imports System.IO.Packaging

    Private Sub CreateZipPackage(ByVal zipFrom As String, ByVal zipTo As String)
        '# Convert system path and file names to Part URIs. 
        Dim ZipUri As String = String.Concat("/", IO.Path.GetFileName(zipFrom))
        Dim partUriDocument As Uri = PackUriHelper.CreatePartUri(New Uri(ZipUri, UriKind.Relative))

        '# Create the Package
        '# If the package file already exists, FileMode.Create will automatically delete it first before creating a new one.
        Using package As Package = package.Open(zipTo, FileMode.Create)
            '# Add the Document part to the Package
            Dim packagePartDocument As PackagePart = package.CreatePart(partUriDocument, System.Net.Mime.MediaTypeNames.Application.Zip, CompressionOption.Normal)

            '# Copy the data to the Document Part
            Using fileStream As New FileStream(zipFrom, FileMode.Open, FileAccess.Read)
                CopyStream(fileStream, packagePartDocument.GetStream())
            End Using '# Close and dispose fileStream.

        End Using '# Close and dispose package.

    End Sub

    Private Shared Sub CopyStream(ByVal source As Stream, ByVal target As Stream)
        '# Copy data from a source stream to a target stream.
        Const bufSize As Integer = &H1000
        Dim buf(bufSize - 1) As Byte
        Dim bytesRead As Integer = 0
        bytesRead = source.Read(buf, 0, bufSize)
        Do While bytesRead > 0
            target.Write(buf, 0, bytesRead)
            bytesRead = source.Read(buf, 0, bufSize)
        Loop
    End Sub

Any suggestions would be gratefully received.

[Niya]

What line does the error occur on in both cases ?

[InvisibleDuncan]

I can't remember where it happened on the second version I showed (I had to dig that old version of the code out of TFS), but on the first one it's happening on:

PkgPart.GetStream().Write(FileContents, 0, FileContents.Length)

[Niya]

My first guess would be that you are using large byte arrays. Since streams are involved you should use smaller byte arrays and write incrementally to the stream instead of trying to write all the data at once. Make sure to dispose of streams when you're done using them.

[InvisibleDuncan]

Thanks, Niya. I'll try to work out how to do that and give it a bash.