How to Store Large File

[quddusaliquddus]

Hi all ,
How could I store a large file such that I could send it in chuncks over the internet? What would be the most efficient method to use?

Your opinion is appreciated.

Thanks

Q

[jmcilhinney]

First up, this question has nothing specific to do with WPF, WCF or WF that I can see, so it doesn't belong in this forum. It seems best suited to the .NET A & D forum because it's just a general .NET question that's not specific to any language or platform. I've asked the mods to move the thread.

As for the question, how are you intending to transfer the file? Most likely you'll need to use a Byte array so you either need to read the file into a single Byte array to begin with or else use a FileStream to read it in chunks. If it's a large file then the latter is the most likely option. You simply open a file stream and read the file in chunks, writing out each chunk as you read it. When the amount of data read is smaller than what you asked fro you know the file is finished and you can close the stream.

[si_the_geek]

Thread moved to ".NET Architecture and Design" forum (thanks for letting us know jmc )

[quddusaliquddus]

Thanks. I thot it belonged under wcf - communication being the key word. But I wasnt sure. Thank you si the geek for moving it to the appropriate section.

jmcilhinney - could you please give me a link or some keyword I could look up to find out how to read chunks of a Byte Array? Thanks

[quddusaliquddus]

Is tis what you are talking about? [http://www.java2s.com/Tutorial/VB/02...reamLength.htm]

Imports System.IO


Public Class Tester
    Public Shared Sub Main
        Dim fs As System.IO.FileStream
        Dim r As System.IO.BinaryReader
        Dim buffer(100) As Char
        Dim mylength As Long
        fs = New System.IO.FileStream("test.txt", IO.FileMode.OpenOrCreate)
        r = New System.IO.BinaryReader(fs)
        mylength = fs.Length
        If mylength > 100 Then
            mylength = 100
        End If
        r.Read(buffer, 0, mylength)
        Console.WriteLine(buffer)
        r.Close()
        fs.Close()


    End Sub
End Class

[quddusaliquddus]

Hi. Im having a hard time adaptng the code from the link to copy a non-text file.

[quddusaliquddus]

I found an example in msdn which seems to work.

Would it be feasable to use a web service to transfer bytes from one computer to another?

Imports System
Imports System.IO

Class Window1

    Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.Windows.RoutedEventArgs) Handles Button1.Click

        ' Specify a file to read from and to create.
        Dim pathSource As String = "C:\Users\Me\Desktop\Downloaded Programs\Crawling.mid"
        Dim pathNew As String = "C:\Users\Me\Desktop\Crawling.mid"
        Try
            Using fsSource As FileStream = New FileStream(pathSource, _
                FileMode.Open, FileAccess.Read)
                ' Read the source file into a byte array.
                Dim bytes() As Byte = New Byte((fsSource.Length) - 1) {}
                Dim numBytesToRead As Integer = CType(fsSource.Length, Integer)
                Dim numBytesRead As Integer = 0

                While (numBytesToRead > 0)
                    ' Read may return anything from 0 to numBytesToRead.
                    Dim n As Integer = fsSource.Read(bytes, numBytesRead, _
                        numBytesToRead)
                    ' Break when the end of the file is reached.
                    If (n = 0) Then
                        Exit While
                    End If
                    numBytesRead = (numBytesRead + n)
                    numBytesToRead = (numBytesToRead - n)

                End While
                numBytesToRead = bytes.Length

                ' Write the byte array to the other FileStream.
                Using fsNew As FileStream = New FileStream(pathNew, _
                    FileMode.Create, FileAccess.Write)
                    fsNew.Write(bytes, 0, numBytesToRead)
                End Using
            End Using
        Catch ioEx As FileNotFoundException
            Console.WriteLine(ioEx.Message)
        End Try

    End Sub

End Class

[jmcilhinney]

Windows Communication Foundation (WCF) does not relate to communication in general. It is a specific technology for creating web services. If you're not creating a WCF web service then the WCF forum is not the place to post.

A web service requires a web server. Do you want to transfer data to a web server? If not then a web service is not really useful.

What's the relationship between the two machines you want to transfer data between? If we know that then we can suggest the best way to do it.

In general, reading a large file in chunks and transferring those chunks else where will look something like this:

Dim destination As NetworkStream 'How you get this stream is up to you, e.g. TcpClient.GetStream
 
Using source As New FileStream("file path here", FileMode.Open)
    Const BUFFER_SIZE As Integer = 1024 'Copy the data in 1 KB blocks.
 
    Dim buffer(BUFFER_SIZE - 1) As Byte
    Dim byteCount As Integer
 
    Do
        'Read the next block of data.
        byteCount = source.Read(buffer, 0, BUFFER_SIZE)
 
        'Write as many bytes as were read.
        destination.Write(buffer, 0, byteCount)
    Loop While byteCount = BUFFER_SIZE 'When the buffer is not filled we have reached the end of the file.
End Using

Note that you're only ever storing 1 KB of data in memory at a time.

[quddusaliquddus]

Thanks