Copying from locked file and using chunks with files > 2 GB

[tmighty2]

Hello!

I am looking for a way to quickly copy files with a filesize of over 2 GB which are currently locked (for example a SQLite database).

Purpose:

- Copy a sqlite db while it is in use
- Allow the copying app to still respond
- Allow copying files of over 2 GB
- Be reasonably fast

The only function which fullfills most of the demands is the following

Dim lRet As Long
lRet = CopyFileW(StrPtr(uSource), StrPtr(uDestination), True)

The problem is that it is blocking. I would have to make this call out of process which I have so far shied away from.

Is anybody aware of other options?

[VanGoghGaming]

Easiest method without creating another thread is to run your own executable (with ShellExecute) and provide a command line parameter that will initiate the copy process. When it's done it could signal back to the main process with SendMessage.

[fafalone]

CopyFileExW with a progress routine, if you just need to be able to cancel and update the UI.

The trick's multithreading module and doing it in another thread.

Or doing it manually with ReadFile/WriteFile and overlapped i/o, or ReadFileEx/WriteFileEx with a file io completion port.

[Elroy]

Here's a post where I wrote an application to deal with >2GB files. It leverages some work done by Dilettante to do this.

However, I'm not sure, in the HugeBinaryFile_Input, when calling "OpenFile", if it's already open by another process, what's going to happen. It may error.

You may want to include FILE_SHARE_READ & FILE_SHARE_WRITE in the dwShareMode argument when calling the CreateFile API call. Not positive if that'll let you share reading of an open file or not. Microsoft doesn't make it clear, and it may depend on whether or not the other process specified those flags.

[tmighty2]

Does anybody have a sample which uses thunking? I am just user of Paul Caton's ASM code. I have no idea how to change it to be usable for callbacks for CopyFileWEx.

[VanGoghGaming]

You don't need any thunk for the callback routine, it's just the AddressOf some BAS function you write yourself.

[fafalone]

If you're using Paul Caton's self-subclass/class callback thunks and want to put the callback inside the form instead of the BAS (where as VanGoghGaming points out, you don't need any thunks), you would use scb_SetCallbackAddr to get the address of the callback function in your form/UC/class. You wouldn't need to add any arguments, just the prototype described by the SDK/MSDN.

Make sure to call scb_TerminateCallbacks in the unload or terminate event.

[VanGoghGaming]

I haven't used the subclassing flavor from Paul Caton but I still wouldn't use any thunks with this callback routine. It seems that CopyFileEx allows you to pass an optional argument to the callback routine. This can be a pointer to a custom "ICallback" interface that can be implemented in your form or class.

[fafalone]

Or just a method.

Add a Public Function CopyCallback(...) As Long

Then, you can make the argument in the actual callback ByVal lpData As Form1, pass ByVal ObjPtr(Me) to it, and call the method:

lpData.CopyCallback(...)

[VanGoghGaming]

Then, you can make the argument in the actual callback ByVal lpData As Form1, pass ByVal ObjPtr(Me) to it, and call the method

If the argument is declared as "ByVal lpData As Form1" then you would just pass "Form1" or "Me". In this case the function could be declared as "Friend" instead of "Public" but a custom interface would make it more generic and easier to use in other forms or projects.