[RESOLVED] Problems When Trying to Update Libcurl

[Dev101]

Running into some issues when trying to get Curl to work on an older system - XP SP3. Downloaded example code from a post found here on VBForums, and while the code worked, I wanted to update the Libcurl.dll with a more recent version.

https://www.vbforums.com/showthread....9340-vbLibCurl

In the post it mentions updating libcurl.dll as being a drop in replacement but wasn’t able to get any version other then 7.13.0 to work, in fact most versions failed to even load.



Was able to get version 7.23.1 to load but it had it’s own issues. While I don’t fully understand what’s going on, it looks as if the 7.13.0 version of Libcurl contains additional libraries, OpenSSL, zlib and c-ares.



And while version 7.23.1 did load successfully it didn’t contain any of the other required libraries.



I don’t understand why most versions of Libcurl fail to load when many specifically state they work with XP. And help would be appreciated.

Thanks.

[xiaoyao]

old version maybe like 2MB,new version maybe 15mb
new version some need win7 up

[Dev101]

Not looking to install the most recent DLL’s, just trying to jump up a couple versions if possible. Still don’t fully understand all that’s going on but appears as if vblibcurl has minimal functionality primarily serving as a VB friendly wrapper for libcurl which in turn makes use of three other supporting DLL’s.

Original files from VBForums CodeBank

DLL		Version		File		Date
-----		---------	-----		-------
Vblibcurl	?		vblibcurl.dll	6/18/2005	VB Wrapper
Libcurl		7.13.0		libcurl.dll	2/28/2005	CURL Functions
OpenSSL		0.9.7e		libcurl.dll	2/28/2005	SSL Functions
Zlib		1.2.1		libcurl.dll	2/28/2005	Compression Utilities
C-ares		1.2.0		libcurl.dll	2/28/2005	DNS Support

If the 3 supporting DLL’s were not bundled / included within the libcurl.dll file itself things would be much simpler as I’ve found a later version of libcurl that works only it doesn’t include all required DLL’s.

DLL		Version		File		Date
-----		---------	-----		-------
Libcurl		7.23.1		libcurl.dll	8/6/2016	CURL Functions

I’ve been trying to locate the other 3 supporting files, OpenSSL, Zlib & C-ares as standalone DLL’s so VB can independently call LoadLibrary for each. So far the only one I’ve been able to find and successfully load in VB has been Zlib version 1.2.3.0 from 8/10/2016. Now I just need to find or rebuild compatible version of OpenSSL and C-ares.

One problem I’m running into is that because the 3 supporting DLL’s are bundled within libcurl, I don’t even know there original file names making it difficult to locate online. I think the original file name for OpenSSL was libssl.dll or libssl32.dll and not really sure about C-ares.

I’ve found several versions of the C source code packages for both OpenSSL and C-ares dating back to around the same time as libcurl version 7.23.1 (8/6/2016) but been struggling with the Make files and so far been unable to successfully get them to compile. The questionable part is that I don’t know if rebuilding these DLL’s will even work for compatibility reasons. I’m Hopeful that if they all come from the same time frame, middle to late 2016 all will be good.

[dz32]

I dont really remember the details of this project since it was a while ago.

from ./vbLibCurl/libcurlvb-1_01/README

$Id: README,v 1.2 2005/06/25 21:29:55 jeffreyphillips Exp $

libcurl.vb 1.01 (c) 2005 Jeff Phillips, jeff@jeffp.net
...
This is not a COM object, but a very thin API of functions that
will allow you to use most of libcurl's functionality from Visual
Basic 6

I just grabbed the libcurl dlls precompiled from a 3rd party site. (from main readme.txt)

https://curl.se/windows/
https://curl.se/windows/dl-7.73.0_1/openssl-1.1.1h_1-win32-mingw.zip

 libcurl 7.73.0_1 was built and statically linked with

    OpenSSL 1.1.1h [64bit/32bit]
    brotli 1.0.9 [64bit/32bit]
    libssh2 1.9.0 [64bit/32bit]
    nghttp2 1.41.0 [64bit/32bit]
    zlib 1.2.11 [64bit/32bit]
    zstd 1.4.5 [64bit/32bit] 

The following tools/compilers were used in the build process:

    binutils-mingw-w64-i686 2.35
    binutils-mingw-w64-x86_64 2.35
    clang 9.0.1
    gcc-mingw-w64-i686 10-win32
    gcc-mingw-w64-x86_64 10-win32
    mingw-w64 8.0.0-1

here is the full source bundle with the dlls I used:

http://sandsprite.com/blogs/files/vbLibCurl.zip

this includes:

libcrypto-1_1.dll
Compiled: Tue, Sep 22 2020, 12:55:07 - 32 Bit DLL
Version:  1.1.1.8

libcurl.dll
Compiled: Wed, Oct 14 2020, 5:52:35 - 32 Bit DLL
Version:  7.73.0

libssl-1_1.dll
Compiled: Tue, Sep 22 2020, 12:55:07 - 32 Bit DLL
Version:  1.1.1.8

vblibcurl.dll
Compiled: Sat, Jun 18 2005, 21:37:21 - 32 Bit DLL

The updated libcurl was 15 years newer and was a drop in replacement at least as of the 2020 7.73.0 version.

The newer versions will run on XP SP2 and newer because of the normaliz.Idn2Ascii import and openssl requires Kernel32.ConvertFiberToThread.

The updated libcurl is required to talk to modern ssl servers. The old libcurl would run on win2k or newer. normaliz.dll from xpsp3 will run on win2k.

The problems you encountered are probably just due to how it was compiled and what dependencies it needed. This can also include what imports it needed from system dlls. It can be tricky to figure out exactly why a dll fails to load. At least the version above will bring you up to 2020 with xpsp2 support.

[Dev101]

dz32 – Can’t thank you enough for the post! Downloaded the code and it worked perfectly, greatly appreciated!

[wqweto]

Btw, bundled curl.exe works with TLS 1.3

D:\TEMP\vbLibCurl>curl https://www.howsmyssl.com/a/check
{"given_cipher_suites":["TLS_AES_256_GCM_SHA384","TLS_CHACHA20_POLY1305_SHA256","TLS_AES_128_GCM_SHA256","TLS_ECDHE_ECDSA_WI TH_AES_256_GCM_SHA384","TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384","TLS_DHE_RSA_WITH_AES_256_GCM_SHA384" ,"TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256","TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256","TLS_ DHE_RSA_WITH_CHACHA20_POLY1305_SHA256","TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256","TLS_ECDHE_RSA_WITH _AES_128_GCM_SHA256","TLS_DHE_RSA_WITH_AES_128_GCM_SHA256","TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384" ,"TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384","TLS_DHE_RSA_WITH_AES_256_CBC_SHA256","TLS_ECDHE_ECDSA_WITH _AES_128_CBC_SHA256","TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256","TLS_DHE_RSA_WITH_AES_128_CBC_SHA256"," TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA","TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA","TLS_DHE_RSA_WITH_AES_256 _CBC_SHA","TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA","TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA","TLS_DHE_RSA_W ITH_AES_128_CBC_SHA","TLS_RSA_WITH_AES_256_GCM_SHA384","TLS_RSA_WITH_AES_128_GCM_SHA256","TLS_RSA_WI TH_AES_256_CBC_SHA256","TLS_RSA_WITH_AES_128_CBC_SHA256","TLS_RSA_WITH_AES_256_CBC_SHA","TLS_RSA_WIT H_AES_128_CBC_SHA","TLS_EMPTY_RENEGOTIATION_INFO_SCSV"],"ephemeral_keys_supported":true,"session_ticket_supported":false,"tls_compression_supported":false, "unknown_cipher_suite_supported":false,"beast_vuln":false,"able_to_detect_n_minus_one_splitting":fal se,"insecure_cipher_suites":{},"tls_version":"TLS 1.3","rating":"Probably Okay"}

cheers,
</wqw>

[Dev101]

Having a bit of a problem after running a few iterations of the VB Curl program. All seems to work without issue while the program is actively running, successfully transferring data using SSL but then, at times, when exiting the IDE or when re-running the program things crash. VB will either crash hard the moment F5 is pressed resulting in the IDE terminating immediately or I get the following message.



There doesn’t seem to be any rhyme or reason as to why or when the error message gets displayed vs the IDE just abruptly terminating. Haven’t changed or modified any code just been playing around with different websites. Tried letting the VB Curl program display data to the ListBox or saving it to a file but makes no difference, things still eventually crash.

Looked around thinking maybe something wasn’t being properly shutdown or cleaned up, added a bunch of error checking code but no luck. I have no idea how to make use of the information shown in error message or to know if this is a problem with VB code or something hiding down inside one of the DLL’s.

Quick update, was trying out a few more webistes, all of which were working until I tried exiting the IDE and got this:

[dz32]

[….]

[dz32]

Looking at the post again that sounds pretty typical of a memory corruption issue. Funny I haves used it a fair amount and never hit it. I would try running it gflags /p /enable and look for where she blows up.

https://learn.microsoft.com/en-us/wi...s-and-pageheap

I’ll play with it when I have more time

Is this happening with the unaltered demo program or when you are merging it with other code? Copymemory is always a prime suspect.

[Dev101]

dz32 - Not sure but think I might have found the issue. While in the IDE I repeatedly hit F5 then immediately closed the program quickly cycling through starting & stopping. Doesn't take long before it crashes, 5 maybe 10 iterations and VB will die. Kind of went with a brute force approach of slowly commenting out code til VB no longer crashed. Here is what looks to be the offending code:

List1.AddItem "Protocols: " & Join(libcurlProtocols(), ",")

Only time will tell but ran through dozens of start / stop cycles and VB no longer crashes - at least not yet.


Update: Dug a little deeper and the call to CopyMemory below seems to be the real culprit, commented it out and no more crashing.

Function libcurlProtocols() As String()
    Dim ptr() As Long, vd As Long, i As Long, s() As String, tmp As String
    
    vd = vbcurl_version_info(CURLVERSION_NOW)
    vbcurl_version_protocols vd, ptr
    
    If AryIsEmpty(ptr) Then Exit Function
    
    ReDim s(UBound(ptr))
    For i = 0 To UBound(ptr)
        CopyMemory ByVal VarPtr(tmp), ptr(i), 4 'steal a ref to an existing BSTR so we now own it
        s(i) = tmp
        'Debug.Print ptr(i) & " " & s(i)
    Next
    
    libcurlProtocols = s()
    
End Function

[dz32]

cool thanks for tracking that down, does look shady I will update the main project with something better

The os was xp sp2?

[Dev101]

XP sp3



Do you know if the call to vbcurl_version_protocols returns a zero based array? Was curios if the For loop is stepping outside the bounds of the array?

[dz32]

I am tempted to add vbcurl_num_protocols and vbcurl_protocol_string to the exports (and make stdcall) and implement the
vbcurl_version_protocols on the vb side.

Looks like it is zero based, adding a if ptr(i) > 0 might help before the vb copy memory in case junk is getting in

a debug.print hex(ptr(i)) might be interesting to watch for you.

I ran this in a for i = 0 to 1000: list.additem join(libcurlProtocols(),",") in the IDE and kept starting and stopping it running it like 30 times on win10 and never had it crash here ;-\

'not exported
'static int vbcurl_num_protocols(CURL_VERSION* pv, int offset)
'static BSTR vbcurl_protocol_string(CURL_VERSION* pv, int offset, int nProt)

'void __stdcall vbcurl_version_protocols(CURL_VERSION* pv,
'    SAFEARRAY** ppsa)
'{
'    LONG rgIndex = 0;
'    int nProtocols = 0;
'    if (pv->_magic != VERSION_MAGIC)
'        RaiseException(0xE0000000, EXCEPTION_NONCONTINUABLE, 0, NULL);
'    nProtocols = vbcurl_num_protocols(pv, OFFSET_PROTOCOLS);  'no declare/export for this
'    *ppsa = SafeArrayCreateVector(VT_BSTR, 0, nProtocols);
'    for (rgIndex = 0; rgIndex < nProtocols; rgIndex++)
'    {
'        SafeArrayPutElement(*ppsa, &rgIndex,
'            vbcurl_protocol_string(pv, OFFSET_PROTOCOLS, rgIndex)); 'no declare/export for this
'    }
'}

[Dev101]

Interesting that it doesn't crash on windows 10, the XP system being used is a very solid build, runs uninterrupted for months at a time processing data in real-time. All very odd, the libcurlProtocols Function doesn't look to have any obvious issues, don't understand why its causing things to crash.

Played around quite a bit with CopyMemory commented out and things didn't crash once. Uncommented the code and sure enough VB crashed almost immediately.

Wanted to see how stable things were without CopyMemory, added a timer to the code retrieving a webpage every minute, been running since around 3:20 pm error free. Overall the code is working great, again wanted to thank you for the updated Libcurl.dll and the example code, it's solved a lot of issue with a really old system.

[dz32]

give this a shot

Private Declare Sub vbcurl_version_protocols Lib "vblibcurl.dll" (ByVal hVerInfo As Long, ByRef ary())

Function libcurlProtocols() As Variant()
    Dim ptr(), vd As Long
    vd = vbcurl_version_info(CURLVERSION_NOW)
    vbcurl_version_protocols vd, ptr
    libcurlProtocols = ptr()
End Function

My thanks goes out to Jeff Phillips and the curl developers. I love that the api is so stable that a 15 year newer dll can be dropped right into place. I use this library when i need to do a direct download written to a target folder directly with no cache involved. Also the api is easier than wininet which is convoluted.

[Dev101]

Tried the new protocol code above but got an error - "Variable uses an Automation type not supported in Visual Basic".

Went back to the original code trying one last time to track down exactly what's happening. The version of libcurl.dll being used has 23 different protocols:





The protocol array is zero based and if I stop CopyMemory at index 21, one short of the total, everything works no matter how many times I run the program inside the IDE.



Also played around with padding out the string array receiving the protocol names but didn't help. If I allow CopyMemory to loop through all 23 protocols, VB will inevitably crash on array index 22. You can watch it crash on the 2nd run below:

VBCurl Crash.zip

The 22nd array element looks have a valid address and is a recognizable protocol name, really don't understand why things are crashing. While it'd be nice to be able to display / know the supported protocols it isn't a must have for the project.

[dz32]

Gremlins…

Automation type thing is weird vb6 is the same although service pack may vary.

Power surge took out my main pc :-/

[Dev101]

Worried about the same last week, lost power and my PC sputtered like 3 times as the electricity couldn’t decided if it was staying on or going out – it went out!

Hey was looking at old releases of SSL and it has version 1.1.1 running all the way up to 9/11/2023, version 1.1.1w. The current one being used is version 1.1.1h from 9/22/2020. Think its possible that version ‘w’ would be compatible? If so that could bring things even closer to the current SSL standard.

https://www.openssl.org/source/old/1.1.1/index.html

[dz32]

Yeah I would assume the api is the same if you just find one compiled in an agreeable manner. Look at the import table with something like pe bear or lord pe or cff explorer to see what dlls it requires.

Power out here was weird most devices turned off, but some fluorescent t lights and an arduino device were still powered. It’s like neutral line blew and they could still get some ground through earth connection. I even thought to myself I wonder if I should unplug the computers as the storm was rolling in or use the ups I had sitting there. Oghh well lucky that’s all I lost! Trying a used power supply first hopefully not the motherboard. ?*??

[VanGoghGaming]

Since the "tmp" string was used to steal a BSTR reference, you need to clear it before the end of the function, otherwise VB6 will try to free the string and bad things will happen!

PutMem4 ByVal VarPtr(tmp), 0&

[dz32]

Good catch

[Dev101]

Holy Cow, that fixed it, no more crashing! Thanks VanGoghGaming.

Going to look through the rest of the code as I believe there are other instances where CopyMemory is used in the same way.

[VanGoghGaming]

LoL, rock on mate!

You shouldn't rush ahead and implement the same change elsewhere though because you'll introduce a ton of memory leaks! Just out of curiosity I've glanced over this vbcurlib.dll project and it seems there are many C++ functions that return a BSTR using the "return SysAllocString" syntax so who's gonna release all those BSTRs that just got allocated?

In most cases those strings are released by VB6 using all those stolen refs so don't go crazy changing that since it works as intended! The "libcurlProtocols" function was an isolated case where this change was needed because the same string was going to be released twice at the end of the function when the variables were going out of scope (once by the "tmp" string and once again by the ptr() array containing the same string) so we needed to prevent one of them!

Hope this makes sense...

[Dev101]

Actually it all makes sense now that you’ve educated me. And have to tell you, never in a million years would I have been able to fix the crashing problem on my own and yet seems so obvious once you pointed it out. Thanks again for the help!

[VanGoghGaming]

The more I look at it the more I'm convinced that ptr() should be a string array instead of a long array in order to allow VB6 to free its individual string members when the array goes out of scope. The "vbcurl_version_protocols" function should be declared like this:

'void _stdcall vbcurl_version_protocols(
'                [in] long ver,
'                [out] SAFEARRAY(BSTR)* ppsa);
Private Declare Sub vbcurl_version_protocols Lib "vblibcurl.dll" (ByVal hVerInfo As Long, ByVal ppsa As Long)
Private Declare Function ArrPtr Lib "msvbvm60" Alias "#390" (vArray As Variant) As Long ' helper function to get the ppsa of a string array

and then the "libcurlProtocols" function could be further simplified like this:

Function libcurlProtocols() As String()
Dim vd As Long
    vd = vbcurl_version_info(CURLVERSION_NOW)
    vbcurl_version_protocols vd, ArrPtr(libcurlProtocols)
End Function

No more troublesome stolen refs here!

[Dev101]

Hey VanGoghGaming, just grabbed you're code for getting the protocols and it worked great. Not only avoids stolen refs but anytime I can ditch a call to CopyMemory it leaves me with a good feeling. Thanks.

[wqweto]

This version of ArrPtr in this particular case might work on *empty* String array but note that ANSI<->Unicode conversion is at play with API calls on String arrays too i.e. C/C++ code might receive a temp (ANSI) copy of the array unless a specialized ArrPtr for String arrays is used to retrieve original SAFEARRAY(BSTR) pointer.

cheers,
</wqw>

[VanGoghGaming]

It is the "__vbaRefVarAry" function that takes a variant parameter and works correctly on all kinds of arrays (except UDT arrays since you can't pass a UDT as a variant parameter). It is not subjected to Unicode->ANSI conversions because it doesn't use "As Any" parameters like the other flavor of "ArrPtr" function so in this regard it is superior.

Private Declare Function ArrPtr Lib "msvbvm60" Alias "#390" (vArray As Variant) As Long

[Dev101]

Yikes, you guys are like on another level with your knowledge and understanding of all this. Some of the stuff in the last two posts went clean over my head!

[VanGoghGaming]

Yikes, you guys are like on another level with your knowledge and understanding of all this. Some of the stuff in the last two posts went clean over my head!

There's nothing sinister about it. This is a remnant from the Windows 95 days. Win 95 didn't support Unicode strings but VB6 does so they needed to come up with a compromise (which became obsolete as soon as Win 98 came out but was kept for compatibility reasons).

Whenever you declare an API function that takes string or "As Any" parameters VB6 will convert the passed strings to ANSI and then when the function returns it will convert the strings back to Unicode. So if you rely on the address of those strings to remain constant, you are out of luck.

There are two equivalent versions of the ArrPtr declare:

Private Declare Function ArrPtr Lib "msvbvm60" Alias "__vbaRefVarAry" (vArray As Variant) As Long
Private Declare Function ArrPtr Lib "msvbvm60" Alias "VarPtr" (arr() As Any) As Long

They both work correctly with arrays of numerical types. For string arrays you need to use the first version to avoid the "As Any" parameter. For UDT (user defined type) arrays you need to use the second version because you get an error passing UDTs as a variant parameter.

That about sums it up.

[Dev101]

I hope you didn’t misunderstand my last post, I wasn’t trying to imply anything negative, quite the opposite, I’m thoroughly impressed with the knowledge you guys have and appreciate all the information and help you’ve provided.

[VanGoghGaming]

No worries mate, you said it went over your head, I just tried to offer an explanation about what was going on under the hood but it seems I didn't do a very good job at that.

[Dev101]

Oh that’s good to hear, certainly didn’t want to think I wasn’t appreciative for all the help. And as for stuff flying over my head, that comes from my lack of knowledge not from the explanations given.

While I understand the basics of what’s going on, or at least I think I do, my eyes start to gloss over when hearing unfamiliar things like Variants & SafeArrays. Wish I had a better overall understanding of how things work.

[Dev101]

Wanted to give an update here as things have been running around the clock without interruption for almost a week now without any issues. Vbcurl is fully integrated with the app and has been grabbing information from the web a dozen or so times every hour. Thanks again for all the help.

[xiaoyao]

how to use it like winhttprequest?xmlhttp?

[qvb6]

how to use it like winhttprequest?xmlhttp?

Please search the forum, or just click on the link in the first post.

[dz32]

finally got my main dev PC back in order. updated the main zip with the fix from VanGoghGaming. Thanks!

[qvb6]

FYI, the latest libcurl.dll(v8.9.1, released 2024-07-31) is marked with minimum OS version 6.0, which is Vista. I just tested it. vbLibCurl.dll from 2005 seems to work fine with it on Windows 10.

[dz32]

cool thanks for testing