[Info-vax] Problems with multithreaded calls to libCURL

[Jim Duff]

Versions:

OpenVMS 8.4 on IA64
HP C V7.2-001:
libCURL: 7.4.7

Nearly seven years ago, I wrote some code to utilise libCURL to perform
FTP for a document routing system.  The code was single threaded, and it
quickly became the event driven successor for what was then a polling
system based on CIFS.

Today, the system ships hundreds of thousands of documents a day off the
central ERP system.  All usually goes well, but there is at least one
problem that can seriously impact document delivery times.  This occurs
when an FTP server at a target machine is unavailable.

When this occurs, with the default setting of CURLOPT_CONNECTTIMEOUT, a
5 minute delay can occur as the library waits for the connection to
complete.  Easy fix says I, just set CURLOPT_CONNECTIMEOUT to a smaller
value with a call like

res = curl_easy_setopt (handle, CURLOPT_CONNECTTIMEOUT, 30L);

for a 30 second timeout.

Except this appears to have no effect.

After double checking that no later version of libCURL is currently
available for OpenVMS, I embarked on multithreading the code so that
there would be one worker thread per server and target directory
combination.  That way, if an FTP server was unavailable, only documents
destined for that server would be impacted by the connect timeout issue
as only the affected thread(s) would stall.

To my consternation, on stress testing the multithreaded code, I'm
occasionally getting an access violation in the depths of libCURL:

%SYSTEM-F-ACCVIO, access violation, reason mask=00, virtual
address=000000000000
0028, PC=FFFFFFFF84088EB0, PS=0000001B
%TRACE-F-TRACEBACK, symbolic stack dump follows
image     module    routine               line      rel PC           abs PC
LIBRTL  LIB$$UNWIND  restoreReg          22380 00000000000007A0
FFFFFFFF84088EB0
LIBRTL  LIB$$UNWIND  step_final_phase    25871 0000000000010652
FFFFFFFF84098D62
LIBRTL  LIB$$UNWIND  UC_step             24860 000000000000C682
FFFFFFFF84094D92
LIBRTL  LIB$$UNWIND  UC_currentContext
                                         21987 0000000000000602
FFFFFFFF84088D12
LIBRTL  LIB$IPF_CALLING_STANDARD  lib$i64_get_curr_invo_context
                                         19476 00000000000009A2
FFFFFFFF8407FAB2
DECC$SHR  C$SETJMP  sigsetjmp            17136 00000000000008B2
FFFFFFFF84B6DBE2
GNV$LIBCURL  hostip  Curl_resolv_timeout
                                         82084 0000000000001382
000000000020E322
GNV$LIBCURL  url  create_conn            92703 00000000000145E2
00000000001FFDD2
GNV$LIBCURL  url  Curl_connect           93456 0000000000014D12
0000000000200502
GNV$LIBCURL  multi  multi_runsingle      83372 0000000000003002
00000000001DCF62
GNV$LIBCURL  multi  curl_multi_perform
                                         84076 0000000000006C52
00000000001E0BB2
GNV$LIBCURL  easy  curl_easy_perform     87297 0000000000000C52
00000000001CCC52
FTP_SENDER  FTP_SENDER  do_ftp           62939 00000000000027C2
00000000000227C2
FTP_SENDER  FTP_SENDER  worker           63195 00000000000047D2
00000000000247D2
PTHREAD$RTL  THD_THREAD  thdBase        245274 0000000000005C52
FFFFFFFF844D4ED2
PTHREAD$RTL                                  0 0000000000054012
FFFFFFFF844AE012
PTHREAD$RTL  THD_INIT  pthread_main     244961 0000000000000492
FFFFFFFF8448A492
                                             0 FFFFFFFF80B026D2
FFFFFFFF80B026D2
DCL                                          0 000000000007D072
000000007AE21072
%TRACE-I-END, end of TRACE stack dump

Enclosing the calls to curl_easy_perform () in a mutex makes the access
violation go away, making me think that libCURL's claim to being thread
safe is not so rock solid.

Reading https://curl.haxx.se/libcurl/c/threadsafe.html I see there are
some exceptions to its thread safe claim, so I duly implemented them:

1) don't share handles between threads.  I wasn't doing this in the
first place.
2) I'm not using the share interface.
3) disable signals with CURLOPT_NOSIGNAL set to 1L
4) call curl_global_init () in the main thread before creating
additional threads.
5) don't use CURLOPT_DNS_USE_GLOBAL_CACHE.  I wasn't.

I'd love to debug this myself, but I have so much on my plate that I
have no time to do so.

If anyone has any ideas, or has implemented a multithreaded system
employing libCURL, I'd love to hear from you.

Jim