[Info-vax] Announcement: A new system library (plibsys) is ported to OpenVMS
Alexander Saprykin
xelfium at gmail.com
Fri Oct 7 09:23:22 EDT 2016
четверг, 6 октября 2016 г., 16:22:19 UTC+3 пользователь Craig A. Berry написал:
> On 10/3/16 9:12 AM, Alexander Saprykin wrote:
> > Hi folks,
> >
> > I'm glad to announce that after several months of work I've ported my
> > personal project to OpenVMS. I'm talking about the plibsys library which
> > is available at GitHub:
> >
> > https://github.com/saprykin/plibsys
> >
> > This is a cross-platform system library with many capabilities provided:
> >
> > - Platform independent data types
> > - Threads: POSIX, Solaris and Win32
> > - Mutexes: POSIX, Solaris and Win32
> > - Condition variables: POSIX, Solaris and Win32
> > - Read-write locks: POSIX, Solaris and Win32
> > - System-wide semaphores: POSIX, System V and Win32
> > - System-wide shared memory: POSIX, System V and Win32
> > - Optimized spinlock
> > - Atomic operations
> > - Socket support (UDP, TCP, SCTP)
> > - Hash functions: MD5, SHA-1, SHA-2, SHA-3, GOST (R 34.11-94)
> > - Binary trees: BST, red-black, AVL
> > - INI file parser
> > - High resolution time profiler
> > - Files and directories
> > - Shared library loading
> > - Useful routines for linked lists, strings, hash tables
> > - Macros for OS and compiler detection
> >
> > The main idea is to use native code wherever is possible. Library supports
> > lot of operating systems and compilers, the full list you can check at the
> > project's page.
> >
> > As discussed here
> > (https://groups.google.com/forum/#!topic/comp.os.vms/ySlKal2n51Y) POSIX
> > semaphores and shared memory unfortunately do not work now due to a bug
> > in the implementation.
> >
> > All the library modules are covered with unit tests. I have only an FreeAXP
> > emulator with OpenVMS 8.4 to run tests, where they are run fine.
> >
> > I would appreciate if someone could run tests on IA64 platform as I do not
> > have access to it. This was actually my first experience working with
> > OpenVMS, so any advice or fixes are appreciated.
> >
> > Build instructions for OpenVMS: https://github.com/saprykin/plibsys/tree/master/platforms/vms-general
> >
> > I hope it will be useful for someone, especially who works with
> > cross-platform projects with OpenVMS as on of the target systems.
>
> I gave it a spin with:
>
> $ cc/vers
> HP C V7.3-020 on OpenVMS IA64 V8.4
> $ cxx/vers
> HP C++ V7.4-005 on OpenVMS IA64 V8.4
>
> First a side note: since you require v8.4 or later, you can use
> DELETE/TREE and shouldn't need the deltree.com procedure you've included.
>
> It's hard to say which commit it was exactly because I downloaded a zip
> file, but it was probably e2eff372bc250dd05b or thereabouts.
>
> Of the two Boost versions included, I first tried with Boost 1.60.0, but
> it fell down hard with:
>
> Compiling test executables...
> [CXX ] patomic_test.cpp
>
> reinterpret_cast<std::size_t>(vtable) &
> ~static_cast<std::size_t>(0x01));
> .............^
> %CXX-W-NARROWPTR, conversion from pointer to smaller integer
> at line number 706 in file
> D0:[craig.plibsys-master.boost.boost_1_60_0_mini.boost.function]function_base.hpp;1
>
> return reinterpret_cast<std::size_t>(vtable) & 0x01;
> ...........^
> %CXX-W-NARROWPTR, conversion from pointer to smaller integer
> at line number 710 in file
> D0:[craig.plibsys-master.boost.boost_1_60_0_mini.boost.function]function_base.hpp;1
>
> #include <fenv.h>
> ...................^
> %CXX-F-SRCFILNOOPEN, could not open source file "fenv.h"
> at line number 97 in file
> D0:[craig.plibsys-master.boost.boost_1_60_0_mini.boost.detail]fenv.hpp;1
>
> %CXX-I-MESSAGE, 1 catastrophic error detected in the compilation of
> "D0:[craig.plibsys-master.tests]patomic_test.cpp;1".
>
> %CXX-I-MESSAGE, Compilation terminated.
>
> So I tried it with Bost 1.34.1, i.e., like so:
>
> $ @build_vms TESTS
> "BOOST_ROOT=/d0/craig/plibsys-master/boost/boost_1_34_1_mini"
>
> It failed as well with a different error:
>
> [CXX ] patomic_test.cpp
>
> return ::boost::unit_test::unit_test_main( argc, argv );
> .....................................................^
> %CXX-E-INCOMPATIBLEPRM, argument of type "char * __ptr32 * __ptr32 " is
> incompatible with parameter of type "char * __ptr64 * __ptr64 "
> at line number 99 in file
> D0:[craig.plibsys-master.boost.boost_1_34_1_mini.boost.test.impl]unit_test_main.ipp;1
>
> %CXX-I-MESSAGE, 1 error detected in the compilation of
> "D0:[craig.plibsys-master.tests]patomic_test.cpp;1".
>
> That can be fixed as follows (gnu unified diff):
>
> --- build_vms.com;-0 2016-10-05 16:45:36 -0500
> +++ build_vms.com 2016-10-05 21:58:03 -0500
> @@ -366,7 +366,7 @@ $ cc_params = cc_params + "/FLOAT=IEEE/I
> $!
> $ if build_64 .eqs. "1"
> $ then
> -$ cc_params = cc_params + "/POINTER_SIZE=64"
> +$ cc_params = cc_params + "/POINTER_SIZE=64=ARGV"
> $ else
> $ cc_params = cc_params + "/POINTER_SIZE=32"
> $ endif
> @@ -490,7 +490,7 @@ $ cxx_params = cxx_params + "/FLOAT=IEEE
> $!
> $ if build_64 .eqs. "1"
> $ then
> -$ cxx_params = cxx_params + "/POINTER_SIZE=64"
> +$ cxx_params = cxx_params + "/POINTER_SIZE=64=ARGV"
> $ else
> $ cxx_params = cxx_params + "/POINTER_SIZE=32"
> $ endif
> [end]
>
> Before even getting to the things that fail, though, there was a warning:
>
> [CC] psocket.c
>
> if (P_UNLIKELY (ioctl (fd, FIONBIO, (ppointer) &arg) < 0)) {
> ............^
> %CC-W-MAYLOSEDATA2, In this statement, "(ppointer)&arg" has a larger
> data size than "short pointer to void". Assignment can result in data loss.
> at line number 114 in file D0:[craig.plibsys-master.src]psocket.c;1
>
> And the compiler warning in that module caused linker warnings for
> everything linked against the plibsys.exe shareable image. It turns out
> that ioctl is one of the functions in the CRTL that does not support
> 64-bit pointers. See Table 1-6 at:
>
> <http://h41379.www4.hpe.com/commercial/c/docs/5763p006.html>
>
> Nevertheless, the build continued to completion. The output of running
> all the tests is below. There were a lot of memory allocation failures
> that didn't seem to have much to do with whether the test failed. The
> socket test is a little less noisy if run with privileges (which are
> needed to change socket buffer size) but still has the SO_KEEPALIVE failure.
>
> Hope this helps.
>
>
> $ run patomic_test.EXE
> Running 1 test case...
>
> *** No errors detected
> $ run pcondvariable_test.EXE
> Running 2 test cases...
> ** Error: PCondVariable::p_cond_variable_new: failed to allocate memory **
>
> *** No errors detected
> $ run pcryptohash_test.EXE
> Running 13 test cases...
> ** Error: PCryptoHash::p_crypto_hash_new: failed to allocate memory **
> ** Error: PCryptoHash::p_crypto_hash_new: failed to allocate memory **
> ** Error: PCryptoHash::p_crypto_hash_new: failed to allocate memory **
>
> *** No errors detected
> $ run pdir_test.EXE
> Running 2 test cases...
>
> *** No errors detected
> $ run perror_test.EXE
> Running 3 test cases...
>
> *** No errors detected
> $ run pfile_test.EXE
> Running 1 test case...
>
> *** No errors detected
> $ run phashtable_test.EXE
> Running 4 test cases...
> ** Error: PHashTable::p_hash_table_new: failed(1) to allocate memory **
> ** Error: PHashTable::p_hash_table_insert: failed to allocate memory **
>
> *** No errors detected
> $ run pinifile_test.EXE
> Running 3 test cases...
>
> *** No errors detected
> $ run plibraryloader_test.EXE
> Running 2 test cases...
> D0:[craig.plibsys-master.tests]plibraryloader_test.cpp;1(59): fatal
> error in "plibraryloader_nomem_test": critical check
> boost::unit_test::framework:
> :master_test_suite().argc > 1 failed
> D0:[craig.plibsys-master.tests]plibraryloader_test.cpp;1(105): fatal
> error in "plibraryloader_general_test": critical check
> boost::unit_test::framewo
> rk::master_test_suite().argc > 1 failed
>
> *** 2 failures detected in test suite "plibraryloader_test"
> $ run plist_test.EXE
> Running 3 test cases...
> ** Error: PList::p_list_append: failed to allocate memory **
> ** Error: PList::p_list_prepend: failed to allocate memory **
>
> *** No errors detected
> $ run pmacros_test.EXE
> Running 1 test case...
> ** Warning: Test warning output **
> ** Error: Test error output **
> ** Debug: Test debug output **
> ** Debug: Likely condition triggered **
>
> *** No errors detected
> $ run pmain_test.EXE
> Running 3 test cases...
> ** Error: MAIN::p_libsys_init_full: failed to initialize memory table **
>
> *** No errors detected
> $ run pmem_test.EXE
> Running 1 test case...
>
> *** No errors detected
> $ run pmutex_test.EXE
> Running 2 test cases...
> ** Error: PMutex::p_mutex_new: failed to allocate memory **
>
> *** No errors detected
> $ run pprocess_test.EXE
> Running 1 test case...
>
> *** No errors detected
> $ run prwlock_test.EXE
> Running 3 test cases...
> ** Error: PRWLock::p_rwlock_new: failed to allocate memory **
>
> *** No errors detected
> $ run psemaphore_test.EXE
> Running 3 test cases...
> D0:[craig.plibsys-master.tests]psemaphore_test.cpp;1(155): fatal error
> in "psemaphore_general_test": critical check sem != 0 failed
> D0:[craig.plibsys-master.tests]psemaphore_test.cpp;1(188): fatal error
> in "psemaphore_thread_test": critical check sem != 0 failed
>
> *** 2 failures detected in test suite "psemaphore_test"
> $ run pshmbuffer_test.EXE
> Running 4 test cases...
> D0:[craig.plibsys-master.tests]pshmbuffer_test.cpp;1(232): fatal error
> in "pshmbuffer_general_test": critical check buffer != 0 failed
> D0:[craig.plibsys-master.tests]pshmbuffer_test.cpp;1(279): fatal error
> in "pshmbuffer_thread_test": critical check buffer != 0 failed
>
> *** 2 failures detected in test suite "pshmbuffer_test"
> $ run pshm_test.EXE
> Running 4 test cases...
> D0:[craig.plibsys-master.tests]pshm_test.cpp;1(145): fatal error in
> "pshm_general_test": critical check shm != 0 failed
> D0:[craig.plibsys-master.tests]pshm_test.cpp;1(262): fatal error in
> "pshm_thread_test": critical check shm != 0 failed
>
> *** 2 failures detected in test suite "pshm_test"
> $ run psocketaddress_test.EXE
> Running 3 test cases...
> ** Warning: PSocketAddress::p_socket_address_to_native: invalid buffer
> size for IPv4 **
> ** Error: PSocketAddress::p_socket_address_to_native: invalid buffer
> size for IPv6 **
> ** Error: PSocketAddress::p_socket_address_new: failed to allocate memory **
> ** Error: PSocketAddress::p_socket_address_new_any: failed to allocate
> memory **
> ** Error: PSocketAddress::p_socket_address_new_loopback: failed to
> allocate memory **
> ** Warning: PSocketAddress::p_socket_address_new_from_native: invalid
> IPv4 native size **
> ** Warning: PSocketAddress::p_socket_address_new_from_native: invalid
> IPv6 native size **
>
> *** No errors detected
> $ run psocket_test.EXE
> Running 7 test cases...
> D0:[craig.plibsys-master.tests]psocket_test.cpp;1(841): error in
> "psocket_general_udp_test": check p_socket_set_buffer_size (socket,
> P_SOCKET_DIRECTI
> ON_RCV, 72 * 1024, 0) == 1 failed
> D0:[craig.plibsys-master.tests]psocket_test.cpp;1(842): error in
> "psocket_general_udp_test": check p_socket_set_buffer_size (socket,
> P_SOCKET_DIRECTI
> ON_SND, 72 * 1024, 0) == 1 failed
> ** Warning: PSocket::p_socket_set_keepalive: setsockopt() with
> SO_KEEPALIVE failed **
> D0:[craig.plibsys-master.tests]psocket_test.cpp;1(927): error in
> "psocket_general_tcp_test": check p_socket_set_buffer_size (socket,
> P_SOCKET_DIRECTI
> ON_RCV, 72 * 1024, 0) == 1 failed
> D0:[craig.plibsys-master.tests]psocket_test.cpp;1(928): error in
> "psocket_general_tcp_test": check p_socket_set_buffer_size (socket,
> P_SOCKET_DIRECTI
> ON_SND, 72 * 1024, 0) == 1 failed
> ** Warning: PSocket::p_socket_set_keepalive: setsockopt() with
> SO_KEEPALIVE failed **
>
> *** 4 failures detected in test suite "psocket_test"
> $ run pspinlock_test.EXE
> Running 2 test cases...
> ** Error: PSpinLock::p_spinlock_new: failed to allocate memory **
>
> *** No errors detected
> $ run pstring_test.EXE
> Running 5 test cases...
>
> *** No errors detected
> $ run ptimeprofiler_test.EXE
> Running 2 test cases...
> ** Error: PTimeProfiler: failed to allocate memory **
>
> *** No errors detected
> $ run ptree_test.EXE
> Running 4 test cases...
> ** Error: PTree::p_tree_new_full: failed to allocate memory **
> ** Error: PTree::p_tree_new_full: failed to allocate memory **
> ** Error: PTree::p_tree_new_full: failed to allocate memory **
>
> *** No errors detected
> $ run ptypes_test.EXE
> Running 6 test cases...
> 0x100
> 0xffffffffffffff00
> 0x200
> 0xfffffffffffffe00
> 0x400
> 0xfffffc00
> 0x800
> 0xfffff800
> 0x1000
> 0xfffffffffffff000
> 0x2000
> -256
> 256
> 512
> -512
> 1024
> -1024
> 2048
> -2048
> 4096
> -4096
> 8192
>
> *** No errors detected
> $ run puthread_test.EXE
> Running 5 test cases...
> ** Error: PUThread::p_uthread_create_internal: failed to allocate memory **
> ** Error: PUThread::p_uthread_create_internal: failed to allocate memory **
> ** Error: PUThread::pp_uthread_get_tls_key: failed to allocate memory **
> ** Error: PUThread::p_uthread_current: failed to allocate memory **
> ** Error: PUThread::p_uthread_local_new: failed to allocate memory **
> ** Error: PUThread::pp_uthread_get_tls_key: failed to allocate memory **
> ** Error: PUThread::p_uthread_current: failed to allocate memory **
> ** Error: PUThread::pp_uthread_get_tls_key: failed to allocate memory **
> ** Error: PUThread::pp_uthread_get_tls_key: failed to allocate memory **
> ** Error: PUThread::pp_uthread_get_tls_key: failed to allocate memory **
> ** Error: PUThread::pp_uthread_get_tls_key: failed to allocate memory **
> ** Warning: PUThread::p_uthread_exit: p_uthread_exit() cannot be called
> from an unknown thread **
>
> *** No errors detected
Hi Craig,
Thank you for checking it out.
1. Recent versions of Boost seems lack support for OpenVMS, that's why I have also provided an old version.
2. Thanks for pointing out for [ARGV] value, added to the build script (I borrowed a code to detect whether a compiler supports it from OpenSSL, by Richard Levitte).
3. I also surrounded ioctl() call with pointer size pragmas when building for 64-bit, after that it compiles without warnings.
4. About lot of errors while running tests: it is ok, because all tests simulate "out-of-memory" situation. It is better to also use the build script to run tests:
@build_vms.com NOLIB RUN_TESTS
It hides all output and only prints final result.
All fixes should be available tonight (forget to push them :( ).
Could you please also try to build 32-bit version of the library and tests? For now I experience a problem while building 32-bit version of the tests. Maybe you can point a direction where to dig.
Best Regards,
Alexander Saprykin
More information about the Info-vax
mailing list