linux uses boost. python to call the c++ dynamic library method

preface

Recently, robot framework has been used to test c++ dynamic libraries. robot framework runs on windows and c++ dynamic libraries run on remote linux hosts. The test is to have robot framework execute the python script on the remote machine through the SSHLIbrary library, and the python script calls the C++ dynamic library. So the trick now is to get python to call the c++ dynamic library.

python calls c++ dynamic libraries in two ways

After looking up information on the Internet and consulting colleagues, I got two methods: The first method encapsulated C++ dynamic library into C interface, and let python call C language interface. Since python can only call the C interface and cannot call the C++ interface directly, a layer of encapsulation is required. Encapsulation method: use extern "C" declaration mode to encapsulate 1 layer of C language interface on top of C++ interface. This approach has been tried and found that the pure C call works, but the python call does not, for reasons explained below. The second method is to use c++ boost library to generate the interface for python invocation. It is feasible after testing, but the process is very tortude-prone. The problems encountered and solutions will be explained in detail below.

Understand the essence of extern C

Before I go into the first approach, let me briefly introduce the role of the extern "C" approach. Specific explanation can refer to this blog, very detailed, recommended reading. For example, in c, there is a function

int add(int a,int b);

If you use the gcc compiler, the name of the compilation is add, but if you use the g++ compiler, you might get something like ABaddCD, which contains the function name, number of arguments, type, and return value. So, if c++ also defines one overloaded

float add(float a,float b);

The compile-generated name might be something like EFaddGH, which also contains the function name, input, return value, etc., so c++ can be overloaded. Imagine using the gcc compiler, call it add, and you won't be able to tell which function is which, so you won't be able to overload. Then, the function of extern "C" is to tell g++ compiler to compile int add(int a,int b) to add, instead of compiling ABaddCD, because add can only be recognized by C language, ABaddCD cannot be recognized by C language, and C language will consider add as 'undefined symbol'. Therefore, we can also see from here that extern "C" can only be used in c++ code. In addition, for the overloaded c++ function, two different functions need to be written to call separately to ensure that the name is not repeated.

python uses extern "C" to call c++ dynamic library

Knowing the nature of extern "C", we encapsulate in this way. I take the source code of c++ dynamic library directly, encapsulate 1 layer of C interface above the source code, and then generate dynamic library. Suppose the add function is encapsulated as addc, the C++ dynamic library is called A, and the dynamic library generated after encapsulating the 1-layer C interface is called B. If you write a test code of ES109en.c, use pure C code to verify the dynamic library B and call addc function, the result is feasible and successful. However, python was used to check the dynamic library B and call the addc function. It was found that the following error was reported:

[

AttributeError: B.so: undefined symbol: add

]

The add function is still not recognized. use

nm B.so | grep add 

To be able to get

[

addc
ABaddCD

]

As a result, the first addc must be recognized by python, and the second ABaddCD, which is the name generated by g++ compilation, cannot be called by python. I just give an example of my own, my own C++ dynamic library source code may be written more complex, can not be python successfully called, there are many examples on the Internet, said that can be successfully called. So the reader can experiment on his or her own, and if it can be successfully called, it's best. boost.python is a tricky way to use it.

python calls the c++ dynamic library using ES151en.python

Other third square libraries that resolve c++ dynamic library dependencies

Since my dynamic library relies on other third party library files, such as openssl, uuid, libevent and pthread, no matter which method is used to call c++ dynamic library, IT needs python to load these dynamic libraries. The specific python code is as follows:

from ctypes import *
ctypes.CDLL("libssl.so", mode=ctypes.RTLD_GLOBAL) 
ctypes.CDLL("libcrypto.so", mode=ctypes.RTLD_GLOBAL) 
ctypes.CDLL("libuuid.so", mode=ctypes.RTLD_GLOBAL) 
ctypes.CDLL("/usr/lib64/libevent.so", mode=ctypes.RTLD_GLOBAL) 
#ctypes.CDLL("/usr/lib64/libpthread.so.0", mode=ctypes.RTLD_GLOBAL)

Have 1 can be loaded by default, such as libpthread. so, we don't need to load, others need to manually load, like libssl. so, libuuid. so, all in/usr/lib64 / directory, you can not add path, but libevent library is usr/lib64 directory, and in/usr/lib/directory also has, must add path again. So, if the compilation fails, use whereis libevent.so to see which directory, and then add the absolute path. Sometimes it's still wrong to add an absolute path, like libpthread.so, and still report an error when you add an absolute path

[

'OSError: /usr/lib64/libpthread.so: invalid ELF header'

]

This means that the version number is incorrect. Find the version number of the link to libpthread.so and add the.0 version number, and the error will not be reported.

c++ code to configure the boost environment

On c++ dynamic library on centos6.6 machine, I refer to: python calling C/C++ method dynamic link library configuration and experiment boost under ubuntu. Boost. Python Python C/C++ hybrid programming python c++ function overloading. When configuring the environment, I use the command: yum install boost*, yum install ES225en-ES226en, refer to these two articles to realize boost, basically can pass, encountered problems, there are also. In addition, I also encountered other problems. I found the solution on Stack Overflow, and I will post the result directly below:

Create a new ES232en.cpp. In this cpp we want to define the functions available for python.

In the ES238en. cpp code, include the following code:

//  Need to include boost The header file 

#include <boost/python.hpp> 
#include <boost/python/module.hpp> 
#include <boost/python/def.hpp>

// Overloaded function implementation in my  c++ The code, LOGIN  The function, Synchronize_Request The function, Notify Function has 3 I'm just going to use one of these overloaded functions 1 a LOGIN The function, 1 a Synchronize_Request The function, 2 a Notify Functions, like the following fun3 and fun4 It's just two different things notify . 

// Only functions that have overloads need to be defined like this  fun1 . fun2 . fun3 . fun4 , there is no overloaded function, you can just write the name 

int (*fun1)(const int server_type, const int request_no, std::string& login_result) = &LOGIN;

int (*fun2)(const int server_type, const int request_no,std::string& recv_answer) = &Synchronize_Request; 
int (*fun3)(const int server_type, unsigned int timeout_ms, unsigned int sesscare ) = &Notify;

int (*fun4)(void) = &Notify;

// add  Example of function overloading 
int (*fun5)(int a,int b) = &add;

 
BOOST_PYTHON_MODULE( libB ) //python The module ,libB The name of the  .so  The name of the 1 to  
{ 
 using namespace boost::python;

 //Initialize  The function is not overloaded and can be used directly, not as above 1 Define the sample  fun1 
 def("Initialize",Initialize);
 //Uninitialize  The function is not overloaded, just use it  
 def("Uninitialize",Uninitialize); 
 def("LOGIN",fun1); 
 def("Synchronize_Request",fun2); 
 def("Notify",fun3); 
 def("Notify2",fun4); 
 def("add",fun5); 
 // python  You can call the above def Defined function  
}

Makefile uses the following command:

%.o : %.cpp
 g++ -g -lssl -fPIC -levent -lcrypto -luuid -lpthread -lrt -lboost\_filesystem -lboost\_system -lboost_python -lpython -I/usr/include/python2.7 -o $@ -c $<

The command to generate ES247en.so is:

g++ -shared -Wl,-soname,libB.so -o libB.so *.o -lpython -lboost_python

This dynamic library needs to be introduced in the python script

import libB

print libB.add(10,20)

Follow the above command to write, compile, I can avoid the pit. Note the -ES256en position, not the front. If you don't implement the definition of overloading, use def("LOGIN",LOGIN); The following error error: no matching for call to 'def(const char [15], < unresolved overloaded function type > )' def("LOGIN",LOGIN); To sum up, I spent a whole day on the research results. If there are any mistakes or omissions, please point them out. Thank you.

Supplement: When calling c++ dynamic libraries in boost.python, I cannot handle reference types such as string & recv_answer is used to receive the returned result and is identified as string{lvalue}, while my python passed in an string type and cannot match. So I manually put string & The recv_answer reference, rewritten to char * recv_answer_c format, is changed to the C language style, and then passed in the recv_answer_c parameter as follows to receive the result.

# using  bytes  To allocate space for variables in advance to ensure that there are no segment errors 
temp = bytearray(1000)
recv_answer_c= bytes(temp)

Conclusion: