Python from C

Last year, there was a need to supplement an old project written in C with functionality in python3 . Despite the fact that there are articles on this subject, I suffered in that year and now when I wrote programs for the article. Therefore, I will give my examples on how to work with python3 from C under Linux (with what I used). I will describe how to create a class and call its methods, to access variables. Calling functions and getting variables from the module. And also the problems that I encountered and could not understand.

Packages

We use the standard Python API . Required python packages:

• python3
• python3-dev
• python3-all
• python3-all-dev
• libpython3-all-dev

Work in the interpreter

The simplest thing is loading the python interpreter and working in it.

To work, you must connect the header file:

#include <Python.h>
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

Loading the interpreter:

 Py_Initialize();
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

The following is a block of work with python, for example:

 PyRun_SimpleString("print('Hello!')");
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

Unload the interpreter:

 Py_Finalize();
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

Full example:

 #include <Python.h> void main() { //   Python Py_Initialize(); //     PyRun_SimpleString("print('Hello!')"); //   Python Py_Finalize(); }
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

How to compile and run:

 gcc simple.c $(python3-config --includes --ldflags) -o simple && ./simple Hello!
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

But this will not work:

 gcc $(python3-config --includes --ldflags) simple.c -o simple && ./simple /tmp/ccUkmq57.o: In function `main': simple.c:(.text+0x5): undefined reference to `Py_Initialize' simple.c:(.text+0x16): undefined reference to `PyRun_SimpleStringFlags' simple.c:(.text+0x1b): undefined reference to `Py_Finalize' collect2: error: ld returned 1 exit status
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

This is because python3-config --includes --ldflags expands into this kind of thing:

 -I/usr/include/python3.6m -I/usr/include/python3.6m -L/usr/lib/python3.6/config-3.6m-x86_64-linux-gnu -L/usr/lib -lpython3.6m -lpthread -ldl -lutil -lm -Xlinker -export-dynamic -Wl,-O1 -Wl,-Bsymbolic-functions
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

Here I think the connection order of the -Wl linker is important . Who knows more precisely, write about this in comments, I will supplement the answer.

Explanation from MooNDeaR :

Everything is quite simple - characters are searched in one pass and all unused characters are thrown away. If you put simple.c at the end, it turns out that the linker will see the use of the Py_Initialize () symbol after it looks at the python libraries, all the symbols of which will be discarded by this moment (because they were not used).

An example of calling a function from a python file:

simple.c

 #include <Python.h> void python() { //   Python Py_Initialize(); //     //   sys PyRun_SimpleString("import sys"); //    python PyRun_SimpleString("sys.path.append('./src/python')"); PyRun_SimpleString("import simple"); PyRun_SimpleString("print(simple.get_value(2))"); PyRun_SimpleString("print(simple.get_value(2.0))"); PyRun_SimpleString("print(simple.get_value(\"Hello!\"))"); //   Python Py_Finalize(); } void main() { puts("Test simple:"); python(); }
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

simple.py

 #!/usr/bin/python3 #-*- coding: utf-8 -*- def get_value(x): return x
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

But these are simple and uninteresting things, we do not get the result of the function.

Work with functions and module variables

This is a bit trickier.

Loading the python interpreter and the func.py module into it:

 PyObject * python_init() { //   Python Py_Initialize(); do { //   sys sys = PyImport_ImportModule("sys"); sys_path = PyObject_GetAttrString(sys, "path"); //     python folder_path = PyUnicode_FromString((const char*) "./src/python"); PyList_Append(sys_path, folder_path); //  func.py pName = PyUnicode_FromString("func"); if (!pName) { break; } //    pModule = PyImport_Import(pName); if (!pModule) { break; } //      pDict = PyModule_GetDict(pModule); if (!pDict) { break; } return pDict; } while (0); //   PyErr_Print(); }
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

Releasing python interpreter resources:

 void python_clear() { //    Py_XDECREF(pDict); Py_XDECREF(pModule); Py_XDECREF(pName); Py_XDECREF(folder_path); Py_XDECREF(sys_path); Py_XDECREF(sys); //   Python Py_Finalize(); }
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

Work with variables and module functions.

 /** *          */ char * python_func_get_str(char *val) { char *ret = NULL; //   get_value  func.py pObjct = PyDict_GetItemString(pDict, (const char *) "get_value"); if (!pObjct) { return ret; } do { //  pObjct  . if (!PyCallable_Check(pObjct)) { break; } pVal = PyObject_CallFunction(pObjct, (char *) "(s)", val); if (pVal != NULL) { PyObject* pResultRepr = PyObject_Repr(pVal); //     ,     python   . //   pResultRepr     . ret = strdup(PyBytes_AS_STRING(PyUnicode_AsEncodedString(pResultRepr, "utf-8", "ERROR"))); Py_XDECREF(pResultRepr); Py_XDECREF(pVal); } else { PyErr_Print(); } } while (0); return ret; } /** *       int */ int python_func_get_val(char *val) { int ret = 0; //     val pVal = PyDict_GetItemString(pDict, (const char *) val); if (!pVal) { return ret; } //    long if (PyLong_Check(pVal)) { ret = _PyLong_AsInt(pVal); } else { PyErr_Print(); } return ret; }
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

Let's dwell on this in more detail.

 pVal = PyObject_CallFunction(pObjct, (char *) "(s)", val);
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

"(s)" means that 1 parameter of type char * is passed as an argument to get_value (x) . If we would need to pass several arguments to the function, it would be like this:

 pVal = PyObject_CallFunction(pObjct, (char *) "(sss)", val1, val2, val3);
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

If you need to pass int , then i would be used, all possible data types and their designations can be found in the python documentation .

 pVal = PyObject_CallFunction(pObjct, (char *) "(i)", my_int);
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

func.py:

 #!/usr/bin/python3 #-*- coding: utf-8 -*- a = 11 b = 22 c = 33 def get_value(x): return x def get_bool(self, x): if x: return True else: return False
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

( Problem resolved )

The problem I encountered and could not understand yet:

 int main() { puts("Test func:"); if (!python_init()) { puts("python_init error"); return -1; } puts("Strings:"); printf("\tString: %s\n", python_func_get_str("Hello from Python!")); puts("Attrs:"); printf("\ta: %d\n", python_func_get_val("a")); printf("\tb: %d\n", python_func_get_val("b")); printf("\tc: %d\n", python_func_get_val("c")); python_clear(); return 0; }
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

If I want to get b or c from func.py , then on:

 Py_Finalize();
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

I get a segmentation fault . There is no such problem with getting only a .

When getting class variables, there are no problems either.

Explanation from pwl :

PyObject PyDict_GetItemString (PyObject p, const char * key)

Return value: Borrowed reference. Nothing needs to be done for a borrowed reference.

The problem was that I was calling Py_XDECREF () for PyDict_GetItemString () . It is unnecessary to do this for this function, leading to a segmentation fault .

Class work

There is still a little more complicated.

Loading the python interpreter and class.py module into it.

 PyObject * python_init() { //   Python Py_Initialize(); do { //   sys sys = PyImport_ImportModule("sys"); sys_path = PyObject_GetAttrString(sys, "path"); //     python folder_path = PyUnicode_FromString((const char*) "./src/python"); PyList_Append(sys_path, folder_path); //  Unicode   UTF-8  pName = PyUnicode_FromString("class"); if (!pName) { break; } //   class pModule = PyImport_Import(pName); if (!pModule) { break; } //      pDict = PyModule_GetDict(pModule); if (!pDict) { break; } //   Class  class.py pClass = PyDict_GetItemString(pDict, (const char *) "Class"); if (!pClass) { break; } //  pClass  . if (!PyCallable_Check(pClass)) { break; } //   Class pInstance = PyObject_CallObject(pClass, NULL); return pInstance; } while (0); //   PyErr_Print(); }
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

Passing a string as an argument and getting the string back

 char * python_class_get_str(char *val) { char *ret = NULL; pVal = PyObject_CallMethod(pInstance, (char *) "get_value", (char *) "(s)", val); if (pVal != NULL) { PyObject* pResultRepr = PyObject_Repr(pVal); //     ,     python   . ret = strdup(PyBytes_AS_STRING(PyUnicode_AsEncodedString(pResultRepr, "utf-8", "ERROR"))); Py_XDECREF(pResultRepr); Py_XDECREF(pVal); } else { PyErr_Print(); } return ret; }
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

There were no problems here; everything works without errors. There are examples in the source how to work with int , double , bool .

While writing material I refreshed my knowledge)

I hope it will be useful.

References

Source codes for examples

Next C / C ++ article from Python