.. .rst: expy tutorial file, created by Yingjie Lan, Tue May 26, 2009 A Dive-In Tutorial on EXPY ========================== #. :ref:`funcs` #. :ref:`tags` #. :ref:`terse` #. :ref:`keywords` #. :ref:`retwrap` #. :ref:`attrs` #. :ref:`exception` #. :ref:`private` #. :ref:`vtoken` #. :ref:`pigtail` .. _funcs: A module with some functions ---------------------------- Suppose we are writing a very simple extension module that provides some global functions, such as a function giving the smaller of two doubles: .. literalinclude:: module1.py :linenos: Note that the decorator '@function(double)' declares the return type as 'double'. More options may be passed to this decorator, such as access, keywords etc.). The Python function *dmax* takes two double arguments *x* and *y*, as specified by the default values *double* (if no default values given, the argument type is taken as *object*). When the default value is an instance (the case with argument *y*), then both the type and the default value for that argument is determined. The call *dmax(-1)* should be equivalent to *dmax(-1,0)* (in other words, when used with one argument, *dmax* gives the positive part of a double value). The module and the function can have their documentations, which will be automatically taken into the generated code, such that the final python extension will come with proper documentation, exactly the Python way! Below is a mapping between python types and C types, so the Python function can be mechanically translated into a C function prototype (more can be added as needed): +------------------+---------------------+-------------------------------+ | Python Type | C Type | C and Python type map | +==================+=====================+===============================+ | :attr:`int` | int | int <==> int | +------------------+---------------------+-------------------------------+ | :attr:`long` | long int | long int <==> int | +------------------+---------------------+-------------------------------+ | :attr:`str` | char \* | char* <==> str | +------------------+---------------------+-------------------------------+ | :attr:`nullstr` | char \* | And: NULL <==> None | +------------------+---------------------+-------------------------------+ | :attr:`unicode` | Py_UNICOCE \* | Py_UNICODE\* <==> unicode | +------------------+---------------------+-------------------------------+ | :attr:`object` | PyObject\* | PyObject\* <==> object | +------------------+---------------------+-------------------------------+ | :attr:`float` | float | float <==> float | +------------------+---------------------+-------------------------------+ | :attr:`Int` | unsigned int | unsigned int <==> int | +------------------+---------------------+-------------------------------+ | :attr:`Long` | unsigned long | unsigned long <==> int | +------------------+---------------------+-------------------------------+ | :attr:`byte` | char | char <==> int | +------------------+---------------------+-------------------------------+ | :attr:`Byte` | unsigned char | unsigned char <==> int | +------------------+---------------------+-------------------------------+ | :attr:`char` | char | char <==> str (length=1) | +------------------+---------------------+-------------------------------+ | :attr:`short` | short int | short int <==> int | +------------------+---------------------+-------------------------------+ | :attr:`Short` | unsigned short | unsigned short <==> int | +------------------+---------------------+-------------------------------+ | :attr:`llong` | long long | long long <==> int | +------------------+---------------------+-------------------------------+ | :attr:`Llong` | unsigned long long | unsigned long long <==> int | +------------------+---------------------+-------------------------------+ | :attr:`double` | double | double <==> float | +------------------+---------------------+-------------------------------+ | :attr:`void` | void | void <==> None | +------------------+---------------------+-------------------------------+ For those types that do not have a Python correspondence, they usually takes a Python type whose C type may be cast into that type. For example, a 'short' integer would actually take an 'int', and then discard the higher bits (Ref: http://docs.python.org/c-api/arg.html). If there are some C types that is not provided above, a special type 'rawtype' can be used to specify C types literally. That is often used in special cases where the prototype must contain some native raw types, such as in special functions of extension types. We will discuss that in more details later on. Suppose the expy package is in place, fire up Python and do this:: >>> import cxpy, expy_module1 as modu >>> cxpy.generate(modu) generating source file: module1.c ...success! Did you notice the "expy\_module1" module we never defined here? The special prefix "expy\_" is prepended to the module name 'module1' here so that the module 'module1' get inspected and prepared for code generation. Also note that this happens *AFTER* we import 'cxpy', as the inspector should be set up in place first. If you forgot to do this, an exception would be thrown when you call *cxpy.generate*. Let's take a look at the generated source file 'module1.h' and 'module1.c'. .. literalinclude:: module1.h :language: c :linenos: .. literalinclude:: module1.c :language: c :linenos: The generated code is actually quite *readable*. A little bit name mangling is used here: the function 'dmax' now becomes 'module1_dmax', and we also have a wrapper funciton 'module1_dmax', to conform to the Python C API. The general rule is that the C function name is prefixed with the module name, connected by an underscore '_', and the wrapper function will be appended with '_wrap'. In the future, this behavior should be configurable. The name mangling is mainly due to the lack of name space in C, for other languages, this might not be necessary. So much for the side talks! Now, let's enjoy our little fruit of labor with the help of the distutils package: .. literalinclude:: module1_setup.py :linenos: Note that the source files are in the *Doc* directory. Now proceed as follows:: ylan@montop:~/expy/Doc$ python module1_setup.py build generating file: /home/ylan/expy/Doc/module1.c ...success! running build running build_ext building 'module1' extension gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.5 -c module1.c -o build/temp.linux-i686-2.5/module1.o gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions build/temp.linux-i686-2.5/module1.o -o build/lib.linux-i686-2.5/module1.so ylan@montop:~/expy/Doc$ cd build/lib.linux-i686-2.5/ ylan@montop:~/expy/Doc/build/lib.linux-i686-2.5$ python Python 2.5.2 (r252:60911, Jul 31 2008, 17:28:52) [GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> from module1 import * >>> help(dmax) >>> dmax(2, 3) 3.0 .. _tags: Insert code using tags ----------------------- If we plan to expose the 'sqrt()' function provided in the library, we need to include the library. Here is the simple code to get this easily done: .. literalinclude:: module1a.py :linenos: The definition of *sqrt(x)* is in the , which is explicitly included here in the document string for the module. The special doc-tag '@head:' *in the beginning of a line* in the document string indicates what follows should be included in the declarations part of the generated code. Similarly, the '@body:' tag indicates what follows should be included in the implementation part of the generated code. Both tags can also be used in the document strings of functions, methods, and classes, with similar effects (what follows the '@head:' tag will be literally included into the declaration part immediately before the generated code for that function, method or class; what follows the '@body:' tag will be literally included into the implementation part immediately before the generated code for that function, method or class. Look also in other examples for the use of these tags. .. _terse: Expressions v.s. code blocks ----------------------------- If the implementation for a function is only a simple expression (such as a simple call to a library function), the expression can be provided instead of a code block. In the example above, we may do something like this:: @function(double) def sqrt(x=double): return "sqrt(x)" Note, this time we only provide an *expression*, instead of a code block that starts with a curly brace "{". With that, the C function "module1a_sqrt" will not be generated, and the wrapper function "module1a_sqrt_wrap" will directly use the expression as is. Still more tersely, we can simply use the Python keyword 'pass', when the function name and parameters are the same as the function to be wrapped (suppose it is already implemented):: @function(double) def sqrt(x=double): pass The 'pass' here is equivalent to providing the expression 'sqrt(x)'. .. _keywords: Allow keywords arguments --------------------------- One more important feature of function calling in Python is that you can have either positional arguments or keywords arguments. By default, cxpy will generate code for positional arguments calling to a function, yet when you choose to enable keywords arguments for your function, simply provide keywords=True option in the declaration. .. literalinclude:: module1b.py :linenos: Take a look at the generated source file 'module1b.c'. .. literalinclude:: module1b.c :language: c :linenos: When you have the extension built, you can call it with keywords like this:: sayHi("Jack", "Tom") sayHi(me="Jack", to="Tom") Of course, in the first call Tom says hello to Jack, while in the second it is Jack who says hello to Tom. .. _retwrap: Customize returned values -------------------------- If you don't like to have cxpy automatically wrap up your returned value, you can wrap up your own by declaring the return type to be 'object', and in your implementation code, you must return a Python object (or, in case of an expression, evaluate to a Py_Object*): .. literalinclude:: module1c.py :linenos: Notice that the returned integer value is wrapped up as an PyInt object, this way the automatic wrap up by expy-cxpy can be avoided. This is very useful when you have some special object type to return. Similarly, if you have some special object type to pass as an argument to your function, you should declare it as an object (the default type). In the same example, we have a function 'getTypeStr' that checks on the type of the passed object and return a string describing the type. This is very useful as you can pass anything as an object to the function (for example, you can pass a tuple to simulate an array). Here is the generated source file 'module1c.c'. .. literalinclude:: module1c.c :language: c :linenos: Note that if you choose to wrap your own returned object, expy automaticlaly take care of increasing the reference count on the returned object. .. _attrs: Add global attributes ------------------------ Defining a global variable in a module is done this way with expy-cxpy: .. literalinclude:: module2.py :linenos: And this is the generated C code: .. literalinclude:: module2.c :language: c :linenos: This is a very simple module just having some global variables. Note, the convenience function create_globals( globals(), ...) creates a bunch of global variables in just one call, and the number of keyword parameters is unlimited. In the next topic, we will discuss how to declare custom exceptions and how to raise exceptions. .. _exception: Exception for arguments/returns ------------------------------- You can raise exceptions based on argument or returned values. The syntax is quite intuitive and it is probably not too hard for you to understand this example below: .. literalinclude:: module1f.py :linenos: Firstly, in the @throws decortor a condition (the first argument 'when') specifies when an exception should be thrown. The condition can either involve arguments or returned values (but not both, thus it is easy to distinguish them). Arguments are specified simply by their names, while the returned value is represented by the string '$ret'. When the condition does not contain '$ret', it is for an argument value exception, and is dealt before knowing the returned value. If the condition has '$ret' there, it is for a returned value exception and is dealt with after the returned value is obtained. You can also specify 'what' to throw (default is 'Exception' in Python). Currently, all standard Python Exceptions/Erros can be thrown. EXPY/CXPY also support custom exceptions (Don't forget to specify the base class, it must be an exception, such as Python 'Exception'). There are four degrees of freedom to throw exceptions, with different values to the three arguments ('when', 'msg', 'code') to the @throw decorator: #. only provide a condition (in 'when') and a fixed message (in 'msg', and the 'code' argument must be at default in this case); #. provide a condition and a format string (in 'msg') and the values (in 'code') separated by comma; #. provide a condition and some native code (in 'code') so that you can throw your custom exceptions (the 'msg' argument must be at default in this case); #. customize condition logic and exception, where the 'when' argument must be either True (for argument excaption) or False (for returned value exception), and the native code is supplied in the 'code'. Note: the order in which the @throws appear is preserved in the generated code for arguments/returns respectively (exceptions on arguments always goes before those on returns, of course). .. literalinclude:: module1f.c :language: c :linenos: Finally, it is certainly OK to manage exceptions in your own block of function code, and the generated wrapper function would first check if any exception occurred first, and would return NULL immediately upon any exception. The above example is only for the purpose of illustration, below is a more efficient implementation: .. literalinclude:: module1g.py :linenos: .. _private: Functions invisible to Python ------------------------------ If you would like to define a C function that is invisible to Python, declare it as a private function:: from expy import * @function(int, private) def funct(x=int): return """{ // A private function, Python won't know it. // Implementation goes here... return 0; }""" @function(int) def expose(x=int): return "util_funct(x)" In the example above, the first function is declared as private, so that it will not be exposed to Python, and there is only the implementation 'util_funct' generated in the C code. Then in the second function, the C function is called, and its functionality is ported to Python in an indirect way. .. _vtoken: Use 'rawtype' for default values ---------------------------------- If you would like to use a literal expression in C (involving some static variables or macros) as your default value to your argument, you can use the special 'rawtype' thing in your prototype. .. literalinclude:: module1d.py :linenos: Note that a macro '_LEN_' is defined to represent a double constant in the prologue, which is then used as a default value to the argument 'side'. This design can be also used to expose some constants to Python (via global fields or class fields, which will be discussed in more details later). Here is the generated C code: .. literalinclude:: module1d.c :language: c :linenos: Another good use of 'rawtype' is when you need to specify the C type by giving the literal:: def __print__(me, file=rawtype('FILE*'), flags=int): this is often used for special methods in writing classes/types extensions (see the next topic). .. _pigtail: Add pigtails to string arguments --------------------------------- When parsing a string, its length can also be determined. The Python C API provides this convenience. To use this feature, expy-cxpy provides a special use of the 'rawtype', that will help take advantage of this feature. .. literalinclude:: module1e.py :linenos: Note that the pigtail argument is not considered a real argument to the Python function prototype, but it is a real argument to the implementation prototype in C. Here is the generated C code: .. literalinclude:: module1e.c :language: c :linenos: