Tutorials that extend the Python program and Zope server using the C language

There are several reasons why you might want to extend Zope with C. Most likely, you have an existing C library that already does something for you, but you're not interested in converting it to Python. Also, because Python is an interpreted language, any Python code that is heavily called will slow you down. Therefore, even if you have already written some of the extensions in Python, you should still consider writing the most frequently invoked parts in C instead. Either way, extending Zope starts with extending Python. In addition, extending Python gives you other benefits because your code will be accessible from any Python script, not just from Zope. The only caveat here is that at the time of this writing, Python's current version is 2.1, but Zope still only works with Python 1.5.2. There is no change between the two versions of the C extension, but if you are interested in wrapping your libraries in Python and want them to work under Zope, you should be careful not to use anything newer than 1.5.2.
What is Zope?

Zope stands for "Z Object Publishing Environment (Z object publishing environment)", which is an application server implemented with Python. "Great," you say, "but what exactly does an application server mean?" An application server is a long-running process that serves "active content." The Web server calls the application server at run time to build the page.
Extending Python: fun and helpful

To extend Zope, you first need to extend Python. While extending Python isn't as complicated as "brain surgery," it isn't as leisurely as "walking in the park." There are two basic components for the Python extension. The first is obviously the C code. I'll talk about that in a second. The other section is the installation file. The installation file describes the module by providing the module name, the location of the module's C code, and any compiler flags you may need. This file is preprocessed to create makefile (on UNIX) or MSVC++ engineering files (MSVC++ project file, on Windows). Let's start with 1. Python on the Windows is actually compiled using the Microsoft compiler. People at Python.org also recommend MSVC++ for compiling extensions. Obviously, you should be able to convince the editors of GNU, but I haven't tried it myself.

Anyway, let's define a module called 'foo'. The 'foo' module will have a function called' bar'. When we want to use it, we can use import foo; To import this function into the Python script, just as you would import any module 1. The installation file is very simple:
Listing 1. A typical installation file

# You can include comment lines. The *shared* directive indicates
# that the following module(s) are to be compiled and linked for
# dynamic loading as opposed to static: .so on Unix, .dll on Windows.
*shared*
# Then you can use the variables later using the $(variable) syntax
# that 'make' uses. This next line defines our module and tells
# Python where its source code is.
foo foomain.c

Write the code

So how do we actually write code that Python knows how to use, you ask? The foomain.c (you can name it however you like, of course) file contains three things: a method table, an initialization function, and the rest of the code. The method table simply associates function names with functions and informs the Python of the parameter passing mechanism used by individual functions (you can choose to use a 1-like list of location parameters or a mixed list of location parameters and keyword parameters). Python calls the initialization function when the module loads. The initialization function does all the initialization required by the module, but more importantly, it also returns a pointer to the method table to Python.

Let's take a look at the C code for our small foo module.
Listing 2. A typical Python extension module

#include <Python.h>
/* Define the method table. */
static PyObject *foo_bar(PyObject *self, PyObject *args);
static PyMethodDef FooMethods[] = {
  {"bar", foo_bar, METH_VARARGS},
  {NULL, NULL}
};
/* Here's the initialization function. We don't need to do anything
  for our own needs, but Python needs that method table. */
void initfoo()
{
  (void) Py_InitModule("foo", FooMethods);
}
/* Finally, let's do something ... involved ... as an example function. */
static PyObject *foo_bar(PyObject *self, PyObject *args)
{
  char *string;
  int  len;
  if (!PyArg_ParseTuple(args, "s", &string))
    return NULL;
  len = strlen(string);
  return Py_BuildValue("i", len);
}

Further study of

Let's look at this code for a second. First, note that you must include Python.h. Unless you have set the path to the file in the include path (include path), you may need to include the -I flag in the installation file to point to the file.

The initialization function must be named init < Module name > , in our example, initfoo. The name of the initialization function is, no doubt, all that Python knows about the module when it is loaded, which is why the name of the initialization function is so rigid. By the way 1, the initialization function must be the only global identifier in the file where 1 is not declared as static. This is more important for static links than for dynamic links, because the non-static identifier will be globally visible. This isn't a huge problem for dynamic linking, but if you're going to link everything at compile time and don't declare static as static for everything that can be declared as static, you're likely to have name conflicts.

Now let's look at the actual code to see how the parameters are handled and how the return value is passed. Of course, all the 1 cuts are objects on the PyObject and Python heap. What you get from the argument is a reference to the "this" object (this for object methods, NULL for older, parameterless functions like bar()) and a parameter tuple stored in args. You retrieve your parameters with PyArg_ParseTuple, and return the results with Py_BuildValue. These functions (and many more) are documented in the "Python/C API" section of the Python documentation. Unfortunately, there is no simple list of functions arranged by name; the documentation is arranged by topic.

Also notice that the function returns NULL in the case of an error. Returning NULL indicates an error; If you want Python to do better, you should throw an exception. I'll refer you to the documentation on how to do this.

Compile the extension

All that remains now is the matter of compiling the module. You can do this in two ways. The first is to run make-f Makefile.pre.in boot as instructed in the documentation, which will compile one Makefile using your Setup. You then compile your project with this Makefile. This only works for UNIX. For Windows, there is a script called "compile.py" (see resources later in this article). The original script is hard to find; I found a heavily altered copy of Robin Dunn (the people behind wxPython) from a mailing list. This script works on UNIX and Windows; On Windows, it will compile the MSVC++ project file from your Setup.

To compile, you must make both the included files and libraries available. The standard Zope installation of Python does not include these files, so you will need to install the regular installation of Python from www.python.org (see resources). On Windows, you must also get the config.h file from the PC directory where the source code is installed; It is a manual version of config.h installed for you. So, on UNIX, you should already have it.

Once this is done, you will have a file with the extension ".pyd ". Put this file in the "lib" directory under the Python installation directory (under Zope, Python is in the "bin" directory, so your extension ends up in the "bin/lib" directory, oddly enough). You can then call it, just as you would any source-generated Python module 1.

 >>> import foo;
 >>> foo.bar ("This is a test");
 14

When I got there, my first question was how do I use C to define classes that are visible from Python. In fact, I may have asked the wrong question. In the examples I've looked at, the Python-specific 1 cuts are all done with Python only, and they all call C functions that are exported from your extension.

Take it to Zope

Once you have completed your Python extension, the next step is to make Zope work with it. There are several ways you can choose, but to a certain extent, how you want your extension to work with Zope 1 will affect the way you compile your extension in the first place. The basic way to use Python (and the extension made with C) code from within Zope is:

If the function is simple, you can think of it as a variable. These are called "external methods."       is a more complex class that can be called from the Zope script (this is a new feature in Zope 2.3).       you can define one Zope Product, then extend it with ZClass (1 set of ready-made, Web accessible objects), use it in scripts, and publish it according to its own permissions (its instances are treated as pages).

Of course, your own applications can use a combination of these approaches.

Creating external methods

The easiest way to call Python from Zope is to make your Python code an external method. The external method is the Python function placed in the "Extensions" directory under the Zope installation directory. Once you have such an Python file in one day, you can go to any folder, select "add external methods," and add variables to call the function you want to use. You can then add the DTML field to any page in the folder that displays the result of the call. Let's look at a simple example that USES the Python extension, foo.bar, defined above.

First, let's look at the extension itself: let's put it in a file called foo.pyd, for example. Remember, this file is in the Extensions directory under Zope. In order for this to work, of course, the foo.pyd we created above must be in the Python library located at bin/lib. A simple package for this purpose might look like this:
Listing 3. A simple external method (file: Extensions/ foo.py)

import foo
def bar(self,arg):
  """A simple external method."""
  return 'Arg length: %d' % foo.bar(arg)

Simple, isn't it? It defines an external method "bar" that can be attached to any folder using the Zope administrative interface. To call our extension from any page in the folder, we simply insert a reference to the DTML variable, as shown below:

 <dtml-var bar('This is a test')>

When the user views our page, the DTML field is replaced by the text "Arg length: 14". That's how we extended Zope with C.

Zope script: Cliff Notes version

The Zope script is a new feature of Python 2.3 that is intended to replace external methods. It does everything external methods can do, and it integrates better with security and management systems, provides more flexibility in integration, and it has access to all of the Zope capabilities exposed in Zope API.

A script is basically a short Python program. It can define classes or functions, but it does not have to. It is installed as an object in the Zope folder, and can then be called as an DTML variable or call (like an external method) or "from Web" (meaning in Zope it will be called as a page). Of course, this means that scripts can generate responses to form submissions just as CGI programs do, but without the overhead of CGI. It's a great feature. In addition, the script has access to the caller or caller object (via the "context" object), the folder where the object resides (via the "container" object), and other bits and pieces. For more information on scripting, see chapter 1 of "advanced Zope scripting (Advanced Zope Scripting)" in the Zope manual (see resources).

You may mistakenly think that you can simply import foo directly from the script and use foo.bar (I know I did make this mistake). But that's not the case. Due to security constraints, only Product can be imported, not any module. 1 generally speaking, the designers of Zope believe that any scripting requires access to the file system, and since script objects are managed by Web using the Zope administration interface, they are not fully trusted. So I'm going to stop there and not show you the sample script, but talk about Product and the base classes.

Focus on Product

Product is a powerful tool for extending Zope. At the installation directory level, Product is one of the directories in the "lib/python/Products" directory located in the Zope directory. You can see many examples of product in your own Zope installation directory, but essentially, the smallest Product consists of only two files located in that directory: one arbitrarily named code file and one Zope called s 312en__ which is used to initialize Product at startup time.py. (note: Zope only reads Product files at startup, which means you must be able to stop and restart the Zope process for testing). This article is just trying to provide as many hints as possible about what you can do with Zope Product.

Be aware that Product encapsulates one or more classes that can be used from ZClass, scripts, or directly from URL on Web. (of course, in the last case, instances of Product are treated as folders; The last section of URL specifies the method to be called, which returns any HTML. You don't have to treat Product as an "addable" object, although that's its primary purpose. For a good, real-world example, look at the ZCatalog implementation, which is part 1 of the standard Zope distribution. There you can see a very simple installation script in s331en__.py, and you can see the ZCatalog class in ZCatalog.py, which provides many distribution methods. Note that Zope USES a strange convention to determine which methods can access a wok through Web. If a method contains an doc string, it can be accessed through Web. Otherwise, it is considered private.

Anyway, let's look at a very simple Product that USES the C module (which we defined above). First of all, s very simple s s 344en__. Notice that it only does one thing, which is to tell Zope the name of the class we are installing. More complex initialization scripts can do much more, including declaring global variables maintained by the server and setting access rights. For more details, refer to the Zope developer's guide in the online documentation, and also check out the Product already available in your Zope installation directory. As you might have guessed, our example Product is called "Foo". You will then create an Foo subdirectory in the lib/python/Products directory.
Listing 4. The basic Product initialization script

import Foo
def initialize(context):
  context.registerClass(
    Foo.Foo, 
    permission='Add Foo',
    constructors=Foo.manage_addFoo
    )

Now notice that this initialization script not only imports that class, making it accessible to other parts of Zope, but also registers the class as "addable." The context.registerClass call does this by first naming the class we imported and then specifying the name of the method that can be used to add an instance (this method must display a management page, and the method will automatically integrate with the Zope administration interface). Cool.

Let's summarize this short, simple Product. It exposes our foo.bar functions to the script and ZClass, and it also has a small interface as an "addable" object, and that's all.
Listing 5. A simple Zope Product

import foo
class Foo(SimpleItem.Item):
 "A Foo Product"
 meta_type = 'foo'
 def bar(self, string):
   return foo.bar(string)
 def __init__(self, id):
   "Initialize an instance"
   self.id = id
 def index_html(self):
   "Basic view of object"
   return '
My id is %s and its length is %d.
' % (self.id, foo.bar(self.id))
 def manage_addFoo(self, RESPONSE):
   "Management handler to add an instance to a folder."
   self._setObject('Foo_id', Foo('Foo_id'))
   RESPONSE.redirect('index_html')

This is just one of the simplest Product. It's not absolutely the smallest possible Product, but it's close. However, it does illustrate some key features of Product. First, notice the "index_html" method: it is called to display an object instance, which is done by building HTML. It's actually one page. The manage_addFoo method is the Zope object management interface. We have quoted it in the above s 386en__.py. The "s 388en__" method initializes the object; All it really has to do is record the instance's 1-only identifier.

This tiny Product does not interact with Zope security. It doesn't do a lot of administration. It has no interaction. So you can add a lot of things to it (even useful features it doesn't have). I hope this is a good start for you.

What should I do next

A brief introduction to Zope Product has shown you how to change C language functions from C code to Zope. To learn how to write Product, you'll have to read more documentation (many of which are still being refined) and, frankly, study the existing Product to see how they work. The Zope model has a lot of power and flexibility, both of which are worth exploring.

I'm currently working on a big project to integrate C and Zope: integrating my workflow toolkit (workflow toolkit). Before this article is published, I'd like to see it in its early form. It's listed in the resources below, check it out; By the time you read this article, you should be able to find an example of an extension. Wish me luck.