WAF in GNOME

I have file GNOME bug #503907, which will hopefully allow me to be the one to introduce WAF into the first couple of GNOME modules. Let us hope this marks the start of a new revolution with the intent of ending the reign of autotools in the GNOME realm :-)

PyBindGen 0.8

Today I have unleashed PyBindGen version 0.8 to the world:

=== pybindgen 0.8 ===
- Support C++ instance attributes through getter/setter methods
- Support functions as methods of C++ classes
- Support the PyObject* type
- Support unsigned int, C strings (char*) (from Mark Lee)
- Add basic support for enum types
- New experimental automatic module generator based on C/C++
header file scanner and annotations in comments, using pygccxml
- Some bug fixes

pybindgen + pygccxml

Last weekend I started a pygccxml based header file scanner for pybindgen. When it's finished, it will allow pybindgen to automatically scan a module's definition directly from header files, with little human intervention, similarly like Py++.

Progress has been good, but of course I have little time to work on it. I'll report back on this.

pybindgen almost feature complete

During August I got sick, and the weather was lousy anyway, so I decided to continue development of PyBindGen. Since then I have finished most features that will be needed at least for NS-3, which is very C++ demanding. The list of features implemented is summarised in the project page, and I won't bother to repeat them here.

So what's left to do now?

1. Testing and bug fixing; I'm sure there are plently of bugs in there waiting to be found and fixed, since the generator has had very little real world testing so far;
2. Adding support for some Python object special methods and protocols, such as __str__, __add__, __sub__, etc.; also mapping C++ iterators to Python iterators and vice versa, etc;
3. Start thinking of a automatic code scanning layer on top of the pybindgen API, which scans C/C++ header files and tries to automatically 'guess' the definitions, a la Py++.

Fortunately the hardest part, the foundation, is already done, and I'm pretty happy with it: it is 100% Python (except the generated code), it's well structured and modular, and uses the standard/simple/well document Python/C API, rather than obscure C++ template metalanguage.

PyBindGen Type Narrowing

Today I finished another important feature in my python bindings generator:
automatic type narrowing of C++ class pointers. It was implemented with a std::map<const char*, PyTypeObject*>, mapping type names to Python wrappers, and the typeid C++ operator, which uses RTTI to return type information in runtime, including type name.

This comes on the heels of supporting function/method/constructor overloading, finished a few days ago. I'm getting increasingly optimistic about pybindgen. It does few things, but does them well. It will hopefully make the (python) world a better place to live in :P

Ubuntu Dell PCs in Europe

Nice to see Dell Ubuntu PCs being sold in Europe. However, one has to wonder why the laptops being sold in Europe are at most Intel Dual Core and Intel 950 graphics, noticeable inferior to those being sold in the U.S., namely Intel Core2 Duo and Intel 965 graphics. What's wrong? Do Europeans require less powerful hardware? Are we being discriminated? *sigh*

GEdit with Emacs Colors

Following up on Paolo Borelli's post about a Dark Tango gedit color scheme, I decide to write a color scheme based on the default GNU Emacs colors circa RedHat 5.2. It basically has a dark slate grey background, and wheat foreground. All these years, since RedHat 5.2, my first Linux distribution, I got used to GNU Emacs as my editor of choice for serious tasks (programming, LaTeX).

I have customized a few things from the default Emacs color scheme, most notably getting rid of the awful red foreground for comments (ugh!) in favour of a less intrusive grey.
The other notable change that I made was the addition of syntax highlighting for symbols, including operators, parenthesis, brackets, curly braces, etc., because I believe it really helps read the code better. Emacs in fact does not natively support highlighting of symbols, but since it is highly extensible I was able to add support for it.

Anyway, my emacs color scheme for gedit can be found here. Playing a bit with gedit with this color scheme, once again I tried to get the feeling on whether I could ever switch from emacs to gedit, and the answer is sadly still no. Here's some of the reasons:

1. Gedit still does not have the notion of indentation offset, and only has tab width. In emacs, you can have tab width set to 8, but if you set indentation offset to 4 then pressing tab indents with 4 spaces, not one tab. In gedit you have to change the tab width to 4, which has disastrous side effects on source files that are using real tab characters for indentation. This is a really serious problem IMHO.
   
2. Gedit is generally not very smart about indentation. Emacs usually knows how to indent a line based on syntax analysis of the surrounding context. I guess I can leave without it, but it would be really nice to have. But, yes, I realize this is very hard to implement for only a little gain.
   
3. Gtk's default cursor is very thin and easy to lose track of it. In contrast, Emacs' cursor is big and blocky, but at least you never lose track of it. Fortunately I was able to kind of work around with a gtkrc style file, although it's not perfect.
4. I already mentioned the symbols/operators problem. Gedit does not natively support operators in most language files; at least not C/C++/Python, which is the ones I care about. I'm not sure if I can fix this easily. I can define color schemes, but I wonder if I can customize language definitions, add new information to them in a inheritance style?
5. The modelines plugin needs more work; it keeps ignoring many of the modelines on my files.
   

In conclusion, there are some problems in gedit and I will not switch to it yet. However, it does support Python plugins, and therefore most of the problems should be fixable given a bit of time and patience. Lucky for me I'm on vacations, so I might yet revisit this issue before the end of the month.

pysed

In case anyone finds this useful, I made a small sed-like program in Python. I call it pysed.

The rationale for this tiny program is that I often want to make regexp substitutions but find the classic regular expressions syntax of sed (and emacs) outdated and not as powerful as Python's. For example, suppose you want replace ".m_" with "." except when preceded by "m_state". You do:

pysed -i -e '(?<!m_state)\.m_' -s '.' file.cc

With traditional sed, or with gnu emacs, it does not look like this is possible, as far as I can tell...

python bindings generator

For the ns-3 project, I've been trying for some time to create some Python bindings. I've been using Py++, which uses gccxml to parse C++ header files and automatically generates Boost.Python code that tries to wrap it.

Py++ is really nice. Unfortunately Boost.Python isn't nearly at the same level. Boost.Python is heavily built on C++ templates. Mastering C++ templates is Hard™. Everything Boost does is practically black magic to me. While using Py++, Boost.Python has been fighting with me every step of the way, and winning too!

I decided to move away from Boost once and for all. Some of the non-boost alternatives I could find were:

1. Python-SIP: the code generator itself is written in C++; that doesn't make any sense! I want something I can easily hack;
2. SWIG: also made in C/C++, and in addition generates very ugly code, almost unreadable with so much macro abuse;
3. The PyGtk code generator. I know this one very well :) Unfortunately it has some problems, the biggest of which is that it doesn't support C++ classes, while the ns-3 code I want to wrap is pure C++.

So, in the end, there really isn't any good python bindings generator tool out there. I decided to create my own. I've set myself the following goals for my forthcoming tool:

• Must be written in pure Python, and in good pylint-checked style;
• Must have unit tests;
• It should have a simple python API to generate the code, other frontends (such as header file scanning) optional and layered on top of the python API;
• Must be easily extensible to handle new types, with strong focus on correct memory management of type convertion code;
  

The project is called PyBindGen (yes, I'm from a school of thought that believes project names should be descriptive of their function). First pre-alpha release is ready, supporting generation of modules with only functions, and only the basic python types. However it already supports something not supported by pygtk-codegen: out/inout parameters and, consequently, multiple return values.


bzr (0.15) branch: http://telecom.inescporto.pt/~gjc/pybindgen/bzr
Releases: http://telecom.inescporto.pt/~gjc/pybindgen/releases
Launchpad (bug reporting, etc.): https://launchpad.net/pybindgen/

The project uses WAF to build; Running "./waf" should be enough to build the example. "./waf check" should be able to run the unit tests.

Publication

Yesterday I received a notice:

I am glad to inform you that the revision of your paper, “QoS Abstraction Layer in 4G Access Networks”(TS2006-69) was approved for publication in Telecommunication Systems.

/me is very happy.

OpenOffice.org still sucks, only a bit less

So it seems that OpenOffice.org 2.1, present in Ubuntu Feisty, finally has a native AMD64 executable, which should improve its memory requirements and make copy-paste less buggy. Unfortunately it seems you still are unable to insert cross-references to numbered items in enumerated lists, which is the technique of choice of MS Word users to insert bibliography references and lists. And of course creating a "Bibliography Database" is both non-trivial and very buggy. I tried it again but had to give up.

Thankfully I can still use LaTeX, which has much better quality anyway...

dijkstra in python

Given an arbitrary network, I knew that Dijkstra's algorithm computes the shortest path between any pair of nodes of the network. In a distributed routing protocol, Dijkstra is often used to compute shortest paths to destinations. I already knew, but wanted to confirm, that for any shortest path computed centrally from one edge node to another, you could apply the same Dijkstra algorithm to each intermediate node in the path and obtain the same result. So I put my Python coding skills to good use and created a small program that generates lots of random graphs with lots of nodes and lots of links with random "distances" and, for each such graph picks a couple of nodes and checks the assertion above. I only had to convert the Dijkstra algorithm expressed as pseudo-code in wikipedia to Python, which was pretty easy. The assertion is confirmed.
/me loves Python.

GNOME CVS to Subversion migration

CVS is dead, long live Subversion!