Rendering engine for Vy (part 1)

I am working on the rendering engine of a Vy, a vi engine written in python.

The line rendering is a fundamental piece of text editor. The result of the rendering is the one thing that the user sees. If it is broken, or it looks ugly, the editor is not worth its line count: a programmer will spend hours, days and nights in front of its editor. He wants good rendering.

Another aspect is that rendering needs to be quick. Rendering occurs every time the programmer enters few characters, every time he adds or remove a line, every time he scroll his window. So basically, that's all the time. If that's slow, that's impeding the programmer's productivity, which is the opposite goal of an editor.

What is the proper way of dealing with a piece of code that must be good and quick, and that is the core of a functionality ? For me, the answer is simple. I need to isolate that piece of code from the rest of my code. And I need to test it! And test it again; as much as possible; from every possible angle.

Rendering can be complicated. There are lot of corner cases. So I need one test for every corner case, because I want to be sure that I handle all of them properly. And each time I see a bug for a corner case that I did not think of, I'll write a new test.

Isolating rendering code is very important. In Yzis, the rendering is done inside paint events, so it is quite difficult to test and validate it. Wrong! The good approach that I am taking is to isolate the rendering code in a class. The rendering consists in taking a few inputs, the buffer content, the screen size, a few rendering options, and producing a representation of the final text with information on how to paint it on screen. In my case, that representation is a list of rendering lines, and a rendering line is a list of references to some text (the initial text) and attributes on how to draw this text. With such isolation, I can fully test my rendering. And painting it will be also simpler.

Optimizing the rendering is important. But optimization can not start before code isolation and testing. Once I have a piece of code that is isolated from the rest of my application, with a full test suite, I can start tweaking the code for optimization. Actually, I will never start by tweaking the code. My internal rules for optimization are:

1. Measure. I can not optimize anything if I don't know how much time it takes. So, first step is to measure the operation I want to work on.

2. Build a usage scenario. I want to optimize for real use case, not for hypothetical use case. For example, a user can not type more than 20 characters in one second, but he can past 100 characters at once. That's an important information to measure rendering performance. I will handle typing and pasting with a different algorithm.

3. Analyze. I want to analyze my code flow, I understand it. Why is action X done at step A and action Y done at step B ? Profiling can be a part of the analysis. But don't reduce code optimization to profiling. Profiling is useless if you don't know why you are doing some operation. In many cases, optimization is about changing the global code flow, not changing how a small part of it behaves.

4. Finally, Optimize. Work a new algorithm. Try out new stuff. I have my use cases, my analysis data, my test suite, I can try all the stuff I want. Computer science is a nice area where common sense and a few heuristics can perform better than the mathematically optimized solution. So I am checking my use cases to see if you can grab an extra trick where things will speed up in a often encountered situation, that is simpler to handle than the general situation.

In my case, the rendering is done in python. And python may be slow. So rendering code is the number one candidate for rewrite in C++. But before that, I will try many different algorithm, and I will measure, measure, measure. Because my rendering code is isolated in a class, at any time, I can substitute my python class with a C++ class and measure again. I will develop all my rendering algorithm in python. Python is easy to code and slow to execute. That's the perfect environment for trying new optimization, and evaluate the impact of their results. When I know that an algorithm is working, I will probably recode it in C++. But not before I have the working and tested python version.

Windres problem and solution

A very technical post as first post on my blog. I publish the solution to a problem that we encountered yesterday, in the hope that it will help others.

Windres is an utility part of mingw or cygwin, that one uses on Windows to compile and add resources to an executable. An example of resources are the application icon, or the version information. There are more types but this is usually the one we are concerned about.

Our windres generation line part of our cmake build process is here:

ADD_CUSTOM_COMMAND( 
    OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/qyzis_ico.obj
    COMMAND windres.exe
        -I${CMAKE_CURRENT_SOURCE_DIR}
        -o ${CMAKE_CURRENT_BINARY_DIR}/qyzis_ico.obj
        -i${CMAKE_CURRENT_SOURCE_DIR}/qyzis_ico.rc
)

The problem we had was with path containing spaces. On windows, those are common beast. If you do a checkout on the desktop, the actual path of your checkout is C:\Document and vSettings\[User Name]\Desktop which contain spaces. If you run windres on a file in such a path, you get an error about gcc not finding the file. Why gcc while we are talking windres. The man page of windres gives the hint:

--preprocessor program
  When  windres  reads  an  rc  file,  it  runs  it  through  the   C
  preprocessor  first.   This  option  may  be  used  to  specify the
  preprocessor to use, including any leading arguments.  The  default
  preprocessor argument is gcc -E -xc-header -DRC_INVOKED.

Oh, windres runs the rc files through the gcc preprocessor. But it does not quote them properly so gcc gets confused. We tried several combination of adding quotes and backslash to solve the problem, but it did not work. windres expects the filename unquoted and pass it like that to gcc.

Luckily, there is a way out of this. Two ways actually.

First solution :

There is a bug reported in the mingw bug tracker (I can find the reference now though) about this problem and a proposed fix. The bug has been closed and a fix is included in the mingw CVS. So, for the brave, you can recompile mingw and probably get rid or the problem.

Second solution :

The fix was quite trivial:

ADD_CUSTOM_COMMAND( 
    OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/qyzis_ico.obj
        COMMAND windres.exe
        -I${CMAKE_CURRENT_SOURCE_DIR}
        -o ${CMAKE_CURRENT_BINARY_DIR}/qyzis_ico.obj
        < ${CMAKE_CURRENT_SOURCE_DIR}/qyzis_ico.rc
) 

Instead of specifying a file path to windres, we just pipe the file through its standard input. Works like a charm. You can thank orzel for that solution.