Rendering engine for Vy (part 1)

I am working on the rendering engine of a Vy, a vi engine written in python.

The line rendering is a fundamental piece of text editor. The result of the rendering is the one thing that the user sees. If it is broken, or it looks ugly, the editor is not worth its line count: a programmer will spend hours, days and nights in front of its editor. He wants good rendering.

Another aspect is that rendering needs to be quick. Rendering occurs every time the programmer enters few characters, every time he adds or remove a line, every time he scroll his window. So basically, that's all the time. If that's slow, that's impeding the programmer's productivity, which is the opposite goal of an editor.

What is the proper way of dealing with a piece of code that must be good and quick, and that is the core of a functionality ? For me, the answer is simple. I need to isolate that piece of code from the rest of my code. And I need to test it! And test it again; as much as possible; from every possible angle.

Rendering can be complicated. There are lot of corner cases. So I need one test for every corner case, because I want to be sure that I handle all of them properly. And each time I see a bug for a corner case that I did not think of, I'll write a new test.

Isolating rendering code is very important. In Yzis, the rendering is done inside paint events, so it is quite difficult to test and validate it. Wrong! The good approach that I am taking is to isolate the rendering code in a class. The rendering consists in taking a few inputs, the buffer content, the screen size, a few rendering options, and producing a representation of the final text with information on how to paint it on screen. In my case, that representation is a list of rendering lines, and a rendering line is a list of references to some text (the initial text) and attributes on how to draw this text. With such isolation, I can fully test my rendering. And painting it will be also simpler.

Optimizing the rendering is important. But optimization can not start before code isolation and testing. Once I have a piece of code that is isolated from the rest of my application, with a full test suite, I can start tweaking the code for optimization. Actually, I will never start by tweaking the code. My internal rules for optimization are:

1. Measure. I can not optimize anything if I don't know how much time it takes. So, first step is to measure the operation I want to work on.

2. Build a usage scenario. I want to optimize for real use case, not for hypothetical use case. For example, a user can not type more than 20 characters in one second, but he can past 100 characters at once. That's an important information to measure rendering performance. I will handle typing and pasting with a different algorithm.

3. Analyze. I want to analyze my code flow, I understand it. Why is action X done at step A and action Y done at step B ? Profiling can be a part of the analysis. But don't reduce code optimization to profiling. Profiling is useless if you don't know why you are doing some operation. In many cases, optimization is about changing the global code flow, not changing how a small part of it behaves.

4. Finally, Optimize. Work a new algorithm. Try out new stuff. I have my use cases, my analysis data, my test suite, I can try all the stuff I want. Computer science is a nice area where common sense and a few heuristics can perform better than the mathematically optimized solution. So I am checking my use cases to see if you can grab an extra trick where things will speed up in a often encountered situation, that is simpler to handle than the general situation.

In my case, the rendering is done in python. And python may be slow. So rendering code is the number one candidate for rewrite in C++. But before that, I will try many different algorithm, and I will measure, measure, measure. Because my rendering code is isolated in a class, at any time, I can substitute my python class with a C++ class and measure again. I will develop all my rendering algorithm in python. Python is easy to code and slow to execute. That's the perfect environment for trying new optimization, and evaluate the impact of their results. When I know that an algorithm is working, I will probably recode it in C++. But not before I have the working and tested python version.