Mentor

Joe Groff

Summary

Factor's graphical user interface library, written completely in Factor, currently uses OpenGL as its cross-platform basis. Since OpenGL is geared mainly toward low-level access to GPU hardware, it is in many ways unsuitable for this purpose: driver bugs, problems dealing with multiple-monitor setups, and complicated management of GPU resources for optimal UI performance have all affected the Factor UI in the past. Modern desktop platforms have native vector graphics libraries with fairly congruent APIs (CoreGraphics on Mac OS X, cairo on X11, GDI+ on Windows XP and later, and Direct2D on Windows 7). Factor also eventually hopes to target mobile platforms, many of which either don't have OpenGL at all or have only OpenGL ES, but provide advanced 2D graphics APIs comparable to CoreGraphics or Cairo. Developing a new UI framework based on a cross-platform wrapper for these vector APIs would provide the Factor UI with a simpler, more robust, more portable, and more powerful foundation.

Existing work

There are bindings to the Cairo library, along with support libraries to render Cairo to an image and display the output as an OpenGL texture. Since Cairo is available on all platforms Factor supports, it could be used as a starting basis for the vector UI until bindings to GDI+ and Core Graphics are developed. Factor also already uses native libraries for text rendering (pango on X11, Core Text on Mac, and Uniscribe on Windows). All of these text libraries are designed to readily integrate with their corresponding graphics library. (However, a DirectWrite backend for text rendering is currently lacking, which we would want to use to render text with a Direct2D vector backend.)

Technical details

There are 3 broad phases to the project.

Bindings to vector APIs

This involves using the Factor FFI to bind to Core Graphics on OS X and GDI+/Direct2D on Windows to accompany the existing Cairo binding. This phase could potentially be postponed, with only one API (such as Cairo) being used as a backend to start out.

Design Factor vector API

All of these vector APIs are very similar:

• set up a graphics context attached to a window or offscreen buffer
• set context properties
• - affine transform
• - clipping shape
• - stroke width, color, miter, and cap
• - fill color or pattern
• - push/pop context state
• draw paths: move to, line to, spline to
• draw text glyphs (can interface with existing ui.text backends)
• draw images
• record and play back patterns

The student would design a Factor API for these operations mapping to platform-specific library calls.

Build the UI on top of vector API

At a high level, the Factor UI consists of a hierarchy of gadget objects. This API can likely remain unchanged. What will need to change is the code that draws these gadgets. In many cases, the new drawing code will be much simpler; the extensive use of images in the current Factor UI could be replaced with vector drawing code. The text framework could also be streamlined, since it would not need to go through the intermediate step of drawing text to an OpenGL texture. At a lower level, the code that opens and initializes windows on each platform will need to change to make views suitable for 2D rather than for OpenGL rendering.

Bonus work

We would of course like to retain the ability to create OpenGL and DirectX windows for use in Factor games. The window protocol could be extended to allow for different window initialization modes (for vectors, for OpenGL, for DirectX, etc.) Integrating support for rendering vector images (and by extension, entire gadgets) to GPU framebuffers or textures would be handy so that the UI framework could still be easily integrated with games.

Benefit to student

The student learns about modern 2D graphics APIs and UI framework design and development.

Benefit to community

Factor has a more robust cross-platform UI foundation.