[FOSDEM 2014] Utilizing GPUs to accelerate 2D content

By: TalksDump

10   0   1703

Uploaded on 02/11/2014

Utilizing GPUs to accelerate 2D content

Speaker: Bas Schouten

Over the last 15 years, GPUs have gone from being a piece of hardware found almost exclusively on the machines of gamers to being present in almost every single desktop and laptop computer. This hardware presents opportunities to greatly improve power usage and performance for graphics applications. Over the last 5 years GPU utilization in the desktop application world for accelerating 2D graphics has slowly moved forward, however their intended use for video games also presents us with a number of limitations.

Over the last 15 years GPUs have gone from being a piece of hardware found almost exclusively on the machines of gamers, to being present in almost every single desktop and laptop computer. This hardware presents opportunities to greatly improve power usage and performance for graphics applications. Over the last 5 years GPU utilization in the desktop application world for accelerating 2D graphics has slowly moved forward, however their intended use for video games also presents us with a number of limitations.

In this presentation I will talk about what GPUs are, why we want to use them, in what different ways they can be put to use, and some of the challenges we've encountered when using them at Mozilla. I will also try and touch on some of the technical details on the different tradeoffs that the most common algorithms present.

Comments (2):

By anonymous    2017-09-20

Theory

Why it is hard

Popular font formats like TrueType and OpenType are vector outline formats: they use Bezier curves to define the boundary of the letter.

Image source.

Transforming those formats into arrays of pixels (rasterization) is too specific and out of OpenGL's scope, specially because OpenGl does not have non-straight primitives (e.g. see Why is there no circle or ellipse primitive in OpenGL?)

The easiest approach is to first raster fonts ourselves on the CPU, and then give the array of pixels to OpenGL as a texture.

OpenGL then knows how to deal with arrays of pixels through textures very well.

Texture atlas

We could raster characters for every frame and re-create the textures, but that is not very efficient, specially if characters have a fixed size.

The more efficient approach is to raster all characters you plan on using and cram them on a single texture.

And then transfer that to the GPU once, and use it texture with custom uv coordinates to choose the right character.

This approach is called a https://en.wikipedia.org/wiki/Texture_atlas and it can be used not only for textures but also other repeatedly used textures, like tiles in a 2D game or web UI icons.

The Wikipedia picture of the full texture, which is itself taken from freetype-gl, illustrates this well:

I suspect that optimizing character placement to the smallest texture problem is an NP-hard problem, see: What algorithm can be used for packing rectangles of different sizes into the smallest rectangle possible in a fairly optimal way?

The same technique is used in web development to transmit several small images (like icons) at once, but there it is called "CSS Sprites": https://css-tricks.com/css-sprites/ and are used to hide the latency of the network instead of that of the CPU / GPU communication.

Non-CPU raster methods

There also exist methods which don't use the CPU raster to textures.

CPU rastering is simple because it uses the GPU as little as possible, but we also start thinking if it would be possible to use the GPU efficiency further.

This FOSDEM 2014 video https://youtu.be/LZis03DXWjE?t=886 explains other existing techniques:

Fonts inside of the 3D geometry with perspective

Rendering fonts inside of the 3D geometry with perspective (compared to an orthogonal HUD) is much more complicated, because perspective could make one part of the character much closer to the screen and larger than the other, making an uniform CPU discretization (e.g. raster, tesselation) look bad on the close part. This is actually an active research topic:

enter image description here

Distance fields are one of the popular techniques now.

Implementations

The examples that follow were all tested on Ubuntu 15.10.

Because this is a complex problem as discussed previously, most examples are large, and would blow up the 30k char limit of this answer, so just clone the respective Git repositories to compile.

They are all fully open source however, so you can just RTFS.

FreeType solutions

FreeType looks like the dominant open source font rasterization library, so it would allow us to use TrueType and OpenType fonts, making it the most elegant solution.

Examples / tutorials:

Other font rasterizers

Those seem less good than FreeType, but may be more lightweight:

Anton's OpenGL 4 Tutorials example 26 "Bitmap fonts"

The font was created by the author manually and stored in a single .png file. Letters are stored in an array form inside the image.

This method is of course not very general, and you would have difficulties with internationalization.

Build with:

make -f Makefile.linux64

Output preview:

enter image description here

opengl-tutorial chapter 11 "2D fonts"

Textures are generated from DDS files.

The tutorial explains how the DDS files were created, using CBFG and Paint.Net.

Output preview:

enter image description here

For some reason Suzanne is missing for me, but the time counter works fine: https://github.com/opengl-tutorials/ogl/issues/15

FreeGLUT

GLUT has glutStrokeCharacter and FreeGLUT is open source... https://github.com/dcnieho/FreeGLUT/blob/FG_3_0_0/src/fg_font.c#L255

OpenGLText

https://github.com/tlorach/OpenGLText

TrueType raster. By NVIDIA employee. Aims for reusability. Haven't tried it yet.

ARM Mali GLES SDK Sample

http://malideveloper.arm.com/resources/sample-code/simple-text-rendering/ seems to encode all characters on a PNG, and cut them from there.

SDL_ttf

enter image description here

Source: https://github.com/cirosantilli/cpp-cheat/blob/d36527fe4977bb9ef4b885b1ec92bd0cd3444a98/sdl/ttf.c

Lives in a separate tree to SDL, and integrates easily.

Does not provide a texture atlas implementation however, so performance will be limited: Rendering fonts and text with SDL2 efficiently

Related threads

Original Thread

Submit Your Video

If you have some great dev videos to share, please fill out this form.