The Raster Tragedy at Low-Resolution Revisited:
Opportunities and Challenges beyond “Delta-Hinting”
In this chapter we will have a look at some of the idiosyncrasies of the TrueType rasterizer that ships with Microsoft Windows. At first glance this may not seem all that useful, or at least not generally so, because it is very specific to one particular implementation of turning fonts into pixels. But this rasterizer is widely used, and it comes with a few unobvious challenges at making fonts look “nice.” Therefore, we need to know these challenges and see how they may be addressed in the constraint system.
TrueType fonts use integer numbers to represent the coordinates of the control points. There are no fractional parts to the coordinates. For instance, in the previous chapter we have looked at the height of the UC ‘H’ of Arial. It was quoted as 1466 font design units, not 1466.3, nor 1466.2357. This may seem to be a rather crude way to approximate the original art work.
However, all these coordinates have to be understood in terms of the em-height (the height of the font)—itself an integer number. For instance, for Arial the em-height is 2048 font design units. In other words, every coordinate in Arial is defined to the nearest 1/2048 of the font height. Two coordinates can differ by as little as approximately one half of one tenth of a percent of the font height.
To put this into perspective, for a 72 pt headline, this difference is about 0.035 pt or 0.012 mm (or 12 µm). This is less than the diameter of the thinnest human hair. To literally see this difference, the healthy human eye would have to observe it from a distance of about 41 mm (1 5/8 in), which is way too close to focus. You’ll need a powerful loupe to have a chance. Accordingly, at this type size and em-height, integers should be precise enough to represent the original art work.
These perspectives change dramatically once we are looking at text sizes and actual screen resolutions. For instance, at 96 DPI and 12 pt, the font height is a mere 16 pixels. Accordingly, a pixel more or less represents a big difference—to wit, 1/16 of the font height. Integer numbers are no longer precise enough to approximate the original art work—or so it seems.
The solution is to look at a suitable fraction of the integer numbers. For instance, a quarter dollar cannot be represented by an integer number of dollars—it is, after all, a quarter. But it can be represented by an integer number of cents. A quarter dollar equals exactly 25 cents, which is an integer number. Seen this way, this would allow to represent coordinates to the nearest 1/100 of a pixel.
The TrueType rasterizer does just that. But instead of decimal fractions (tenths, hundredths, thousandths, etc.), it uses binary fractions. Binary fractions are one half, one quarter (one half of one half), one eighth (one half of one quarter), one sixteenth (one half of one eighth), and so forth. This makes it easy and efficient for the computer to convert to and from fractions.
If the same fraction is used for all calculations, it is called a fixed point binary number. For instance, to represent scaled coordinates, the rasterizer always uses 26 binary digits (or bits) for the “number part” and 6 bits for the “fraction part” (abbreviated as “26.6 number”). With 6 bits for the fractions, every coordinate can be represented to the nearest 1/64 of a pixel. At 96 DPI screen resolution, this corresponds to about 4 µm, which is again precise enough.
The rasterizer uses different fixed point binary numbers for different purposes. As mentioned already, scaled coordinates use 26.6 numbers. By contrast, the scaling factor (cf , , and ) uses a 16.16 number, because it needs more precision, but less “size.” Similarly, the x- and y-components of TrueType’s projection and freedom vectors (the direction of the constraint and the direction of compliance with the constraint) use 2.14 numbers.
By now we have introduced fixed point binary numbers. Their purpose is to continue to use “integers” to represent coordinates, scaling factors, and TrueType’s projection and freedom vectors, but at a higher precision than integers. But working with fixed point binary numbers is not always the same as working with integers.
Adding and subtracting fixed point binary numbers is straightforward. All we are doing is adding or subtracting whole numbers of fractions, like balancing a check book where every check was written in cents, as opposed to dollars-and-cents. As long as we have enough bits for the sum, the results will be exact (cf ).
The problem is the division and—potentially unexpectedly so—the multiplication. Two integers multiply and the result is again an integer. If there are enough bits for the product, the result will be exact. But once we use fractions, the result may become inexact. Here is why (to illustrate the point, I’ll use decimal fractions, which come a bit more naturally to humans than binary fractions, but the argument is the same).
Let’s assume we want to multiply 3.25 by 5.25. The exact product of these numbers is 17.0625. But if two decimal places are all we have, like dollars-and-cents, then this number can not be represented in our fixed point format. It must be rounded to the nearest number that can be represented, which is 17.06.
Think of currency exchange, if this helps, say from dollars to euros. The bank will round the dollar-and-cents amount you pay for the euros—and usually in its own favor. By contrast, at least the rasterizer is fair. It will round the result to the nearest fraction. For instance, it will round coordinates (26.6 numbers) to the nearest 1/64 of a pixel.
Last but definitely not least, always beware of the division. Two integers hardly ever divide into another integer. There will always be fractions. But aren’t we using fractions with these fixed point formats? We sure are, but there are hardly ever enough fractions!
For instance, dividing the above 3.25 by 5.25, the pocket calculator tells me the result is 0.619047619047619047619047619047… If you look at the distractingly long row of digits hard enough, you’ll notice a pattern. This pattern repeats indefinitely, but we only have a (very) finite number of decimal places. Once more, the result must be rounded to the nearest number that can be represented, which is 0.62.
At first glance this may sound like an extreme case of splitting hair. Really! Why obsess over 0.619047… vs 0.62 pixels? Because this tiny rounding error can compound with other seemingly negligible rounding errors, until the result is no longer negligible. Oftentimes, when this happens, it comes as a complete surprise, because the compounding happened “in hiding.” We will look at this next.
In the following scenario we will have a look at the hidden rounding error compounding of the TrueType rasterizer. Specifically, we will constrain the width of a diagonal stroke in the context of a character, such as a lowercase ‘v.’ The illustration below captures this scenario at the point where the rasterizer is about to process the instructions for constraining the diagonal.
Constraining the width of a diagonal stroke
The above sketch illustrates the diagonal stroke in light gray. On the right edge of the stroke there are two control points labeled P0 and P1. These points function as “parent” points. At the top of the left edge of the stroke there is a control point labeled C. This point functions as “child” point. Furthermore, there are two vectors (“arrows”) labeled p and f, representing the direction of the constraint (“projection vector”) and the direction of compliance with the constraint (“freedom vector”).
The goal of the constraint is to establish a given distance between the child point C and the parent points P0 and P1. This distance is determined perpendicularly to the edge denoted by P0 and P1, which is parallel to p (“measured across the stroke”). If, at this point in the process, the given distance has not yet been established, then the child point C has to “give way” relative to the parent points P0 and P1, in order to comply with the constraint. The direction in which the child point is allowed to “give way” is denoted by f.
For instance, if the child point is too close to the parent points, it will have to move farther away, and vice-versa. But it is allowed to do so only in the direction prescribed by f. If it were allowed to move in any other direction, it would invalidate a prior constraint that establishes C to be constrained e.g. to the x-height. In the process, the final position of C is denoted by C′, along with the given distance d′, while d denotes the initial distance between C and the parent points P0 and P1.
Readers familiar with TrueType may recognize this scenario by a sequence of instructions similar to the following code fragment
SRP0[], P0
SFVTCA[X]
SDPVTL[R], P0, P1
MIRP[M>RBl], C, d′
where a CVT number would assume the role of d′.
Before delving into the numerical minutiae, we need some numbers for the relative positions of the involved control points. I’ll assume the following distances between P1, P0, and C.
distance between… | …in pixels |
P1 and P0 in x-direction | 2 56/64 |
P1 and P0 in y-direction | 5 |
P0 and C in x-direction | −25/64 |
P0 and C in y-direction | 0 |
Distances between the control points involved in constraining the width of the above diagonal stroke
In a first step the rasterizer must determine the x- and y-components of the freedom vector f. This is easy, because it is in x-direction. Hence, as 2.14 numbers
f = (16384/16384, 0/16384).
In a second step the rasterizer must determine the x- and y-components of the projection vector p. This is harder. To do so it must establish a vector from P1 to P0, rotate it by 90° (counter-clockwise), normalize it (divide each component by the length of the vector), and finally represent the result as 2.14 numbers (remembering that the vector and its length will be 26.6 numbers). Hence
p = (−320/64, 184/64)/(369/64) = (−14208/16384, 8170/16384)
where 369/64 is the length l of the vector from P1 to P0
l = (1/64)·((−320)2 + 1842)1/2 = 369/64.
In a third step the rasterizer must determine the initial distance d between the child point C and the parent points P0 and P1. To do so it can “project” the vector from P0 to C onto the projection vector p using the scalar product (inner product, or dot product), and represent the result as 26.6 number. Hence
d = (−25/64, 0/64)•(−14208/16384, 8170/16384) = 22/64.
In other words, in our scenario the stroke is initially a mere 22/64 pixels wide.
To continue we will assume that the given distance d′ is one full pixel, whether this is the result of using a cvt number, or a minimum distance criterion, or a combination thereof. Thus, initially, the child point C misses its target by
d′ − d = 64/64 − 22/64 = 42/64.
In a fourth and final step the rasterizer must therefore move the child point C by a distance of 42/64 when measured along the projection vector p, but it can only move it along the freedom vector f. This distance m is longer than the one measured along the projection vector p, but it can be computed as follows.
m = (d′ − d)/(f•p)
= (42/64)/((16384/16384, 0/16384)•(−14208/16384, 8170/16384))
= (42/64)/(−14208/16384)
= −48/64
where f•p is the scalar product of the freedom and projection vectors.
Multiplied by the freedom vector f, and then represented as 26.6 numbers, and in turn added to the initial child point C, this yields the final child point C′
C′ = C + m·f
= (−25/64, 0) + (−48/64)·(16384/16384, 0/16384)
= (−1 9/64, 0/64).
In other words, the final C′ ends up 1 9/64 pixels to the left of P0.
Let’s “double-check” this result. To do so we will use the same logic that we have used already to determine the initial distance d between the child point C and the parent points P0 and P1, except that now we will use the final child point C′. Hence
d″ = (−1 9/64, 0/64)•(−14208/16384, 8170/16384) = 63/64.
If none of the preceding calculations had introduced any rounding errors, this should be 64/64 pixels (1 full pixel), because that is what we “asked for.” But instead it is 63/64. This is 1/64 of a pixel “short” of the target.
A couple of comments may be in order here. First and foremost, I won’t claim that the above example reflects exactly how the rasterizer processes the associated code fragment. I am writing this from memory. But you are safe to assume that this example is at least representative of the inner workings of the rasterizer. Also, some of the math used in the process shall be left unexplained. It is simply beyond the general scope of this website. Last but not least, keen TrueType-savvy readers may have noticed that the code fragment uses the dual projection vector, while the rest of the text simply refers to the projection vector. This is neither a typo in the code nor in the text, but the associated subtlety is again beyond the scope of this website.
TrueType subtleties and math aside, the key take-away from this example should be the following. Fixed point binary numbers can and should be used whenever possible, because they are (or were) more efficient than the more general floating point numbers, and because the rounding “behavior” of a single arithmetic operation is predictable, which is (or was) not always the case with floating point numbers. However, for maximum precision, intermediate rounding steps must be avoided—fixed point or integer! If this is not—or cannot be—avoided, then the result is in jeopardy, which is why I call this the “double rounding jeopardy.”
At the time the TrueType rasterizer was designed, 32 bit numbers were “top-of-the-line” and floating point numbers were either prohibitively time-consuming to emulate or required an expensive floating point co-processor. In turn, with “only” 32 bits, many TrueType instructions must be broken down into multiple arithmetical operations on fixed point binary numbers. Each of these operations may introduce a rounding error, particularly if the projection and freedom vectors point in a diagonal direction. Accordingly, the result may not be as precise as it could be. This is simply an engineering trade-off between speed and precision in the context of available processor power at the time.
Is the TrueType rasterizer “broken?” Not really. Granted, the above example could be implemented by re-arranging the entire math into a single (!) fraction or rational number. Combined with clever intermediate use of 64 bit fixed point binary numbers available nowadays this may get a precise result. But it is hardly ever exact because the final division still has to round to the nearest 26.6 number. We simply have to learn how to deal with the hidden rounding error compounding.
Thus, to reiterate: Think before you round more than once! Your results will be in jeopardy. But if you have to do it—or if the rasterizer makes you do it—be prepared for the consequences. We will have a look at a simple example of dealing with these consequences next, and a more complex one later in this chapter (cf ).
The TrueType rasterizer maintains two sets of outlines that are available to the programmer, the original outlines and the instructed outlines. The original outlines represent the original, unconstrained artwork that has been scaled down to the targeted point size and device resolution. The instructed outlines reflect any differences that executing the TrueType instructions may have caused. Both sets of outlines are defined by their control points whose coordinates are represented as 26.6 numbers.
The seemingly innocent act of representing the original but scaled outlines as 26.6 numbers has a potentially unexpected consequence. To illustrate this point we will have a look at a single stem, its designed outlines, its scaled outlines, and the resulting scaled stroke width.
Outlines of the “vertical bar” of Arial (U+007C)
I have “doctored” these outlines ever so slightly to cause the problem I’d like to illustrate. The table below summarizes these alterations.
188 | 190 | 189 | |
345 | 343 | 342 | |
157 | 153 | 153 | |
188 | 190 | 189 | |
187 | 189 | 190 | |
532 | 532 | 532 |
Outline data (in font design units) of the above “vertical bar” of Arial (U+007C): original and 2 variants, “doctored” purely for demonstration purposes
As can be seen, the alteration is a small decrease in stem width, and the 2 variants differ by one font design unit in their side-bearings. The two side-bearings add up to an odd number, hence both “spacings” represent an attempt to center the stem.
The unexpected consequence of attempting to center this stem, one way or the other, is that at 20 ppem we get two different scaled stem widths, as compiled in the table below.
1 55/64 | 1 54/64 | |
3 22/64 | 3 22/64 | |
1 31/64 | 1 32/64 |
Outline data of the above 2 variants of the “vertical bar” (U+007C), scaled to 20 ppem (in pixels)
The x coordinates of the left edges of the 2 variants scale to two different 26.6 numbers, while those of the right edges scale to the same number—the difference is lost in rounding to the nearest 1/64 of a pixel.
Don’t believe it? Here is how I got there. The math to translate font design units (fUnits) into pixels, given the emHeight and the ppem size, is the following:
fUnits × ppem / emHeight = pixels.
This yields a floating point number. To get a 26.6 number, we have to round the floating point number to the nearest 1/64 of a pixel. To do so we simply multiply pixels by 64, round it to the nearest integer, and then divide it by 64. Example:
343 fUnits × 20 ppem × 64 / 2048 fUnits = 214.375
Round(214.375) = 214
214 / 64 = 3 22/64 pixels.
Likewise:
342 fUnits × 20 ppem × 64 / 2048 fUnits = 213.750
Round(213.750) = 214 (again)
214 / 64 = 3 22/64 pixels.
Like I said, the difference is lost in rounding to the nearest 1/64 of a pixel. Two identical stem widths scale to a different number of pixel fractions. Nevertheless, this is what the TrueType rasterizer calls the "original" outlines.
Somehow, this doesn’t seem right, and strictly speaking, it isn’t. Now, as I was preparing this example, I thought I knew just exactly which stops to pull to illustrate the consequences. Turns out that I knew only sort of. Quite to my surprise, the TrueType rasterizer managed to puzzle me with this example, even after 15 years of exposure.
I used a few simple TrueType instructions to constrain the stem width using 3 slightly different methods. They all have in common that they first constrain the left edge to the nearest pixel boundary, and then constrain the right edge relative to the left one, like below for method 1:
SVTCA[X]
MDAP[R], leftEdge
MDRP[m>RBl], rightEdge
IUP[X]
Instead of
MDRP[m>RBl], rightEdge
method 2 uses
MIRP[m>RBl], rightEdge, cvt
with an exact CVT (a CVT value of 153, corresponding to the exact stem width) and a sufficiently large CVT cut-in (8 pixels) to ensure this CVT overrides the “original” outline’s distance between the two edges. Method 3 also uses
MIRP[m>RBl], rightEdge, cvt
but intentionally with a wrong CVT number (having a CVT value of 2048) and zero CVT cut-in to prompt the instruction to fall back on to the “original” outline’s distance between the two edges.
The following table compiles the stem widths (in pixels) that each of the 3 methods rendered using both of the 2 outline variants.
MDRP[…] (no CVT, no CVT cut-in) | 2(*) | 2 |
MIRP[…] correct CVT, large CVT cut-in | 2(*) | 2(*) |
MIRP[…] wrong CVT, no CVT cut-in | 1 | 2 |
(for reference) | 1 31/64 (rounds to 1) | 1 32/64 (rounds to 2) |
(*) 1 pixel expected |
Pixel data of the above 2 variants of the “vertical bar” (U+007C), rendered stem width (in pixel) at 20ppem, constrained by 3 slightly different methods
Before we delve into the minutiae of the above results, recall , where we talked about surrogates representing clusters of sufficiently similar stroke weights. These surrogates are used in lieu of the actual outline’s stem widths—the type size and device resolution permitting. This can ensure that substantially equal strokes are rendered with an equal number of pixels or samples.
TrueType’s terminology for the surrogate is the control value. Control values are tabulated and identified by a number, hence the term control value table number (or CVT number for short). To ensure that CVT numbers are used only if the ppem size permits, TrueType uses a threshold called CVT cut-in. If the scaled value associated with a CVT number (the scaled CVT value) is within said threshold of the (scaled) “original” outline’s distance, then the (scaled) CVT value will be used to constrain the distance, else the (scaled) “original” distance. This permits to render stem widths in all their artistic individuality at large sizes while regularizing them at small sizes.
I have used this mechanism in methods 2 and 3 above, and quite deliberately so. For method 2, I chose an exact CVT value of 153 fUnits to represent my “cluster” of one stroke, and I chose a more than large enough CVT cut-in to ensure the (scaled) CVT value will be applied. Conversely, for method 3, I chose a CVT number that is “way-off” and a CVT cut-in of 0 (zero). This combination should ensure that the “original” distance is used to constrain the stem. Method 1 does not involve any CVT numbers, hence this should use the “original” distance by default.
Now then, let’s go back to the above table to see how the 3 methods perform against their specifications. I’ll start with method 3, because it works without surprises. The selected CVT value and CVT cut-in had the constraint fall back on to the “original” stem width, which rounds to 1 pixel for variant A and 2 pixels for variant B. Not that the two differing “original” stem widths, scaled and rounded to 26.6 from one and the same designed stem width, are particularly intuitive, but at least they can be explained (cf ).
For method 2 we need to translate the CVT value, which is in font design units, into pixels, like we did previously. Hence
153 fUnits × 20 ppem / 2048 fUnits ≅ 1.494 pixels.
This rounds down to 1 pixel. However, the rasterizer represents the above 1.494 pixels as 26.6 number, like any other coordinate. Hence
153 fUnits × 20 ppem × 64 / 2048 fUnits = 95.625
Round(95.625) = 96
96 / 64 = 1 32/64 pixel
which the subsequent MIRP[m>RBl]
rounds to the nearest pixel. That’s how we end up getting 2 pixels—a case of “double rounding jeopardy” split between two rounding agents. A little math and some memories of the rasterizer, and the results of method 2 can be explained, as well.
However, as far as method 1 is concerned, I’m at a loss. I cannot explain how MDRP[m>RBl]
determines the “original” distance and gets variant A to render with 2 pixels, instead of 1. There is no CVT involved, and hence no CVT cut-in. It should behave like the fall back code path of MIRP[m>RBl]
. Maybe it’s a case of “double code path jeopardy?”
MIRP
s and MDRP
s aside, the key take-away is that TrueType’s “original” outlines are somewhat compromised in that, once scaled to the targeted type size and device resolution, they are rounded to the nearest 1/64 of a pixel. Don’t rely on equally designed stems to be represented equally. Consider using CVTs instead. In edge cases, like the carefully constructed example above, they may not get you the expected pixel count, but at least they’ll get you the same pixel count in every instance. This should help consistency.
So… isn’t the TrueType rasterizer “broken?” Not really. It’s once more an engineering trade-off between speed and precision in the context of prevailing processor power. Early processors had to use a form of “long division” to divide two numbers, one bit at a time. This took a lot of time, relatively speaking. Therefore, trying to “cache” the division by storing the scaled coordinates and control values made sense.
Nowadays the trade-off may look different. Compared to all the “GUI sugar” of current operating systems, not “caching” the division most likely would go unnoticed. However, given how seemingly “fragile” or “fault-intolerant” font rendering is, what may get noticed may very well be an attempt to improve the internal precision of the rasterizer, however illogical this may sound at first.
Think of it this way. With the existing rasterizer, a particular character may render in a particular way at a particular ppem size. The author of the constraints (or “hints”) may see no fault with the rendered pixel pattern. But he or she may be oblivious to the fact that it took the rasterizer a seemingly serendipitous chain of tiny rounding errors to produce this particular result.
Now, if any single one of these rounding errors is “improved,” the outcome may change easily, as we have just seen. And since the font looked “good” before, chances are that it will look “worse” afterwards. We simply have to learn how to deal with the available precision—it’s hardly ever going to be exact, anyway.
In the preceding two chapters we have discussed potential benefits of sub-pixel anti-aliasing methods. In particular, in and we have looked at opportunities such as fractional stroke weights and positions. However, when ClearType was announced at the November 1998 COMDEX, pretty much none of the existing TrueType fonts constrained stroke edges to fractional pixel boundaries compatible with the requirements of ClearType.
Accordingly, at the time we were looking for some way of “shoehorning” existing TrueType fonts into ClearType duty. Therefore, in this section we will have a look at a number of challenges associated with this endeavor. Notice that in this context, TrueType chiefly refers to a way of constraining outlines, such as to generate like sample counts for like features, while ClearType refers to a method of converting multiple samples into pixels or RGB values.
The simplest way to “tease” more resolution out of existing TrueType fonts is to “pretend” a higher device resolution in x-direction than in y-direction. Even before ClearType, this is a property that the TrueType rasterizer is supposed to be able to handle gracefully. Fax machines and inkjet printers continue to be prime examples of this requirement long after the first “Hercules Graphics Card”.
Recall , where we determined ClearType’s oversampling rate to be 6x, that is, the outlines are sampled at 6 times the rate of pixels in x-direction. Accordingly, we can ask the rasterizer to render fonts at 576 DPI in x-direction (6×96 DPI = 576 DPI) combined with 96 DPI in y-direction. These numbers produce bitmaps like the one illustrated below:
Rendering previously “hinted” characters with different device resolutions in x- and y-direction (Times New Roman lc ‘m,’ 12 pt, 576 DPI = 6×96 DPI in x-direction, 96 DPI in y-direction)
⇒ Hover your mouse over the above illustration to see the “unhinted” rendition
Conceptually, we can proceed with rectangles of 6×1 (W×H) of these “skinny” bi-level pixels, interpret each of these pixels as an “in” or “out” sample, and downsample each sextuplet of samples into an RGB value according to the ClearType anti-aliasing filter.
In the process, the higher DPI in x-direction directly translates to extra rendered detail when compared to plain bi-level rendering. For instance, the 3 stems are 8 samples wide each, which translate to 1 1/3 pixels, while the 2 counterforms translate to 18 samples or 3 pixels. This provides for regular stem spacing while preserving the overall proportions of the lc ‘m’.
In practice, this simple method almost worked: Almost—but not on every character at every point size. In a seemingly random fashion some characters appeared to “explode” at some sizes, as illustrated below:
Rendering previously “hinted” characters with different device resolutions in x- and y-direction (Palatino Linotype lc ‘b,’ 12 pt, 720 DPI [= 6 × 120 DPI] in x-direction, 120 DPI in y-direction)
⇒ Hover your mouse over the above illustration to see the “unhinted” rendition
The primary cause of such “explosions” is the orientation of TrueType’s projection and freedom vectors p and f relative to one another. Recall the example illustrating the constraints of a diagonal stroke as shown in . The projection vector p was set perpendicular to the edge of the diagonal, while the freedom vector f was set in x-direction.
Constraining the width of a diagonal stroke
Once we “pretend” a much higher device resolution in x-direction than in y-direction, all of TrueType’s “original” x-coordinates will be larger by the same rate. In turn, this causes the angles of the projection and freedom vectors p and f to get skewed rather violently, as illustrated below.
Constraining the width of a diagonal stroke with a much higher DPI in x-direction than in y-direction
Recall how the width was measured “across” the stroke, or parallel to the projection vector p, and that the child point C was limited to complying with the constraint by “giving way” parallel to the freedom vector f. Importantly, also recall that this meant that the child point C had to move by a slightly longer distance than when measured along the projection vector p.
Once the projection and freedom vectors p and f get skewed as illustrated above, this slightly longer distance turns into a much longer distance. In fact, in the extreme case where the projection and freedom vectors p and f are perpendicular, the child point C could move as far left or right as it wanted to, but it would never comply with the constraint! It would just move “sideways.” Think about this for a moment.
Luckily, we do not need that much extra detail. But still, there is a limit as to how far we can push this approach. Specifically, in the computation of the final child point C′, we divided by f•p, the scalar product of the projection and freedom vectors p and f. If these vectors are perpendicular, then the scalar product is zero—that’s the mathematical definition of perpendicular. If they are “almost” perpendicular, then the scalar product is “almost” zero.
In turn, this means that we would divide by zero, which is “illegal,” or we would divide by a very small number, which creates a very large result that may no longer be represented by a 26.6 number. The TrueType rasterizer is smart enough to detect these situations. In fact, if the projection and freedom vectors p and f are within about 3.58° of being perpendicular, you will get a warning in VTT. Heed it!
Alas, in the above example illustrating an “exploding” character, VTT didn’t give me a warning. This doesn’t mean that there is no problem with the projection and freedom vectors p and f. All that this tells me is that they never got close enough to trigger a warning. But they may still be close enough to cause a problem.
Other causes of “explosions” are italic characters in general. Mathematically speaking, italics aren’t all that much different from “romans,” except that the y-axis is “italicized.” This simple aspect is often missed—both by the rasterizer itself, and by the author of the constraints (or “hints”). We will get to the rasterizer’s share of the italic challenges later in this chapter (cf and ).
Last but not least, details may go amiss in the thicket of TrueType instructions, or there is insufficient understanding of certain concepts of constraining outlines, or a combination thereof. Within a list of cryptic mnemonics and a commensurate supply of numbers, it is easy to miss that e.g. both parent points must have been constrained in both x- and y-direction before it makes any sense to set the (dual!) projection vector perpendicular to these points. Likewise, it is easy to miss the introduction of a cycle in the hierarchy of constraints, such as a constraint that depends on another constraint which in turn depends on the completion of the constraint about to be introduced.
Which of the above points ultimately caused the “explosion” I do not remember. By the time I was alerted to this particular example, I had seen my fair share of “unusual” ways to constrain (or “hint”) characters, hence I was not surprised. Most likely, I haven’t even looked at the source code anymore, which would explain why I don’t remember any details. Rest assured though that for practical intents and purposes this example is representative of the challenges we were facing upon pushing existing fonts into ClearType duty.
Granted, I didn’t expect anybody to have anticipated—and therefore to have tested against—such “lopsided” resolutions in x- and y-direction. But bogus code is bogus code, even though the results may just “happen to look right” in those cases that were actually looked at. Informally put, think of two “wrongs” that happen to make a “right” and then unravel one of the “wrongs.” This presented a formidable challenge. Fix all the affected fonts or “jury rig” the rasterizer?
Let’s recap what we have tried so far to “shoehorn” existing TrueType fonts into ClearType duty. The core step was to scale the outlines to a higher DPI in x-direction than in y-direction. I’ll call this overscaling. Along with overscaling I’ll define the overscaling factor as the rate by which the higher DPI exceeds the lower DPI. For instance, if the DPI in x-direction is 6 times the DPI in y-direction, then the overscaling factor is 6x.
With this definition, what we have tried so far is overscaling as the first of the following four steps:
- Overscale outlines in x-direction by 6x
- Apply constraints (“hints”) to overscaled outlines
- Sample constrained overscaled outlines into overscaled bitmaps
- Apply anti-aliasing filter to downsample overscaled bitmaps
The show-stopping disadvantage is the chance for the seemingly random “explosions” as previously illustrated. To avoid these “explosions” the obvious alternative is to swap steps 1 and 2 above:
- Apply constraints (“hints”) to outlines
- Overscale constrained outlines in x-direction by 6x
- Sample overscaled constrained outlines into overscaled bitmaps
- Apply anti-aliasing filter to downsample overscaled bitmaps
Accordingly I was looking for an approach that combines the advantages of overscaling before constraint application with the “safety” of overscaling after constraint application. To do so I “jury rigged” the rasterizer to dynamically re-interpret all rounding instructions to mean “round to sample boundary” if the rendering method is ClearType and if the direction of the projection vector p is not parallel to the long side of the LCD sub-pixels, else to keep the meaning “round to pixel boundary” as before:
- Apply contextually interpreted constraints (“hints”) to outlines
- Overscale constrained outlines in x-direction by 6x
- Sample overscaled constrained outlines into overscaled bitmaps
- Apply anti-aliasing filter to downsample overscaled bitmaps
Alas, while this “jury rigged” approach worked really well to eliminate the “explosions,” it also brought to daylight some of the dirty secrets of day-to-day “hinting.” Way back when bi-level rendering was the only rendering method, it was not uncommon for constraints to completely mangle the outlines, as long as this would yield the desired pixel patterns. Like with the “lopsided” resolutions in x- and y-direction, most likely this was not anticipated to ever change, but now presented a host of formidable challenges. We will discuss two areas of challenges, along with their “jury rigs,” next.
The TrueType instruction set comes with a concept called Delta Exception. A delta exception is an instruction that permits to dislocate a single control point by an arbitrary fractional number of pixels in an arbitrary direction and at one specific ppem size at a time. For instance, a delta instruction can move control point 28 by 13/16 of a pixel in the positive x-direction at 16 ppem. This is a bit of a mixed blessing.
On the one hand, it permits authors of constraints to obtain any pixel pattern even if everything else fails to do so. On the other hand, the relative ease with which delta exceptions can be added to the constraints does not encourage the development of ppem size independent solutions. Particularly the help of a graphical user interface such as in VTT is tempting to “solve” any problem with deltas.
Moreover, at the time, the deltas were completely “hard-wired” into the code, since there was no alternate rendering method requiring to do otherwise. This often led to outlines like the one illustrated below.
The effect of delta instructions on outlines and rendered pixels (Times New Roman UC ‘W,’ 12 pt, 96 DPI, RTM version of the font rendered in bi-level)
⇒ Hover your mouse over the above illustration to easily see which pixels would be “out” if it weren’t for TrueType’s dropout control feature
Notice that there are several pixels whose centers are “out.” If it weren’t for TrueType’s mechanism for restoring “missed” pixels (dropout control mechanism), these pixels wouldn’t even show up in the final bitmap! Be sure to hover your mouse over the above illustration to see which pixels would be “missing.”
This example illustrates the delicate interplay between deltas deliberately distorting the outline and the seeming serendipity of TrueType’s dropout control mechanism for the purpose of obtaining a particular pixel pattern. Maybe certain usages of deltas are considered artful, clever, cunning, shrewd, or whatever… But in terms of pixel coverage (cf ), obviously, this set of outlines is a “wreck.”
What to do? These deltas show up in ClearType—and not to its advantage. I could “jury rig” the rasterizer to bypass all delta instructions. This would cure the symptoms in the above example. But like the Hydra rearing one of its many heads, bypassing all deltas would eventually show up in other examples.
The positioning of crossbars appears to be a prime example where authors of constraints “solve” the erroneous positioning with delta instructions (cf ). Since delta instructions apply to one specific ppem size only, each ppem size at which the crossbar’s position is “off” will have to be fixed by a separate delta instruction—usually one delta will not be enough.
Accordingly, from within the rasterizer, I tried to identify the purpose of each delta instruction. Is it meant for positioning features, and is the projection vector p parallel to the long side of the LCD sub-pixels? In other words, is it meant to position an aliased stroke or similar in ClearType? Then the delta instruction will execute as specified. For all (!) other cases the delta instruction will be bypassed. These cases include the positioning of sub-pixel anti-aliased strokes along with all deltas to distort the outlines with the original purpose of obtaining a particular bi-level rendering pixel pattern.
But—you might object—how can you identify the purpose of a delta instruction? Strictly speaking, I cannot. But I can make an educated guess based upon the sequential position of the delta instruction within the stream of TrueType instructions (notice that the following applies to ClearType only, bi-level and full-pixel anti-aliasing are not affected):
- If the delta instruction appears after the
IUP[X]
orIUP[Y]
instruction for the respective projection vector p, then the delta is meant to “fix” a pixel pattern, hence it will be skipped. VTT calls this a post-IUP delta. - If the delta instruction appears before the
IUP[X]
orIUP[Y]
instruction, but there are no other constraints depending on the particular control point, given the respective projection vector p, then the delta is once more meant to “fix” a pixel pattern, and hence it will be skipped, too. VTT calls this a pre-IUP delta. - If, however, the delta instruction appears before the
IUP[X]
orIUP[Y]
instruction and there are other constraints that depend on the particular control point, given the respective projection vector p, then the delta is meant to “fix” a feature position, hence it will not be skipped if and only if the projection vector p is parallel to the long side of the LCD sub-pixels. VTT calls this an inline delta (for any orientation of the projection vector p).
The effect of delta instructions on outlines and rendered pixels (same as above except rendered in ClearType). Observe how the “dents” in the outline are gone, the stroke design contrast reflects the original design more closely, but the serifs at the top are still aligned with the cap height
⇒ Hover your mouse over the above illustration to revert to bi-level rendering)
Be sure to hover your mouse over the above illustration to see the difference between the shapes of the outlines in ClearType and in bi-level rendering!
This illustration may give rise to the impression that all instructions in x-direction are ignored—one of the opinions I keep reading in internet discussion forums and similar sources of collective wisdom. But this is not true! What you see is the difference between fractional and full-pixel stroke weights and the absence of unnecessary outline distortions for controlling untamable pixel patterns. So let me repeat this:
ClearType rendering does not generally ignore TrueType instructions in x-direction. It merely rounds them differently, and it bypasses distorting deltas.
Now, before you lament that you’re no longer in control of the outcome of your artwork, think about how you would round a control point to the nearest sample boundary, instead of the nearest pixel boundary, using TrueType instructions. TrueType’s RTDG[]
(“round-to-double-grid”) is not “fine grained” enough, and however long and hard I looked at it, TrueType’s SROUND[]
(“super-round”) won’t do it, either (cf also ). Seems easier to let the rasterizer do this, doesn’t it?
But if you really need to delta ClearType beyond inline deltas in y-direction, there is a switch to get you most of the original TrueType functionality back—the main exception being the rounding. In your Control Program place the following code snippet right at the beginning:
ASM("
#PUSHOFF
/* test if CT avail on this rasterizer */
#PUSH, 37, 1
GETINFO[]
LTEQ[]
#PUSH, 64, 1
GETINFO[]
GTEQ[]
AND[]
IF[] /* avail, switch to native CT mode */
#PUSH, 4, 3
INSTCTRL[]
EIF[]
#PUSHON
")
If you don’t use VTT’s Control Program, that is, if you use a simple Control Value Table along with your own Pre-Program template, skip the ASM("…")
bracket and place the TrueType code at the beginning of your Pre-Program.
Beyond that, consider limiting deltas to the respective rendering modes. Don’t “hard-wire” them into the code. For instance, if a delta applies to bi-level rendering only, use the following VTT Talk instructions (requires VTT’s Font Program template):
XBDelta(…)
YBDelta(…)
This will compile to a function that applies a delta instruction exclusively when in bi-level (“black-and-white”) rendering. The same functionality is available from VTT’s graphical user-interface by using the Delta Tool, provided you are initiating the delta with a “right-click” while in bi-level rendering.
For further information on “deltaing” ClearType, please refer to Microsoft’s recent publication. Said document lists additional stop-gap solutions I implemented to overcome a number of “unusual” ways of authoring constraints.
With the “explosions” and deltas out of the way, there was one more noteworthy concern. Some combinations of outlines and constraints lead to fairly “skinny” strokes. In bi-level rendering at typical text sizes on prevalent screen resolutions, this was not much of a problem. The minimum distance criterion or the dropout control mechanism would reliably take care of the “emaciated” appearance of these strokes.
Not so with a much improved notion of pixel coverage. The first time this sort of problem showed up in ClearType was with the font Courier New. Legend has it that the outlines of Courier New were taken from the “golf ball” type element of an IBM Selectric typewriter. Striking the type element’s raised letters against the ribbon caused the ribbon’s ink to spread on paper, hence the type element’s letters were “thinned” to compensate.
This compensation was not reversed after digitizing the outlines. Rendered with ClearType, it turned the font into Courier “Light”—particularly after gamma correction (cf ), as illustrated below.
The result of previously “hinted” “skinny” characters when rendered in ClearType (Courier New lc ‘m,’ 10 pt, 96 DPI, corresponding to a 12 CPI [characters per inch, 12 pitch] “golf ball” type element, gamma correction set to 2.2 [cf ])
⇒ Hover your mouse over the above illustration to see how the “unhinted” renders at the same size. It is a “skinny design”
The problem was exacerbated by the addition of anti-aliasing along the long side of the LCD sub-pixels (cf ). Together with the respective downsampling filters, and after gamma “over-correction” (cf ), this made parts of characters look “washed out,” as illustrated below (CAUTION: To properly view the following illustration, be sure to double-check all your settings as per the check-list introduced in ).
The result of previously “hinted” “skinny” characters when rendered in “anti-aliased ClearType” (hybrid sub-pixel anti-aliasing, 8 pt through 16 pt at 96 DPI, same gamma correction as above)
⇒ Hover your mouse over the above illustration to see the same anagrams rendered in plain bi-level. Notice the difference in stroke rendering contrast: Both illustrations nominally render black text against a white background
To understand the extent of the problem I’ll revisit the table in but replace the assumed “typical” stem widths by those of Courier New (84 or 85 fUnits). Here are the numbers:
1 5/6 (≅1.83) |
|
44…45 | |
33…34 | |
27 | |
22 |
Minimum stroke widths (in pixels) to render Courier New’s stems with a “black core” (top row), followed by the corresponding ppem and point sizes for a range of current screen resolutions
In other words, to render Courier “Light” with a “black core” on a screen with a typical resolution of 96 DPI, we would need a type size of 33 point or more. My old “№ 5 Underwood Standard Typewriter” could do better than that, given a fresh ribbon…
To address the challenge of Courier “Light,” we are effectively looking at a combination of problems:
- The outlines are “designed” too light.
- With any of the anti-aliasing methods, including ClearType, TrueType’s dropout control mechanism inserts missing samples, not missing pixels.
- To minimize distortions caused by aggressive bi-level constraints, it was eventually decided that ClearType rendering should reduce the minimum distance criterion to one half of the value specified by the author of the constraints.
Notice that it is not always trivial to apply a minimum distance constraint just about anywhere on the outlines in order to assert minimal pixel coverage and hence a minimum stroke rendering contrast. It may require a suitable pair of control points to do so, which the outlines may not have in the desired location, and it generally requires a geometrically sound set of constraints (or “hints”). Without suitable constraints, even a well-designed font can get badly mangled, as illustrated below.
The result of characters previously “hinted” to become quite “skinny” after rendering in ClearType (Palatino Linotype lc ‘n,’ 8 pt, 96 DPI, gamma correction reset to 1.0 [cf ])
⇒ Hover your mouse over the above illustration to revert to bi-level rendering and observe how an acceptable set of pixels is obtained irrespective of the badly mangled outline
Notice that if there were any deltas involved in the above illustration, they would have been “skipped” according to the strategy discussed in . Notice also that the illustrated problem is not specific to sub-pixel anti-aliasing methods such as ClearType. Due to their improved notion of pixel coverage, any of the anti-aliasing methods will render with similar results, as illustrated below for full-pixel anti-aliasing:
The result of characters previously “hinted” to become quite “skinny” after rendering in full-pixel anti-aliasing (otherwise same example as above)
⇒ Hover your mouse over the above illustration to revert to bi-level rendering and observe how an acceptable set of pixels is obtained irrespective of the badly mangled outline
The reason why bi-level rendering yields a set of pixels that may be acceptable in its own context (while any form of anti-aliasing seemingly doesn’t) is a subtle difference in the way TrueType’s dropout control mechanism works in anti-aliasing vs bi-level rendering. In bi-level (“black-and-white”) rendering, dropout control inserts entire pixels deemed missing, while in anti-aliasing it “merely” inserts missing samples. To my intuition this seems logical, but at the same time it may yield unexpected results, hence let’s have a closer look at one pixel of the above illustrations where the outline gets really “skinny:”
One pixel of the above illustration where the outline gets really “skinny”
In full-pixel anti-aliasing (“gray-scaling”) the pixel in question renders in a light shade of gray. Now recall oversampling as introduced and illustrated in . With 4x4y oversampling (as used in Windows’ “Standard Font-Smoothing”) most likely the pixel in question is sampled like so:
Sampling the pixel of the above illustration where the outline gets really “skinny.”
4 out of 16 samples are “in” yielding a 25% level of gray.
⇒ Hover your mouse over the above illustration to see that level of gray
Only 4 out of 16 samples are “in” and hence the pixel renders at 25% gray. These 25% represent the pixel coverage—albeit very loosely so given this badly mangled outline.
If dropout control had anything to do with this outcome, it may be the result of “dropping in” the one sample deliberately turned off in the following illustration:
Sampling the pixel of the above illustration where the outline gets really “skinny.”
3 out of 16 samples are “in” yielding an 18.75% level of gray.
⇒ Hover your mouse over the above illustration to see that level of gray
It’s an “edge case” and hence the “may be” above. But regardless, in case you haven’t noticed it by now: that single “dropped in” sample will contribute only 1/16 of the final pixel “darkness,” as illustrated below:
One pixel of the above illustration where the outline gets really “skinny” and potentially rendered without dropout control applied to anti-aliasing
⇒ Hover your mouse over the above illustration to see the modest difference a single “dropped in” sample will make
The difference a single “dropped in” sample will make and hence the effectiveness of “anti-aliased dropout control” may indeed be surprisingly small at first. But considering that any form of anti-aliasing—including ClearType—can render subtleties that bi-level’s big pixels simply can’t, it would seem natural that “anti-aliased dropout control” operates in much the same subtle way. The initial surprise merely reflects a challenge with the opportunities afforded by a more accurate representation of fractional pixel coverage.
Except that this finding doesn’t help with existing fonts “hinted” to become “skinny” while relying on “bi-level dropout control” to insert missing pixels. Accordingly I was looking for a “jury rig” to address this challenge on the level of samples, rather than second-guessing the purpose of even more instructions. I experimented with various heuristics targeted at augmenting the number of “in” samples along excessively thin parts of glyphs, which provided some degree of relief, as illustrated below.
The result of characters previously “hinted” to become quite “skinny,” after rendering in ClearType, and after applying some form of sample count augmentation (same outline and conditions as above, except for sample count augmentation)
⇒ Hover your mouse over the above illustration to revert to the previous illustration without sample count augmentation
CAUTION: To properly view the following illustration, be sure to double-check all your settings as per the check-list introduced in
Same conditions as above, except applied to a popular anagram
⇒ Hover your mouse over the above illustration to revert to the version without sample count augmentation
Readers familiar with VTT may turn on the above heuristics by enabling “CT (anti-aliased)” and “CT (dropout ctrl)” in the “Rasterizer Options.” Depending on the font and type size, the effect may be more or less subtle than above. The purpose of this approach was to increase pixel coverage without turning each and every font into a monoline font.
As far as I remember, a variant of this sample count augmentation made its way into production code, at least into WPF (Windows Presentation Foundation). I do not recall the exact method by which Courier “Light” was addressed in GDI (Graphics Device Interface) in the end. Suffice it to say that the preferred method would be outlines designed to represent the correct stroke weights combined with a set of appropriate constraints (or “hints”).
Attempting to “shoehorn” existing TrueType fonts into ClearType duty represents only part of the challenge. The other part is assembling individual characters into words and phrases. Once rendered with ClearType, the seemingly inconspicuous act of “putting one character after another” had surprisingly conspicuous consequences to anything from dialog boxes to entire websites.
Recall through where we discussed character proportions, layout contexts, fractional advance widths, and fractional ppem sizes. Depending on how we prioritized the constraints, we were able to preserve character proportions or advance widths, but not necessarily both. While a difficult decision on its own, knowing the layout context helped to make an educated choice.
Unfortunately, many existing text layout applications provided a context that was “tailored” to bi-level rendering. Words and phrases rendered in ClearType caught these contexts off guard. Surprisingly, even DirectWrite’s “natural” modes leave room for improvement. Therefore, in this section we will have a look at glyph metrics or advance widths in the context of text layout applications.
Consider the fragments of text in a dialog box, such as the text next to a radio button or a group of check boxes. The designer of the dialog box may have been lucky that a particular fragment of 8 point text at 96 DPI “just about” fit into the available space. Never mind that it may not be good design practice to omit margins or to rely on the dialog box never being used at any other screen resolution. As far as the proud designer is concerned, his or her dialog box is “pixel perfect.”
Enter ClearType, with all the above efforts at skipping deltas to avoid unnecessary distortions. As a “side-effect,” these efforts often restored the characters’ advance widths that bi-level constraints may have compromised, as illustrated below.
Comparing advance widths from “historic” bi-level rendering with advance widths from of ClearType rendering: the green lines represent the constrained (or “hinted”) advance width while the blue lines represent the designed (or “unhinted”) advance width (Tahoma lc ‘m,’ 8 pt, 96 DPI—the default UI font of Windows XP, at its default size)
⇒ Hover your mouse over the above illustration to see how this renders in ClearType and observe the different advance widths
Be sure to hover your mouse over the above illustration to see the dramatic difference between bi-level and ClearType rendering.
Bi-level constraints prioritized equal stroke positioning and combined the limited side-bearing space into one pixel on the right. In the context of bi-level rendering, this is as good as it gets. But it accumulates a rounding error of approximately 1 pixel, which is absorbed in the advance width. If many characters have to do this, and if the dialog box “design” relies on the extra character or two it thus can squeeze onto a line, then the more natural advance widths of ClearType create a problem: The text will be truncated.
ClearType just broke the dialog box! And not just this one dialog box; many more were affected. In fact, it affected many applications, including the browser rendering many “pixel perfect” websites. Never mind that these dialog boxes or websites simply revealed their lack of scalability. As far as their respective designers were concerned, ClearType just broke their “design.” It was again a case of two “wrongs” making a “right” with ClearType fixing one of the “wrongs.”
What to do? Initially, GDI had a simple solution. They added an x-direction scaling step between steps 1 and 2 of the process described in :
- Apply contextually interpreted constraints (“hints”) to outlines
+ Scale in x-direction to “fit” outlines into bi-level advance width - Overscale constrained outlines in x-direction
- Sample overscaled constrained outlines into overscaled bitmaps
- Apply anti-aliasing filter to downsample overscaled bitmaps
Simple indeed… except that along with “squeezing and stretching” characters back into their bi-level widths, it also “squeezed and stretched” stroke weights. Recall how the contextually interpreted constraints emulated rounding to sample boundaries, as opposed to pixel boundaries. This enabled fractional stroke weights and positions. But the ensuing “squeezing and stretching” changed those fractional stroke weights. In turn, characters with like strokes were rendered with unlike stroke weights!
Accordingly, I was looking for an “alternative.” Now, if there was ever an appropriate usage of the term “lesser evil,” this may be it. What I was looking into was to compress or expand the stroke positions without affecting the stroke weights. In the process, one character may get compressed, while the other one may get expanded. This is evil! But I felt that this was still the lesser evil because at least it left the stroke weights unchanged.
From within the rasterizer, I tried to identify patterns of TrueType code conventionally associated with constraining stroke weights and positions. Sometimes these patterns were consistent, sometimes not, in which case I tried to somehow “double-check:” Maybe this “white link” was meant to be a “black link?” Sometimes the existing system of constraints was simply too inconsistent.
With the strokes and their positions identified, I could implement a form of intelligent scaling: Apply the scaling in x-direction to the stroke positions only, but not to the stroke weights.
- Apply contextually interpreted constraints (“hints”) to outlines
+ Intelligently scale in x-direction to “fit” outlines into bi-level AW - Overscale constrained outlines in x-direction
- Sample overscaled constrained outlines into overscaled bitmaps
- Apply anti-aliasing filter to downsample overscaled bitmaps
Comparing advance widths from ClearType rendering with advance widths from “historic” bi-level rendering (same conditions as above, except starting out from ClearType instead of bi-level rendering)
⇒ Hover your mouse over the above illustration to see the same outline rendered to an advance width “compatible” with bi-level rendering. Notice that the green lines represent the constrained (or “hinted”) advance width while the yellow lines represent the “fitted” or “compatible” advance width. Notice in particular that the stem widths remain unchanged in the process
Be sure to hover your mouse over the above illustration to see how the character gets compressed but the strokes keep their weight.
Sometimes the existing constraints weren’t consistent enough to identify all stems, or there were cycles in the constraints that made it hard to “second-guess” their intended positioning and hence their re-positioning, and sometimes the “prescribed” advance width was simply way too tight, as in the following example.
Limitations of making ClearType advance widths “compatible” with bi-level advance widths: for some combinations of characters and point sizes there is simply no intuitive way to “give way.” (Tahoma lc ‘l,’ 11 pt, 96 DPI)
⇒ Hover your mouse over the above illustration to see what it would look like without any further attempts at “squeezing”
Without counterforms to absorb the “squeeze” there is nothing particularly intelligent that could be done to fit this character into the available space. Depending on the text, this can have fairly dramatic consequences, as illustrated below.
Limitations of making ClearType advance widths “compatible” with bi-level advance widths: Without counterforms to absorb the “squeeze” there is nothing particularly “smart” that could be done to “squeeze” the respective character into the available space
⇒ Be sure to hover your mouse over the above illustration to see what this would look like if we didn’t have to “do the squeeze.”
This looks very odd to me, too. But it would fit the “pixel perfect” dialog box or website “designed” around bi-level rendering… For less dramatic cases, what I’ve tried to do is some form of damage control.
Granted, this is far from optimal. Dialog boxes and websites should become more “flexible.” But at the time they weren’t, and “breaking” them was deemed unacceptable, hence our efforts at making it work—somehow. Never mind that nowadays websites have a much better chance at being “broken” simply by zooming in or out (cf ) or by setting a minimum font size, both for reasons of accessibility. Last but not least, if you are creating your own software, there is a switch to turn off bi-level “compatible” advance widths.
As we have just seen, assembling words and phrases by “putting one character after another” can break “pixel-perfect layouts” if they aren’t ready for advance widths to ever change. But even without “compatible” advance widths “squeezed” into previously defined layouts, insufficient understanding of constraining outlines combined with incomplete knowledge of the rasterizer’s idiosyncrasies can make or break optimal inter-character spacing. We will have a look at an innocent example thereof next.
“Natural” advance widths are the result of turning off “compatible” advance widths. Like the latter, they are laid out by full-pixel positioning, but they are not compromised by “backwards compatibility requirements.” They can be as precise as the size of the pixel allows without fractional pixel positioning (cf and ). In short, “natural” advance widths provide a straightforward yet attractive default.
Unfortunately, “natural” advance widths are also a victim of the “double rounding jeopardy.” I counted 5 (!) rounding operations that are cumulatively applied to the designed advance widths before a glyph appears on screen. One rounding operation on top of another rounding operation on top of … etc. This is an accident waiting to happen!
To illustrate one such accident, I’ll use Calibri, one of the fonts of the ClearType font collection and the new default font for Office 2007. In particular, I’ll have a look at the lowercase ‘e’ at 11 point on a 96 DPI display. This is the most frequent letter in the English language, used at the default font size in Office, and displayed on what is likely the dominant display resolution. Here is what happens:
- The advance width of the lowercase ‘e’ is 1019 fUnits. At 11 point and 96 DPI, this translates to 7 pixels as follows:
advance width × point × DPI / (emHeight × 72)
= 1019 × 11 × 96 / (2048 × 72) ≅ 7.297526
Round(7.297526) = 7 pixels. - 11 point at 96 DPI translates to 14 2/3 ppem, a fractional ppem size. The font restricts ppem sizes to integers, hence the rasterizer rounds the ppem size to 15 ppem. In turn, this translates the advance width—rounded to the nearest 1/64 of a pixel to represent it as 26.6 number—as follows:
advance width × ppem × 64 / emHeight
= 1019 × 15 × 64 / 2048 = 477.65625
Round(477.65625) = 478
478 / 64 = 7 30/64 pixels. - The rasterizer re-rounds this advance width to the nearest 1/16 of a pixel, probably in a well-meant but misguided attempt to provide the author of the constraints with the most natural rendition of the advance width in terms of the 16x oversampling rate in use at the time. In turn, this translates the advance width to:
(7 30/64) × 16 = 119.5
This is what the rasterizer presents to both the author and the interpreter of the constraints.
Round(119.5) = 120
120 / 16 = 7 8/16 pixels. - Once the constraints have been applied, the rasterizer re-rounds the advance width once more, to the nearest full-pixel, because it is going to be used in full-pixel positioning:
7 8/16 = 7.5
Round(7.5) = 8 pixels.
Cumulative rounding operations compromise ClearType’s “natural” advance widths (Calibri 11 pt, 96 DPI, “hinted” outlines)
Refer to the table below for a detailed description of the colors
blue | what the application asked for | 7.297526 |
green | what the font asked for | 7.46875 |
red | what the rasterizer shows | 7.5 |
black | what the rasterizer finally delivers | 8 |
There are two separate accidents in this particular chain of rounding operations:
- The “natural” advance width is off by more than 0.7 pixels. The resulting 8 pixels correspond to a font size of 12 point, not 11 point. This compromises spacing in a scalable layout context.
- The rasterizer silently adds 0.5 pixels of “padding” to the right side-bearing space without “telling” the author of the constraints. This compromises spacing in a reflowable layout context.
Cumulative rounding operations on the advance widths compromise the inter-character spacing even when laying out text one character after another (Calibri 11 pt, 96 DPI, ClearType’s “natural” advance widths, top: enlarged, bottom: original size)
⇒ Hover your mouse over the above illustrations to see the inter-character spacing without accumulated rounding operations
There are two key take-aways from this example:
- “Natural” widths aren’t quite as “natural” as they purport to be.
- It’s not always the “evil WYSIWYG algorithm” that ruins spacing.
In the preceding sub-section we have discussed how the “double rounding jeopardy” (cf ) can compromise inter-character spacing even when using “natural” widths. While “natural” widths allow to minimize “blur” at small type sizes and low resolutions, few professional “hinters” are knowledgeable enough to make the respective efforts in (ppem size independent!) TrueType code. There are simply too many variables involved, and few text layout applications currently use “natural” widths, anyway.
Instead, text layout applications switch to using DirectWrite’s fractional advance widths (cf ) in hopes that this solves all the remaining “spacing issues.” Theoretically, if “natural” widths can position characters to the nearest ±1/2 of a pixel, fractional widths should be able to do so to the nearest ±1/2 of a sample, or ±1/12 of a pixel, given ClearType’s 6× oversampling rate (cf ).
Spacing errors of this small magnitude should go unnoticed, because even on a screen with a mere 96 DPI resolution, 1/12 of a pixel is less than 1/1000 of an inch (or about 22 µm). The healthy human eye can’t resolve this level of detail from a distance beyond 7.5 cm (3 inches), and the adult eye cannot accommodate to such a short viewing distance. To confirm this claim, let’s have a look at the illustration below:
Windows 7 RTM DirectWrite |
“rehinted” ‘mno’ VTT (fract AW) |
Spacing test string “nnmnonomoommm” taken from VTT and rendered with DirectWrite (left) and in VTT (right), both using fractional advance widths and fractional pixel positioning (Calibri, 8 pt at 96 DPI). Each line differs from the previous one by a horizontal shift of 1/6 of a pixel to the right.
⇒ Hover your mouse over either of the illustrations to enlarge it to 400%
At first glance, the difference between DirectWrite and VTT may seem subtle, and they may be partially “masked” by different characters getting “blurred” to different degrees as a result of using fractional pixel positioning. Therefore the following illustration repeats the previous one, but with the role of the “mouse-over” changed to revert to the RTM version:
Same test string as above, but both rendered in VTT with the “rehinted” characters ‘mno’ at original size (left) and enlarged to 400% (right).
⇒ Hover your mouse over either of the illustrations to revert to the Windows 7 RTM version
Be sure to hover your mouse over the above illustration to revert to the Windows 7 RTM version, and try not to get overly sidetracked by different degrees of “blur” on different characters.
Notice how—when reverting to the RTM version— the characters appear to get wider, and how e.g. the last ‘o’ on the first and second rows seem to touch the adjacent ‘m’?
What you see are integer ppem size advance widths “squeezed” into fractional ppem size metrics!
In case of the 8 pt Calibri above, rendered at 96 DPI, this means that the font has been “hinted” at 11 ppem but now is “squeezed” into metrics or advance widths meant for 10 2/3 ppem (cf table towards the end of ).
At this combination of point size and device resolution, “1/3 ppem” can make a difference far beyond the 1/12 of a pixel we were hoping for in order to solve all remaining “spacing issues.” To see this, the following illustration repeats one of the above lc ‘m’ along with the respective advance widths.
Integer ppem sizes “squeezed into fractional advance widths: the green lines represent the advance width as rendered, while the blue lines represent the advance widths as expected (Calibri, 8 pt at 96 DPI).
⇒ Hover your mouse over the illustration to see the same character rendered with fractional ppem sizes.
Be sure to hover your mouse over the above illustration. The difference between the above advance widths is about 1/3 of a pixel! This is a lot more than the 1/12 of a pixel we were hoping for. In fact, it represents about 40% of the right side-bearing space as designed! What’s more, these 40% are simply “chopped off” of the right side-bearing without “telling” the author of the constraints (or “hints”), which does not sound entirely unlike the accidents we have discussed with “natural widths” (cf ).
Once more, there is no easy “fix” to overcome the shortcomings of fractional advance widths rendered in DirectWrite. It can be done, as I have illustrated above, but it requires some serious efforts in terms of TrueType coding. The main reason for these efforts is the inevitable use of fractional ppem sizes (cf ) which in turn eliminates any hopes at using “deltas” for “fixing” unwanted pixel patterns—or any other “situation” that a professional “hinter” may not be able to address in a ppem size independent way!
Fractional advance widths and fractional pixel positioning represent a unique opportunity to get the inter-character spacing right, but this comes at a price: “Blurring” can no longer be minimized by carefully positioning individual stems. Just have a look at any of the lc ‘m’ above, particularly adjacent ones, and observe how they are rendered with different levels of “blurring.” At the very least, end-users willing to pay that price should get inter-character spacing without “chopping off” some 40% of the right side-bearing!
In the preceding sub-sections we have looked at issues with inter-character spacing due to accumulated rounding errors and forcing (rigid) characters into prescribed advance widths. To make it easy to see these issues, I deliberately chose fairly conspicuous examples. Depending on the actual font, individual character, and combination of point size and resolution (DPI), these issues may be less prominent, if not all but absent.
Nevertheless, a few prominent accidents can impact the gestalt of the entire page or paragraph, as illustrated below.
Advance widths compared: bi-level (aka “black-and-white”, left) vs ClearType using “compatible” advance widths (right). Line breaks and number of lines match, but the page color in ClearType’s “compatible” widths is uneven (Tahoma, 11 pt at 96 DPI).
⇒ Hover your mouse over either of the above illustrations to see the character-by-character match.
Be sure to hover your mouse over the above illustration. Notice how “compatible” advance widths indeed produces a text layout that perfectly matches that of bi-level (aka “black-and-white”). For the purpose of dialog boxes and many websites at the time, it simply had to—without compromise [sic]—or else ClearType would not have been released.
Amongst the many issues with ClearType’s “compatible” widths, notice the uneven page color produced by clusters of very narrow lc ‘i’ and ‘l’ characters (cf also end of ). Owing to their distinct fur markings I called this the “Dalmation Dog Effect.” If you squint your eyes a little bit, you may notice mildly darker spots even in bi-level rendering.
At the time I came up with “compatible” widths as the “lesser evil,” it was our understanding that this will be used only “temporarily”—until the respective applications become smart enough to account for advance width variations due to different rendering methods, device resolutions, or “zoom” factors (cf also end of ). Alas, 10 years after the 2001 release of Windows XP, this turned into a mostly “permatemp” solution.
The emerging alternative (Internet Explorer 9, Firefox 5) is DirectWrite’s implementation of fractional advance widths (cf ). The illustration below shows how this can stack up against the other alternative, “natural” widths (cf ).
Advance widths compared: “natural” widths (GDI, left) vs fractional widths (DirectWrite, right). While the page color in either rendition looks fairly even, line breaks and number of lines are different, and the “sharpness” of strokes appears a bit irregular in DirectWrite, as if there were random smudges on the display (Tahoma, 9 pt at 96 DPI).
⇒ Hover your mouse over either of the above illustrations to see the difference between the two advance widths.
Notice how “natural” widths make Tahoma appear “sharper” and with a higher degree of stroke rendering contrast (cf ), while fractional widths make it look as if there were some thin veil of dust on the display. To me, this looks a bit as if I needed to clean my eye-glasses. Of course, cleaning displays or eye-glasses won’t help to explain the above difference; it merely shows the inherent disparity between fractional and integer pixel positioning (cf ).
To permit making your own comparisons, the illustration below allows you to choose a combination of advance widths and a point size for rendition at 96 DPI.
Advance widths compared: use the above buttons to arrange for your own custom comparison between 2 rendering modes at a particular point size (Tahoma RTM, rendered at 96 DPI in GDI for bi-level, compatible, and natural advance widths, and in DirectWrite for fractional advance widths).
Be sure to use the above buttons to customize your comparisons. Some of these combinations can make one and the same Tahoma look like a different font!
Of particular interest may be comparing compatible widths with fractional widths since this illustrates the move of Internet Explorer 9 and Firefox 5 to using DirectWrite (fractional widths). By contrast, Chrome 10 continues to use compatible widths (GDI) and coincidentally, as far as I can tell, Safari 5 uses integer widths but allows fractional ppem sizes (cf column “fractional ppem sizes, integer widths” in the table at the beginning of ).
Notice that between GDI and DirectWrite, the identifiers used for different advance widths may be a bit confusing, as several of them contain the designation NATURAL
. Following is a table for comparison purposes.
NONANTIALIASED_ |
DWRITE_RENDERING_MODE_ |
|
CLEARTYPE_ |
DWRITE_RENDERING_MODE_ |
|
CLEARTYPE_NATURAL_ |
DWRITE_RENDERING_MODE_ |
|
n/a |
DWRITE_RENDERING_MODE_ |
Advance widths compared: designations of the advance widths as used in the programming platforms GDI (left) and DirectWrite (right).
If you are in the business of creating software that involves rendering text, and which software doesn’t, you don’t have to choose “compatible” advance widths. Instead, you can choose whichever set of advance widths best fits your layout application. Moreover, if your application does not impose a hard constraint on scalability, such as a “zoomable” print preview, you could choose to leave this choice to the end-users (cf also and ).
In the previous two sections we have discussed various obstacles with “shoehorning” existing fonts into ClearType duty, and in turn, into existing applications. Working around these obstacles entailed several “jury rigs” in the rasterizer to defuse the respective problems. Understanding that these problems aren’t really solved, merely patched up with big “band-aids,” future fonts should strive for more adaptive constraints or “hints.”
Alas, even if you have the luxury to start constraining a font from scratch, duly turning off all “jury rigs” as far as possible, there are a few more obstacles. In this section, I will present some of the obstacles I encountered trying to do things “properly.” Notice that there are workarounds for all of these obstacles, but they require extremely cumbersome and tedious TrueType code, and plenty of it!
Rounding to sample boundaries, as opposed to pixel boundaries, is a fundamental prerequisite to implementing fractional stroke weights and stroke positions (cf and , also cf and ). Unfortunately, however flexible SROUND[]
portrays to be, it limits the “rounding granularity” to 0.5 period
, where period
is the spacing of the pixel grid. Anything “finer grained” is not available.
To implement constraints compatible with the various anti-aliasing methods introduced in Chapter 2, what is needed is the possibility to select a period
of the form 1/n, where n is the oversampling rate in x- or y-direction, as appropriate. This would allow to round to the nearest 1/4, 1/5, or 1/6 of a pixel—or whatever oversampling rate the future may require.
Mathematically, the only difference between italic and upright characters is an “italic” y-axis. Accordingly, it stands to reason that setting the projection vector p perpendicular to the “italic” y-axis is all that is needed to constrain italic characters. Readers familiar with VTT may recognize the concepts of the italic angle and the “adjusted” italic angle, encoded by a single or double forward slash, respectively.
Inconveniently, this simple mathematical model seems to break down in the presence of asymmetric overscaling, or any non-square aspect ratio for that matter, as illustrated below.
Interpolating control points with the projection vector p set perpendicular to the “italic” y-axis (Arial lc ‘l’)
For the purpose of illustrating the problem, I didn’t use any CVTs, and I deliberately left every control point “unrounded” in x-direction, just to make sure I won’t mask the problem. Here is the code:
GlyphStrokeAngle(23,22) |
CALL[], 23, 22, 88 |
VTT Talk and corresponding TrueType source code used to illustrate the problem with interpolating control points when the projection vector p is set perpendicular to the “italic” y-axis
Interpolating control points with the projection vector p set perpendicular to the “italic” y-axis (Arial lc ‘l’, 11 pt, 72 DPI in y direction, combined with the following resolutions in x-direction: 72, 96, 120, 144, 216, 288, and 432 DPI)
⇒ Be sure to push any of the above buttons to test the various non-square aspect ratios and asymmetric overscaling factors
The more lopsided the aspect ratio or the asymmetric overscaling, the more erroneous the angle of the italic stroke gets. Turns out that this “misbehavior” is specific to the IP[]
instruction (“interpolate point”). Other instructions I’ve tried appear to work fine. This would suggest a case of “double code path jeopardy.”
IUP[X]
and IUP[Y]
(“interpolate untouched points”) are two “shortcuts” that constrain any previously unconstrained (“untouched”) control points between consecutive pairs of constrained points on the same outline. Specifically, if the unconstrained points are physically between the constrained points, then the shortcuts are equivalent to an IP[]
instruction (“interpolate point”) applied to all the unconstrained points on the outline between the pairs of constrained points.
For instance, in the following illustration of a lowercase ‘n,’ points 2 and 12 may have been constrained in x-direction while points 3 through 11 are between 2 and 12, both logically (along the same outline) and physically (all x-coordinates are within the range of x-coordinates spanned by points 2 and 12). Likewise for points 14 through 21 between points 13 and 22:
Using the instruction IUP[X]
on a roman character (Arial lc ‘n’ 12 pt, 96 DPI)
In this case, IUP[X]
will “interpolate” points 3 through 11 between 2 and 12, and likewise 14 through 21 between 13 and 22. Notice that in this example, the direction of the constraints—the projection vector p—is in x-direction. “Physically between” is determined against this projection vector p.
Once we apply the same logic to an italic lowercase ‘n,’ the direction of the projection vector p changes. It is now perpendicular to the “italic” y-axis, like in the preceding example. However, the IUP[]
instruction is “hard-wired” to use either the x-direction or the y-direction. If IUP[X]
is used, instead of a corresponding “italic variant,” the erroneous projection vector p can be noticed:
Using the instruction IUP[X]
on an italic character throws parallel italic edges “out-of-parallel” (Arial Italic lc ‘n’ 12 pt, 96 DPI)
⇒ Hover your mouse over the above illustration to see the results of a hypothetic IUP[I]
instruction (‘I’ for “italic”). Notice how this puts all italic edges back to their parallel design
For bi-level rendering, the absence of an IUP[I]
instruction (‘I’ for “italic”) may not be all that critical, but for any kind of anti-aliasing, this may show up as uneven stroke weights.
TrueType implements a way of assembling glyphs out of other glyphs or parts of glyphs. TrueType calls this concept composite glyphs. In theory, this harbors the potential for asserting consistency by concept, rather than by industriousness.
For instance, any character with a diacritical mark can be represented as some kind of a blue print that specifies how the respective mark combines with the associated base character. This should assert that e.g. a lc ‘ü’ looks exactly like a lc ‘u’ with an “umlaut” (“dieresis”) on top, or that the “umlaut” on the ‘ü’ looks exactly like the “umlaut” on the ‘ä’ or on any other character it may occur.
In practice, the implementation of this “feature” does not come across as thoroughly thought through, though. The blue print essentially superimposes two or more glyphs, as illustrated below:
Composite glyphs in TrueType: components are essentially superimposed on top of each other (basic blue print of Arial lc ‘ü’)
To use the same “umlaut” on more than one base character, TrueType allows for an individual translation vector (offset in x- and y-direction) or transformation matrix for each of the components. Conceptually, this allows to design characters out of predesigned components.
Translation Vector: x-direction: 220 fUnits y-direction: 0 fUnits |
Translation Vector: x-direction: 393 fUnits y-direction: 286 fUnits |
Composite glyphs in TrueType: individual translation vectors or transformation matrices can be applied to each component to re-use a component in different glyphs (actual blue print of Arial lc ‘ü’ and UC ‘Ü’)
⇒ Hover your mouse over the above illustrations to see the respective superpositions without offsets
But it does not necessarily render the entire assembly as intended.
Composite glyphs in TrueType: the assembled blue print does not necessarily render as intended (Arial lc ‘ü,’ 9 pt, 96 DPI [top, enlarged] and 8 pt through 36 pt [bottom, actual bitmaps])
⇒ Hover your mouse over the above illustrations to see more appropriate renditions of the same character
As can be seen in the above illustration, the problem of “ill-positioned umlauts” runs all the way from 8 pt up to 36 pt—and beyond on either side! Some “umlauts” appear pushed to one side, some appear pushed too far up, and some appear too close together.
The reason behind these “ill-positioned” renditions is the following: Both components (the base character ‘u’ and the “umlaut” ‘¨’) were constrained (“hinted”) individually within their own contexts. In the process, both the ‘u’ and the ‘¨’ are independently positioned relative to e.g. their respective left- and right side-bearing points. As a result, their individual positions may have shifted left or right, and—in case of the ‘¨’ that was not designed to “sit” on the baseline like the ‘u’—up or down.
Moreover, to be used in the blue print, the above offsets in x- and y-direction are scaled and rounded to the targeted point size and device resolution. This keeps all the involved components “on-the-grid.” But, although it erstwhile permitted to correctly design characters out of predesigned components, now the translation vector is usually “bogus.”
Think of it this way: To correctly inset e.g. the left “dot” of the “umlaut” relative to the left stroke of the ‘u,’ there are 3 design variables involved—the left side-bearing of the ‘u,’ the “left side-bearing” of the “umlaut,” and the offset in x-direction. In terms of font design units, these 3 variables “add up,” but in terms of (rounded!) pixels they don’t!
We have discussed this before. Remember where we were looking at strokes being rendered with one pixel count here and another one there? At that time we had a left edge plus a stroke width define a right edge of a stem, in order for these 2 variables to “add up.” Now we simply have 3 variables, and we use add or subtract equivalently (with the potential caveat about negative numbers mentioned in ).
But now for the real problem: The concept of “original” outlines, whatever limitations they may have (cf ), seems unusable in the blue prints of composite characters. TrueType instructions such as MDRP[]
, MIRP[]
, or IP[]
will not perform as expected when interspersed with the rest of the blue print code. At the same time the offsets of the blue print are available to the TrueType programmer in design units only, not in scaled device units like the “original” outlines.
This makes for a really cumbersome and tedious workaround. As you can see by hovering your mouse over the above illustration, it is possible to position the “umlaut” at its natural distance above the base character, to center it, and to balance the distance between the individual “dots” with their inset relative to the black-body width of the base character.
The implemented strategy does not require any deltas (cf ), but a little bit of “thinking outside the box,” and—like any of the problems illustrated in this section—plenty of code for TrueType’s font program.
*****
In this chapter we have looked at some of the imperfections of the TrueType rasterizer that ships with Microsoft Windows. This involved delicate issues of computer geometry along with obvious errors and—in hindsight—omissions. By the same standards, most of the fonts aren’t perfect, either. They contain obvious errors and—again in hindsight—very obvious omissions.
What to do? Fix and upgrade the rasterizer, and then fix and upgrade all (!) the fonts, painstakingly agonizing over pixel-by-pixel backwards compatibility in bi-level rendering? Not a chance. Merely fixing the rasterizer could have “broken” many fonts (cf ). Instead, I tried to patch the biggest holes by shifting some of the burden from the fonts to the rasterizer.
Understanding that this deprives font makers of part of the control over their artwork, we have discussed the “switches” to turn off (most of) these patches for future fonts. In turn, this puts the burden back on the font makers. They will have to develop a very clear concept of the purpose of their constraints (or “hints”) while being prepared to look for alternatives in case their preferred implementation hits a limitation of the rasterizer (cf ).
In the next chapter we will have a look at a number of challenges beyond fixing the rasterizer and the fonts, and after discontinuing “compatible” advance widths. These challenges occur even with “perfect” fonts interpreted by a “perfect” rasterizer and laid out by “perfect” applications.
← previous chapter, ↑ top, next chapter →