2 Jan 2005 09:26

## Re: UTF8 and (La)TeX backend?

```
> > how do you think we should handle encoding for the TeX backend?  I
> > am tempted to just dump
> >
> >   \usepackage[utf8]{inputenc}
> >
> > [...]
> >
> > we still need to do something with encoding, to
> >
> > 1. dump the appropriate \usepackage headings
> >
> > 2. select a reasonable guesstimate for the initial rendering run
> >    used by -f texstr
>
> In addition, it would be nice if normal (i.e. utf8 encoded) input
> files keep working and produce similar results, both in -f tex and
> -f ps.

I believe that `-f tex' and `-f ps' are fundamentally different, and
that we shouldn't invest time to `unify' both interfaces.  We have
different fonts, different encodings, different font attributes, etc.
As discussed before, lilypond should pass the text strings for the TeX
backend unmodified to TeX.

It probably makes sense to pronounce the differences between `-f tex'
and `-f ps' w.r.t. font handling even more -- what about renaming the
TeX font attributes to `tex-font-...'?

Werner
```

2 Jan 2005 12:48

### Re: UTF8 and (La)TeX backend?

```wl <at> gnu.org writes:
>
> > > how do you think we should handle encoding for the TeX backend?  I
> > > am tempted to just dump
> > >
> > >   \usepackage[utf8]{inputenc}
> > >
> > > [...]
> > >
> > > we still need to do something with encoding, to
> > >
> > > 1. dump the appropriate \usepackage headings
> > >
> > > 2. select a reasonable guesstimate for the initial rendering run
> > >    used by -f texstr
> >
> > In addition, it would be nice if normal (i.e. utf8 encoded) input
> > files keep working and produce similar results, both in -f tex and
> > -f ps.
>
> I believe that `-f tex' and `-f ps' are fundamentally different, and
> that we shouldn't invest time to `unify' both interfaces.  We have

I disagree. We want both to look equal or as similar as possible. For
example, we can have people develop a score using GNOME point and
click, then print samples using PS output, and do final touchup on the
SVG output.

Of course, we can decide that TeX is the odd one out, but then we have
to figure out a way to make lilypond-book use the -f ps backend
```

3 Jan 2005 19:25

### Re: UTF8 and (La)TeX backend?

```> > I believe that `-f tex' and `-f ps' are fundamentally different,
> > and that we shouldn't invest time to `unify' both interfaces.
>
> I disagree.  We want both to look equal or as similar as possible.
> For example, we can have people develop a score using GNOME point
> and click, then print samples using PS output, and do final touchup
> on the SVG output.

Sorry to say but I think this is not realistic.  GNOME, PS, and SVG
are all based on Pango, while TeX isn't (and never will be).

> Of course, we can decide that TeX is the odd one out, but then we
> have to figure out a way to make lilypond-book use the -f ps backend
> (because we want the manual to use the preferred backend, of
> course).

lilypond-book should support both `-f ps' and `-f tex'.

> We could do so by dumping systems as separate EPS files, and dumping a
> .tex file saying
>
>   \includegraphics{system1.eps}
>   \includegraphics{system2.eps}
>   \includegraphics{system3.eps}
>   \includegraphics{system4.eps}
>
> the last actually has my preference, since I am not a fan of the tex
> backend.

```

4 Jan 2005 23:32

### Re: UTF8 and (La)TeX backend?

```wl <at> gnu.org writes:
>
> > How do we prevent bigcheese20.otf from ending up hundreds of times
> > in the output?
>
> It shouldn't be too difficult:
>
>   . LilyPond produces EPS files which contain
>
>       %%DocumentNeededResources: font bigcheese20
>
>     but doesn't contain the font itself.  (I'm not sure whether such
>     files can be called `EPS' since they aren't fully self-contained.)
>
>   . The CFF part of bigcheese20.otf is extracted and put into a PS
>     resource which is loaded with the -h option of dvips.  If that
>     doesn't work, lilypond-book simply emits
>     \special{header=bigcheese.cff} at the beginning of the document.

OK.  I just had a change of plans: I'm going to retain the classic
font selection scheme, but for the non-tex backends, the selection
scheme does not produce a PFA font name, but rather a
PangoFontDescription as a string. For lilypond-book, I'm going to
implement the above, setting --backend=ps as the default.

This is not perfect. Ideally, we'd use FontConfig for fontselection
wholesale, but I don't see how we can mix that with Pango, Design
sizes and the TeX backend.

--

--
```

8 Jan 2005 17:39

### Re: UTF8 and (La)TeX backend?

```hanwen <at> xs4all.nl writes:
>
> OK.  I just had a change of plans: I'm going to retain the classic
> font selection scheme, but for the non-tex backends, the selection
> scheme does not produce a PFA font name, but rather a
> PangoFontDescription as a string. For lilypond-book, I'm going to
> implement the above, setting --backend=ps as the default.

This is implemented in current CVS and it seems to work. For the
non-TeX backend, the text font defaults to 12pt URW Century
Schoolbook.

Suggestions for a better default are appreciated.

--

--

Han-Wen Nienhuys   |   hanwen <at> xs4all.nl   |   http://www.xs4all.nl/~hanwen
```
6 Jan 2005 06:55

### Re: UTF8 and (La)TeX backend?

```
> > . The CFF part of bigcheese20.otf is extracted and put into a PS
> >   resource which is loaded with the -h option of dvips.  If that
> >   doesn't work, lilypond-book simply emits
> >   \special{header=bigcheese.cff} at the beginning of the document.

I was a bit unclear.  Here a reformulation:

. The CFF part of bigcheese20.otf is extracted and put into a PS
resource which is loaded with the -h option of dvips.  If that
doesn't work (this is, if my assumption is incorrect and option -h
can't be used to load a PS resource file at all), lilypond-book
simply emits \special{header=bigcheese.cff} at the beginning of the
document.

Werner
```
9 Jan 2005 23:29

### Re: UTF8 and (La)TeX backend?

```wl <at> gnu.org writes:
>
> > > . The CFF part of bigcheese20.otf is extracted and put into a PS
> > >   resource which is loaded with the -h option of dvips.  If that
> > >   doesn't work, lilypond-book simply emits
> > >   \special{header=bigcheese.cff} at the beginning of the document.
>
> I was a bit unclear.  Here a reformulation:
>
> . The CFF part of bigcheese20.otf is extracted and put into a PS
>   resource which is loaded with the -h option of dvips.  If that
>   doesn't work (this is, if my assumption is incorrect and option -h
>   can't be used to load a PS resource file at all), lilypond-book
>   simply emits \special{header=bigcheese.cff} at the beginning of the
>   document.

BTW, is there an option to wrap the CFF as ASCII data? The binary data
not only confuses dvips, it also slows down emacs tremendously.

>
>
>      Werner

--

--

Han-Wen Nienhuys   |   hanwen <at> xs4all.nl   |   http://www.xs4all.nl/~hanwen
```
10 Jan 2005 15:01

### Re: UTF8 and (La)TeX backend?

```
> BTW, is there an option to wrap the CFF as ASCII data?  The binary
> data not only confuses dvips, it also slows down emacs tremendously.

Yes, this is possible.  See below how it looks like -- it's a CID
font, not a CFF, but you should get the idea.  Details can be found in

Werner

======================================================================

%%DocumentNeededResources: procset CIDInit
%%IncludeResource: procset CIDInit
%%Version: 1

/CIDInit /ProcSet findresource begin

20 dict begin

/CIDFontVersion 1 def
/CIDFontType 0 def

/CIDSystemInfo 3 dict dup begin
/Registry (mozilla_printout) def
/Ordering (zdN0NM2+APEIGSpvdLhD/xU+epg=) def
/Supplement 0 def
```

27 Jan 2005 00:01

### Re: UTF8 and (La)TeX backend?

```wl <at> gnu.org writes:
>
> > BTW, is there an option to wrap the CFF as ASCII data?  The binary
> > data not only confuses dvips, it also slows down emacs tremendously.
>
> Yes, this is possible.  See below how it looks like -- it's a CID
> font, not a CFF, but you should get the idea.  Details can be found in

Actually,  I don't get the idea. Which parts do I need to supply?
All of the  /CFFdefinitions or just the

> %%BeginData: 1314 Binary Bytes
> (Hex)        632 StartData

part?

--

--

Han-Wen Nienhuys   |   hanwen <at> xs4all.nl   |   http://www.xs4all.nl/~hanwen
```
27 Jan 2005 10:15

### Re: UTF8 and (La)TeX backend?

```
> > Yes, this is possible.  See below how it looks like -- it's a CID
> > font, not a CFF, but you should get the idea.  Details can be found in
>
> Actually,  I don't get the idea. Which parts do I need to supply?
> All of the  /CFFdefinitions or just the
>
> 	> %%BeginData: 1314 Binary Bytes
> 	> (Hex)        632 StartData
>
> part?

Oops!  I've just checked the PostScript reference, and I now see that
`StartData' for CFFs doesn't allow the `(Hex)' keyword.  Sorry.  It
seems that we have to live with binary data in the PS files.

Werner
```
6 Jan 2005 23:44

### Re: UTF8 and (La)TeX backend?

```wl <at> gnu.org writes:
>
> > > . The CFF part of bigcheese20.otf is extracted and put into a PS
> > >   resource which is loaded with the -h option of dvips.  If that
> > >   doesn't work, lilypond-book simply emits
> > >   \special{header=bigcheese.cff} at the beginning of the document.
>
> I was a bit unclear.  Here a reformulation:
>
> . The CFF part of bigcheese20.otf is extracted and put into a PS
>   resource which is loaded with the -h option of dvips.  If that
>   doesn't work (this is, if my assumption is incorrect and option -h
>   can't be used to load a PS resource file at all), lilypond-book
>   simply emits \special{header=bigcheese.cff} at the beginning of the
>   document.

dvips doesn't get the CFF embedding. Perhaps there's something wrong
in our routine? It's in scm/framework-ps.scm, function ps-embed-cff

byrd:~/usr/src/lilypond\$ dvips  lily-1406216824
' TeX output 2005.01.06:2339' -> lily-1406216824.ps
dvips: Font bigcheese20 used in file lily-1406216824-1.eps is not in the mapping file.
dvips: Font unknown used in file lily-1406216824-1.eps is not in the mapping file.
<texc.pro><bigcheese20.cff>
dvips: ! expected to see %%EndBinary at end of data

dvips complains both if I use -h and \special{header=}

Also can you check if the .EPS files are correct wrt. document
```

3 Jan 2005 23:34

### Re: UTF8 and (La)TeX backend?

```wl <at> gnu.org writes:
>     by Pango.  Even if they use TeX fonts, the information is based on
>     the PS outlines.  I fully support your idea to replace
>     ec-fonts-mftraced with cmsuper or something similar, converted to
>     OpenType so that the TFM files are no longer needed.  Ideally,
>     those OpenType TeX fonts should behave similar to any other
>     OpenType font; this means the loss of italic correction (which
>     isn't available in OpenType fonts), for example, but it assures
>     that the font interface doesn't have to cope with special cases.

That's not going to work completely, not only do we loose italic
correction, but also design size. Can't we agree with other parties
(tetex?) on a standard SFNT table for TeX specific information?

--
Han-Wen Nienhuys   |   hanwen <at> xs4all.nl   |   http://www.xs4all.nl/~hanwen
```
4 Jan 2005 09:09

### Re: UTF8 and (La)TeX backend?

```
> >     Ideally, those OpenType TeX fonts should behave similar to any
> >     other OpenType font; this means the loss of italic correction
> >     (which isn't available in OpenType fonts), for example, but it
> >     assures that the font interface doesn't have to cope with
> >     special cases.
>
> That's not going to work completely, not only do we loose italic
> correction, but also design size.  Can't we agree with other parties
> (tetex?) on a standard SFNT table for TeX specific information?

That's an excellent idea, so I'm forwarding this to the relevant
mailing lists.  George already has started with three SFNT tables in
fontforge:

ITLC  italic correction
TCHL  TeX charlist
TEXL  TeX extension list

Maybe we should add a fourth table to hold the missing bits...

Werner
```
4 Jan 2005 17:04

### Re: Re: UTF8 and (La)TeX backend?

```On Tue, 2005-01-04 at 00:09, Werner LEMBERG wrote:
> > >     Ideally, those OpenType TeX fonts should behave similar to any
> > >     other OpenType font; this means the loss of italic correction
> > >     (which isn't available in OpenType fonts), for example, but it
> > >     assures that the font interface doesn't have to cope with
> > >     special cases.
> >
> > That's not going to work completely, not only do we loose italic
> > correction, but also design size.  Can't we agree with other parties
> > (tetex?) on a standard SFNT table for TeX specific information?
>
> That's an excellent idea, so I'm forwarding this to the relevant
> mailing lists.  George already has started with three SFNT tables in
> fontforge:
>
>   ITLC  italic correction
>   TCHL  TeX charlist
>   TEXL  TeX extension list
ITLC is a GPOS format 1 feature (allows positioning changes to
single glyph)
TCHL is a GSUB format 3 feature (replaces one glyph with one of
many glyphs)
TEXL is a GSUB format 2 feature (replaces one glyph with many).
>
> Maybe we should add a fourth table to hold the missing bits...
I'd be happy to add a 'TEX ' table (or something similar) to hold TeX
font parameters. What should go in it? (I'd rather have a separate
meaning for each parameter slot than overload them as the tfm file
does).
```

5 Jan 2005 10:54

### Re: Re: UTF8 and (La)TeX backend?

```On 04 Jan 2005 08:04:23 -0800, George Williams <gww <at> silcom.com> wrote:
> > Maybe these things ought to be discussed on the OpenType mailing list too?
> I brought up ITLC on the OpenType list some time ago, hoping to get it
> added to the standard feature list. I was unable to do so. Perhaps if
> other people expressed interest we could bring it up again?

I didn't even remember that; but looking at the archive it seems there
just wasn't much of a discussion. I'd suggest we try again. Note that
the Adobe people may not feel the necessity of italic correction with
an optical kerning feature built into InDesign.
A fair amount of discussion could follow; I, for one, think that there
ought to be italic correction on the left side as well.

Regards,
Rogier

-------------------------------------------------------
The SF.Net email is sponsored by: Beat the post-holiday blues
Get a FREE limited edition SourceForge.net t-shirt from ThinkGeek.
It's fun and FREE -- well, almost....http://www.thinkgeek.com/sfshirt
```
4 Jan 2005 23:11

### Re: [Fontforge-devel] Re: UTF8 and (La)TeX backend?

```gww <at> silcom.com writes:
> > mailing lists.  George already has started with three SFNT tables in
> > fontforge:
> >
> >   ITLC  italic correction
> >   TCHL  TeX charlist
> >   TEXL  TeX extension list
> 	ITLC is a GPOS format 1 feature (allows positioning changes to
> 		single glyph)
> 	TCHL is a GSUB format 3 feature (replaces one glyph with one of
> 		many glyphs)
> 	TEXL is a GSUB format 2 feature (replaces one glyph with many).
> >
> > Maybe we should add a fourth table to hold the missing bits...
> I'd be happy to add a 'TEX ' table (or something similar) to hold TeX
> font parameters. What should go in it? (I'd rather have a separate
> meaning for each parameter slot than overload them as the tfm file
> does).

Hmm. I'd say it should contain all info from the TFM

* metrics (these are entirely unrelated to the extents of the
outlines)

* 30 spacing parameters

* checksum

* design size
```

5 Jan 2005 00:19

### Re: Re: UTF8 and (La)TeX backend?

```On Tue, 2005-01-04 at 14:11, Han-Wen Nienhuys wrote:
> Hmm. I'd say it should contain all info from the TFM
>
>   * metrics (these are entirely unrelated to the extents of the
>     outlines)
The advance widths have their natural place.
Height/Depth information:
Per glyph Bounding box information is stored before every glyph in a
tt file (more accurately than in a tfm file). These data are not
stored in a cff file but are easily calculated from outlines.
>
>   * 30 spacing parameters
I'm not sure what you mean by this. Are these the font parameter words?
If so there may be 8 or 23 or 14... Oh, perhaps I see, if you don't
overload them then there are 30 of them.
>
>   * checksum
Why? It has no meaning in an sfnt. Sfnts have their own checksums.
>
>   * design size
As Rogier van Dalen points out this should be stored as a feature.

So I the only things I hear as being needed in a separate table are the
spacing parameters -- which I presume to be the font parameter words --
and possibly the checksum -- if someone can explain why this is
relevant.

Any thoughts on a tag for the table? "TEX ", "TeX ", "TFM " ...

-------------------------------------------------------
```

5 Jan 2005 00:40

### Re: [Fontforge-devel] Re: UTF8 and (La)TeX backend?

```gww <at> silcom.com writes:
> On Tue, 2005-01-04 at 14:11, Han-Wen Nienhuys wrote:
> > Hmm. I'd say it should contain all info from the TFM
> >
> >   * metrics (these are entirely unrelated to the extents of the
> >     outlines)
> The advance widths have their natural place.
> Height/Depth information:
>    Per glyph Bounding box information is stored before every glyph in a
>    tt file (more accurately than in a tfm file). These data are not
>    stored in a cff file but are easily calculated from outlines.

Then they have to be there in the CFF separately. The H/D can be
different from the outlines, eg. due to overshoot.

> >   * 30 spacing parameters
> I'm not sure what you mean by this. Are these the font parameter words?
> If so there may be 8 or 23 or 14... Oh, perhaps I see, if you don't
> overload them then there are 30 of them.

in my TFM reader it says,

/* The maximum number of global font parameters we allow.  */
#define TFM_MAX_FONTDIMENS 30

I guess they're font parameter words.

> >   * checksum
> Why? It has no meaning in an sfnt. Sfnts have their own checksums.

```

5 Jan 2005 02:36

### Re: Re: UTF8 and (La)TeX backend?

```On Tue, 2005-01-04 at 15:40, Han-Wen Nienhuys wrote:
> Then they have to be there in the CFF separately. The H/D can be
> different from the outlines, eg. due to overshoot.
What do you mean by overshoot?

Remember tfm metrics are not perfect. There are only 16 possible values
for depth or height, and one of these is required to be 0. So in a font
with many different glyph sizes the tfm file has already done some kind
of grouping operation which can lose a lot of information.

All the information about the glyph's shape is contained in its outline.
It's going to be far more accurate than a rounded tfm value. Now
admittedly there an outline may not reach it's extrema at an on-curve
point, but finding extrema on a cubic/quadratic is fairly trivial.

-------------------------------------------------------
The SF.Net email is sponsored by: Beat the post-holiday blues
Get a FREE limited edition SourceForge.net t-shirt from ThinkGeek.
It's fun and FREE -- well, almost....http://www.thinkgeek.com/sfshirt
```
5 Jan 2005 16:35

### Re: [Fontforge-devel] Re: UTF8 and (La)TeX backend?

```
> Remember tfm metrics are not perfect.  There are only 16 possible
> values for depth or height, and one of these is required to be 0.
> So in a font with many different glyph sizes the tfm file has
> already done some kind of grouping operation which can lose a lot of
> information.

Whatever the TeX height and depth values look like, they should be
exactly represented in the OpenType font -- remember that those values
are *completely independent* from the real glyph shape.  Replacing
them with bbox values is not possible.

Werner

```
5 Jan 2005 14:56

### Re: Re: [Fontforge-devel] Re: UTF8 and (La)TeX backend?

```"GW" == George Williams writes:

>> Then they have to be there in the CFF separately. The H/D can be
>> different from the outlines, eg. due to overshoot.
GW> What do you mean by overshoot?

GW> Remember tfm metrics are not perfect. There are only 16 possible
GW> values for depth or height, and one of these is required to be
GW> 0. So in a font with many different glyph sizes the tfm file has
GW> already done some kind of grouping operation which can lose a lot
GW> of information.

GW> All the information about the glyph's shape is contained in its
GW> outline.  It's going to be far more accurate than a rounded tfm
GW> value.

No, the glyph height, for example, is used by TeX to put calculate
accent positioning over the glyph. And by design, some glyphs like "o"
have small "overshoots", which should not lead to a change in
caculated accent position. So having glyph dimentions as separate
entities (rather than obtained from glyph outline) seems the right
thing to do.

(added Cc to tex-fonts <at> math.utah.edu, as this discussion is most
relevant to that list).

Best,
v.

```

4 Jan 2005 10:37

### Re: Re: UTF8 and (La)TeX backend?

```On Tue, 04 Jan 2005 09:09:16 +0100 (CET), Werner LEMBERG <wl <at> gnu.org> wrote:
>
> > >     Ideally, those OpenType TeX fonts should behave similar to any
> > >     other OpenType font; this means the loss of italic correction
> > >     (which isn't available in OpenType fonts), for example, but it
> > >     assures that the font interface doesn't have to cope with
> > >     special cases.
> >
> > That's not going to work completely, not only do we loose italic
> > correction, but also design size.  Can't we agree with other parties
> > (tetex?) on a standard SFNT table for TeX specific information?
>
> [...]
> Maybe we should add a fourth table to hold the missing bits...

FWIW, design size information is already in the OpenType specs: "size"
feature (rather than a table).
Maybe these things ought to be discussed on the OpenType mailing list too?

Regards,
Rogier

-------------------------------------------------------
The SF.Net email is sponsored by: Beat the post-holiday blues
Get a FREE limited edition SourceForge.net t-shirt from ThinkGeek.
It's fun and FREE -- well, almost....http://www.thinkgeek.com/sfshirt
```
3 Jan 2005 23:21

### Re: UTF8 and (La)TeX backend?

```wl <at> gnu.org writes:
>
> > * LilyPond needs to determine a font both in TeX and PS.
> > * LilyPond needs to load those metrics to get a rough estimate of
> >   sizes.
>
> Why do you need a rough estimate of sizes for the TeX backend?  I
> thought this is done by `-f texstr'.

Because -f texstr runs a full formatting run. To make that one run
sanely, we have to have dimensions which are somewhat realistic.

> To avoid misunderstandings, let me recapitulate:
>
>   . All backends except TeX directly produce output using font
>     information provided by fontconfig, and string dimensions provided
>     by Pango.  Even if they use TeX fonts, the information is based on
>     the PS outlines.

Yes. - and if possible, I would also like to do font selection for TeX
with pango.

> I fully support your idea to replace
>     ec-fonts-mftraced with cmsuper or something similar, converted to

Yes.

>     OpenType so that the TFM files are no longer needed.  Ideally,
>     those OpenType TeX fonts should behave similar to any other
>     OpenType font; this means the loss of italic correction (which
```

4 Jan 2005 09:29

### Re: UTF8 and (La)TeX backend?

```> > As perverse as it probably sounds in the very first moment, what
> > do you think of always create an EPS file, even for the TeX
> > backend, but without the text strings?
>
> Hmmm; that's an interesting idea.  I agree it sounds perverse at the
> beginning, but on 2nd thought it actually makes sense, because there
> most of the TeX backend escapes to PS anyway, via the \embeddedps{}
> macro.  Nevertheless, I don't see what practical advantage it would
> deliver.

The advantage is that we only have to externally handle the creation
of text strings, everything else is internal to lilypond.  With other
words, we no longer need the feta/parmesan/... TFM and PFB files,
since TeX will never see those glyphs -- only lilypond uses them.

Compare this to metapost's handling of labels (mpman.ps, pages 20ff)
which also works with both groff and TeX.

Werner
```

Gmane