Line length for hyphenated lines

[PhilterPaper]

While experimenting with new examples for text file output (effectively a constant width font), I came across a minor problem. It appears that when KP looks at the accumulated line length (when determining where to split), it is failing to account for the hyphen that may need to be added to the end of the line (i.e., the line ends with a word fragment). The issue arises when the line (with fragment) is already close to being full (r = 0). This is not major issue for proportional fonts, although it is possible that a line that was already a bit tight (r approaching -0.5) could end up excessively tight (r < -0.5) once the hyphen is added. This is not applicable if "hanging punctuation" is used (the hyphen ends up past the right margin anyway).

For constant width fonts (such as flat text output), where r must be ≥ 0, the effect of not taking that hyphen into account can be that the line ends up one character too long. A proportional font can largely hide this by tightening the line a bit more than it should, but for constant width, the interword spaces can't be reduced, so the line ends up one too long.

Most typesetting is done with proportional fonts, so this is not a serious shortcoming, but at some point it should be addressed. Possibly, any word fragment could have its width increased by a hyphen, but only if it's likely that this fragment will be at the end of a line. It will take some diving into the algorithm to see what the best course of action is.

[PhilterPaper]

Just a note that, unless I missed something, @bramstein / typeset and other implementations of Knuth-Plass may also suffer from this glitch. Comments are welcome as to whether the KP algorithm may need tweaking to deal with this.

Also, I would like to know how other implementations handle constant width fonts (or text output). Do they set the maximum glue shrinkage to 0? This is used as a divisor, so I had to check for 0 shrinkage and handle that case explicitly (avoiding the division). Otherwise, it seems to work.

[bramstein]

This is usually on purpose, the hyphens are allowed to "hang" out of the text column. See: https://www.fonts.com/content/learning/fontology/level-4/fine-typography/hanging-punctuation

You can avoid this by some combination of boxes, glue, and penalty (though I don't recall which, sorry).

[PhilterPaper]

I'm assuming here that "hanging punctuation" is not always desired (it is not the default for Text::KnuthPlass, anyway). To "mechanically align" the right edge, added hyphens need to be kept within the hard margin, which causes problems if the line is already as tight as it can be. Certainly, it's a bit easier to let the added hyphen hang out over the edge (especially for constant width text), but then, for consistency most or all other punctuation should also hang over (presumably just the last mark if there are several in a row). There's discussion here and there on what needs to be done with KP to achieve this. It might be desirable when using constant width text to encourage the use of hanging punctuation.

The whole subject of optical alignment on both margins versus a strict mechanical alignment involves not just punctuation hanging over (a bit), but also some letters overhanging a bit on both margins. That can get pretty complicated, but is presumably possible with KP. I've seen tables of suggested percentage overhang for different punctuation marks and letters, but that's probably font-dependent.

Again, thanks much for contributing to the discussion with resource links. Adding hanging punctuation (at right margin) is on my "to do" list for Text::KnuthPlass.

[PhilterPaper]

It occurs to me that one way to treat this situation would be, if setting for text file or constant width font, to reduce the allowed line length by one character, to make room for an added hyphen. If no hyphen is added, it's a tiny bit of wasted space, but as most lines seem to need to expand by at least one character (r > 0) anyway, it's a small cost to avoid having an added hyphen hang over into the margin (assuming you're not doing hanging punctuation).

[PhilterPaper]

That didn't work, so I am going to go ahead and release 1.07 with this minor bug still present. I'll get back to it sometime -- I'm sure it can be fixed.