test result formats

So we’ve got a little issue I earlier worried about with the test
suite results format: the mixed-content HTML. In short, there are many
ways to achieve particular visual output results in HTML, but the
current format would require everyone to agree on one.

I don’t really know how to solve this problem, but perhaps we need to
consider the possibility that we simply can’t reasonably test the rich
formatting options, and should just test plain text output?

Any opinions or ideas on this?

Bruce

So we’ve got a little issue I earlier worried about with the test
suite results format: the mixed-content HTML. In short, there are many
ways to achieve particular visual output results in HTML, but the
current format would require everyone to agree on one.

I don’t really know how to solve this problem, but perhaps we need to
consider the possibility that we simply can’t reasonably test the rich
formatting options, and should just test plain text output?

Any opinions or ideas on this?

Formatting needs to be tested, unfortunately. One of the hardert
things to implement has been second-field-align, which I think is
going to require some pretty arcane twisting and turning in any
implementation. … The punctuation order issue has to have formatting
in there so you can see sequencing. … Layout has to capture its
affixes with formatting … there are probably others.

The idea we had at the start was to allow the fixture to define
multiple results, and pass if one of them matches. That’s easy enough
to set up in the fixture. It would take a little work to set up in
each test suite, but it’s possible to do.

An alternative would be to have alternative “result sets” defined
separately, and allow an implementation to select one with a
parameter. It could work a bit like locale fallback, with the result
written into the fixture serving as default. That way when an
implementation encounters a failing test on valid output, the
implementer could register the valid result in the suite and move on.

I think I like the alternative solution, immediately above.

?

First, I’d like again to suggest that other implementers spend some
time integrating the test suite and offering feedback on these sorts
of issues. I’m a little frustrated that Frank and I are really the
only people working on this …

So we’ve got a little issue I earlier worried about with the test
suite results format: the mixed-content HTML. In short, there are many
ways to achieve particular visual output results in HTML, but the
current format would require everyone to agree on one.

I don’t really know how to solve this problem, but perhaps we need to
consider the possibility that we simply can’t reasonably test the rich
formatting options, and should just test plain text output?

Any opinions or ideas on this?

Formatting needs to be tested, unfortunately. One of the hardert
things to implement has been second-field-align, which I think is
going to require some pretty arcane twisting and turning in any
implementation. …

“Second-field align”?

The punctuation order issue has to have formatting
in there so you can see sequencing. …

This I don’t follow. For sake of argument, what if you had the affixes
include obviously delimiters (“|”) to signal the transitions?

Layout has to capture its affixes with formatting … there are probably others.

I don’t deny this might be important; I just don’t see ATM how to do
it. In my implementation, for example, I have no control over things
like whitespace, or attribute order (which is not significant in xml).
I just let the tostring function in the xml library takes care of all
these details (along with namespaces and prefixes).

But as I said, it’d be helpful if others would look into this.

The idea we had at the start was to allow the fixture to define
multiple results, and pass if one of them matches. That’s easy enough
to set up in the fixture. It would take a little work to set up in
each test suite, but it’s possible to do.

An alternative would be to have alternative “result sets” defined
separately, and allow an implementation to select one with a
parameter. It could work a bit like locale fallback, with the result
written into the fixture serving as default. That way when an
implementation encounters a failing test on valid output, the
implementer could register the valid result in the suite and move on.

I think I like the alternative solution, immediately above.

Bruce

That’s the one! Here’s the comment I was working from:

second-field-align is useful for formatting a numbered list. It aligns

any subsequent lines of a bibliography entry with the beginning of the

second field. For example, if the first field is

On Sat, Jul 11, 2009 at 7:32 AM, Bruce D’Arcus<@Bruce_D_Arcus1> wrote:

First, I’d like again to suggest that other implementers spend some
time integrating the test suite and offering feedback on these sorts
of issues. I’m a little frustrated that Frank and I are really the
only people working on this …

On Fri, Jul 10, 2009 at 6:09 PM, Frank Bennett<@Frank_Bennett> wrote:

On Sat, Jul 11, 2009 at 2:23 AM, Bruce D’Arcus<@Bruce_D_Arcus1> wrote:

So we’ve got a little issue I earlier worried about with the test
suite results format: the mixed-content HTML. In short, there are many
ways to achieve particular visual output results in HTML, but the
current format would require everyone to agree on one.

I don’t really know how to solve this problem, but perhaps we need to
consider the possibility that we simply can’t reasonably test the rich
formatting options, and should just test plain text output?

Any opinions or ideas on this?

Formatting needs to be tested, unfortunately. One of the hardert
things to implement has been second-field-align, which I think is
going to require some pretty arcane twisting and turning in any
implementation. …

“Second-field align”?

1. Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do

eiusmod

If set to “margin,” then the first field is put in the margin and all

subsequent lines of text are aligned with the margin. This is useful for

IEEE style.

second-field-align-opt = attribute name { “second-field-align” },
attribute value { “true” | “margin” }

To implement that, you need to put a wrapper around (take a breath)
the remaining top-level rendering objects after the first one. It’s
not going to be obvious how to code that in any implementation, so
there should be a test to check that it’s working.

The punctuation order issue has to have formatting
in there so you can see sequencing. …

This I don’t follow. For sake of argument, what if you had the affixes
include obviously delimiters (“|”) to signal the transitions?

Sorry, I should have unpacked that a bit. It’s important to check
the nesting sequence (punctuation should not get boldfaced). It’s
easy to get that part right, but it’s harder to get that right when
moving punctuation inside quotes, so testing it is a good idea. Not a
bit issue, it’s just one example you can think of.

Layout has to capture its affixes with formatting … there are probably others.

I don’t deny this might be important; I just don’t see ATM how to do
it. In my implementation, for example, I have no control over things
like whitespace, or attribute order (which is not significant in xml).
I just let the tostring function in the xml library takes care of all
these details (along with namespaces and prefixes).

Are there libraries around for doing XML diffs?

“Second-field align”?

That’s the one! Here’s the comment I was working from:

second-field-align is useful for formatting a numbered list. It aligns

any subsequent lines of a bibliography entry with the beginning of the

second field. For example, if the first field is

1. Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do

eiusmod

If set to “margin,” then the first field is put in the margin and all

subsequent lines of text are aligned with the margin. This is useful for

IEEE style.

second-field-align-opt = attribute name { “second-field-align” },
attribute value { “true” | “margin” }

I’m not sure this is as complicated as you think, particularly when
dealing with HTML and CSS. From the CSS perspective, I think we’re
saying whether the list item has a negative margin or not. It’s for
numbered styles where the number goes in the margin.

To implement that, you need to put a wrapper around (take a breath)
the remaining top-level rendering objects after the first one. It’s
not going to be obvious how to code that in any implementation, so
there should be a test to check that it’s working.

But this is really about the rendering engine; not sure we really can.

The punctuation order issue has to have formatting
in there so you can see sequencing. …

This I don’t follow. For sake of argument, what if you had the affixes
include obviously delimiters (“|”) to signal the transitions?

Sorry, I should have unpacked that a bit. It’s important to check
the nesting sequence (punctuation should not get boldfaced). It’s
easy to get that part right, but it’s harder to get that right when
moving punctuation inside quotes, so testing it is a good idea. Not a
bit issue, it’s just one example you can think of.

Layout has to capture its affixes with formatting … there are probably others.

I don’t deny this might be important; I just don’t see ATM how to do
it. In my implementation, for example, I have no control over things
like whitespace, or attribute order (which is not significant in xml).
I just let the tostring function in the xml library takes care of all
these details (along with namespaces and prefixes).

Are there libraries around for doing XML diffs?

Good question; I’ll check.

Bruce

“Second-field align”?

That’s the one! Here’s the comment I was working from:

second-field-align is useful for formatting a numbered list. It aligns

any subsequent lines of a bibliography entry with the beginning of the

second field. For example, if the first field is

1. Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do

eiusmod

If set to “margin,” then the first field is put in the margin and all

subsequent lines of text are aligned with the margin. This is useful for

IEEE style.

second-field-align-opt = attribute name { “second-field-align” },
attribute value { “true” | “margin” }

I’m not sure this is as complicated as you think, particularly when
dealing with HTML and CSS. From the CSS perspective, I think we’re
saying whether the list item has a negative margin or not. It’s for
numbered styles where the number goes in the margin.

All you need to test is whether the right tag wrappers are going in
the right place – the specifics of the formatting attributes can all
be kicked out to CSS, in HTML mode at least. But the tag wrappers do
need to be tested, to be sure the engine is capable of handling the
option.

It’s true that it’s not traumatic to handle or anything, but there is
some coding involved, and untested is broken code and all that.

To implement that, you need to put a wrapper around (take a breath)
the remaining top-level rendering objects after the first one. It’s
not going to be obvious how to code that in any implementation, so
there should be a test to check that it’s working.

But this is really about the rendering engine; not sure we really can.

The punctuation order issue has to have formatting
in there so you can see sequencing. …

This I don’t follow. For sake of argument, what if you had the affixes
include obviously delimiters (“|”) to signal the transitions?

Sorry, I should have unpacked that a bit. It’s important to check
the nesting sequence (punctuation should not get boldfaced). It’s
easy to get that part right, but it’s harder to get that right when
moving punctuation inside quotes, so testing it is a good idea. Not a
bit issue, it’s just one example you can think of.

Layout has to capture its affixes with formatting … there are probably others.

I don’t deny this might be important; I just don’t see ATM how to do
it. In my implementation, for example, I have no control over things
like whitespace, or attribute order (which is not significant in xml).
I just let the tostring function in the xml library takes care of all
these details (along with namespaces and prefixes).

Are there libraries around for doing XML diffs?

Good question; I’ll check.

If there is something for Python, that could be a way forward, using a
testing script written in Python that runs arbitrary processors via a
standard API. Citerpoc-js will have trouble with that in the short
term, because I can’t find a way of setting the locale across the
python-spidermonkey bridge to anything other than “C”. But someone
will eventually sort that out, I’m sure, or another bridge module will
come along.

But this sort of highlights my point, I don’t intend to to do anything
special with this feature in terms of output markup, and my solution
may be as simple as:

    ...

… and leave it to CSS to render. There’s just no way to test this adequately.

Bruce

“Second-field align”?

That’s the one! Here’s the comment I was working from:

second-field-align is useful for formatting a numbered list. It aligns

any subsequent lines of a bibliography entry with the beginning of the

second field. For example, if the first field is

1. Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do

eiusmod

If set to “margin,” then the first field is put in the margin and all

subsequent lines of text are aligned with the margin. This is useful for

IEEE style.

second-field-align-opt = attribute name { “second-field-align” },
attribute value { “true” | “margin” }

I’m not sure this is as complicated as you think, particularly when
dealing with HTML and CSS. From the CSS perspective, I think we’re
saying whether the list item has a negative margin or not. It’s for
numbered styles where the number goes in the margin.

All you need to test is whether the right tag wrappers are going in
the right place – the specifics of the formatting attributes can all
be kicked out to CSS, in HTML mode at least. But the tag wrappers do
need to be tested, to be sure the engine is capable of handling the
option.

It’s true that it’s not traumatic to handle or anything, but there is
some coding involved, and untested is broken code and all that.

But this sort of highlights my point, I don’t intend to to do anything
special with this feature in terms of output markup, and my solution
may be as simple as:

    ...

… and leave it to CSS to render. There’s just no way to test this adequately.

To quote Monty Python, “This isn’t argument, it’s contradiction!” We
don’t have to throw up our hands over this. It’s doable.

For the example using the ol tag and CSS, you won’t want to implement
a render bibiliography in that way. Think of how far removed it is
from the original CSL structures. Also, remember that it must also
render in RTF, and in formats that don’t have CSS behind them.

For the example using the ol tag and CSS, you won’t want to implement
a render bibiliography in that way.

Well, I haven’t made a decision yet, but this is certainly a perfectly
reasonable approach: use the classes, and provide a little CSS file
for developers to include in their documents.

Think of how far removed it is from the original CSL structures. Also,
remember that it must also render in RTF …

RTF support is not a requirement of CSL at all.

Most formats that are important these days (HTML, ODF, OOXML, etc.;
maybe even RTF) support lists, and some way to style them like this
using classes, and so without having to get low-level.

So I don’t think we can design the test suite around that assumption.

Bruce

For the example using the ol tag and CSS, you won’t want to implement
a render bibiliography in that way.

Well, I haven’t made a decision yet, but this is certainly a perfectly
reasonable approach: use the classes, and provide a little CSS file
for developers to include in their documents.

It’s not the use of classes for styling – that’s all good – it’s
that using ol as the top-level wrapper would have CSS providing the
citation-number value automagically, on the assumption that it will
correspond to the values used in CSL to generate citations. That
might work, at least in current use cases, although it does seem
risky. A tougher question is how you would map the formatting
attributes on the first-to-render
element through to the CSS in a robust way. CSL might change the
formatting of the element depending on the reference item content
(boldface for books, say – never mind whether it seems likely or a
good idea, the language allows it, and the language needs to behave
correctly).

You’ll want to keep the output markup as close as possible to the CSL
layout structure. It’s simpler all around that way.

Frank

True; I’ll need to look out for this before making any final
decisions. The key test is whether I can more-or-less easily modify
the wrappers around a list item using CSS or other other styling
language. I know I can modify other characteristics of the list mark.

Bruce