Syntax proposal: conditions [take two]

CSL Development
1

Thanks to everyone who contributed to the previous thread on this:

http://xbiblio-devel.2463403.n2.nabble.com/Syntax-proposal-conditions-td7578516.html

I’d like to reboot the discussion based on a revised proposal. I’ll
just give a brief illustration in this post, with a note of its
advantages and disadvantages. A link to the original test case,
revised and refactored on the basis of this proposal, is at the bottom
of the post.

(I should perhaps mention that this has become a front-burner issue
for CSL-m, the extended schema used for the MLZ legal styles. An MLZ
user has reported that one cite form in the MLZ American Law Style
crashes the processor on her machine, with an error message declaring
“excessive recursion”. There is in fact no recursion issue in the
processor. The error seems to be triggered by deep conditional
nesting, which is exactly what the syntax proposed here is meant to
help bring under control.)***

The syntax proposed here, based on a suggestion by Rintze, looks like
this (I have substituted the entirely made-up term “anynot” for
Rintze’s tentative suggestion of “nand”):

Example

Summary

The extended syntax only kicks in when cs:if or cs:else-if have no
attributes. Otherwise, the current syntax is applied. Under the
extended syntax, cs:conditions must be an immediate child of cs:if or
cs:else-if. cs:conditions carries an (optional) match attribute, and
must have one or more cs:condition children. The cs:condition elements
are each evaluated exactly as current cs:if or cs:else-if elements,
and the set of cs:condition results are evaluated according to the
match attribute on cs:conditions. The cs:condition elements may not
have sub-elements.

Advantages

From an XML standpoint, this avoids the messiness of “not:” prefixes
in the arguments with which Bruce took issue. That should also
simplify things on the implementation side.

From an implementation standpoint, it is simple to identify
extended-syntax nodes by the absence of attributes on cs:if or
cs:else-if, and the cs:conditions parent reflects the structure of the
logic demanded – evaluation of children, followed by top-level
evaluation according to the cs:conditions match attribute. The
symmetry of the extended logic requires very little additional code
for implementation.

From a style coding standpoint, this has no impact on existing styles.
When the extended syntax is used, it does not alter the indenting of
style code, making for clean patches in the repository.

Disadvantages

As Sylvester indicates, interaction between nodes can be a headache,
and this proposal does involve that.

There is a cost in the need for additional documentation for the new
functionality, and need to consider how the additional functionality
should be deployed, and at what version level it should be introduced.

Versioning

As this is backward-compatible, deployment in a CSL 1.0.2 version
should be possible (but I have no strong opinion).

Here is a test fixture that exercises the syntax (Unfortunately
BitBucket does not support interlineary comments):

https://bitbucket.org/fbennett/citeproc-js/src/54e0952d27e76231a319bce0dd70732af272b2c4/tests/fixtures/local/condition_PlusMinus.txt?at=default#cl-10

Frank

2

I personally think this structure is the ‘cleanest’ yet, and I think
it can be clearly described and easily understood. It is not as
expressive as some of the other schemes, but Frank seems content with
it, and I don’t expect that we’ll come across much harder cases than
American Law Style. I would have no problems introducing this in CSL.

(whether or not we truly need “nand”/“anynot” as a value for the
"match" attribute seems like a separate issue to me. Frank, please let
us know if you come across cases where it would be useful)

Rintze

3

I like it too. It is easy to read, and allows for a lot of expression. It takes into account Bruce’s concerns. Finally, the fact that it meets Frank’s demands means it fulfills the initial needs that prompted the discussion.

I am not sure about ‘anynot’ and I overlooked that part in the previous discussion. If it’s supposed to be a ‘nand’, then ‘nand’ is a better word for it.

Charles

4

I’d like to ask an orthogonal question. See below …

As this is backward-compatible, deployment in a CSL 1.0.2 version
should be possible (but I have no strong opinion).

Do we have a common understanding of how we define
“backward-compatible” for CSL?

Do we maybe need to make that explicit if it’s not now?

Bruce

5

I’d like to ask an orthogonal question. See below …

As this is backward-compatible, deployment in a CSL 1.0.2 version
should be possible (but I have no strong opinion).

Do we have a common understanding of how we define
“backward-compatible” for CSL?

Do we maybe need to make that explicit if it’s not now?

Didn’t we have this discussion already when I prepared the release of CSL 1.0.1?

From http://en.wikipedia.org/wiki/Backward_compatibility: “If products
designed for the new standard can receive, read, view or play older
standards or formats, then the product is said to be
backward-compatible”. So for us, I would define backward-compatibility
as the ability of CSL processors written for one specific version
(e.g. “1.0.1”) to process CSL styles written in an earlier CSL version
(e.g. “1.0”).

If this proposal is accepted, there would be two ways to encode
conditions in CSL styles: the “old” way, where all conditions and the
“match” attribute are loaded on cs:if’s and cs:else-if’s, and the
“new” way, where we use the XML structure from the current proposal.
Styles would be able to mix & match at the level of individual cs:if’s
and cs:else-if’s (e.g., using the “old” structure for one cs:if, and
the “new” structure for the following cs:else-if). Since we keep the
“old” structure around, CSL 1.0.1 styles will keep working. Hence the
change would be backward-compatible, hence the change would qualify
for a 1.0.2 release.

Rintze

6

I’d like to ask an orthogonal question. See below …

As this is backward-compatible, deployment in a CSL 1.0.2 version
should be possible (but I have no strong opinion).

Do we have a common understanding of how we define
“backward-compatible” for CSL?

Do we maybe need to make that explicit if it’s not now?

Didn’t we have this discussion already when I prepared the release of CSL 1.0.1?

From http://en.wikipedia.org/wiki/Backward_compatibility: “If products
designed for the new standard can receive, read, view or play older
standards or formats, then the product is said to be
backward-compatible”. So for us, I would define backward-compatibility
as the ability of CSL processors written for one specific version
(e.g. “1.0.1”) to process CSL styles written in an earlier CSL version
(e.g. “1.0”).

I think the following are two separate things:

  • the spec is backwards compatible
  • a CSL processor is backwards compatible

Note that that no matter what you write in the spec a processor can always opt to be backwards compatible (unless you outright forbid it) by analyzing a given style and choosing between alternative algorithms.

Note also that you can always make new features backwards compatible in the spec by doing exactly what you propose below: explicitly specifying alternatives by saying there is a ‘new’ way and an ‘old’ way. In my opinion this should not be part of a spec – by all means, you can add a note urging implementers to also support an ‘old’ way, but why force them? I find this extremely disrespectful – sure, it’s easy for an established implementation to switch to a new feature and keep the old one around for compatibility’s sake, but think for a moment what you’re saying to developers working on a new implementation (I understand that this is something to be encouraged) – essentially, to implement a feature twice, or to wrap their head around how they can reconcile two approaches into a single algorithm – precisely what the spec fails to do in the first place.

If a feature differs so much that the underlying syntax changes or that new elements need to be created, I don’t think it’s fair to call it backwards compatible. Such a feature should go into a major release and the legacy feature should be dropped completely. By doing this you’re actually helping implementers to provide backwards compatibility, because they can simply look at a style’s version and tell whether they need to follow the ‘new’ or ‘old’ way.

A new feature that can be implemented in such a way that the same algorithm can still render old styles – this is something I would call backwards-compatible (as far as the spec is concerned). The original proposal for the feature in question was just that by the way – I would see no problem to include it in a minor release. This is not the case with the current proposal.

Sylvester

7

When you put it this way, I agree “backward-compatibility” is more of a trick than a real thing.

However, it got me thinking: do we want to get rid of the “old” syntax, where the predicates are in the <if> element? I would think no, we want to keep them. This is still the shortest and most readble way to write conditionals in most cases. So maybe in this case, we are in fact truly backward-compatible :slight_smile:

Charles

8

When you put it this way, I agree “backward-compatibility” is more of a trick than a real thing.

However, it got me thinking: do we want to get rid of the “old” syntax, where the predicates are in the <if> element? I would think no, we want to keep them. This is still the shortest and most readble way to write conditionals in most cases. So maybe in this case, we are in fact truly backward-compatible :slight_smile:

Charles, you’re right, if we still want to encourage its use that’s a compelling reason to keep the old way around, of course. (I wanted to make the case firmly against keeping old features around just for the sake of compatibility.)

But if this is the case, it gives me even more reason to believe that the original proposal was actually the better one – if only a handful of styles need the advanced syntax anyway it seems much more reasonable to me for it to be an extension of the normal syntax rather than a completely different one. Especially since you decided to lose the ‘not:’ in favour of ‘nand’ matchers anyway.

Do we really want to add two new elements for an alternate syntax when most styles use attributes on cs:if anyway?

Thinking about how to implement this, my best approach was to simply convert the new conditions to the lists which can be used with the original/extended rendering algorithm. But this just tells me that all we’re accomplishing here is introducing a bloated XML syntax to describe simple lists.

Looking at the example:

What I would do here, in order to evaluate the statement is translate the conditions to something that can be reduced efficiently. Something like this:

[‘all’, [‘type’, ‘all’, [‘book’]], [‘variable’, ‘nand’, [‘edition’]]]

This is the information I need to process the conditions – I personally think that using 4 XML nodes to describe this list makes testing a nightmare.

Also, think of the headaches you’re creating for tools that create styles like the style editor: if you have a condition in the ‘old’ way and you change a little detail – does it stay in the old way or is it converted? If you try to keep it in the old way then once you add a feature that can’t be expressed in the old way you need to convert everything etc. – or if tools cannot handle this sort of thing, as a style author you’re forced to convert all conditions to the new syntax if you want to add something that was not supported previously. With the original approach you can always add new conditions without having to change the old ones.

Sylvester

9

The conclusion on the original proposal was that it was hard to read (from a human perspective) and had the problem of moving some of the logic to the attribute values (which from an implementation point of view, and for validation purposes, is a problem).

This proposal is more verbose, but also more readable.

[‘all’, [‘type’, ‘all’, [‘book’]], [‘variable’, ‘nand’, [‘edition’]]]

Only programmers can parse that :wink:

This is the information I need to process the conditions – I personally think that using 4 XML nodes to describe this list makes testing a nightmare.

It’s again a trade-off between being friendly/readable to style implementors, and being friendly to the processors. Though I am not sure where the nightmare comes from in this case. Did you mean it’s harder to build test cases? In which way?

Charles

10

The conclusion on the original proposal was that it was hard to read (from a human perspective) and had the problem of moving some of the logic to the attribute values (which from an implementation point of view, and for validation purposes, is a problem).

Isn’t this problem eliminated by introducing the nand matcher?

As I understood it the new proposal does not support the individual negation either but uses the ‘nand’ matcher for this purpose – you can apply this to the original proposal just as easily. The example becomes:

This proposal is more verbose, but also more readable.

[‘all’, [‘type’, ‘all’, [‘book’]], [‘variable’, ‘nand’, [‘edition’]]]

Only programmers can parse that :wink:

Indeed – it wasn’t supposed to be a new syntax proposal, just to explain what sort of information we as programmers need to extract from the style. : )

This is the information I need to process the conditions – I personally think that using 4 XML nodes to describe this list makes testing a nightmare.

It’s again a trade-off between being friendly/readable to style implementors, and being friendly to the processors. Though I am not sure where the nightmare comes from in this case. Did you mean it’s harder to build test cases? In which way?

Is it being friendly to style implementors though? Personally, I don’t enjoy writing so much XML. Nightmare is perhaps a little strong, granted, but look: I want to have test cases for each individual element – it helps limit errors when I am able to do that. With the new syntax the sole purpose of the if or conditions element is to fetch the conditions from its child elements – that’s something you cannot test in isolation unless you mock all the child elements. However, since the logic involved here is so trivial it isn’t really worth the trouble to write mocks – and your tests will, in all likelihood not be isolated anymore.

Furthermore, I am writing a processor specifically to be able to deal with CSL fragments, because sometimes users want to overrule a specific element on the fly or because they don’t need a full CSL style, but just want to format, say a title, or a name – for these cases, and for testing, it is extremely useful to just create a new node on the fly and feed it to the processor. Having child elements always complicates matters because you need to create those first.

Of course, in many cases this is necessary – but is that true for a condition? More so, as conditions are something that, conceivably, you might want to quickly change on the fly if you’re working with a CSL processor programmatically.

I have the feeling that CSL processors tend to become these monolithic black-boxes that take the style, and the locale, and the citation data and give you the output – but are extremely difficult for other developers to work with from the outside – this is certainly what happened to citeproc-ruby and this is why I started to re-write it from scratch. Having to handle an XML sub-tree to hold a simple condition is precisely something that encourages this tendency in my opinion.

I realize that at the end of the day it is largely a matter of taste: do you prefer more attributes or more elements? – But to actively support both approaches is a really bad idea in my opinion. Most processors parse the style and convert it into some kind of internal representation – but all tools that also want to convert the internal presentation back into a valid style are in a really bad position now: the easy way out is of course, to always convert to the more expressive syntax – but if, as you say, the idea is to keep the old one for the simple cases because it is shorter and more readable, this defeats the purpose.

Sylvester

11

The conclusion on the original proposal was that it was hard to read (from a human perspective) and had the problem of moving some of the logic to the attribute values (which from an implementation point of view, and for validation purposes, is a problem).

Isn’t this problem eliminated by introducing the nand matcher?

As I understood it the new proposal does not support the individual negation either but uses the ‘nand’ matcher for this purpose – you can apply this to the original proposal just as easily. The example becomes:

This proposal is more verbose, but also more readable.

[‘all’, [‘type’, ‘all’, [‘book’]], [‘variable’, ‘nand’, [‘edition’]]]

Only programmers can parse that :wink:

Indeed – it wasn’t supposed to be a new syntax proposal, just to explain what sort of information we as programmers need to extract from the style. : )

This is the information I need to process the conditions – I personally think that using 4 XML nodes to describe this list makes testing a nightmare.

It’s again a trade-off between being friendly/readable to style implementors, and being friendly to the processors. Though I am not sure where the nightmare comes from in this case. Did you mean it’s harder to build test cases? In which way?

Is it being friendly to style implementors though? Personally, I don’t enjoy writing so much XML. Nightmare is perhaps a little strong, granted, but look: I want to have test cases for each individual element – it helps limit errors when I am able to do that. With the new syntax the sole purpose of the if or conditions element is to fetch the conditions from its child elements – that’s something you cannot test in isolation unless you mock all the child elements. However, since the logic involved here is so trivial it isn’t really worth the trouble to write mocks – and your tests will, in all likelihood not be isolated anymore.

That’s true. But the problem isn’t limited to this syntax. cs:label
inside cs:names can’t be tested without its parent and sibling
elements, and children of a sibling cs:substitute node. Implicit
suppression in cs:group can’t be tested without its children. cs:macro
can’t be tested without children.

Furthermore, I am writing a processor specifically to be able to deal with CSL fragments, because sometimes users want to overrule a specific element on the fly or because they don’t need a full CSL style, but just want to format, say a title, or a name – for these cases, and for testing, it is extremely useful to just create a new node on the fly and feed it to the processor. Having child elements always complicates matters because you need to create those first.

Of course, in many cases this is necessary – but is that true for a condition? More so, as conditions are something that, conceivably, you might want to quickly change on the fly if you’re working with a CSL processor programmatically.

I have the feeling that CSL processors tend to become these monolithic black-boxes that take the style, and the locale, and the citation data and give you the output – but are extremely difficult for other developers to work with from the outside – this is certainly what happened to citeproc-ruby and this is why I started to re-write it from scratch. Having to handle an XML sub-tree to hold a simple condition is precisely something that encourages this tendency in my opinion.

I agree that it is less cumbersome to have the entire logic on the
cs:if or cs:else-if node, but working through some initial cleanup in
one of the legal styles has taught me that second proposal is
necessary to produce more compact code in a complex style.

Here is what the title macro from the American Law style now looks like:

Title macro from the American Law Style, as a worked example from a production environment · GitHub

Welcome to my nightmare: as you can see, the underlying style is quite
a disaster zone, but we’re stuck with it (and not just us – pity the
poor paralegals who are currently tasked with typing this stuff out by
hand). The conditions seem to cope: the current set of 1760
integration tests for the style now pass.

Through this consolidation I was able to remove four macros and 60
lines of code. Any mis-rendered title in the style will now originate
from this macro, which should save a lot of debugging time going
forward. With the initial proposal, the result would not have been
this compact.

Implementing the syntax wasn’t too much work, since it was just a
matter of nesting functions that had already been coded. But that’s
just one experience: I hear you when you say that spreading logic
across nodes has costs. I just don’t see how logic of several of the
conditions in the sample could be squeezed onto a single node.

I realize that at the end of the day it is largely a matter of taste: do you prefer more attributes or more elements? – But to actively support both approaches is a really bad idea in my opinion. Most processors parse the style and convert it into some kind of internal representation – but all tools that also want to convert the internal presentation back into a valid style are in a really bad position now: the easy way out is of course, to always convert to the more expressive syntax – but if, as you say, the idea is to keep the old one for the simple cases because it is shorter and more readable, this defeats the purpose.

I confess that the idea of converting the internal representation back
to XML never occurred to me, and I’m curious. What would be the use
case for that?

12

With the CSL 1.0.1 release, we had a similar issue with the algorithm
for ordinals, as you’re well aware. But if we had done what you
propose, and drop the “old” algorithm from the spec, we would either
have had to:

  • drop the feature from CSL 1.0.1 and release it as a 'clean’
    backwards compatible release;
    or
  • release 1.0.1 as a backwards incompatible release. That would have
    involved the need to update all the existing styles in the repository
    to the new logic. Also, all CSL-implementations (like Zotero and
    Mendeley) would have to make sure from that point on to feed their CSL
    processors 1.0.1 styles, either by internally updating styles, or by
    complaining that 1.0 styles are no longer supported, which would be a
    headache for users with customized and/or not-auto-updating styles.

I understand the downside of having to implement two algorithms, but
from a user perspective the transition from CSL 1.0 to 1.0.1 was much
smoother than from 0.8.1 to 1.0, and I thought you rather liked the
new ordinal logic (to which you contributed yourself).

Rintze

13

The conclusion on the original proposal was that it was hard to read (from a human perspective) and had the problem of moving some of the logic to the attribute values (which from an implementation point of view, and for validation purposes, is a problem).

Isn’t this problem eliminated by introducing the nand matcher?

As I understood it the new proposal does not support the individual negation either but uses the ‘nand’ matcher for this purpose – you can apply this to the original proposal just as easily. The example becomes:

This proposal is more verbose, but also more readable.

[‘all’, [‘type’, ‘all’, [‘book’]], [‘variable’, ‘nand’, [‘edition’]]]

Only programmers can parse that :wink:

Indeed – it wasn’t supposed to be a new syntax proposal, just to explain what sort of information we as programmers need to extract from the style. : )

This is the information I need to process the conditions – I personally think that using 4 XML nodes to describe this list makes testing a nightmare.

It’s again a trade-off between being friendly/readable to style implementors, and being friendly to the processors. Though I am not sure where the nightmare comes from in this case. Did you mean it’s harder to build test cases? In which way?

Is it being friendly to style implementors though? Personally, I don’t enjoy writing so much XML. Nightmare is perhaps a little strong, granted, but look: I want to have test cases for each individual element – it helps limit errors when I am able to do that. With the new syntax the sole purpose of the if or conditions element is to fetch the conditions from its child elements – that’s something you cannot test in isolation unless you mock all the child elements. However, since the logic involved here is so trivial it isn’t really worth the trouble to write mocks – and your tests will, in all likelihood not be isolated anymore.

That’s true. But the problem isn’t limited to this syntax. cs:label
inside cs:names can’t be tested without its parent and sibling
elements, and children of a sibling cs:substitute node. Implicit
suppression in cs:group can’t be tested without its children. cs:macro
can’t be tested without children.

Of course you need integration tests, too – my point is that the cost of switching from attributes to elements means that condition evaluation (note that I’m not talking about actual rendering/processing, but just evaluating the condition) now requires mocks for unit testing too.

In trying to write a modular processor, every inter-dependency between elements hurts. Obviously we have this elsewhere, too, but this adds one more instance to the list – I’m not against it on principle, but if all we get from this is improved legibility in a handful of complex styles I wonder if it is really worth it. You have a much better grasp of both sides – styles and processor – so I’ll gladly take your word on it, if you say that it is worth it after all. I just wanted to emphasize that I do feel it is a steep price to pay.

This is not even considering the issue of dealing with two syntax variants, which adds a considerable burden I think.

Furthermore, I am writing a processor specifically to be able to deal with CSL fragments, because sometimes users want to overrule a specific element on the fly or because they don’t need a full CSL style, but just want to format, say a title, or a name – for these cases, and for testing, it is extremely useful to just create a new node on the fly and feed it to the processor. Having child elements always complicates matters because you need to create those first.

Of course, in many cases this is necessary – but is that true for a condition? More so, as conditions are something that, conceivably, you might want to quickly change on the fly if you’re working with a CSL processor programmatically.

I have the feeling that CSL processors tend to become these monolithic black-boxes that take the style, and the locale, and the citation data and give you the output – but are extremely difficult for other developers to work with from the outside – this is certainly what happened to citeproc-ruby and this is why I started to re-write it from scratch. Having to handle an XML sub-tree to hold a simple condition is precisely something that encourages this tendency in my opinion.

I agree that it is less cumbersome to have the entire logic on the
cs:if or cs:else-if node, but working through some initial cleanup in
one of the legal styles has taught me that second proposal is
necessary to produce more compact code in a complex style.

Here is what the title macro from the American Law style now looks like:

Title macro from the American Law Style, as a worked example from a production environment · GitHub

Welcome to my nightmare: as you can see, the underlying style is quite
a disaster zone, but we’re stuck with it (and not just us – pity the
poor paralegals who are currently tasked with typing this stuff out by
hand). The conditions seem to cope: the current set of 1760
integration tests for the style now pass.

Ouch! This reminds me of the BibTeX style manual – ‘taming the beast’ : )

Through this consolidation I was able to remove four macros and 60
lines of code. Any mis-rendered title in the style will now originate
from this macro, which should save a lot of debugging time going
forward. With the initial proposal, the result would not have been
this compact.

Looking at this style I do see the advantages. It is mostly that this allows you to include different combinations of condition-types in the same statement, right? (The one on line 130 is a good example) Whereas with the original proposal you would have to split that into individual else-ifs all targeting the same macro?

How many styles like that are there, though? There’s no doubt that this application obviously makes the syntax more powerful – if this outweighs the costs in your opinion, I’m all for adding it.

Implementing the syntax wasn’t too much work, since it was just a
matter of nesting functions that had already been coded. But that’s
just one experience: I hear you when you say that spreading logic
across nodes has costs. I just don’t see how logic of several of the
conditions in the sample could be squeezed onto a single node.

I don’t think that it is a problem to implement this syntax: I thought it added unwarranted overhead. If there are enough benefits for enough styles, I agree that it is worth it.

I realize that at the end of the day it is largely a matter of taste: do you prefer more attributes or more elements? – But to actively support both approaches is a really bad idea in my opinion. Most processors parse the style and convert it into some kind of internal representation – but all tools that also want to convert the internal presentation back into a valid style are in a really bad position now: the easy way out is of course, to always convert to the more expressive syntax – but if, as you say, the idea is to keep the old one for the simple cases because it is shorter and more readable, this defeats the purpose.

I confess that the idea of converting the internal representation back
to XML never occurred to me, and I’m curious. What would be the use
case for that?

For example, Rintze asked me the other day if we could detect styles that had superfluous strip-periods attributes. This involves a kind of mini cite processor because you need to translate all possible combinations of term translations. You can see the barebones of how this could work here: Analyze CSL Styles Strip-Periods · GitHub – this is much easier to do with a dedicated library than in pure XML. Once you detect the changes, it would be great to simply save them back as XML but for that you need to convert the internal representation back to XML. When doing that, there are already many subtleties (e.g., how to treat attributes which are set to default values) – having to keep track of different output variants doesn’t make the job easier.

Sylvester

14

You only need to update the styles, because you’re not allowing a version number of 1.0.1 on the style; or, alternatively, because you’re releasing it as 1.0.1 instead of 1.1.

Every processor that supports the new version can still support the older version. By checking the CSL version number of each style it is easy to differentiate between the two. And what is the big problem of having both 1.0 and 1.1 styles in the repository?

If the problem is that there were critical bugfixes in 1.0.1 which required all styles to be updated then there should actually be two releases: one bugfix 1.0.1 and a new stable release 1.1 which also includes the new features. This way only styles or locales that require the new feature need to be updated to the latest release version.

The ordinal solution now is awkward – the processor can’t make use of the version number and needs to look closely at the ordinal terms to decide which algorithm to use. Why do we even use version numbers if we can’t use them?

Again, all I keep saying is this:

Even if the spec is not backwards compatible, the cite processors still can and probably will support multiple versions. You keep citing the problems of CSL-implementations and their users – but this is not really the problem of the specification, but of the implementations. All the implementers are here in this forum, it’s not like anyone would be taken unawares by a release. And don’t get me wrong: I think you’re doing a great job by caring so much for backwards compatibility – if I say we shouldn’t enforce it in the spec it doesn’t mean that we can’t share strategies etc. of how to keep the implementations backwards compatible. If you put all that into the spec, however, it will become more convoluted with every release.

Sylvester

15

Every processor that supports the new version can still support the older version.

Is supporting one version of CSL that requires support for two
different algorithms worse than having to support two CSL versions?

And what is the big problem of having both 1.0 and 1.1 styles in the repository?

In a single branch? We have trouble enough as it is to explain to
style contributors how to submit their styles. Would we have 1.0 and
1.1 versions of the same style? How is that not a nightmare to
maintain? And if you’re proposing to use multiple branches, we already
do that: https://github.com/citation-style-language/styles/tree/1.0
(which is about as much proof as I can give you that nobody is
interested in maintaining 1.0 styles).

The ordinal solution now is awkward – the processor can’t make use of the version number and needs to look closely at the ordinal terms to decide which algorithm to use.

I guess we could have allowed for a “1.0.1” value on the “version”
attribute. But that would have meant either bifurcating the schema to
support validation of both 1.0 and 1.0.1 styles/locales, or using
separate versions of the schema for validate the respective versions
of styles and locales. Both would be a headache.

Why do we even use version numbers if we can’t use them?

For “major” updates, like a 1.1 release. (which might not be the right approach)

to keep the implementations backwards compatible. If you put all that into the spec, however, it will become more convoluted with every release.

We would strip any “old” algorithms from the spec on every “major”
backwards-incompatible release.

Rintze

16

So this is the kind of conversation about compatibility, in relation
to change management, that I was hoping to prompt with my question.

But we basically have two (or more) related, but also parallel,
discussions and issues going on here, and I think we need to clearly
separate them.

Suggestion: can we marshal this into some document, say on the wiki,
that we can refer to later?

Bruce

17

Every processor that supports the new version can still support the older version.

Is supporting one version of CSL that requires support for two
different algorithms worse than having to support two CSL versions?

For the CSL implementation it is the same thing, but it gives you the choice to support only the new version. The CSL implementation becomes simpler.

And what is the big problem of having both 1.0 and 1.1 styles in the repository?

In a single branch? We have trouble enough as it is to explain to
style contributors how to submit their styles. Would we have 1.0 and
1.1 versions of the same style?

No. As long as the style does not require any 1.1 features it stays at 1.0.

If it needs 1.1 features it is updated – the 1.0 version can keep existing in a separate branch like it does now if anyone wants it.

That’s why I said that if there are any changes which absolutely must be updated by all styles, they should always go into a ‘truly’ backwards compatible release.

How is that not a nightmare to maintain?

The difference to what we have now is that you mark the style as updated by updating it’s version number and that you must update all backwards incompatible parts of a style. This can be a burden, true, but what are the backwards compatible parts? The ordinals in locales? The conditionals?

I realize that maintenance is a big issue – we can make it part of the decision of whether or not to include a backwards incompatible feature. For example, we could try to write a script which updates the feature in question ahead of time and only if we know that we can update styles automatically do we agree to drop support for a legacy feature.

For example, ordinals and conditions: if we have a script that updates locales and styles to the new syntax, we could drop the old syntax from a 1.1 release. CSL implementations could still implement 1.0 in order to keep supporting old styles – it’s essentially the same as specifying two variants in the spec but going forward the spec is lighter and implementations can make use of the version number to have much cleaner separations of alternative algorithms.

And if you’re proposing to use multiple branches, we already
do that: https://github.com/citation-style-language/styles/tree/1.0
(which is about as much proof as I can give you that nobody is
interested in maintaining 1.0 styles).

I think the legacy branches are a good pragmatic approach.

The ordinal solution now is awkward – the processor can’t make use of the version number and needs to look closely at the ordinal terms to decide which algorithm to use.

I guess we could have allowed for a “1.0.1” value on the “version”
attribute. But that would have meant either bifurcating the schema to
support validation of both 1.0 and 1.0.1 styles/locales, or using
separate versions of the schema for validate the respective versions
of styles and locales. Both would be a headache.

Like with the styles, what is the problem of keeping a 1.0 version of the schema around? When validating a style you must check the style’s version and validate against the appropriate schema. It’s not a problem at all to built this into the QA tests – in fact I had it implemented already, because I had been unaware that the 1.0.1 value was not used.

Why do we even use version numbers if we can’t use them?

For “major” updates, like a 1.1 release. (which might not be the right approach)

to keep the implementations backwards compatible. If you put all that into the spec, however, it will become more convoluted with every release.

We would strip any “old” algorithms from the spec on every “major”
backwards-incompatible release.

I was not aware of that : )

This makes it much easier for me to swallow the ‘backwards compatible by retaining old variants’ approach for minor releases.

Sylvester

18

Suggestion: can we marshal this into some document,
say on the wiki, that we can refer to later?

I would appreciate this. I haven’t had time to fully read and digest
this thread yet. The gist Frank posted above really does make me
wonder quite what kind of rabbit hole we’re going down.

Regards,
Rob.

19

If the latest proposal for conditionals is adopted, I’d strongly
suggest keeping the old syntax around, since it’s more compact and
will suffice for most conditionals.

Also, if law styles are the main benefactors of this change, we could
also just keep CSL as it is for the short term, since Frank already
maintains a fork (CSL-m). (he has expressed the desire to have his
CSL-m modifications merged back into CSL, but that’s going to be a
long road, regardless)

Rintze

20

I think the most productive way would be to start by identifying the
various possible release strategies (rolling vs. versioned updates,
etc.), taking into account how they would work with maintenance of
styles/locales in the repositories and those in the wild. I’ll leave
it to others to take the lead on this, though.

Rintze