Upcoming CSL meetup context

[Bruce_D_Arcus1]

The following presents context for discussion at the upcoming virtual chat, scheduled for July 21. But I post it here both in case people want to comment, but won’t be coming, and also for folks to have some context ahead of the discussion.

Current status and problem statement

CSL has been wildly successful.

1. multiple compliant processors
2. thousands of available styles
3. diverse end-user applications, with millions of users

In the past year alone, we’ve seen three processors, and integration with classic open source tools like TeX and Emacs.

This success, however, brings with it a problem: it’s difficult to maintain and evolve all of this (including documentation) in sync. This is particularly the case when much of the labor to do this work is distributed, and done on the inconsistent free time of busy people.

Recent changes and next steps

Last year we tagged a minor v1.0.2 release, with bug fixes and new types and variables. This was the first release in almost a decade.

Draft v1.1 changes

While triaging the issue tracker to identify these smaller changes, we also discovered feature requests that required more structural changes in styling and/or input schema. We identified a subset of these to implement in the draft v1.1 branch, which includes the following changes.

Style XML schema

1. Independent formatting of titles and subtitles.
2. Independent formatting of in-text and in-parentheses author lists
   (required in APA).
3. Richer EDTF dates (intervals, times, etc.).
4. Related-items.

Input JSON/YAML schema

1. Convert array to object, with metadata and a required version.
2. Allow titles to be an object, with main and sub titles.
3. Prefer dates as an EDTF+ string.
4. Related-items.
5. experimental rich-text schema

Aside: a lot of work went into the above, surely almost exclusively a consequence of the free time opened up for some of us early in the pandemic. I doubt that will happen again (hopefully not!).

Questions to resolve

1. How do we get this to release? Do we, for example, just open up a public comment period? If yes, where, and how? Should we call it 1.1, or 2.0?
2. How, more generally, should we manage change in CSL going forward? What other barriers (say in terms of resources, including time/labor) impact our answer to this question, and how might we reduce those barriers?

[larsgw]

@Bruce_D_Arcus1

Should we call it 1.1, or 2.0?

I think a thing to consider is the compatibility of the input schema and the style schema. Right now, without specific processor magic, 1.0.1 input data is already not compatible with 1.0.2 styles (and vice versa), due to event-title and software. So although 1.0.2 and 1.1 styles can probably still be used by 1.0.1 processors, 1.0.2 and 1.1 input cannot be used by 1.0.1 versions of software that relies on specific variables such as the styles but also other applications like Zotero, without data loss. But then, decoupling the data and style versions is probably unwanted too. And data loss is not breaking in the same sense, it likely does not result in crashes.

@Bruce_D_Arcus1

The following presents context for discussion at the upcoming virtual chat, scheduled for July 21.

Is this a public virtual chat?

[Bruce_D_Arcus1]

@larsgw

Is this a public virtual chat?

I haven’t publicized it yet. I’m not sure how best to do that, while keeping it secure, but easy to access and such.

I’ll message you the details for now.

[Sebastian_Karcher]

@Bruce_D_Arcus1

I haven’t publicized it yet. I’m not sure how best to do that, while keeping it secure, but easy to access and such.

I think we should treat this as a “by invitation” event but extend invitations liberally – including to most anyone posting here who is interested in joining, but I don’t think there’s a real benefit to making this public public.

[bwiernik]

Right now, without specific processor magic, 1.0.1 input data is already not compatible with 1.0.2 styles (and vice versa), due to event-title and software

I’m not sure I understand what you mean. Could you elaborate?

[larsgw]

If you have CSL-JSON data with event-title, a 1.0.1 style that use <text variable="event"/> would not work and CSL-JSON data with event does not work with a 1.0.2 style using <text variable="event-title"/>. Same for <if type="software"/> and <if type="book" variable="version"/>. Unless the processor transforms the CSL-JSON data to make it compatible with the style version (though the style only says 1.0 making it difficult to know whether it’s compatible).

[larsgw]

I see there’s some discussion in [Feedback CSL 1.0.2]: Publication type "event" - #31 by Philipp_Zumstein already. That would help but the version strings need to be more precise in this case.

[Bruce_D_Arcus1]

Meeting notes (still open for comments)

(I tried to quickly add them here, but formatting got screwed up)

High-level summary

Great discussion, but we didn’t really resolve the key questions. Hopefully we can in the coming months.

Any further thoughts on how to do that, please post them below.

My further thoughts and ideas

YAML tests?

One small thing I was wondering: should we switch to YAML (or TOML) for the test suite, per @cormacrelf’s example?

Define two (or more) change types, and match them with GitHub review teams

One bigger question, that is about process and labor:

If we do decide we want to release a bigger change, and so have a plan to evolve CSL going forward, perhaps we could have github teams to review different kinds of changes?

For example, processor developers could be on a “Processor reviewers” team, and we could have a separate team for variables/type/term additions and such (adding strings).

Issue triage could then classify and auto-assign reviewers.

And if we don’t want to be bothered, we should just say so and declare CSL effectively frozen, aside from minor changes.

But per @Sebastian_Karcher’s point during the meeting, I’d prefer not to give up the goal of being able to evolve CSL, despite all of the inertia.

Is a schema with more modular features possible?

On that point, I guess the even bigger question is if it’s technically possible to do what some were musing about: having modular features that processors could support, or ignore without consequence for backward compatibility.

I haven’t ever really looked into this idea, but I’m skeptical it is possible.

But that would solve a lot of problems if it was.

Certainly some of what we added to “1.1” could be ignored by a 1.0 processor. But we’d basically need rules for that; e.g. “Processors shall ignore unknown nodes.” And with that, it may constrain some of what we added.

But I’m not sure; maybe we should look into that more?

Off the top of my head, for example:

• cs:intext would work, while this alternative would not
• split titles almost surely would not
• EDTF dates: not sure
• related-items: no

[retorquere]

My apologies for the no-show, y’all.

[John_MacFarlane]

Hi all, sorry I could not come to the meeting. A few thoughts:

• I’m glad to see the discussion about an automatic test suite for the major styles.
• When CSL is updated, it would be good to publish an “archived” style repository for the older version, to support people using processors that only support the older version.
• I like the idea that styles that support the new version could still work with the old version; but I don’t have a good enough idea how much of a constraint that would be on improving CSL.
• I’ve often wished for a tool that can “prune” CSL in a test case so that it’s only as big as it needs to be. (I’m often too lazy to do this, so my test cases are often big and contain many copies of the same CSL style.) It should be possible to add tooling to a CSL processor that keeps track of which macros have been used; this information could be input to a tool that removes unnecessary macros.

Edit: One more thing came to mind. When I was working on citeproc-hs, I observed that the test suite contained a lot of tests for minor corner cases that went beyond anything described in the CSL spec. My README.md contains the following:

Although this library is much more accurate in implementing the CSL spec than pandoc-citeproc was, it still fails some of the tests from the CSL test suite (67/862). However, most of the failures are on minor corner cases, and in many cases the expected behavior goes beyond what is required by the CSL spec. (For example, we intentionally refrain from capitalizing terms in initial position in note styles. It makes more sense for the calling program, e.g. pandoc, to do the capitalization when it puts the citations in notes, since some citations in note styles may already be in notes and in this case their rendering may not require capitalization. It is easy to capitalize reliably, hard to un-capitalize reliably.)

Indeed, in a lot of these corner cases I think citeproc-hs produces better output than the expected test output. I seem to recall discussing this with one of you at the time. Anyway, the gist of my comment is that it would be nice to separate out true CSL conformance tests (which test behavior specifically mandated by the spec) from tests that were just in there as citeproc-js regression tests. The latter tests may still be useful, but they should be separate.

[Bruce_D_Arcus1]

@John_MacFarlane

… the gist of my comment is that it would be nice to separate out true CSL conformance tests (which test behavior specifically mandated by the spec) from tests that were just in there as citeproc-js regression tests. The latter tests may still be useful, but they should be separate.

We actually have tried to do this, I think in response to your earlier feedback.

Do you have a list of these remaining tests that you would propose?

Also, while on this topic, any thoughts on changing these to use YAML, like this?

[John_MacFarlane]

Bruce_D_Arcus1
July 22
John_MacFarlane:
… the gist of my comment is that it would be nice to separate out true CSL conformance tests (which test behavior specifically mandated by the spec) from tests that were just in there as citeproc-js regression tests. The latter tests may still be useful, but they should be separate.

We actually have tried to do this, I think in response to your earlier feedback.

Great! Looking it over, it looks better than I remember. I still see some things which, as far as I can see, don’t test anything mentioned in the spec. Here are some examples of failures from citeproc-hs. In each case I may have simply missed a relative part of the spec.

Does the spec specify these behaviors for quotes?

[FAILED]   test/csl/flipflop_OrphanQuote.txt
--- expected
+++ actual
@@
-Nation of "Positive Obligations " of State under the European Convention on Human Rights (1)
+Nation of “Positive Obligations ” of State Under the European Convention on Human Rights (1)

[FAILED]   test/csl/quotes_QuotesUnderQuotesFalse.txt
--- expected
+++ actual
@@
 <div class="csl-bib-body">
-  <div class="csl-entry"> 'Title with ‘quotes’ in it',.</div>
+  <div class="csl-entry"> ’Title with ‘quotes’ in it’,.</div>
 </div>

Does the spec say to do this? (As noted in my last mail, it’s not always desirable.)

[FAILED]   test/csl/magic_CapitalizeFirstOccurringTerm.txt
--- expected
+++ actual
@@
-And
+and

Does the spec specify these things?

[FAILED]   test/csl/sort_OmittedBibRefMixedNumericStyle.txt
--- expected
+++ actual
@@
   <div class="csl-entry">1. Anderson, Book One</div>
-  <div class="csl-entry">2. [CSL STYLE ERROR: reference with no printed form.]</div>
+  <div class="csl-entry">[CSL STYLE ERROR: reference with no printed form.]</div>
   <div class="csl-entry">3. Crane, Book Two</div>

[FAILED]   test/csl/sort_OmittedBibRefNonNumericStyle.txt
--- expected
+++ actual
@@
   <div class="csl-entry">Anderson, Book One</div>
+  <div class="csl-entry">[CSL STYLE ERROR: reference with no printed form.]</div>
   <div class="csl-entry">Crane, Book Two</div>

Here I wonder if the absence of space between the Latin and Chinese names is really intended?

[FAILED]   test/csl/name_EtAlWithCombined.txt
--- expected
+++ actual
@@
   <div class="csl-entry">John Doe。</div>
-  <div class="csl-entry">——著，Ziggy Zither等點校。</div>
+  <div class="csl-entry">——著，Ziggy Zither 等點校。</div>

[Edit: added later] In this case I don’t understand why the space before Frinkle (which is present in the reference database) disappears:

[FAILED]   test/csl/sort_LeadingApostropheOnNameParticle.txt
--- expected
+++ actual
@@
   <div class="csl-entry">d’Wander, W</div>
-  <div class="csl-entry">de’ Frinkle, B</div>
+  <div class="csl-entry">de’Frinkle, B</div>
   <div class="csl-entry">in ’t Horvath, P A B</div>

Also, while on this topic, any thoughts on changing these to use YAML, like this?

I guess I’m not sold on the advantages. The existing format is very straightforward to parse, and no special quoting or escaping is needed in the different sections. YAML has some complex quoting and escaping rules, which sometimes trip people up. In addition, you’d have to indent CSL when pasting it into the test. None of this is a big deal, though; if there are substantial advantages to switching to YAML, I wouldn’t object.

John

[Sebastian_Karcher]

@John_MacFarlane

When CSL is updated, it would be good to publish an “archived” style repository for the older version, to support people using processors that only support the older version.

We did this for 1.0.1–> 1.0.2 (the 1.0.1 branch of the repository) and would do the same moving forward.

@John_MacFarlane

I’ve often wished for a tool that can “prune” CSL in a test case so that it’s only as big as it needs to be. (I’m often too lazy to do this, so my test cases are often big and contain many copies of the same CSL style.) It should be possible to add tooling to a CSL processor that keeps track of which macros have been used; this information could be input to a tool that removes unnecessary macros.

Ooh I like that

[Denis_Maier]

First of all, I’d really would like not to freeze CSL and see it evolving. But we could freeze CSL 1, and make a fresh start with CSL 2.0 while keeping CSL 1 around to keep the vast amount of available styles usable. Maybe we could still add the intext feature to a version of CSL 1 as this is more or less already there.

As I’ve said in the meeting I think the current release model entails a number of drawbacks. I’m still unsure if there are good and practical alternatives, but I wanted to post them before I forget.

Would regular (yearly?) releases be an option?

Instead of aiming for a complete release we could release whatever we have a certain time. E.g. once a year in August. Or maybe every other year in August. A comment period could then start in January before the next release.

CSL → meta-CSL ?

Right now, the CSL operates on a number of different levels. E.g., adding a new term is totally different from adding a new citation mode. Maybe we could make the distinction more explicit. CSL in terms of a DSL could specify that there are terms, variables of certain types (names, dates, etc.), but not what these things are. That could then be specified elsewhere. Also, processors could perhaps just pull in such a list from somewhere instead maintaining a list of variables themselves. Terms and variable only updates would then be much easier.

Add a beta channel for new features

Could we add a say beta channel to try out new features. Of course, these would have to be implemented somewhere by someone. We could then try out new stuff and codify afterwards if things work out.

Make the language more powerful on a lower level

Instead of adding complex features, we could give users more power to make certain things themselves, e.g. with additional testing features.

[Bruce_D_Arcus1]

@Denis_Maier

we could freeze CSL 1, and make a fresh start with CSL 2.0 while keeping CSL 1 around to keep the vast amount of available styles usable.

I really like this idea (and the others as well).

We could then take a longer time to get it right, and maybe figure out the technical challenges.

I also think it’s cleaner from a marketing and communication POV; and of course potentially also much cleaner from a technical POV.

Instead of adding complex features, we could give users more power to make certain things themselves, e.g. with additional testing features.

Do you have any idea of what that might look like practically?

[Bruce_D_Arcus1]

@Sebastian_Karcher

We did this for 1.0.1–> 1.0.2 (the 1.0.1 branch of the repository) and would do the same moving forward.

@John_MacFarlane - WDYM by “archived repository”? Do you mean that literally?

I worry a little about using branching as a strategy for this with such a large and monolithic repository, but I’m not sure if that worry is misplaced.

[Bruce_D_Arcus1]

Conversation has died down, but I think we might draw a few high-level conclusions from the discussion so far:

1. There may be some process things we can do in the short-run (like my suggestion of review teams), that would be easy, and make sense. But if we set up those teams, for example, folks will have to be willing to accept invites, and occasionally do reviews.
2. There are some technical questions (raised most notably by Denis and I here, but also the Zotero folks during the virtual chat) that will only really be answered with the help of processor developers; people actually coding CSL. So we might encourage them to think about some of the questions we’ve raised, and weigh in if and when they can. It may well be possible we can find a technical solution that reduces some of the other inertial barriers to moving CSL forward?
3. I think in any case we should rename 1.1 to 2.0, and prepare for a cleaner break, a bit farther out. I hope we’re talking a year or so, rather than multiple years.

[Denis_Maier]

@Bruce_D_Arcus1

Instead of adding complex features, we could give users more power to make certain things themselves, e.g. with additional testing features.

Do you have any idea of what that might look like practically?

Not really. Citavi, for example, allows users to use arbitrary C# code in their style to code their own tests. We could make something similar, but IDK if that will work given the variety of implementations.

[Bruce_D_Arcus1]

That’s the rub; the whole point of CSL is that it’s language-independent.

[Denis_Maier]

Yes, sure. I was contemplating whether using a language such as Lua might be an option here… But that as well might work well for some implementations (pandoc’s citeproc, citeproc-rs, citeproc-lua), but I don’t know about the implementations in ruby, elisp and php.
And, even if it was technically feasible, it would still open a can of worms as it goes against the whole “xml can be validated” kind of thing.
Anyway, that’s probably a discussion for a different thread, if at all …