# csl editor testing?

**URL:** <https://discourse.citationstyles.org/t/csl-editor-testing/874>\
**Category:** CSL Development\
**Created:** [July 30, 2010, 8:51pm UTC](https://discourse.citationstyles.org/t/csl-editor-testing/874 "2010-07-30T20:51:16Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![Bruce\_D\_Arcus1](https://avatars.discourse-cdn.com/v4/letter/b/4bbf92/32.png) [@Bruce\_D\_Arcus1](https://discourse.citationstyles.org/u/Bruce_D_Arcus1)\
**Post date:** [July 30, 2010, 8:51pm UTC](https://discourse.citationstyles.org/t/csl-editor-testing/874/1 "2010-07-30T20:51:16Z")

</div>

Ian,

You have any thoughts about when you want to consider the CSL Editor  
ready for testing, and how you want to run the tests?

Bruce

---

<div class="post-metadata">

**Author:** ![Bruce\_D\_Arcus1](https://avatars.discourse-cdn.com/v4/letter/b/4bbf92/32.png) [@Bruce\_D\_Arcus1](https://discourse.citationstyles.org/u/Bruce_D_Arcus1)\
**Post date:** [July 30, 2010, 9:03pm UTC](https://discourse.citationstyles.org/t/csl-editor-testing/874/2 "2010-07-30T21:03:34Z")

</div>

Related, what kind of feedback (if any) do you want from those of us  
on this list, when?

For example, I could quickly say that:

1. We need to make sure that the “author” formatting block includes  
support for the “substitute” element (since it’s critical for sorting  
in many styles), but this may be a known issue (?).

2. I don’t like the “info” popup, but it may be temporary (?).

Bruce

---

<div class="post-metadata">

**Author:** ![Ian\_Mulvany](https://avatars.discourse-cdn.com/v4/letter/i/a9adbd/32.png) [@Ian\_Mulvany](https://discourse.citationstyles.org/u/Ian_Mulvany)\
**Post date:** [July 30, 2010, 10:31pm UTC](https://discourse.citationstyles.org/t/csl-editor-testing/874/3 "2010-07-30T22:31:17Z")

</div>

I think it’s ready for testing now. I’m hoping to have it ready not for our next client release at the end of august, but for the following one towards the end of September.

I’ve not thought about testing scenarios. I think it’s going to be impossible to prevent a growth in instances of CSL variants, however if we can set up a canonical repository of styles and encourage instances of the editor to point there it should help.

With that in mind any suggestions you have for testing are welcome.

---

<div class="post-metadata">

**Author:** ![Bruce\_D\_Arcus1](https://avatars.discourse-cdn.com/v4/letter/b/4bbf92/32.png) [@Bruce\_D\_Arcus1](https://discourse.citationstyles.org/u/Bruce_D_Arcus1)\
**Post date:** [July 30, 2010, 11:40pm UTC](https://discourse.citationstyles.org/t/csl-editor-testing/874/4 "2010-07-30T23:40:08Z")

</div>

> I think it’s ready for testing now. I’m hoping to have it ready not for our next client release at the end of august, but for the following one towards the end of September.
> 
> I’ve not thought about testing scenarios. I think it’s going to be impossible to prevent a growth in instances of CSL variants, however if we can set up a canonical repository of styles and encourage instances of the editor to point there it should help.

This is one of the reasons I really wanted to specify some  
requirements language and use cases upfront. If we don’t require that  
the editor:

1. make it easy for users to quickly find existing styles to use,  
and/or to base new styles on those, and …

2. make it possible to comment on and easily edit existing styles …

… then we’re guaranteed not only to have an explosion of different  
style variants (think 100 different versions of APA), but to force  
users to create new styles when they see a problem.

That explosion of styles would in turn make it much more difficult to  
find existing styles that work for them.

So I strongly suggest we take things slowly and get it right, and that  
we try to avoid requiring an unnecessary proliferation of style  
variants.

I also suggest as part of this that we clearly lay out: what do we  
expect to see at the end of this hosted at [citationstyles.org](http://citationstyles.org)? When a  
user goes to that site, what should they see? If they know they need  
“X” style, how will they find, or know that it’s not there and they  
need to create it? How will they know what styles are really close to  
the style they need?

> With that in mind any suggestions you have for testing are welcome.

My main thought was just to make sure that we have:

1. a diversity of testers, based on both expertise (beginner vs.  
experts) and field (sciences vs. humanities vs. social sciences; each  
of which have different priorities in citation styling, and so will  
stress different parts of the design).

2. a reasonably manageable pool of testers (maybe 20-30 at first?) so  
we get good comments

3. some guidelines and user scenarios that we want people to test  
(creating a new style from scratch according to a style guide, for  
example; basing one on an existing style; editing an existing style to  
correct a mistake)

Am thinking the demo/testing instance ought to have support for user  
registration (since it’s required for any kind of sane repository  
system, and can be used to keep a handle on testing), and that we ask  
for volunteers that we register from both the Mendeley and Zotero  
communities: both the non-technical users that have been chomping at  
the bit to have a GUI to easily create their styles with, and  
experienced style authors who know the ins-and-outs of both citation  
styling in general, and CSL in particular.

The only thing I wasn’t sure about is how to solicit feedback once we  
get things started. Mailing list? Emails? Forum?

Bruce

---

<div class="post-metadata">

**Author:** ![Bruce\_D\_Arcus1](https://avatars.discourse-cdn.com/v4/letter/b/4bbf92/32.png) [@Bruce\_D\_Arcus1](https://discourse.citationstyles.org/u/Bruce_D_Arcus1)\
**Post date:** [July 30, 2010, 11:44pm UTC](https://discourse.citationstyles.org/t/csl-editor-testing/874/5 "2010-07-30T23:44:27Z")

</div>

Here’s an example of the kind of thing the style editor should make  
drop-dead easy:

[http://forums.zotero.org/discussion/11028/style-request-journal-of-the-american-water-resources-association/](http://forums.zotero.org/discussion/11028/style-request-journal-of-the-american-water-resources-association/)

E.g. user know they need a style, they can quickly see “my style is  
almost exactly like style Y, except for A, B, C details” and then  
quickly create that style.

If we do that well, people will love it!

Bruce

---

<div class="post-metadata">

**Author:** ![Frank\_Bennett](https://yyz1.discourse-cdn.com/flex029/user_avatar/discourse.citationstyles.org/frank_bennett/32/198_2.png) [@Frank\_Bennett](https://discourse.citationstyles.org/u/Frank_Bennett)\
**Post date:** [July 31, 2010, 2:56am UTC](https://discourse.citationstyles.org/t/csl-editor-testing/874/6 "2010-07-31T02:56:13Z")

</div>

> > I think it’s ready for testing now. I’m hoping to have it ready not for our next client release at the end of august, but for the following one towards the end of September.
> > 
> > I’ve not thought about testing scenarios. I think it’s going to be impossible to prevent a growth in instances of CSL variants, however if we can set up a canonical repository of styles and encourage instances of the editor to point there it should help.
> 
> This is one of the reasons I really wanted to specify some  
> requirements language and use cases upfront. If we don’t require that  
> the editor:
> 
> 1. make it easy for users to quickly find existing styles to use,  
> and/or to base new styles on those, and …

We’re already suffering on this; even with the rollover examples, it’s  
pretty cumbersome for a user coming to the repository cold to find a  
style that is close to what they want.

One thing we might do is derive some metadata for use in categorizing  
or searching of the site content. For citation and bibliography,  
citeproc-js can be tweaked to report out the sequence of fields  
rendered against sample items (say, a simple article and a simple  
book, for a start), and maybe some other, limited details that a user  
is likely to find useful as a search term (quotes or styling on the  
title? name forms used? sort keys?).

Metadata could be dumped out to an auto-generated file to accompany  
each style in the repository. With a little tick-box interface and a  
visual example next to each, it might help people to zero in on a  
limited set of styles that are reasonably close to what they are  
after.

---

<div class="post-metadata">

**Author:** ![Bruce\_D\_Arcus1](https://avatars.discourse-cdn.com/v4/letter/b/4bbf92/32.png) [@Bruce\_D\_Arcus1](https://discourse.citationstyles.org/u/Bruce_D_Arcus1)\
**Post date:** [July 31, 2010, 2:39pm UTC](https://discourse.citationstyles.org/t/csl-editor-testing/874/7 "2010-07-31T14:39:04Z")

</div>

> > > I think it’s ready for testing now. I’m hoping to have it ready not for our next client release at the end of august, but for the following one towards the end of September.
> > > 
> > > I’ve not thought about testing scenarios. I think it’s going to be impossible to prevent a growth in instances of CSL variants, however if we can set up a canonical repository of styles and encourage instances of the editor to point there it should help.
> > 
> > This is one of the reasons I really wanted to specify some  
> > requirements language and use cases upfront. If we don’t require that  
> > the editor:
> > 
> > 1. make it easy for users to quickly find existing styles to use,  
> > and/or to base new styles on those, and …
> 
> We’re already suffering on this; even with the rollover examples, it’s  
> pretty cumbersome for a user coming to the repository cold to find a  
> style that is close to what they want.
> 
> One thing we might do is derive some metadata for use in categorizing  
> or searching of the site content. For citation and bibliography,  
> citeproc-js can be tweaked to report out the sequence of fields  
> rendered against sample items (say, a simple article and a simple  
> book, for a start), and maybe some other, limited details that a user  
> is likely to find useful as a search term (quotes or styling on the  
> title? name forms used? sort keys?).

This would be helpful, but the really valuable low-hanging fruit is  
the style metadata: in particular the field(s), but also categories  
like “author-date”, “footnote” and such.

> Metadata could be dumped out to an auto-generated file to accompany  
> each style in the repository. With a little tick-box interface and a  
> visual example next to each, it might help people to zero in on a  
> limited set of styles that are reasonably close to what they are  
> after.

Yes. Which I guess underlines a point I was making that we really need  
some explicit user scenarios/stories to test the interface against.  
This one is basically the same as the zotero forum link I posted: user  
needs a style; how do they find it, or something close?

Bruce

---

<div class="post-metadata">

**Author:** ![Carles\_Pina](https://avatars.discourse-cdn.com/v4/letter/c/dbc845/32.png) [@Carles\_Pina](https://discourse.citationstyles.org/u/Carles_Pina)\
**Post date:** [August 1, 2010, 6:22pm UTC](https://discourse.citationstyles.org/t/csl-editor-testing/874/8 "2010-08-01T18:22:37Z")

</div>

Hello,

> > I think it’s ready for testing now. I’m hoping to have it ready not for our next client release at the end of august, but for the following one towards the end of September.
> > 
> > I’ve not thought about testing scenarios. I think it’s going to be impossible to prevent a growth in instances of CSL variants, however if we can set up a canonical repository of styles and encourage instances of the editor to point there it should help.
> 
> This is one of the reasons I really wanted to specify some  
> requirements language and use cases upfront. If we don’t require that  
> the editor:
> 
> 1. make it easy for users to quickly find existing styles to use,  
> and/or to base new styles on those, and …

I’ll say a couple of ideas. Will not fix the problem, but maybe we  
like some of that ideas. If you don’t like any of that ideas it’s not  
a problem 🙂 . Anyway, it’s Sunday afternoon.

Problem to be solved: we want the users to re-use or change the  
existing styles instead of creating new styles from the scratch. So we  
should help/force the user to search on the existing styles in the  
repository (IMHO, I think that the users will really search before  
start the major work of creating and testing a new style. But I can be  
wrong).

Naive idea:  
1.- Before allowing the user to use the style editor the user should  
write the name of the style / name of the journal / name that the new  
style is based on (if the user knows). The style editor will search in  
“fuzzy way” for all styles and dependent styles to see if something is  
similar.  
“fuzzy way”: not for an exact match, but substring maching, soundex,  
Levenshtein, etc. etc. whatever method could give good results.

More original idea (I think so…)  
2.- Do you know the game 20q? The standard one can be played in  
[http://www.20q.net/](http://www.20q.net/) press on “Think in American” and then “Classic  
20Q”.  
The idea is to ask 5 or 10 easy features about the new style (like  
which separator it use in the authors bibliography, if it contains the  
URL, DOI, which date format, etc. everything very visual, and multiple  
choice questions).

Having the above information we could select 5 or 10 styles that the  
user maybe could use, we could show examples about that styles. The  
user could use one of that styles or just edit one of the styles and  
adapt it.

About the questions to ask: in the 20Q each question is different up  
to the previous questions, to try to guess the thing that the user has  
thought (e.g. if it’s an animal it will ask if it can fly, if it’s a  
food it will ask something else). Maybe our case doesn’t need it to  
use this sophistication, or maybe it can use it. The problem is that  
it can be fuzzy (it’s exactly some style but with a different author  
separator, so we cannot eliminate styles too early) (  
[20Q - Wikipedia](http://en.wikipedia.org/wiki/20Q) ).

Last approach, derived from 2.-  
3.- We give the meta data of some document to the user. The user  
should format it as the style will look like and write back to the  
system. We can compare with our existing files to search for the ones  
that looks more similar, we show this ones to the user and the user  
can use or edit that ones.

I haven’t thought how feasible the approaches are…

---

<div class="post-metadata">

**Author:** ![Bruce\_D\_Arcus1](https://avatars.discourse-cdn.com/v4/letter/b/4bbf92/32.png) [@Bruce\_D\_Arcus1](https://discourse.citationstyles.org/u/Bruce_D_Arcus1)\
**Post date:** [August 1, 2010, 7:11pm UTC](https://discourse.citationstyles.org/t/csl-editor-testing/874/9 "2010-08-01T19:11:55Z")

</div>

> Hello,
> 
> > > I think it’s ready for testing now. I’m hoping to have it ready not for our next client release at the end of august, but for the following one towards the end of September.
> > > 
> > > I’ve not thought about testing scenarios. I think it’s going to be impossible to prevent a growth in instances of CSL variants, however if we can set up a canonical repository of styles and encourage instances of the editor to point there it should help.
> > 
> > This is one of the reasons I really wanted to specify some  
> > requirements language and use cases upfront. If we don’t require that  
> > the editor:
> > 
> > 1. make it easy for users to quickly find existing styles to use,  
> > and/or to base new styles on those, and …
> 
> I’ll say a couple of ideas. Will not fix the problem, but maybe we  
> like some of that ideas. If you don’t like any of that ideas it’s not  
> a problem 🙂 . Anyway, it’s Sunday afternoon.
> 
> Problem to be solved: we want the users to re-use or change the  
> existing styles instead of creating new styles from the scratch. So we  
> should help/force the user to search on the existing styles in the  
> repository (IMHO, I think that the users will really search before  
> start the major work of creating and testing a new style. But I can be  
> wrong).

I think it’s absolutely the case that users really don’t want to bother with:

a) style creation, unless the style doesn’t exist  
b) style editing, unless their style doesn’t work as expected

In other words, what “we” want here is fully consistent with what  
users want (and indeed, many of us are users too!). People have better  
things to do than hassle with citation styles.

Also, as I’ve mentioned before, if we do a good job creating this idea  
of styles and community editing, we may be able to start to drawing in  
participation from publishers. Imagine, for example, a case where  
Rintze takes responsibility as primary editor for the APA style, and  
then someone from APA says “how can we help.” We could in theory add  
them as an editor for the style.

> Naive idea:  
> 1.- Before allowing the user to use the style editor the user should  
> write the name of the style / name of the journal / name that the new  
> style is based on (if the user knows). The style editor will search in  
> “fuzzy way” for all styles and dependent styles to see if something is  
> similar.  
> “fuzzy way”: not for an exact match, but substring maching, soundex,  
> Levenshtein, etc. etc. whatever method could give good results.
> 
> More original idea (I think so…)  
> 2.- Do you know the game 20q? The standard one can be played in  
> [http://www.20q.net/](http://www.20q.net/) press on “Think in American” and then “Classic  
> 20Q”.  
> The idea is to ask 5 or 10 easy features about the new style (like  
> which separator it use in the authors bibliography, if it contains the  
> URL, DOI, which date format, etc. everything very visual, and multiple  
> choice questions).
> 
> Having the above information we could select 5 or 10 styles that the  
> user maybe could use, we could show examples about that styles. The  
> user could use one of that styles or just edit one of the styles and  
> adapt it.

Yes, this is exactly what I’ve been advocating.

It’s based loosely on the makebst commandline utility. So makebst  
already shows this approach is very helpful.

Moreover, if you look at the existing evidence in the 1000+ styles in  
the [zotero.org](http://zotero.org) repo, the vast majority are dependent styles, and what  
those styles are based on is really consistent depending on field (or  
broader area; social sciences vs. humanities, etc.). So if you know  
what field someone is from and the type of style they need  
(author-date, etc.), that severely constrains the pool of possible  
styles that would match what the user needs. Most history styles are  
some variant of Chicago, for example, most social science styles are  
variants of APA, etc.

So a big +1 on your brain-storming!

> About the questions to ask: in the 20Q each question is different up  
> to the previous questions, to try to guess the thing that the user has  
> thought (e.g. if it’s an animal it will ask if it can fly, if it’s a  
> food it will ask something else). Maybe our case doesn’t need it to  
> use this sophistication, or maybe it can use it. The problem is that  
> it can be fuzzy (it’s exactly some style but with a different author  
> separator, so we cannot eliminate styles too early) (  
> [20Q - Wikipedia](http://en.wikipedia.org/wiki/20Q) ).

We’d start with the basics:

- field(s)
- type

We’d want to look at makebst’s question, but you can do things like say:

- How should the bibliography be sorted? [examples]
- How should the publisher be rendered? [examples]
- etc.

> Last approach, derived from 2.-  
> 3.- We give the meta data of some document to the user. The user  
> should format it as the style will look like and write back to the  
> system. We can compare with our existing files to search for the ones  
> that looks more similar, we show this ones to the user and the user  
> can use or edit that ones.

Yeah, Dan Stillman suggested this idea a couple of time, and he seemed  
to think it would quite feasible. Am not exactly sure of the details  
of what he was thinking in terms of implementation though.

In any case, on a related note, we need to fill out the preview data  
to cover a wider range of sources.

Bruce

---

<div class="post-metadata">

**Author:** ![Bruce\_D\_Arcus1](https://avatars.discourse-cdn.com/v4/letter/b/4bbf92/32.png) [@Bruce\_D\_Arcus1](https://discourse.citationstyles.org/u/Bruce_D_Arcus1)\
**Post date:** [August 2, 2010, 12:02am UTC](https://discourse.citationstyles.org/t/csl-editor-testing/874/10 "2010-08-02T00:02:25Z")

</div>

…

> It’s based loosely on the makebst commandline utility. So makebst  
> already shows this approach is very helpful.

If any one is interested, I believe the best place to look for  
documentation on makebst is here:

[http://mirrors.med.harvard.edu/ctan/macros/latex/contrib/custom-bib/merlin.pdf](http://mirrors.med.harvard.edu/ctan/macros/latex/contrib/custom-bib/merlin.pdf)

The series of parameter options (on which the questions are based) starts on p9.

Worth keeping in mind that bibtex was designed by hard scientists, so  
this list may or may not be as comprehensive as it should be to work  
for humanities people in particular.

Bruce