Ian,
You have any thoughts about when you want to consider the CSL Editor
ready for testing, and how you want to run the tests?
Bruce
Related, what kind of feedback (if any) do you want from those of us
on this list, when?
For example, I could quickly say that:
-
We need to make sure that the “author” formatting block includes
support for the “substitute” element (since it’s critical for sorting
in many styles), but this may be a known issue (?). -
I don’t like the “info” popup, but it may be temporary (?).
Bruce
I think it’s ready for testing now. I’m hoping to have it ready not for our next client release at the end of august, but for the following one towards the end of September.
I’ve not thought about testing scenarios. I think it’s going to be impossible to prevent a growth in instances of CSL variants, however if we can set up a canonical repository of styles and encourage instances of the editor to point there it should help.
With that in mind any suggestions you have for testing are welcome.
I think it’s ready for testing now. I’m hoping to have it ready not for our next client release at the end of august, but for the following one towards the end of September.
I’ve not thought about testing scenarios. I think it’s going to be impossible to prevent a growth in instances of CSL variants, however if we can set up a canonical repository of styles and encourage instances of the editor to point there it should help.
This is one of the reasons I really wanted to specify some
requirements language and use cases upfront. If we don’t require that
the editor:
-
make it easy for users to quickly find existing styles to use,
and/or to base new styles on those, and … -
make it possible to comment on and easily edit existing styles …
… then we’re guaranteed not only to have an explosion of different
style variants (think 100 different versions of APA), but to force
users to create new styles when they see a problem.
That explosion of styles would in turn make it much more difficult to
find existing styles that work for them.
So I strongly suggest we take things slowly and get it right, and that
we try to avoid requiring an unnecessary proliferation of style
variants.
I also suggest as part of this that we clearly lay out: what do we
expect to see at the end of this hosted at citationstyles.org? When a
user goes to that site, what should they see? If they know they need
“X” style, how will they find, or know that it’s not there and they
need to create it? How will they know what styles are really close to
the style they need?
With that in mind any suggestions you have for testing are welcome.
My main thought was just to make sure that we have:
-
a diversity of testers, based on both expertise (beginner vs.
experts) and field (sciences vs. humanities vs. social sciences; each
of which have different priorities in citation styling, and so will
stress different parts of the design). -
a reasonably manageable pool of testers (maybe 20-30 at first?) so
we get good comments -
some guidelines and user scenarios that we want people to test
(creating a new style from scratch according to a style guide, for
example; basing one on an existing style; editing an existing style to
correct a mistake)
Am thinking the demo/testing instance ought to have support for user
registration (since it’s required for any kind of sane repository
system, and can be used to keep a handle on testing), and that we ask
for volunteers that we register from both the Mendeley and Zotero
communities: both the non-technical users that have been chomping at
the bit to have a GUI to easily create their styles with, and
experienced style authors who know the ins-and-outs of both citation
styling in general, and CSL in particular.
The only thing I wasn’t sure about is how to solicit feedback once we
get things started. Mailing list? Emails? Forum?
Bruce
Here’s an example of the kind of thing the style editor should make
drop-dead easy:
E.g. user know they need a style, they can quickly see “my style is
almost exactly like style Y, except for A, B, C details” and then
quickly create that style.
If we do that well, people will love it!
Bruce
I think it’s ready for testing now. I’m hoping to have it ready not for our next client release at the end of august, but for the following one towards the end of September.
I’ve not thought about testing scenarios. I think it’s going to be impossible to prevent a growth in instances of CSL variants, however if we can set up a canonical repository of styles and encourage instances of the editor to point there it should help.
This is one of the reasons I really wanted to specify some
requirements language and use cases upfront. If we don’t require that
the editor:
- make it easy for users to quickly find existing styles to use,
and/or to base new styles on those, and …
We’re already suffering on this; even with the rollover examples, it’s
pretty cumbersome for a user coming to the repository cold to find a
style that is close to what they want.
One thing we might do is derive some metadata for use in categorizing
or searching of the site content. For citation and bibliography,
citeproc-js can be tweaked to report out the sequence of fields
rendered against sample items (say, a simple article and a simple
book, for a start), and maybe some other, limited details that a user
is likely to find useful as a search term (quotes or styling on the
title? name forms used? sort keys?).
Metadata could be dumped out to an auto-generated file to accompany
each style in the repository. With a little tick-box interface and a
visual example next to each, it might help people to zero in on a
limited set of styles that are reasonably close to what they are
after.
I think it’s ready for testing now. I’m hoping to have it ready not for our next client release at the end of august, but for the following one towards the end of September.
I’ve not thought about testing scenarios. I think it’s going to be impossible to prevent a growth in instances of CSL variants, however if we can set up a canonical repository of styles and encourage instances of the editor to point there it should help.
This is one of the reasons I really wanted to specify some
requirements language and use cases upfront. If we don’t require that
the editor:
- make it easy for users to quickly find existing styles to use,
and/or to base new styles on those, and …We’re already suffering on this; even with the rollover examples, it’s
pretty cumbersome for a user coming to the repository cold to find a
style that is close to what they want.One thing we might do is derive some metadata for use in categorizing
or searching of the site content. For citation and bibliography,
citeproc-js can be tweaked to report out the sequence of fields
rendered against sample items (say, a simple article and a simple
book, for a start), and maybe some other, limited details that a user
is likely to find useful as a search term (quotes or styling on the
title? name forms used? sort keys?).
This would be helpful, but the really valuable low-hanging fruit is
the style metadata: in particular the field(s), but also categories
like “author-date”, “footnote” and such.
Metadata could be dumped out to an auto-generated file to accompany
each style in the repository. With a little tick-box interface and a
visual example next to each, it might help people to zero in on a
limited set of styles that are reasonably close to what they are
after.
Yes. Which I guess underlines a point I was making that we really need
some explicit user scenarios/stories to test the interface against.
This one is basically the same as the zotero forum link I posted: user
needs a style; how do they find it, or something close?
Bruce
Hello,
I think it’s ready for testing now. I’m hoping to have it ready not for our next client release at the end of august, but for the following one towards the end of September.
I’ve not thought about testing scenarios. I think it’s going to be impossible to prevent a growth in instances of CSL variants, however if we can set up a canonical repository of styles and encourage instances of the editor to point there it should help.
This is one of the reasons I really wanted to specify some
requirements language and use cases upfront. If we don’t require that
the editor:
- make it easy for users to quickly find existing styles to use,
and/or to base new styles on those, and …
I’ll say a couple of ideas. Will not fix the problem, but maybe we
like some of that ideas. If you don’t like any of that ideas it’s not
a problem 
. Anyway, it’s Sunday afternoon.
Problem to be solved: we want the users to re-use or change the
existing styles instead of creating new styles from the scratch. So we
should help/force the user to search on the existing styles in the
repository (IMHO, I think that the users will really search before
start the major work of creating and testing a new style. But I can be
wrong).
Naive idea:
1.- Before allowing the user to use the style editor the user should
write the name of the style / name of the journal / name that the new
style is based on (if the user knows). The style editor will search in
“fuzzy way” for all styles and dependent styles to see if something is
similar.
“fuzzy way”: not for an exact match, but substring maching, soundex,
Levenshtein, etc. etc. whatever method could give good results.
More original idea (I think so…)
2.- Do you know the game 20q? The standard one can be played in
http://www.20q.net/ press on “Think in American” and then “Classic
20Q”.
The idea is to ask 5 or 10 easy features about the new style (like
which separator it use in the authors bibliography, if it contains the
URL, DOI, which date format, etc. everything very visual, and multiple
choice questions).
Having the above information we could select 5 or 10 styles that the
user maybe could use, we could show examples about that styles. The
user could use one of that styles or just edit one of the styles and
adapt it.
About the questions to ask: in the 20Q each question is different up
to the previous questions, to try to guess the thing that the user has
thought (e.g. if it’s an animal it will ask if it can fly, if it’s a
food it will ask something else). Maybe our case doesn’t need it to
use this sophistication, or maybe it can use it. The problem is that
it can be fuzzy (it’s exactly some style but with a different author
separator, so we cannot eliminate styles too early) (
20Q - Wikipedia ).
Last approach, derived from 2.-
3.- We give the meta data of some document to the user. The user
should format it as the style will look like and write back to the
system. We can compare with our existing files to search for the ones
that looks more similar, we show this ones to the user and the user
can use or edit that ones.
I haven’t thought how feasible the approaches are…
Hello,
I think it’s ready for testing now. I’m hoping to have it ready not for our next client release at the end of august, but for the following one towards the end of September.
I’ve not thought about testing scenarios. I think it’s going to be impossible to prevent a growth in instances of CSL variants, however if we can set up a canonical repository of styles and encourage instances of the editor to point there it should help.
This is one of the reasons I really wanted to specify some
requirements language and use cases upfront. If we don’t require that
the editor:
- make it easy for users to quickly find existing styles to use,
and/or to base new styles on those, and …I’ll say a couple of ideas. Will not fix the problem, but maybe we
like some of that ideas. If you don’t like any of that ideas it’s not
a problemProblem to be solved: we want the users to re-use or change the
existing styles instead of creating new styles from the scratch. So we
should help/force the user to search on the existing styles in the
repository (IMHO, I think that the users will really search before
start the major work of creating and testing a new style. But I can be
wrong).
I think it’s absolutely the case that users really don’t want to bother with:
a) style creation, unless the style doesn’t exist
b) style editing, unless their style doesn’t work as expected
In other words, what “we” want here is fully consistent with what
users want (and indeed, many of us are users too!). People have better
things to do than hassle with citation styles.
Also, as I’ve mentioned before, if we do a good job creating this idea
of styles and community editing, we may be able to start to drawing in
participation from publishers. Imagine, for example, a case where
Rintze takes responsibility as primary editor for the APA style, and
then someone from APA says “how can we help.” We could in theory add
them as an editor for the style.
Naive idea:
1.- Before allowing the user to use the style editor the user should
write the name of the style / name of the journal / name that the new
style is based on (if the user knows). The style editor will search in
“fuzzy way” for all styles and dependent styles to see if something is
similar.
“fuzzy way”: not for an exact match, but substring maching, soundex,
Levenshtein, etc. etc. whatever method could give good results.More original idea (I think so…)
2.- Do you know the game 20q? The standard one can be played in
http://www.20q.net/ press on “Think in American” and then “Classic
20Q”.
The idea is to ask 5 or 10 easy features about the new style (like
which separator it use in the authors bibliography, if it contains the
URL, DOI, which date format, etc. everything very visual, and multiple
choice questions).Having the above information we could select 5 or 10 styles that the
user maybe could use, we could show examples about that styles. The
user could use one of that styles or just edit one of the styles and
adapt it.
Yes, this is exactly what I’ve been advocating.
It’s based loosely on the makebst commandline utility. So makebst
already shows this approach is very helpful.
Moreover, if you look at the existing evidence in the 1000+ styles in
the zotero.org repo, the vast majority are dependent styles, and what
those styles are based on is really consistent depending on field (or
broader area; social sciences vs. humanities, etc.). So if you know
what field someone is from and the type of style they need
(author-date, etc.), that severely constrains the pool of possible
styles that would match what the user needs. Most history styles are
some variant of Chicago, for example, most social science styles are
variants of APA, etc.
So a big +1 on your brain-storming!
About the questions to ask: in the 20Q each question is different up
to the previous questions, to try to guess the thing that the user has
thought (e.g. if it’s an animal it will ask if it can fly, if it’s a
food it will ask something else). Maybe our case doesn’t need it to
use this sophistication, or maybe it can use it. The problem is that
it can be fuzzy (it’s exactly some style but with a different author
separator, so we cannot eliminate styles too early) (
20Q - Wikipedia ).
We’d start with the basics:
- field(s)
- type
We’d want to look at makebst’s question, but you can do things like say:
- How should the bibliography be sorted? [examples]
- How should the publisher be rendered? [examples]
- etc.
Last approach, derived from 2.-
3.- We give the meta data of some document to the user. The user
should format it as the style will look like and write back to the
system. We can compare with our existing files to search for the ones
that looks more similar, we show this ones to the user and the user
can use or edit that ones.
Yeah, Dan Stillman suggested this idea a couple of time, and he seemed
to think it would quite feasible. Am not exactly sure of the details
of what he was thinking in terms of implementation though.
In any case, on a related note, we need to fill out the preview data
to cover a wider range of sources.
Bruce
…
It’s based loosely on the makebst commandline utility. So makebst
already shows this approach is very helpful.
If any one is interested, I believe the best place to look for
documentation on makebst is here:
http://mirrors.med.harvard.edu/ctan/macros/latex/contrib/custom-bib/merlin.pdf
The series of parameter options (on which the questions are based) starts on p9.
Worth keeping in mind that bibtex was designed by hard scientists, so
this list may or may not be as comprehensive as it should be to work
for humanities people in particular.
Bruce