\
**Category:** CSL Development\
**Created:** [June 17, 2009, 10:18pm UTC](https://discourse.citationstyles.org/t/quotes-localization-was-html-entities/667 "2009-06-17T22:18:12Z")\
**Posts on this page:** 12\
**Page:** 2
**Author:**  [@Frank\_Bennett](https://discourse.citationstyles.org/u/Frank_Bennett)\
**Post date:** [July 1, 2009, 11:45pm UTC](https://discourse.citationstyles.org/t/quotes-localization-was-html-entities/667/21 "2009-07-01T23:45:02Z")
> …
>
> > Actually, configuration of quotes is closely related to configuration
> > of inline markup.
>
> Also has some connection to dates localization, which we also need to deal with.
>
> > If a solution can be found that covers both, that
> > would be splendid.
> >
> > Just as a style needs to know what to do with balanced " characters,
> > it will need to know what decorations to apply with (say) XXX, or XXX. So
> > there needs to be a mapping from semantic tag names to formatting
> > attributes+value sets. The obvious place for those parameters to be
> > provided is the locale. How would you like to represent it?
>
> I don’t know. We need someone (say Zotero) to commit to this before we
> settle on how to do it. We also need to know how complex this mapping
> needs to be (again, assuming someone implements it).
I’m ready to work on implementation in the processor. The question is
over support in the Zotero UI?
---
**Author:**  [@Frank\_Bennett](https://discourse.citationstyles.org/u/Frank_Bennett)\
**Post date:** [July 1, 2009, 11:53pm UTC](https://discourse.citationstyles.org/t/quotes-localization-was-html-entities/667/22 "2009-07-01T23:53:35Z")
> > I’d be inclined to go with tag-based markup in Zotero, since I really don’t
> > think we want to pollute stored(/synced/displayed) data with arbitrary
> > inline markup, and we already have mechanisms in various places to deal with
> > HTML. Simon and I have discussed using a stripped-down version of TinyMCE
> > for title fields to handle this. My only concern would be performance, since
> > the TinyMCE-based rich-text notes in 2.0 have a bit of a display lag, and
> > that’d be more of an issue for title fields.
>
> Any chance a lighter-weight alternative would work?
>
> > A few French and German users in the Zotero forums have expressed the need
> > for citation-level control over quote style based on the original language
> > of the source, but that idea was mostly dismissed as crazy and prohibitively
> > difficult.
>
> I’m more wondering if users working in other languages are complaining
> about the quotes output in the bibliographies. Or, to turn it around,
> am wondering if there are emerging trends towards standardizing on
> English conventions.
Quotes and formatting decorations need to be placed under the control
of the locale before multi-lingual citations can be contemplated. For
example, in an English-language title that calls for italicization, a
parenthetical English translation of the title should be italicized,
but the Japanese characters should not be (italicization can be
applied by , but it is not a Japanese typesetting convention
and looks really weird on the page).
The first step to getting there is to set formatting and
quote-handling attributes in the locale of the style. At a later
stage, that could be extended to apply decorations as appropriate to
the language of the field, if things develop in that direction.
We’re not seeing these issues at the moment because this behavior is a
core requirement in multilingual environments, so Zotero doesn’t get
used there. Personally, I would like to encourage movement toward
multi-lingual support, of course, because there is a strong demand for
it among our students.
---
**Author:**  [@Frank\_Bennett](https://discourse.citationstyles.org/u/Frank_Bennett)\
**Post date:** [July 2, 2009, 12:01am UTC](https://discourse.citationstyles.org/t/quotes-localization-was-html-entities/667/23 "2009-07-02T00:01:14Z")
> A decision will need to be made at the UI level about how quotes will
> be represented in the data. Inline markup will need to recognize
> these characters and treat them as markup, so the processor needs to
> know what characters are in the set, so it can identify them.
> Possibilities are to require typewriter quotes everywhere (fragile,
> probably not a good idea), or recognize everything (better, but
> requires someone to identify what all the possible quotation marks in
> the world are – if I should use the wikipedia entry linked by Bruce,
> let me know).
>
> In data, this why I prefer using XML:
>
> Here's some quote
>
> Tag markup would be easier to implement in the processor, because you
> would not need to handle courier-style quotes, which are identical for
> open and close. But is Zotero going to implement markup for quotes at
> the database level in the short term? If the answer is no, then we
> need to handle quote characters in the processor if we want it
> deployed in Zotero.
>
> I’ll look forward to seeing how it turns out.
>
> There are a lot of things up-in-the-air here.
>
> Is there any indication of how Zotero intends to deal with this (is
> there a ticket, for example)?
>
> I’d be inclined to go with tag-based markup in Zotero, since I really don’t
> think we want to pollute stored(/synced/displayed) data with arbitrary
> inline markup, and we already have mechanisms in various places to deal with
> HTML. Simon and I have discussed using a stripped-down version of TinyMCE
> for title fields to handle this. My only concern would be performance, since
> the TinyMCE-based rich-text notes in 2.0 have a bit of a display lag, and
> that’d be more of an issue for title fields.
Sorry, Dan, we might be talking at cross purposes. The reference to
tag-based versus characters was specific to quotes – I think everyone
is agreed that wysiwyg is preferable to wiki-style markup. The
question is whether quotes should be represented as tags, rather than
as Unicode characters, with me thinking out loud that it would be
simpler to allow users to enter them normally via the keyboard.
---
**Author:**  [@Dan\_Stillman](https://discourse.citationstyles.org/u/Dan_Stillman)\
**Post date:** [July 2, 2009, 2:59am UTC](https://discourse.citationstyles.org/t/quotes-localization-was-html-entities/667/24 "2009-07-02T02:59:09Z")
But are you suggesting that a user would override the locale-specified
quote characters that appeared around a title in bibliographic output by
editing the data value in Zotero? If so, that doesn’t make much sense to
me, because then the data would essentially be locale-specific, and we’d
need to strip or interpret those characters anywhere we wanted to
process or display the values.
For quotes that appear around titles, I don’t see why we’d need anything
in the actual data values. For quotes that appear within a title, a
tag-based approach that separates structure from presentation makes more
sense to me.
Or perhaps I’m completely misunderstanding the issue…
---
**Author:**  [@Frank\_Bennett](https://discourse.citationstyles.org/u/Frank_Bennett)\
**Post date:** [July 2, 2009, 5:50am UTC](https://discourse.citationstyles.org/t/quotes-localization-was-html-entities/667/25 "2009-07-02T05:50:05Z")
> I’d be inclined to go with tag-based markup in Zotero, since I really don’t
> think we want to pollute stored(/synced/displayed) data with arbitrary
> inline markup, and we already have mechanisms in various places to deal with
> HTML. Simon and I have discussed using a stripped-down version of TinyMCE
> for title fields to handle this. My only concern would be performance, since
> the TinyMCE-based rich-text notes in 2.0 have a bit of a display lag, and
> that’d be more of an issue for title fields.
>
> Sorry, Dan, we might be talking at cross purposes. The reference to
> tag-based versus characters was specific to quotes – I think everyone
> is agreed that wysiwyg is preferable to wiki-style markup. The
> question is whether quotes should be represented as tags, rather than
> as Unicode characters, with me thinking out loud that it would be
> simpler to allow users to enter them normally via the keyboard.
>
> But are you suggesting that a user would override the locale-specified quote
> characters that appeared around a title in bibliographic output by editing
> the data value in Zotero? If so, that doesn’t make much sense to me, because
> then the data would essentially be locale-specific, and we’d need to strip
> or interpret those characters anywhere we wanted to process or display the
> values.
No, sorry if what I wrote gave that impression. The current
quotes=“true” markup in CSL for styles that need quotes is certainly
the right way to go for wrapping titles in quotes during bib
rendering. The next step will be to figure out how to set the
characters used for the quotes. For the outer quotes on a title (if
any) there seem to be two proposals on the table. One is to define
four locale terms, one for each of the opening and closing, single and
double quote characters. The other is to have an option in the
bibliography section that says something like , with permutations for the various
possible combinations of quote marks. Either way, the quote marks to
be used would be defined in the style, and applied by the style.
There would be no tampering with field content.
> For quotes that appear around titles, I don’t see why we’d need anything in
> the actual data values. For quotes that appear within a title, a tag-based
> approach that separates structure from presentation makes more sense to me.
At first blush this looked attractive to me also, but I think that it
turns out to be more complicated to implement common cross-lingual
cases via this route. At the boundary between French and English, you
might have an English review of a book written in French, which uses
French quotes in the title of the _review_:
J. Smith, “A Review of Cholley’s «Je suis un chat»”, Literary Studies
(2000), v. 10, n. 5
I could be wrong, but I think that would be the commonly expected
behavior in this case. The inner quotes aren’t hard-wired: they would
need to flip-flop to singles if the review were cited in a style that
applies French guillemets instead of English quotes to a title. But
in the example above, the guillemets are outside the scope of English
punctuation, so they are just passed through verbatim.
Implementing this in CSL is not difficult, and it wouldn’t require any
changes in Zotero to make it work; quote characters are just entered
as part of literal text of titles in which they appear, and display as
ordinary UTF-8. Any flip-flop mangling at bib render time would be up
to the CSL processor. To make flip-flop work where it’s needed, the
set of quote marks to be recognized in input for the locale of the
style, and the hierarchy of quotes to be used in output, would need to
be fixed, either as locale terms, or using some special element for
that purpose. (Separating input from output quote character sets
would solve any lurking problems with courier-style quote marks
slipping through into typeset output.)
If the inner quotes were tag markup, on the other hand, they would
need to carry a hint of the locale of the source in order to produce
the correct quotation glyphs. You would also need some blocking or
intercept mechanism to prevent the insertion of literal quote
characters via the keyboard … while somehow permitting users to
insert apostrophes … it seems like it could get pretty complicated
pretty quickly.
> Or perhaps I’m completely misunderstanding the issue…
I think we’re all on the same page, it’s just uncharted territory.
---
**Author:**  [@Dan\_Stillman](https://discourse.citationstyles.org/u/Dan_Stillman)\
**Post date:** [July 2, 2009, 6:30am UTC](https://discourse.citationstyles.org/t/quotes-localization-was-html-entities/667/26 "2009-07-02T06:30:01Z")
> > But are you suggesting that a user would override the locale-specified quote
> > characters that appeared around a title in bibliographic output by editing
> > the data value in Zotero? If so, that doesn’t make much sense to me, because
> > then the data would essentially be locale-specific, and we’d need to strip
> > or interpret those characters anywhere we wanted to process or display the
> > values.
>
> No, sorry if what I wrote gave that impression. The current
> quotes=“true” markup in CSL for styles that need quotes is certainly
> the right way to go for wrapping titles in quotes during bib
> rendering. The next step will be to figure out how to set the
> characters used for the quotes. For the outer quotes on a title (if
> any) there seem to be two proposals on the table. One is to define
> four locale terms, one for each of the opening and closing, single and
> double quote characters. The other is to have an option in the
> bibliography section that says something like, with permutations for the various
> possible combinations of quote marks. Either way, the quote marks to
> be used would be defined in the style, and applied by the style.
> There would be no tampering with field content.
OK, got it. Sorry for the confusion.
I can’t think of any particular reason (e.g., from the forums) not to do
this using locale terms.
> > For quotes that appear around titles, I don’t see why we’d need anything in
> > the actual data values. For quotes that appear within a title, a tag-based
> > approach that separates structure from presentation makes more sense to me.
>
> At first blush this looked attractive to me also, but I think that it
> turns out to be more complicated to implement common cross-lingual
> cases via this route. At the boundary between French and English, you
> might have an English review of a book written in French, which uses
> French quotes in the title of the _review_:
>
> J. Smith, “A Review of Cholley’s «Je suis un chat»”, Literary Studies
> (2000), v. 10, n. 5
>
> I could be wrong, but I think that would be the commonly expected
> behavior in this case. The inner quotes aren’t hard-wired: they would
> need to flip-flop to singles if the review were cited in a style that
> applies French guillemets instead of English quotes to a title. But
> in the example above, the guillemets are outside the scope of English
> punctuation, so they are just passed through verbatim.
>
> Implementing this in CSL is not difficult, and it wouldn’t require any
> changes in Zotero to make it work; quote characters are just entered
> as part of literal text of titles in which they appear, and display as
> ordinary UTF-8. Any flip-flop mangling at bib render time would be up
> to the CSL processor. To make flip-flop work where it’s needed, the
> set of quote marks to be recognized in input for the locale of the
> style, and the hierarchy of quotes to be used in output, would need to
> be fixed, either as locale terms, or using some special element for
> that purpose. (Separating input from output quote character sets
> would solve any lurking problems with courier-style quote marks
> slipping through into typeset output.)
That makes sense, and indeed sounds like the better option. I
particularly like the “wouldn’t require any changes in Zotero” part.
---
**Author:**  [@Bruce\_D\_Arcus1](https://discourse.citationstyles.org/u/Bruce_D_Arcus1)\
**Post date:** [July 2, 2009, 11:49am UTC](https://discourse.citationstyles.org/t/quotes-localization-was-html-entities/667/27 "2009-07-02T11:49:04Z")
Another question for the spec:
Do we assume default values for quotes, or are all locales required to
specify the glyphs? The only reason I could for the latter is if it’s
feasible a locale would want nil values (and so no quotes).
Bruce
---
**Author:**  [@Dan\_Stillman](https://discourse.citationstyles.org/u/Dan_Stillman)\
**Post date:** [July 2, 2009, 5:39pm UTC](https://discourse.citationstyles.org/t/quotes-localization-was-html-entities/667/28 "2009-07-02T17:39:23Z")
Are quotes different in this regard from any other locale terms? I would
think that, across the board, we should just distinguish between empty
strings and missing strings, with the latter falling back to English.
---
**Author:**  [@Bruce\_D\_Arcus1](https://discourse.citationstyles.org/u/Bruce_D_Arcus1)\
**Post date:** [July 2, 2009, 5:55pm UTC](https://discourse.citationstyles.org/t/quotes-localization-was-html-entities/667/29 "2009-07-02T17:55:48Z")
So you’re suggesting a “yes” answer to my question?
Bruce
---
**Author:**  [@Frank\_Bennett](https://discourse.citationstyles.org/u/Frank_Bennett)\
**Post date:** [July 3, 2009, 2:15am UTC](https://discourse.citationstyles.org/t/quotes-localization-was-html-entities/667/30 "2009-07-03T02:15:38Z")
> > Another question for the spec:
> >
> > Do we assume default values for quotes, or are all locales required to
> > specify the glyphs? The only reason I could for the latter is if it’s
> > feasible a locale would want nil values (and so no quotes).
>
> Are quotes different in this regard from any other locale terms? I would
> think that, across the board, we should just distinguish between empty
> strings and missing strings, with the latter falling back to English.
I’ve added a couple of tests, and can confirm (after fixing a small
bug that the test revealed) that the new processor behaves in this way
for all terms. So quotes can either be provided in all locales, or a
default can be provided for English, with the other locales providing
a definition only when it differs.
---
**Author:**  [@Frank\_Bennett](https://discourse.citationstyles.org/u/Frank_Bennett)\
**Post date:** [July 3, 2009, 9:34am UTC](https://discourse.citationstyles.org/t/quotes-localization-was-html-entities/667/31 "2009-07-03T09:34:24Z")
> > > Another question for the spec:
> > >
> > > Do we assume default values for quotes, or are all locales required to
> > > specify the glyphs? The only reason I could for the latter is if it’s
> > > feasible a locale would want nil values (and so no quotes).
> >
> > Are quotes different in this regard from any other locale terms? I would
> > think that, across the board, we should just distinguish between empty
> > strings and missing strings, with the latter falling back to English.
>
> So you’re suggesting a “yes” answer to my question?
Is the question whether quotes should be specified in the locale or,
alternatively be coded into the processor with a set of option values
() to modify the default
behavior on a per-style basis?
I would recommend the former, if that’s the question posed. If quotes
are specified as locale terms, it would make it simpler, at a later
stage, to adapt CSL to handle multi-lingual citations correctly (i.e.
cites with translated titles etc).
If the question is whether, when quote characters are specified in the
English locale, they also should be specified in all other locales,
the answer is that they only need to be defined when the default
should be overridden (including nil quote cases, since specifying an
empty term will override the default value and replace it with an
empty string.) It does no harm to define the quote characters
explicitly in every locale, though.
(Hoping that the above is responsive.)
Frank
---
**Author:**  [@Bruce\_D\_Arcus1](https://discourse.citationstyles.org/u/Bruce_D_Arcus1)\
**Post date:** [July 3, 2009, 1:10pm UTC](https://discourse.citationstyles.org/t/quotes-localization-was-html-entities/667/32 "2009-07-03T13:10:08Z")
> > > > Another question for the spec:
> > > >
> > > > Do we assume default values for quotes, or are all locales required to
> > > > specify the glyphs? The only reason I could for the latter is if it’s
> > > > feasible a locale would want nil values (and so no quotes).
> > >
> > > Are quotes different in this regard from any other locale terms? I would
> > > think that, across the board, we should just distinguish between empty
> > > strings and missing strings, with the latter falling back to English.
> >
> > So you’re suggesting a “yes” answer to my question?
>
> Is the question whether quotes should be specified in the locale or,
> alternatively be coded into the processor with a set of option values
> () to modify the default
> behavior on a per-style basis?
No; the question was the one I asked which Dan quoted:
“Do we assume default values for quotes, or are all locales required
to specify the glyphs?”
> I would recommend the former, if that’s the question posed. If quotes
> are specified as locale terms, it would make it simpler, at a later
> stage, to adapt CSL to handle multi-lingual citations correctly (i.e.
> cites with translated titles etc).
>
> If the question is whether, when quote characters are specified in the
> English locale, they also should be specified in all other locales,
> the answer is that they only need to be defined when the default
> should be overridden (including nil quote cases, since specifying an
> empty term will override the default value and replace it with an
> empty string.) It does no harm to define the quote characters
> explicitly in every locale, though.
Yes. My conclusion is the quotes only get specified in the locale if
they are different than what we use in English (e.g. what we’ve been
doing). E.g. my answer to my question is “yes.”
Bruce
[Previous page](https://discourse.citationstyles.org/t/quotes-localization-was-html-entities/667.md?page=1)