# citeproc-py setup?

**URL:** <https://discourse.citationstyles.org/t/citeproc-py-setup/434>\
**Category:** CSL Development\
**Created:** [February 21, 2008, 9:28pm UTC](https://discourse.citationstyles.org/t/citeproc-py-setup/434 "2008-02-21T21:28:33Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![Bruce\_D\_Arcus1](https://avatars.discourse-cdn.com/v4/letter/b/4bbf92/32.png) [@Bruce\_D\_Arcus1](https://discourse.citationstyles.org/u/Bruce_D_Arcus1)\
**Post date:** [February 21, 2008, 9:28pm UTC](https://discourse.citationstyles.org/t/citeproc-py-setup/434/1 "2008-02-21T21:28:33Z")

</div>

Johan,

It may not be ready to play with yet, but how do you set up citeproc-py with a  
standard python install (not py2app); at least for local testing/playing?

Bruce

---

<div class="post-metadata">

**Author:** ![Johan\_Kool2](https://avatars.discourse-cdn.com/v4/letter/j/cab0a1/32.png) [@Johan\_Kool2](https://discourse.citationstyles.org/u/Johan_Kool2)\
**Post date:** [February 21, 2008, 10:19pm UTC](https://discourse.citationstyles.org/t/citeproc-py-setup/434/2 "2008-02-21T22:19:52Z")

</div>

cd citeproc-py/citeproc  
python [citeproc.py](http://citeproc.py) [options]

for details:  
python [citeproc.py](http://citeproc.py) -h

basically:  
python [citeproc.py](http://citeproc.py) --csl /path/to/cslfilein --output /path/to/cslfileout

I am trying to use your cslgallery, but it doesn’t go past  
OperationalError at /no such table: styles\_style even though I ran the  
syncdb command. (I’m using sqlite3.)

Johan

---

<div class="post-metadata">

**Author:** ![Bruce\_D\_Arcus1](https://avatars.discourse-cdn.com/v4/letter/b/4bbf92/32.png) [@Bruce\_D\_Arcus1](https://discourse.citationstyles.org/u/Bruce_D_Arcus1)\
**Post date:** [February 21, 2008, 10:38pm UTC](https://discourse.citationstyles.org/t/citeproc-py-setup/434/3 "2008-02-21T22:38:23Z")

</div>

> cd citeproc-py/citeproc  
> python citeproc.py [options]
> 
> for details:  
> python citeproc.py -h
> 
> basically:  
> python citeproc.py --csl /path/to/cslfilein --output /path/to/cslfileout

Doh! Thanks.

> I am trying to use your cslgallery, but it doesn’t go past  
> OperationalError at /no such table: styles\_style even though I ran the  
> syncdb command. (I’m using sqlite3.)

When you run syncdb, it should report (to stdout) the list of tables it builds.

Perhaps because I’m using the authentication and admin machinery,  
it expects a multi-user db lke postgres or mysql?

I’m using it with postgres, FWIW. These directions had me running  
postgres nicely in about 10 minutes:

[http://www.robbyonrails.com/articles/2008/01/22/installing-ruby-on-rails-and-postgresql-on-os-x-third-edition](http://www.robbyonrails.com/articles/2008/01/22/installing-ruby-on-rails-and-postgresql-on-os-x-third-edition)

In any case, if you have further problems, I’ll give you access to my  
installation, though  
I probably can’t get to it until tomorrow sometime, or Saturday.

Bruce

---

<div class="post-metadata">

**Author:** ![Bruce\_D\_Arcus1](https://avatars.discourse-cdn.com/v4/letter/b/4bbf92/32.png) [@Bruce\_D\_Arcus1](https://discourse.citationstyles.org/u/Bruce_D_Arcus1)\
**Post date:** [February 21, 2008, 10:50pm UTC](https://discourse.citationstyles.org/t/citeproc-py-setup/434/4 "2008-02-21T22:50:05Z")

</div>

BTW, this …

-w …, --citeulike-username your username at [CiteULike.org](http://CiteULike.org)

… is cool!

You might look to make it more generic over time, such that one can  
use different services. Perhaps have separate parameters for  
"service" and “username”?

Bruce

---

<div class="post-metadata">

**Author:** ![Johan\_Kool2](https://avatars.discourse-cdn.com/v4/letter/j/cab0a1/32.png) [@Johan\_Kool2](https://discourse.citationstyles.org/u/Johan_Kool2)\
**Post date:** [February 22, 2008, 1:31am UTC](https://discourse.citationstyles.org/t/citeproc-py-setup/434/5 "2008-02-22T01:31:14Z")

</div>

Hey Bruce,

I didn’t got to look at cslgallery anymore, maybe tomorrow evening.  
That particular styles\_style table was not reported during syncdb. I  
am not quite keen on installing mysql or postgresql on my machine, but  
I could use the mysql server on my website. I think I’ll just look  
into the sqlite3 option more. Is there any particular reason you  
recommended against it in your [setup.example.py](http://setup.example.py)?

The [CiteULike.org](http://CiteULike.org) thing was just a quick idea. I guess it should be  
possible, but I haven’t looked into it much, nor is that my priority.  
My own data is in bibtex, so that’ll be the first format I want to  
support. And it is a lot easier to parse than MODS. 🙂

Goodnight!

Johan—  
[http://www.johankool.nl/](http://www.johankool.nl/)

---

<div class="post-metadata">

**Author:** ![Bruce\_D\_Arcus1](https://avatars.discourse-cdn.com/v4/letter/b/4bbf92/32.png) [@Bruce\_D\_Arcus1](https://discourse.citationstyles.org/u/Bruce_D_Arcus1)\
**Post date:** [February 22, 2008, 2:11am UTC](https://discourse.citationstyles.org/t/citeproc-py-setup/434/6 "2008-02-22T02:11:01Z")

</div>

> I didn’t got to look at cslgallery anymore, maybe tomorrow evening.  
> That particular styles\_style table was not reported during syncdb.

Weird. You could always manually create the table.

> Is there any particular reason you  
> recommended against it in your setup.example.py?

Not particularly; just what I said earlier about the muilti-user stuff.

Let me know if you get it working, and I’ll send you the URL for my  
install when I have it running again.

Bruce

---

<div class="post-metadata">

**Author:** ![Liam\_Magee](https://avatars.discourse-cdn.com/v4/letter/l/7ab992/32.png) [@Liam\_Magee](https://discourse.citationstyles.org/u/Liam_Magee)\
**Post date:** [February 22, 2008, 1:29pm UTC](https://discourse.citationstyles.org/t/citeproc-py-setup/434/7 "2008-02-22T13:29:47Z")

</div>

I’m hoping to finally get back into the Ruby citeproc version in the  
next week or so (I had a stab at this with partial success last year).  
I’m glad to see a Python version underway, and if possible would like to  
collaborate to harmonise the approaches. I have no particularly fixed  
ideas about the implementation, but the approach with citeproc-py looks  
roughly commensurable. Do you mind if we work together on this?

Regards,

Liam.

Bruce D’Arcus wrote:

---

<div class="post-metadata">

**Author:** ![Johan\_Kool2](https://avatars.discourse-cdn.com/v4/letter/j/cab0a1/32.png) [@Johan\_Kool2](https://discourse.citationstyles.org/u/Johan_Kool2)\
**Post date:** [February 22, 2008, 2:57pm UTC](https://discourse.citationstyles.org/t/citeproc-py-setup/434/8 "2008-02-22T14:57:21Z")

</div>

Hello Liam,

I do not mind brainstorming together on how to approach and implement  
our parsers. I am a bit wary to have others working on CiteProc-Py at  
this moment, mostly because it is changing a lot these days. But I  
don’t think that you had the latter in mind anyway. You are of course  
free to read my code and implement it in a similar manner in Ruby. I  
would love to hear where you think my approach is wrong, or could be  
more efficient.

I took a quick glance at the Ruby code and realized how little Ruby I  
speak. One big thing that might be nice to synchronize is the options  
from the command line we take. Than we can share documentation, which  
is after all the most boring part to write.

Johan—  
[http://www.johankool.nl/](http://www.johankool.nl/)

---

<div class="post-metadata">

**Author:** ![Bruce\_D\_Arcus1](https://avatars.discourse-cdn.com/v4/letter/b/4bbf92/32.png) [@Bruce\_D\_Arcus1](https://discourse.citationstyles.org/u/Bruce_D_Arcus1)\
**Post date:** [February 22, 2008, 8:08pm UTC](https://discourse.citationstyles.org/t/citeproc-py-setup/434/9 "2008-02-22T20:08:41Z")

</div>

FWIW, while I’m no expert in either, I think Ruby and Python are more similar  
than they are different. The kinds of idioms that work well in one tend to work  
well in the other.

It might be worth considering coordinating on an API and tests? What  
are the basic  
classes and associated methods, for example. What parameters do they take?

Common documentation, of course, can flow from that.

I updated the test json file in the “data” directory, BTW. Ought to be good for  
testing of basic processing (abstracted from import parsing of data formats,  
for example)…

Bruce

---

<div class="post-metadata">

**Author:** ![Bruce\_D\_Arcus1](https://avatars.discourse-cdn.com/v4/letter/b/4bbf92/32.png) [@Bruce\_D\_Arcus1](https://discourse.citationstyles.org/u/Bruce_D_Arcus1)\
**Post date:** [February 22, 2008, 8:52pm UTC](https://discourse.citationstyles.org/t/citeproc-py-setup/434/10 "2008-02-22T20:52:28Z")

</div>

BTW, looking at the [citeproc.py](http://citeproc.py) file, here’s a pretty fundamental  
question that gets  
at these questions of API. This method:

168 def loadCSL(self, csl):  
169 # Create an instance of the Handler.  
170 handler = CSLDocumentHandler()  
171 # Create an instance of the parser.  
172 parser = make\_parser()  
173 # Set the content handler.  
174 parser.setContentHandler(handler)  
175 inFile = open(csl, ‘r’)  
176 # Start the parse.  
177 parser.parse(inFile) # [10]  
178 # Alternatively, we could directly pass in the file name.  
179 #parser.parse(inFileName)  
180 inFile.close()  
181 return handler.root

As I read this, the method take a CSL file name. I’ve been thinking  
for awhile that  
it should instead take a URI, and that mapping to a particular file  
(on the network,  
or cached locally) should probably be a separate method.

e.g. I’m thinking stuff like:

c = CitaProcessor(csl=“[http://zotero.org/styles/apa](http://zotero.org/styles/apa)”)

… and maybe something like:

c.changeStyle(“[http://ex.net/syles/xyz](http://ex.net/syles/xyz)”)

BTW, for testing in Python ,the doctest support might be good for  
this, since you can  
get the API usage automatically extracted into documentation along with it  
serving double-duty for testing.

Bruce

---

<div class="post-metadata">

**Author:** ![Bruce\_D\_Arcus1](https://avatars.discourse-cdn.com/v4/letter/b/4bbf92/32.png) [@Bruce\_D\_Arcus1](https://discourse.citationstyles.org/u/Bruce_D_Arcus1)\
**Post date:** [February 22, 2008, 8:53pm UTC](https://discourse.citationstyles.org/t/citeproc-py-setup/434/11 "2008-02-22T20:53:07Z")

</div>

Oops, that’d be:

c.change\_style(“[http://ex.net/styles/xzy](http://ex.net/styles/xzy)”)

Bruce

---

<div class="post-metadata">

**Author:** ![Bruce\_D\_Arcus1](https://avatars.discourse-cdn.com/v4/letter/b/4bbf92/32.png) [@Bruce\_D\_Arcus1](https://discourse.citationstyles.org/u/Bruce_D_Arcus1)\
**Post date:** [February 22, 2008, 10:18pm UTC](https://discourse.citationstyles.org/t/citeproc-py-setup/434/12 "2008-02-22T22:18:30Z")

</div>

Another comment Johan:

In looking through the directory and class organization, and comparing  
it to other Python code I’ve seen, wouldn’t it make more sense that an  
example like:

```
 CitationCSLObject.py

```

… would instead be:

```
 csl/citation.py

```

…?\*

E.g. “Object” is redundant, and create a csl directory.

That would result in python import statement like:

import citeproc.csl.citation

Or something like that.

Bruce

- Or even “csl/models/citation.py”.

---

<div class="post-metadata">

**Author:** ![Johan\_Kool2](https://avatars.discourse-cdn.com/v4/letter/j/cab0a1/32.png) [@Johan\_Kool2](https://discourse.citationstyles.org/u/Johan_Kool2)\
**Post date:** [February 22, 2008, 11:41pm UTC](https://discourse.citationstyles.org/t/citeproc-py-setup/434/13 "2008-02-22T23:41:45Z")

</div>

> As I read this, the method take a CSL file name. I’ve been thinking  
> for awhile that it should instead take a URI, and that mapping to a  
> particular file (on the network, or cached locally) should probably  
> be a separate method.

Consider that done.

It can now be either one of these things:

- a URL of a remote XML file
- a filename of a local XML file
- standard input (“-”)
- the actual XML document, as a string

This worked great:

% python citeproc/citeproc.py --csl [http://www.zotero.org/styles/asa](http://www.zotero.org/styles/asa)

> BTW, for testing in Python ,the doctest support might be good for  
> this, since you can get the API usage automatically extracted into  
> documentation along with it serving double-duty for testing.

Do you have a good website for me to read on this topic? I know  
docstrings and I know testing, but not how to combine those.

> In looking through the directory and class organization, and  
> comparing it to other Python code I’ve seen, wouldn’t it make more  
> sense that an example like: CitationCSLObject.py … would instead  
> be: csl/citation.py …?\*

I guess I could do a global find and replace at some point.  
CitationCSLObject is btw very different from Citation. The former is  
the object representing the citation-tag in a CSL file, the latter is  
a citation encountered in a text file. I’ll give it a thought though…

Johan—

> **[Johan Kool](https://johankool.nl/)**
>
> I write apps for iOS and macOS. I am based in Rotterdam, the Netherlands.

---

<div class="post-metadata">

**Author:** ![Bruce\_D\_Arcus1](https://avatars.discourse-cdn.com/v4/letter/b/4bbf92/32.png) [@Bruce\_D\_Arcus1](https://discourse.citationstyles.org/u/Bruce_D_Arcus1)\
**Post date:** [February 23, 2008, 12:31am UTC](https://discourse.citationstyles.org/t/citeproc-py-setup/434/14 "2008-02-23T00:31:37Z")

</div>

> > As I read this, the method take a CSL file name. I’ve been thinking  
> > for awhile that it should instead take a URI, and that mapping to a  
> > particular file (on the network, or cached locally) should probably  
> > be a separate method.
> 
> Consider that done.
> 
> It can now be either one of these things:
> 
> - a URL of a remote XML file
> - a filename of a local XML file
> - standard input (“-”)
> - the actual XML document, as a string
> 
> This worked great:
> 
> % python citeproc/citeproc.py --csl [http://www.zotero.org/styles/asa](http://www.zotero.org/styles/asa)

What happens if the file identified by that URI is stored/cached  
locally (say in a datbase)?

> > BTW, for testing in Python ,the doctest support might be good for  
> > this, since you can get the API usage automatically extracted into  
> > documentation along with it serving double-duty for testing.
> 
> Do you have a good website for me to read on this topic? I know  
> docstrings and I know testing, but not how to combine those.

I ran across it in the context of Django. See:

[http://docs.python.org/lib/module-doctest.html](http://docs.python.org/lib/module-doctest.html)  
[http://www.djangoproject.com/documentation/0.96/testing/](http://www.djangoproject.com/documentation/0.96/testing/)

> > In looking through the directory and class organization, and  
> > comparing it to other Python code I’ve seen, wouldn’t it make more  
> > sense that an example like: CitationCSLObject.py … would instead  
> > be: csl/citation.py …?\*
> 
> I guess I could do a global find and replace at some point.  
> CitationCSLObject is btw very different from Citation. The former is  
> the object representing the citation-tag in a CSL file, the latter is  
> a citation encountered in a text file. I’ll give it a thought though…

Ah, right. Maybe document.citation vs. csl.citation?

Bruce

---

<div class="post-metadata">

**Author:** ![Johan\_Kool2](https://avatars.discourse-cdn.com/v4/letter/j/cab0a1/32.png) [@Johan\_Kool2](https://discourse.citationstyles.org/u/Johan_Kool2)\
**Post date:** [February 23, 2008, 1:02am UTC](https://discourse.citationstyles.org/t/citeproc-py-setup/434/15 "2008-02-23T01:02:47Z")

</div>

> What happens if the file identified by that URI is stored/cached  
> locally (say in a datbase)?

Currently it gets fetched every time AFAIK. If I implement a cache, it  
would work just as with webpages. Just a matter of asking the  
webserver when the URI was last updated and act accordingly.

Johan—

> **[Johan Kool](https://johankool.nl/)**
>
> I write apps for iOS and macOS. I am based in Rotterdam, the Netherlands.

---

<div class="post-metadata">

**Author:** ![Liam\_Magee](https://avatars.discourse-cdn.com/v4/letter/l/7ab992/32.png) [@Liam\_Magee](https://discourse.citationstyles.org/u/Liam_Magee)\
**Post date:** [March 16, 2008, 5:07am UTC](https://discourse.citationstyles.org/t/citeproc-py-setup/434/16 "2008-03-16T05:07:40Z")

</div>

I’ve finally got back to looking at citeproc-rb. It is still very minimal,  
but I’ve committed the changes in any case, since it is a starting point.

I like the idea of a consistent API, test cases, command-line options and  
documentation - judging from a quick review of the Python code, it seems  
there is already some overlap in class names and design.

I couldn’t run the Python code locally incidentally:

$ python [citeproc.py](http://citeproc.py) --csl …/misc/asa.csl -m …/misc/Test.mods -d  
…/misc/docb  
ook-test.xml  
Traceback (most recent call last):  
File “[citeproc.py](http://citeproc.py)”, line 451, in   
main(sys.argv[1:])  
File “[citeproc.py](http://citeproc.py)”, line 446, in main  
k = CiteProcessor(csl, referenceSource, referenceType, documentSource,  
docum  
entType, output)  
File “[citeproc.py](http://citeproc.py)”, line 115, in **init**  
cslfiles = dircache.listdir(csl)  
File “/usr/lib/python2.5/dircache.py”, line 27, in listdir  
list = os.listdir(path)  
OSError: [Errno 20] Not a directory: ‘…/misc/asa.csl’

Regards,

Liam.

---

<div class="post-metadata">

**Author:** ![Bruce\_D\_Arcus1](https://avatars.discourse-cdn.com/v4/letter/b/4bbf92/32.png) [@Bruce\_D\_Arcus1](https://discourse.citationstyles.org/u/Bruce_D_Arcus1)\
**Post date:** [March 16, 2008, 7:35am UTC](https://discourse.citationstyles.org/t/citeproc-py-setup/434/17 "2008-03-16T07:35:52Z")

</div>

> I’ve finally got back to looking at citeproc-rb. It is still very minimal,  
> but I’ve committed the changes in any case, since it is a starting point.

Cool!

Note, your licensing terms are a little vague. I see this in one file …

# Licensed under the same terms as CiteProc.

… but when I go to the main citeproc.rb, I see the same thing. Mght  
it be good too use the BSD-like Ruby license?

> I like the idea of a consistent API, test cases, command-line options and  
> documentation ..

Yes. Nice to see you filling out the tests. The real challenge will be  
in coming up with good tests for the actual formatting (et al  
handling, sorting and related stuff, name-handling).

Bruce

---

<div class="post-metadata">

**Author:** ![Liam\_Magee](https://avatars.discourse-cdn.com/v4/letter/l/7ab992/32.png) [@Liam\_Magee](https://discourse.citationstyles.org/u/Liam_Magee)\
**Post date:** [March 16, 2008, 8:16am UTC](https://discourse.citationstyles.org/t/citeproc-py-setup/434/18 "2008-03-16T08:16:17Z")

</div>

I’m happy with whichever license - I meant really the same terms as  
citeproc-xsl. BSD is fine, or if there is a common license covering CSL, I’m  
happy with that too.

Unfortunately I’m somewhat hazy about what the actual tests should _produce_

- the formatting is not my strong point. However a common syntax for test  
citation data, and some way of specifying how the citations are formatted  
for testing, common to both Python and Ruby, would be ideal. YAML or XML are  
probably candidates for how this information is declared. I’m happy to work  
on this - but as I say I just don’t know enough about what, say, APA  
citations should look like in fine detail.

Regards,

Liam.

---

<div class="post-metadata">

**Author:** ![Johan\_Kool2](https://avatars.discourse-cdn.com/v4/letter/j/cab0a1/32.png) [@Johan\_Kool2](https://discourse.citationstyles.org/u/Johan_Kool2)\
**Post date:** [March 16, 2008, 4:12pm UTC](https://discourse.citationstyles.org/t/citeproc-py-setup/434/19 "2008-03-16T16:12:41Z")

</div>

Hello Lia,

You should now be able to run CiteProc-Py from the commandline. It was  
before expecting a folder with CSL style files so I could easily see  
the output for each of those.

You do not need to know exactly what APA citations look like to be  
able to write tests. I think it is better to test on smaller things,  
like if I set a prefix and suffix, and ask my code to add those, is  
that done correctly? Or, if I set the capitalization to words, does it  
produce the expected output. If you write a test for your whole code  
and expect it to output APA, you cannot know at that point wether it  
is an error in your codes logic or in the APA style file.

As I already have indicated to Bruce off-list. If the need arises, I  
am willing to reconsider other licenses for CiteProc-Py than the  
current GPLv3.

Regards,

Johan—  
[http://www.johankool.nl/](http://www.johankool.nl/)

---

<div class="post-metadata">

**Author:** ![Bruce\_D\_Arcus1](https://avatars.discourse-cdn.com/v4/letter/b/4bbf92/32.png) [@Bruce\_D\_Arcus1](https://discourse.citationstyles.org/u/Bruce_D_Arcus1)\
**Post date:** [March 16, 2008, 5:16pm UTC](https://discourse.citationstyles.org/t/citeproc-py-setup/434/20 "2008-03-16T17:16:40Z")

</div>

ooops; forgot to send this earlier (consistent with Johan’s suggestion) …

Liam Magee wrote:

> Unfortunately I’m somewhat hazy about what the actual tests should  
> _produce_ - the formatting is not my strong point. However a common  
> syntax for test citation data, and some way of specifying how the  
> citations are formatted for testing, common to both Python and Ruby,  
> would be ideal. YAML or XML are probably candidates for how this  
> information is declared.

This is part of why I started the sample data file. I’d imagine having  
tests like the following would be good:

# when given a list of names and CSL parameter does it output correct

# order, initialization, spacing

name\_formatting

# when given a date and CSL parameter does it output correct

# order, shortening, and spacing

date\_formatting

# test for correct “2000a”; insure proper index value among list of refs

author\_date\_suffx

# when given a long author list and CSL et-al-min and et-al-use

# parameters, yields correct boolean value

et\_al

# when given a list of citations, and item within it returns

# the correct boolean value

subsequent

RSpec is nice for this kind of thing, BTW, but feel free to turn this  
into standard tests.

There could probably be a lot more. but this ought to be a good start.

Bruce