-
Notifications
You must be signed in to change notification settings - Fork 2
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
0 parents
commit 47fe4fc
Showing
26 changed files
with
2,694 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,26 @@ | ||
test: test-doc test-call-citeproc test-dont-call-citeproc | ||
|
||
test-doc: | ||
rm -f test/doc-is.html | ||
pandoc --lua-filter ./pandoc-zotxt.lua -F pandoc-citeproc \ | ||
-o test/doc-is.html test/doc.md | ||
cmp test/doc-is.html test/doc-should.html | ||
|
||
test-call-citeproc: | ||
rm -f test/call-citeproc-is.html | ||
pandoc --lua-filter ./pandoc-zotxt.lua \ | ||
-o test/call-citeproc-is.html test/call-citeproc.md | ||
cmp test/call-citeproc-is.html test/call-citeproc-should.html | ||
|
||
test-dont-call-citeproc: | ||
rm -f test/dont-call-citeproc-is.html | ||
pandoc --lua-filter ./pandoc-zotxt.lua \ | ||
-o test/dont-call-citeproc-is.html test/dont-call-citeproc.md | ||
cmp test/dont-call-citeproc-is.html test/dont-call-citeproc-should.html | ||
|
||
performance-comparison: | ||
time pandoc -F pandoc-zotxt -o /dev/null test/long.md | ||
time pandoc --lua-filter ./pandoc-zotxt.lua -o /dev/null test/long.md | ||
|
||
.PHONY: test test-doc test-call-citeproc test-dont-call-citeproc \ | ||
performance-comparison |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,135 @@ | ||
================ | ||
pandoc-zotxt.lua | ||
================ | ||
|
||
``pandoc-zotxt.lua`` looks up sources of citations in Zotero and adds | ||
their bibliographic data to the metadata of the document, where it | ||
can be read by ``pandoc-citeproc``. | ||
|
||
You need the zotxt_ plugin for Zotero. Citations should be inserted | ||
as so-called easy citekeys. See the documentation of *zotxt* for details. | ||
|
||
See the `manual page <man/pandoc-zotxt.lua.rst>`_ for more details. | ||
|
||
|
||
|
||
Installing ``pandoc-zotxt.lua`` | ||
=============================== | ||
|
||
You use ``pandoc-zotxt.lua`` **at your own risk**. You have been warned. | ||
|
||
You need Pandoc_ 2.0 or later. If you are using an older version of Pandoc, | ||
try `pandoc-zotxt <https://github.com/egh/zotxt>`_, | ||
which works with Pandoc 1.12 or later (but also requires Python_ 2.7). | ||
|
||
1. Download the `current release | ||
<https://codeload.github.com/odkr/pandoc-zotxt/tar.gz/v0.1>`_. | ||
2. Unpack it. | ||
3. Copy the whole directory to the ``filters`` | ||
subdirectory of your Pandoc data directory. | ||
|
||
Where your Pandoc data directory is located depends on your operating system. | ||
``pandoc --version`` will tell you. Consult the Pandoc manual for details. | ||
|
||
You may also want to copy the manual page to wherever your system stores manual | ||
pages; typically, this is ``/usr/local/share/man/``. | ||
|
||
If you are using a Unix-ish operating system, you can do all of the above by:: | ||
|
||
PANDOC_DATA_DIR=$(pandoc --version | | ||
sed -n 's/^Default user data directory: //p') | ||
mkdir -p "${PANDOC_DATA_DIR:?}/filters" | ||
cd "${PANDOC_DATA_DIR:?}/filters" | ||
curl https://codeload.github.com/odkr/pandoc-zotxt.lua/tar.gz/v0.1 | | ||
tar -xz | ||
sudo cp pandoc-zotxt.lua-0.1/man/pandoc-zotxt.lua.1 \ | ||
/usr/local/share/man/man1 | ||
|
||
|
||
``pandoc-zotxt.lua`` vs ``pandoc-zotxt`` | ||
======================================== | ||
|
||
I started to write ``pandoc-zotxt.lua`` because I had hoped that I could write | ||
a faster replacement for ``pandoc-zotxt``. Unfortunately, Pandoc_ does *not* | ||
support LuaSocket_ (a library for, among other things, retrieving data via a | ||
network) and only provides a blocking method to fetch data from networks | ||
itself. So, there is no way to retrieve data for multiple citation items | ||
concurrently. As a consequence, ``pandoc-zotxt.lua`` is about as fast as | ||
``pandoc-zotxt``. | ||
|
||
+------------------------------------+---------------------------------------+ | ||
| ``pandoc-zotxt.lua`` | ``pandoc-zotxt`` | | ||
+====================================+=======================================+ | ||
| Requires only Pandoc_ 2.0 | Requires Pandoc_ 1.12 and Python_ 2.7 | | ||
+------------------------------------+---------------------------------------+ | ||
| Apparently, a tiny bit faster | Apparently, a tiny bit faster | | ||
| for long and/or complex documents. | for short and/or simple documents. | | ||
| (But you won't notice.) | (But you won't notice.) | | ||
+------------------------------------+---------------------------------------+ | ||
|
||
|
||
Test suite | ||
========== | ||
|
||
For the test suite to work, you need Zotero_ and the sources that are cited | ||
in the test documents. You can import those sources from the file ``items.rdf`` | ||
in the directory ``test``. To run the test suite, just say:: | ||
|
||
make test | ||
|
||
|
||
Documentation | ||
============= | ||
|
||
See the `manual page <man/pandoc-zotxt.lua.rst>`_ | ||
and the source for details. | ||
|
||
|
||
Contact | ||
======= | ||
|
||
If there's something wrong with ``pandoc-zotxt.lua``, `open an issue | ||
<https://github.com/odkr/pandoc-zotxt.lua/issues>`_. | ||
|
||
|
||
License | ||
======= | ||
|
||
Copyright 2018 Odin Kroeger | ||
|
||
Permission is hereby granted, free of charge, to any person obtaining a copy | ||
of this software and associated documentation files (the "Software"), to deal | ||
in the Software without restriction, including without limitation the rights | ||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||
copies of the Software, and to permit persons to whom the Software is | ||
furnished to do so, subject to the following conditions: | ||
|
||
The above copyright notice and this permission notice shall be included in | ||
all copies or substantial portions of the Software. | ||
|
||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | ||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | ||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | ||
SOFTWARE. | ||
|
||
|
||
Further Information | ||
=================== | ||
|
||
GitHub: | ||
<https://github.com/odkr/pandoc-zotxt.lua> | ||
|
||
|
||
See also | ||
======== | ||
<https://github.com/egh/zotxt> | ||
|
||
|
||
.. _zotxt: https://github.com/egh/zotxt | ||
.. _Zotero: https://www.zotero.org/ | ||
.. _Pandoc: https://www.pandoc.org/ | ||
.. _Python: https://www.python.org/ | ||
.. _LuaSocket: https://github.com/diegonehab/luasocket |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,8 @@ | ||
# 1.2.0 | ||
|
||
* For ease of embedded use, the decoder and the SAX parser are made self-contained one file implementation. | ||
* `1` is decoded as an integer on Lua 5.3. | ||
* Number parsing routine is changed. Now `-?[0-9][-+.A-Za-z0-9]*` is detected as a number, and its conformance to JSON spec is checked. | ||
* Automated testing. | ||
* Cool logo :) | ||
* Bug fixes. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,21 @@ | ||
The MIT License (MIT) | ||
|
||
Copyright (c) 2015-2017 Shunsuke Shimizu (grafi) | ||
|
||
Permission is hereby granted, free of charge, to any person obtaining a copy | ||
of this software and associated documentation files (the "Software"), to deal | ||
in the Software without restriction, including without limitation the rights | ||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||
copies of the Software, and to permit persons to whom the Software is | ||
furnished to do so, subject to the following conditions: | ||
|
||
The above copyright notice and this permission notice shall be included in | ||
all copies or substantial portions of the Software. | ||
|
||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | ||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | ||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN | ||
THE SOFTWARE. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,84 @@ | ||
# ![Lunajson](logo/lunajson.png) | ||
[![CircleCI](https://circleci.com/gh/grafi-tt/lunajson.svg?style=shield)](https://circleci.com/gh/grafi-tt/lunajson) | ||
|
||
Lunajson features SAX-style JSON parser and simple JSON decoder/encoder. It is tested on Lua 5.1, Lua 5.2, Lua 5.3, and LuaJIT 2.0. | ||
It is written only in pure Lua and has no dependencies. Even so, decoding speed matches lpeg-based JSON implementations because it is carefully optimized. | ||
The parser and decoder reject input that is not conformant to the JSON specification (ECMA-404), and the encoder always yields conformant output. | ||
The parser and decoder also handle UTF/Unicode surrogate pairs correctly. | ||
|
||
## Install | ||
luarocks install lunajson | ||
|
||
Or you can download source manually and copy `src/*` into somewhere on your `package.path`. | ||
|
||
## Simple Usage | ||
local lunajson = require 'lunajson' | ||
local jsonstr = '{"Hello":["lunajson",1.5]}' | ||
local t = lunajson.decode(jsonstr) | ||
print(t.Hello[2]) -- prints 1.5 | ||
print(lunajson.encode(t)) -- prints {"Hello":["lunajson",1.5]} | ||
|
||
## API | ||
### lunajson.decode(jsonstr, [pos, [nullv, [arraylen]]]) | ||
Decode `jsonstr`. If `pos` is specified, it starts decoding from `pos` until the JSON definition ends, otherwise the entire input is parsed as JSON. `null` inside `jsonstr` will be decoded as the optional sentinel value `nullv` if specified, and discarded otherwise. If `arraylen` is true, the length of an array `ary` will be stored in `ary[0]`. This behavior is useful when empty arrays should not be confused with empty objects. | ||
|
||
This function returns the decoded value if `jsonstr` contains valid JSON, otherwise an error will be raised. If `pos` is specified it also returns the position immediately after the end of decoded JSON. | ||
|
||
### lunajson.encode(value, [nullv]) | ||
Encode `value` into a JSON string and return it. If `nullv` is specified, values equal to `nullv` will be encoded as `null`. | ||
|
||
This function encodes a table `t` as a JSON array if a value `t[1]` is present or a number `t[0]` is present. If `t[0]` is present, its value is considered as the length of the array. Then the array may contain `nil` and those will be encoded as `null`. Otherwise, this function scans non `nil` values starting from index 1, up to the first `nil` it finds. When the table `t` is not an array, it is an object and all of its keys must be strings. | ||
|
||
### lunajson.newparser(input, saxtbl) | ||
### lunajson.newfileparser(filename, saxtbl) | ||
Create and return a sax-style parser context, which parses `input` or a file specified by `filename`. `input` can be a string to be parsed, or a function that returns the next chunk of a data as a string to be parsed (or `nil` when all data is yielded). An example function for `input` follows (this sample is essentially same as the implementation of `newfileparser`). Note that `input` will never be called once it has returned `nil`. | ||
|
||
local fp = io.open("myfavorite.json") | ||
local function input() | ||
local s | ||
if fp then | ||
s = fp:read(8192) | ||
if not s then | ||
fp:close() | ||
fp = nil | ||
end | ||
end | ||
return s | ||
end | ||
|
||
`saxtbl` is a table of callbacks. It can have the following functions. Those functions will be called on corresponding events, if it is in the table. | ||
|
||
- startobject() | ||
- key(s) | ||
- endobject() | ||
- startarray() | ||
- endarray() | ||
- string(s) | ||
- number(n) | ||
- boolean(b) | ||
- null() | ||
|
||
A parser context maintains the current parse position, initially 1. | ||
|
||
#### parsercontext.run() | ||
Start parsing from current position. If valid JSON is parsed, the position moves to just after the end of this JSON. Otherwise it errors. | ||
|
||
#### parsercontext.tellpos() | ||
Return current position. | ||
|
||
#### parsercontext.tryc() | ||
Return the byte of current position as a number. If input is ended, it returns `nil`. It does not change current position. | ||
|
||
#### parsercontext.read(n) | ||
Return the `n`-length string starting from current position, and increase the index by `n`. If the input ends, the returned string and the updated position will be truncated. | ||
|
||
## Benchmark | ||
Following graphs are the results of the benchmark, decoding [`simple.json`](test/decodeparse/benchjson/simple.json) (about 750KiB) 100 times and encoding [`simple.lua`](test/encode/benchdata/simple.lua) (the decoded result of `simple.json`) 100 times. I conducted benchmarks of lunajson 1.0, [dkjson 2.5](http://dkolf.de/src/dkjson-lua.fsl/home) and [Lua CJSON 2.1.0](http://www.kyne.com.au/~mark/software/lua-cjson.php). Dkjson is a popular JSON encoding/decoding library in Lua, which is written in Lua and optionally uses [lpeg](http://www.inf.puc-rio.br/~roberto/lpeg/) to spped up decoding. Lua CJSON is a JSON encoding/decoding library implemented in C and is inherently fast. | ||
|
||
![The graph of decoding benchmark results](result/decode-simple.png) | ||
|
||
![The graph of encoding benchmark results](result/encode-simple.png) | ||
|
||
This benchmark is conducted in my desktop machine that equips Core i5 3550K and DDR3-1600 memory. Lua implementations and concerning modules are compiled by GCC 4.9.2 with `-O2 -march=ivybridge -mtune=ivybridge` options. The versions of lua implementations are the newest official releases at the time of benchmark. The version of lpeg is 0.12.2. | ||
|
||
In this benchmark Lunajson performs well considering that it is implemented only in standard Lua, especially in LuaJIT 2.0 benchmark. Lunajson also supplies incremental parsing in a SAX-style API, therfore you don't have to load whole large JSON files into memory in order to scan the information you're interested in from them. Lunajson is especially useful when non-standard libraries cannot be used easily or incremental parsing is favored. |
31 changes: 31 additions & 0 deletions
31
lib/luarocks/rocks-5.3/lunajson/1.2-0/lunajson-1.2-0.rockspec
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,31 @@ | ||
package = "lunajson" | ||
version = "1.2-0" | ||
source = { | ||
url = "git://github.com/grafi-tt/lunajson.git", | ||
tag = "1.2" | ||
} | ||
description = { | ||
summary = "A strict and fast JSON parser/decoder/encoder written in pure Lua", | ||
detailed = [[ | ||
Lunajson features SAX-style JSON parser and simple JSON decoder/encoder. It is tested on Lua 5.1, Lua 5.2, Lua 5.3, and LuaJIT. | ||
It is written only in pure Lua and has no dependencies. Even though, since it is carefully optimized, decoding speed even matches to other lpeg-based JSON modules. | ||
The parser and decoder reject inputs not conforms the JSON specification (ECMA-404), and the encoder always yields outputs conforming the specification. | ||
The parser and decoder also handle surrogate pair correctly. | ||
]], | ||
homepage = "https://github.com/grafi-tt/lunajson", | ||
maintainer = "Shunsuke Shimizu", | ||
license = "MIT/X11" | ||
} | ||
dependencies = { | ||
"lua >= 5.1" | ||
} | ||
build = { | ||
type = 'builtin', | ||
modules = { | ||
['lunajson'] = 'src/lunajson.lua', | ||
|
||
['lunajson.decoder'] = 'src/lunajson/decoder.lua', | ||
['lunajson.encoder'] = 'src/lunajson/encoder.lua', | ||
['lunajson.sax' ] = 'src/lunajson/sax.lua', | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,16 @@ | ||
rock_manifest = { | ||
doc = { | ||
["CHANGELOG.md"] = "b173b6220589bd7e512cb5ea1f009060", | ||
LICENSE = "2001d13200cac722635f81768f80463e", | ||
["README.md"] = "c6d5d3393e5d54b40abd764d1eb24dcb" | ||
}, | ||
lua = { | ||
lunajson = { | ||
["decoder.lua"] = "a02440404a53a1c6d7cd70c9362884f4", | ||
["encoder.lua"] = "197d6edcbc03cea7db3d918493c11616", | ||
["sax.lua"] = "bf58ae0c4a877ba4e49ee31b05a14fed" | ||
}, | ||
["lunajson.lua"] = "64bce3b8bcfa53c02e64fd924c1cf4b4" | ||
}, | ||
["lunajson-1.2-0.rockspec"] = "e16bb1378fa36d6a662964ea925efa48" | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,49 @@ | ||
commands = {} | ||
dependencies = { | ||
lunajson = { | ||
["1.2-0"] = { | ||
{ | ||
constraints = { | ||
{ | ||
op = ">=", | ||
version = { | ||
5, 1, string = "5.1" | ||
} | ||
} | ||
}, | ||
name = "lua" | ||
} | ||
} | ||
} | ||
} | ||
modules = { | ||
lunajson = { | ||
"lunajson/1.2-0" | ||
}, | ||
["lunajson.decoder"] = { | ||
"lunajson/1.2-0" | ||
}, | ||
["lunajson.encoder"] = { | ||
"lunajson/1.2-0" | ||
}, | ||
["lunajson.sax"] = { | ||
"lunajson/1.2-0" | ||
} | ||
} | ||
repository = { | ||
lunajson = { | ||
["1.2-0"] = { | ||
{ | ||
arch = "installed", | ||
commands = {}, | ||
dependencies = {}, | ||
modules = { | ||
lunajson = "lunajson.lua", | ||
["lunajson.decoder"] = "lunajson/decoder.lua", | ||
["lunajson.encoder"] = "lunajson/encoder.lua", | ||
["lunajson.sax"] = "lunajson/sax.lua" | ||
} | ||
} | ||
} | ||
} | ||
} |
Oops, something went wrong.