dictate

Background and rationale

Moin dictionaries (" key:: value" listings) are useful for storing values that can be accessed using the keys in various ways, for example, as variables (@key@), retrieved by GetVal macro (<<GetVal(key)>>) and in custom macros (such as my Ref.py macro).

However, there are limited tools for maintaining dictionaries, especially as the listing becomes long. It would be useful to be able to sort dictionaries for display and re-saving purposes. This would allow the inserting of new entries anywhere or appending multiple entries from another dictionary, and sort them afterwards. In dictionaries, duplication of keys is possible with only the first instance being be retrieved (perhaps giving rise to an unexpected value being return), so a simple way to detect duplication would also be helpful.

Additionally, the default rendering of dictionary entries is clear but might not suit all contexts, so a way to display entries in a table might be useful. One might think the csv parser could do this, but it won't accept a two-character delimiter ("::" in this case) as a argument (actually this causes a crash on Moin 1.9.2 Desktop Edition). If the single colon is given, a redundant empty column is generated. Also, the csv parser treats the first line as column headings, setting them in bold, which is not appropriate for dictionary listings. Matt Copper's keyval parser addresses this idea, but not specifically for dictionary " key:: value" listings.

In my case, I wanted to use dictionary for storing reference with name-year keys, so an easy method to sort and check for duplicates would be useful. References such as "Jones, Smith and Ford (2011)" and "Jones, Ford and Smith (2011)" would both have keys "Jones et al. 2010" which need to be distinguished as 2011a and 2011b. Hence, the value in sorting and duplication detection.

Parsers render the text for display but don't change the page (and rightly so). However, if a user needs to permanently sort the listings, this must currently be done manually or by an external program. So having a sorting parser that optionally displays the raw markup text would allow that text to be cut and pasted from the preview into the editor.

Description

A simple parser that reads key:value pairs from a dictionary listings, and optionally (1) sorts entries by keys or values, (2) highlights and counts duplicate keys and values, (3) counts entries and non-conforming lines, (4) formats the output as either normal for dictionary entries or as a simple two-column table, (5) displays the output in final or markup modes, and (5) includes a summary table at the head of the listing.

The parser works as both a processing instruction and page section. The parser name is followed by a list of keywords separated by spaces to select options required.

Keywords

case: perform case sensitive sort. By default, sort is case insensitive.
ascending: enrtries sorted on ascending value of first group of digits in the key, then alphabtically as requested by sort and reverse keywords. Sort must also be set.
descending: enrtries sorted on descending value of first group of digits in the key, then alphabtically as requested by sort and reverse keywords. Sort must also be set.
raw: display the output as raw markup, so it can be cut and pasted into the same or another page. By default, text will be formatted.
reverse: sort in descending order. By default, sorting is in ascending order.
sort: sort entries by keys for display only. By default, entries are unsorted.
summary: display a table at the head of the listing with counts of unique keys, duplicate keys, ignored lines (line that are not in " key:: value" format), and the sorting and layout options used in processing the text. By default, no summary is provided.
tabulate: tabulate the valid " key:: value" entries as a simple two-column table. By default, text will be formatted s normal dictionary entires (as is this list of keywords).
val: perform sort and duplicate detection on val. By default, key is used.
warn: highlight duplicate keys. This is done the insertion of the symbol $/!\$ before the key in the listing. In the case of the dictionary entry formatting the duplicate entries are formatted as normal lines. By default, no warning is given.

Keywords can appear in any order.

Usage

As processing instruction

#format dictate [ascending|descending ][case ][raw ][reverse ][sort ][summary ][tabluate ][val ][warn ]

 key:: value
 key:: value
 ...

As parser section

{{{#!dictate [ascending|descending ][case ][raw ][reverse ][sort ][summary ][tabluate ][val ][warn ]
 key:: value
 key:: value
 ...
}}}

Hint

The parser can be used transiently to sort and validate a listing, then turned off, say after unwanted duplication is corrected or the sorted raw text has been pasted over the original unsorted raw text. If included as a processing instruction, it can be turned off by simply changing the leading # to ##. Likewise, all option keywords can be included and switched off by for particular preview calls by adding a suffix (e.g. !raw). For example, including only the first line below will do a normal sort with warnings and summary only, and it can be turned off, allowing the default wiki parser to take effect, by changing it to the second line below.

#dictate sort !case !val !reverse warn summary !tabulate !raw

##dictate sort !case !val !reverse warn summary !tabulate !raw

Download & Release Notes

Download	Release Version	Moin Version	Release Notes
dictate-1.3.py	1.3	1.9.2

Examples

=== Dictate parser example 1 ===

{{{#!dictate sort warn

 Croll and Mathews 1977:: Croll NA, Mathews BE (1977) ''Biology of Nematodes''. (Wiley: NY, USA) 201 pp.

 Sijmons et al. 1994:: Sijmons PC, Atkinson HJ, Wyss U (1994) Parasitic strategies of root nematodes and associated host cell responses. ''Annual Review of Phytopathology'' '''32''', 235-259.

 McTest 2010:: Testing !WikiName escape handling

ignored line

 Croll and Mathews 1977:: Croll NA, Mathews BE (1977) ''Biology of Nematodes''. (Wiley: NY, USA) 201 pp.
}}}

=== Dictate parser example 2 ===

{{{#!dictate sort ascending warn table summary

 Croll and Mathews 1977:: Croll NA, Mathews BE (1977) ''Biology of Nematodes''. (Wiley: NY, USA) 201 pp.

 Sijmons et al. 1994:: Sijmons PC, Atkinson HJ, Wyss U (1994) Parasitic strategies of root nematodes and associated host cell responses. ''Annual Review of Phytopathology'' '''32''', 235-259.

 McTest 2010:: Testing !WikiName escape handling

ignored line

 Croll and Mathews 1977:: Croll NA, Mathews BE (1977) ''Biology of Nematodes''. (Wiley: NY, USA) 201 pp.
}}}

=== Dictate parser example 3 ===

{{{#!dictate sort raw 

 Croll and Mathews 1977:: Croll NA, Mathews BE (1977) ''Biology of Nematodes''. (Wiley: NY, USA) 201 pp.

 Sijmons et al. 1994:: Sijmons PC, Atkinson HJ, Wyss U (1994) Parasitic strategies of root nematodes and associated host cell responses. ''Annual Review of Phytopathology'' '''32''', 235-259.

 McTest 2010:: Testing !WikiName escape handling

ignored line

 Croll and Mathews 1977:: Croll NA, Mathews BE (1977) ''Biology of Nematodes''. (Wiley: NY, USA) 201 pp.
}}}

History

Developed with inspiration and code from:

Copyright

License

GNU GPL, see COPYING for details.

Known issues and limitations

Non-standard lines, such as " :: blah blah" or " <<blah>>:: blah blah", are assumed not to occur. It is assumed the user will not make such lines. The impact of such lines have not been tested. An entry is only checked for leading space and the occurrence of ":: ".
Sort case sensitive (Z comes before a). Case option needed and, also the option to sort on either key or val. Resolved with Version 1.2.

Hits: 351

dictate.py