1. Java Parser Generator
2. Drivers License Generator Picture
3. C# Parser Generator

A list of Python parsing tools initially imported from @nedbat'sblog post.

Human-parser-generator - A straightforward recursive descent Parser Generator with a focus on 'human' code generation and ease of use.

The List

Standard Modules

The Python standard library includes a few modules for special-purpose parsing problems. These are not general-purpose parsers, but don't overlook them. If your need overlaps with their capabilities, they're perfect:

• shlex lexes command lines using the rules common to many operating system shells.
• ConfigParser implements a basic configuration file parser language which provides a structure similar to what you would find on Microsoft Windows INI files.
• ArgParse makes it easy to write user-friendly command-line interfaces. The program defines what arguments it requires, and argparse will figure out how to parse those out of sys.argv. The argparse module also automatically generates help and usage messages and issues errors when users give the program invalid arguments.
• email provides many services, including parsing email and other RFC-822 structures.parser parses Python source text.
• cmd implements a simple command interface, prompting for and parsing out command names, then dispatching to your handler methods.
• json is a JSON (JavaScript Object Notation) encoder and decoder
• tokenize is a lexical scanner for Python source code, implemented in Python.

Articles

• Simple Top-Down Parsing in Python - A methodology for writing top-down parsers in Python. (7/2008)
• Pysec: Monadic Combinatoric Parsing in Python - An exposition of using monads to build a Python parser. (2/2008)

Licensing and Attribution

Python Parsing Tools by Michael R. Bernstein is licensed under a Creative Commons Attribution 4.0 International License.
Based on a work at https://github.com/webmaven/python-parsing-tools/.

Parsing

The Parsing module implements an LR(1) parser generator, as well as theruntime support for using a generated parser, via the Lr and Glr parserdrivers. There is no special parser generator input file format, but theparser generator still needs to know what classes/methods correspond tovarious aspects of the parser. This information is specified viadocstrings, which the parser generator introspects in order to generate aparser. Only one parser specification can be embedded in each module, butit is possible to share modules between parser specifications so that, forexample, the same token definitions can be used by multiple parserspecifications.

The parsing tables are LR(1), but they are generated using a fast algorithmthat avoids creating duplicate states that result when using the genericLR(1) algorithm. Creation time and table size are on par with the LALR(1)algorithm. However, LALR(1) can create reduce/reduce conflicts that don'texist in a true LR(1) parser. For more information on the algorithm, see:

Parsing table generation requires non-trivial amounts of time for largegrammars. Internal pickling support makes it possible to cache the mostrecent version of the parsing table on disk, and use the table if thecurrent parser specification is still compatible with the one that was usedto generate the pickled parsing table. Since the compatibility checking isquite fast, even for large grammars, this removes the need to use thestandard code generation method that is used by most parser generators.

Parser specifications are encapsulated by the Spec class. Parser instancesuse Spec instances, but are themselves based on separate classes. Thisallows multiple parser instances to exist simultaneously, without requiringmultiple copies of the parsing tables. There are two separate parser driverclasses:

Java Parser Generator

Generalized LR driver, capable of tracking multiple parse treessimultaneously, if the %split precedence is used to mark ambiguousactions. This driver is closely based on Elkhound's design, whichis described in a technical report:

Parser generator directives are embedded in docstrings, and must begin witha '%' character, followed immediately by one of several keywords:

Precedence:

%fail%nonassoc%left%right%split

Token:

%token

Non-terminal:

%start%nonterm

Production:

%reduce

All of these directives are associated with classes except for %reduce.%reduce is associated with methods within non-terminal classes. The Parsingmodule provides base classes from which precedences, tokens, andnon-terminals must be derived. This is not as restrictive as it sounds,since there is nothing preventing, for example, a master Token class thatsubclasses Parsing.Token, which all of the actual token types then subclass.Also, nothing prevents using multiple inheritance.

Folowing are the base classes to be subclassed by parser specifications:

• Precedence
• Token
• Nonterm

Drivers License Generator Picture

The Parsing module implements the following exception classes:

C# Parser Generator

• SpecError - when there is a problem with the grammar specification
• ParsingException - any problem that occurs during parsing
• UnexpectedToken - when the input sequence contains a token that isnot allowed by the grammar (including end-of-input)

In order to maintain compatibility with legacy code, the Parsing moduledefines the following aliases. New code should use the exceptions abovethat do not shadow Python's builtin exceptions.

• Exception - superclass for all exceptions that can be raised
• SyntaxError - alias for UnexpectedToken

• AttributeError

Author: Jason Evans jasone@canonware.com

Github repo: http://github.com/sprymix/parsing