Changeset 469

Show
Ignore:
Timestamp:
11/27/07 19:38:08 (1 year ago)
Author:
athomas
Message:

cly: More documentation updates.

Files:

Legend:

Unmodified
Added
Removed
Modified
Copied
Moved

  • cly/trunk/cly/builder.py

    r468 r469  
    7777    ``match_candidates=False``: boolean 
    7878        The candidates() method returns a list of words that match at the 
    79         current token which are used for completion, but can also be used 
    80         to constrain the allowed matches if match_candidates=True.  Useful 
    81         for situations where you have a general regex pattern (eg. a 
    82         pattern matching files) but a known set of matches at this point 
    83         (eg.  files in the current directory). 
     79        current token, which are then used for completion, but can also be 
     80        used to constrain the allowed matches if match_candidates=True. 
     81        Useful for situations where you have a general regex pattern (eg. a 
     82        pattern matching files) but a known set of matches at this point (eg. 
     83        files in the current directory). 
    8484 
    8585    ``traversals=1``: integer 

  • cly/trunk/doc/developers-guide.rst

    r455 r469  
    77-------- 
    88 
    9 CLY allows developers to quickly and easily add interactive shell environments 
    10 to their applications. 
    11  
    12 It offers a simple syntax and a powerful interactive shell based on readline, 
    13 all with automatic full tab completion and contextual help. Applications that 
    14 don't wish to (or can't) utilise the readline integration may still find the 
    15 CLY parser useful: it powers the readline shell and offers the same completion 
    16 and contextual help, through an API. 
     9CLY's general use-case can be broken down into several phases: 
     10 
     11  1. Defining the grammar. 
     12  2. Creating a parser based on that grammar. 
     13  3. Parsing input: 
     14 
     15    1. Create a context in which to store parse state. 
     16    2. Traverse grammar, tokenising and matching input stream. 
     17    3. Collect variables from input stream. 
     18 
     19  4. Execute actions, passing collected variables to callback. 
    1720 
    1821Constructing a Grammar 
    1922---------------------- 
    2023 
     24Matching Input 
     25~~~~~~~~~~~~~~ 
     26 
    2127The syntax of a shell is defined as a tree of ``Node`` objects. 
    2228 
    23 A node matches a token of user input. 
    24  
    25 .. code-block:: python 
    26  
    27    one=Node('One') 
    28  
    29  
    30 , but may additionally have other behaviour such as 
    31 storing the token for later use. The children of each node define the next set 
    32 of valid tokens. 
    33  
    34 The developer defines their syntax using a hierarchy of objects. Syntactically, 
    35 child nodes are attached to their parent by ''calling'' the parent with the 
    36 children as positional or keyword arguments. The actual key of each keyword 
    37 argument is interpreted as the command at that position in the grammar.  
    38 Additionally, unless overridden, it is treated as the name of that child node. 
    39 Each positional argument is treated as an `anonymous node` and thus should 
    40 usually be reserved for terminal nodes or nodes whose name, pattern and help 
    41 will be explicitly overridden.  
    42  
    43 Here is a simple example: 
     29A node matches a token of user input defined by the ``pattern`` 
     30attribute of the ``Node``, which defaults to the ``Node`` name. For 
     31example, the following grammar would match the token ``one``: 
    4432 
    4533.. code-block:: python 
    4634 
    4735  grammar = Grammar( 
    48       one=Node('1')( 
    49           one_one=Node('1.1')( 
    50             Action(one_one), 
    51           ), 
    52           one_two=Node('1.2')( 
    53             Action(one_two), 
    54           ), 
     36    one=Node('One') 
     37  ) 
     38 
     39Because nodes are hierarchical, child nodes are only considered for 
     40matching after the parent has matched and consumed a token. The 
     41following grammar would match the tokens ``parent child``: 
     42 
     43.. code-block:: python 
     44 
     45  grammar = Grammar( 
     46    parent=Node('Parent')( 
     47      child=Node('Child') 
     48    ) 
     49  ) 
     50 
     51As mentioned, the name of the node is used as the default pattern to 
     52match against input tokens. This can be overridden by passing 
     53``pattern=<regex>`` to the constructor. The following example will match 
     54one or more digits: 
     55 
     56.. code-block:: python 
     57 
     58  grammar = Grammar( 
     59    number=Node('Number', pattern=r'\d+'), 
     60  ) 
     61 
     62 
     63Grammar Branching 
     64~~~~~~~~~~~~~~~~~ 
     65 
     66It's a common requirement that grammar paths be executable multiple 
     67times, whether that be to enter a list of IP addresses, allow multiple 
     68commands at a branch, etc. In CLY this is achieved with the ``Alias`` 
     69node. An alias node, as its name suggests, aliases its target at the 
     70current location, effectively merging two branches of the grammar. 
     71 
     72Here's an example: 
     73 
     74.. code-block:: python 
     75 
     76    grammar = Grammar( 
     77      one=Node('One')( 
     78        Alias('/three'), 
     79        two=Node('Two')( 
     80        ) 
    5581      ), 
    56       two=Node('2')( 
    57         Action(two), 
    58       ), 
    59   ) 
    60  
    61 This CLY grammar is equivalent to the following EBNF-style grammar:: 
    62  
    63   TOP := ONE | TWO 
    64   ONE := 'one', (ONE_ONE | ONE_TWO) 
    65   TWO := 'two' 
    66   ONE_ONE := 'one_one' 
    67   ONE_TWO := 'one_two' 
    68  
    69 Or, in more readable terms, this set of commands:: 
    70  
    71   one one_one 
    72   one one_two 
    73   two 
     82      three=Node('Three'), 
     83    ) 
     84 
     85This will alias the node ``/three`` underneath ``/one``. That is, this 
     86grammar will match the input ``one three``. 
     87 
     88Nodes are referenced by their full path, though globs may be used to 
     89alias multiple nodes at once. 
     90 
     91By default, nodes may only be traversed once per parse run, visited 
     92nodes are stored in the context. This can be overridden by passing  
     93``traversals=<count>`` to node constructors individually or by using the 
     94``Group()`` node to apply this attribute to all descendants. This 
     95construct is discussed in detail later. 
     96 
     97Collecting Variables 
     98~~~~~~~~~~~~~~~~~~~~ 
     99 
     100Matching input is great, but if you want your grammar to be useful 
     101you're going to want it to do something with it. This is where the 
     102``Variable`` class comes in: it stores matching input tokens into the 
     103parse context for later use as arguments to execution callbacks. 
     104 
     105Here's an example of a variable matching a number: 
     106 
     107.. code-block:: python 
     108 
     109  grammar = Grammar( 
     110    number=Variable('Number', pattern=r'\d+'), 
     111  ) 
     112 
     113The matched input token is stored by the ``Variable.selected()`` method, 
     114into the ``vars`` dictionary of the context. 
     115 
     116When parsing terminates on an action the context variables are passed as 
     117keyword arguments to the final callback. Given the input ``1234`` the 
     118above grammar would execute the callback with 
     119``callback(number='1234')`` 
     120 
     121It's as simple as that. 
     122 
     123Type Conversion 
     124~~~~~~~~~~~~~~~ 
     125 
     126By default, variables are passed to callbacks as strings. This is 
     127nice, but CLY allows end users to customise the value passed to the 
     128callback by subclassing and overriding ``Node.parse()``, which by 
     129default simply passes all text that matched the pattern. 
     130 
     131Continuing the example from the previous section: 
     132 
     133.. code-block:: python 
     134 
     135  class Number(Node): 
     136 
     137    pattern = r'\d+' 
     138 
     139    def parse(self, context, match): 
     140      return int(match.group()) 
     141 
     142Now our callback will be executed with ``callback(number=1234)``. Of 
     143course, much more complex conversions can occur, including IP address 
     144parsing, E-Mail parsing, etc. A set of commonly used variable types is 
     145included in ``cly.builder``. 
     146 
     147Note how the ``pattern`` attribute may be a class member as a convenience.  
     148 
     149Executing Commands 
     150~~~~~~~~~~~~~~~~~~ 
     151 
     152Now that our variables are collected into the parse context. 
     153 
     154 
     155Tab-Completion 
     156~~~~~~~~~~~~~~ 
     157 
    74158 
    75159Help! 
    …… 
    83167construct help, either from a single pair: 
    84168 
    85   .. code-block:: python 
    86  
    87     one=Node(Help.pair('one', 'Command 1')) 
     169.. code-block:: python 
     170 
     171  one=Node(Help.pair('one', 'Command 1')) 
    88172 
    89173or from a list of tuples: 
    90174 
    91   .. code-block:: python 
    92  
    93     help = [('one', 'Command 1'), 
    94             ('1', 'Command 1')] 
    95  
    96     one=Node(Help(help), pattern=r'one|1') 
     175.. code-block:: python 
     176 
     177  help = [('one', 'Command 1'), 
     178          ('1', 'Command 1')] 
     179 
     180  one=Node(Help(help), pattern=r'one|1') 
    97181 
    98182Types of Nodes 
    …… 
    139223be traversed in a parse context. 
    140224 
    141 These attributes can be set in three ways: 
     225These attributes can be set in four ways: 
    142226 
    143227As keyword arguments passed to the node constructor: 
    144228 
    145     .. code-block:: python 
    146  
    147         Node('Help', pattern=r'.+') 
     229.. code-block:: python 
     230 
     231  Node('Help', pattern=r'.+') 
    148232 
    149233As keyword arguments passed to the node when it is "called": 
    150234 
    151     .. code-block:: python 
    152  
    153         Node('Help')(pattern=r'.+') 
     235.. code-block:: python 
     236 
     237  Node('Help')(pattern=r'.+') 
     238 
     239By the special ``Group`` node. This node will set attributes on *all* 
     240descendants: 
     241 
     242.. code-block:: python 
     243 
     244  Group(traversals=0)( 
     245    one=Node('One'), 
     246    two=Node('Two'), 
     247  ) 
     248 
    154249 
    155250By subclassing the node and defining the attribute as a class attribute: 
    156251 
    157     .. code-block:: python 
    158  
    159         class Any(Node): 
    160           pattern = r'.+' 
     252.. code-block:: python 
     253 
     254  class Any(Node): 
     255    pattern = r'.+' 
    161256 
    162257For details on what attributes are available for each node class, refer to the