tclsh expand.tcl [options] files...
Any valid Tcl command is a valid macro; this document will use the
term "macro" for Tcl commands intended to be used within an Expand
input file, and "command" for Tcl commands intended to be called from
other Tcl code. Before reading any input files, Expand reads
any exprules.tcl file in the current directory. This
file, called the "rules" file, can define any additional macros
needed by the document being processed.
Output can be written to standard output, directed to a file, or suppressed entirely.
Expand can be used to process any kind of input text, but was specifically designed with HTML preprocessing in mind. It is especially useful in maintaining large websites with a significant amount of boilerplate used by many pages. Changing the boilerplate is as easy as redefining an Expand macro. One could define a navigation bar macro, for example, with one argument: the identity of the current page. The macro would format the navigation bar appropriately, in context.
-help
-rules file
exprules.tcl, if present, will be used.
-out file
nul, no output will be written. If no file is
specified, output will be written to stdout. The output file
can be changed at any time during processing using the
setoutput command.
-errout mode
mode are
nothing, macro, error
or fail; fail is the default.
If mode is nothing, the erring macro
will produce no output. If mode is
macro, the macro and its arguments will be output
unchanged. If mode is error, the
macro and its arguments will be output followed by an error
message; this mode is especially useful for locating errors.
If mode is fail, the error message
will be written to standard error, and Expand will halt.
-web
Expand macros can do anything you like, but the typical use is to format and return text. The following simple example shows how to highlight text with asterisks:
proc highlight {text} {
return "*** $text ***"
}
If some other kind of highlighting is desired later on, this macro can be changed, and all documents that use it can be updated just by running them through Expand.
It's important that the macro return the formatted text, rather than printing it out. Expand replaces text in the input with the result of this macro, and doesn't print it out until the entire file is complete. If this macro wrote its formatted text to the output itself, it would be written out of order.
Sometimes macros are executed for their side effects, rather than to
format output. Such macros should always end with an explicit
return, so that they don't add erroneous output. The
following macro adds a word to global list; if the
return command weren't used, the entire list would be
written to the output:
proc saveword {word} {
global wordList
lappend wordList $word
return
}
User-defined macros are typically placed in a rules file, such as
exprules.tcl.
init_hook
init_hook,
rather than inline in the rules files. A common use for the
init_hook is parsing rules-specific options from
the command line using the getoptions command.
The result of init_hook is never added to the output.
begin_hook
end_hook
begin_file_hook fileName
end_file_hook fileName
raw_text_hook text
init_hook, verifies that no unknown
options remain, and assumes that any remaining command line arguments
are the names of input files. Thus, a rules file can define its own
command line options or in any other way modify the command line in
its init_hook. The easiest way to do this is to use the
getoptions command.
Suppose the rules will output summary information to a file if
the "-summary" option is specified; the information can be
"brief" or "verbose"; furthermore, the rules can be "-strict" in
their interpretation of particular macros. The following
init_hook will parse and remove the necessary options
from the command line.
proc init_hook {} {
global argv strictFlag summaryFile summaryMode
# Parse the options
getoptions argv {
{-strict strictFlag flag}
{-summary summaryFile string ""}
{-mode summaryMode enum brief verbose}
}
# Open the summary file
}
If $argv was "-summary out.text foo.in bar.in" before
the init_hook was called, it will be "foo.in bar.in"
afterward.
Presumably the macros in the rules file modify their behavior
based on the strictFlag, summaryFile,
and summaryMode variables.
setbrackets command
can change the bracket tokens to something more suitable:
"{" and "}", or "(*" and "*)", or even "%" and newline. The
only requirement is that neither token can be the empty string.
section command, it's easy to generate the numbers
automatically. Now consider producing a table of contents at the
beginning of the document, with links to the sections. It cannot
be done in one pass through the input; at least two passes are
needed. The first pass accumulates the table of contents information,
and the second actually formats the document, inserting the table of
contents in its proper place.
Expand supports this kind of processing via the
setpasses and exppass commands.
setpasses is called from the init_hook and sets the
total number of passes (the minimum, obviously, is 1). During
processing, exppass returns the number of the current pass;
the first pass is pass 1. When there is more than one pass, Expand's
algorithm is more complicated:
init_hook.
setoutput; otherwise, set
the output to nul.
begin_hook.
begin_file_hook
end_file_hook
end_hook
Note that by default, no output is produced until the final pass.
The macro used in the input files can modify their behavior based
on the value of exppass; a section macro, for
example, would add information to a global table-of-contents list on
pass 1 and return the formatted section header on pass 2.
Dr. Pangloss, however, thinks that this is the best of all
possible worlds.[footnote "See Candide, by Voltaire"]
The footnote macro would, presumably, assign a number to
this footnote and save the text to be formatted later on. However,
this solution is ugly if the footnote text is long or should contain
additional markup. Consider the following instead:
Dr. Pangloss, however, thinks that this is the best of all
possible worlds.[footnote]See [bookTitle "Candide"], by
[authorsName "Voltaire"], for more information.[/footnote]
Here the footnote text is contained between footnote and
/footnote macros, continues onto a second line, and
contains several macros of its own. This is both clearer and more
flexible; however, in Expand 1.3 there was no easy way to do it. The
footnote text would have been expanded into the output in place. In
Expand 2.0, however, the footnote macro pushes a new
context onto the context stack. Then, all expanded text gets placed
in that new context. /footnote retrieves it by popping
the context. Here's a skeleton implementation of these two macros:
proc footnote {} {
cpush footnote
}
proc /footnote {} {
set footnoteText [cpop footnote]
# Save the footnote text, and return an appropriate footnote
# number and link.
}
The cpush command pushes a new context onto the stack; the
argument is the context's name. It can be any string, but would
typically be the name of the macro itself. Then, cpop
verifies that the current context has the expected name, pops it off
of the stack, and returns the accumulated text.
Expand provides several other tools related to the context stack.
Suppose the first macro in a context pair takes arguments or computes
values which the second macro in the pair needs. After calling
cpush, the first macro can define one or more context
variables; the second macro can retrieve their values anytime before
calling cpop. For example, suppose the document must
specify the footnote number explicitly:
proc footnote {footnoteNumber} {
cpush footnote
csave num $footnoteNumber
# Return an appropriate link
}
proc /footnote {} {
set footnoteNumber [cget num]
set footnoteText [cpop footnote]
# Save the footnote text and its footnoteNumber for future
# output.
}
At times, it might be desirable to define macros that are valid only
within a particular context pair; such macros should verify that they
are only called within the correct context using either
cis or cname.
cget varname
cis name
cname
cpop name
name must match the previous cpush. See
THE CONTEXT STACK.
cpush name
cpop. See
THE CONTEXT STACK.
csave varname value
cvar varname
varname,
suitable for using in Tcl append and
lappend calls.
expandText text
expfile
exppass
setpasses has not been called.
expwrite text
end_hook, needs to write summary output to several
files. The files can be opened using setoutput, and
the information written using expwrite.
getoptions arglist ?-strict? deflist
"arglist" must be the name of a variable in the current scope which contains a list of arguments. It's typically "argv".
If "-strict" is specified, unknown options are flagged as errors.
The "deflist" is a list of option definitions. Each option definition has one of the following forms. In each form, NAME is the option name, which must begin with a "-" character, and VAR is the name of a variable in the caller's scope which will receive the option's value.
{NAME VAR flag}
{NAME VAR enum VAL1 VAL2...}
If the next argument begins with a hyphen, "-",
getoption assumes that the option's value
is missing.
{NAME VAR string DEFVALUE}
If the next argument begins with a hyphen, "-",
getoption assumes that the option's value
is missing.
If any errors are found, an error message is output and the program halts. Otherwise, the parsed options and their values are removed from the argument list variable.
For an example, see PARSING THE COMMAND LINE, above.
include fileName
begin_file_hook and end_file_hook
functions aren't called, and expfile returns the name
of the including file.
lb
popArg listvar
listvar, removing that element from the list. If
the list is empty, returns the empty string.
rb
readFile textFile
setbrackets lb rb
setErrorOutputMode mode
nothing,
macro, error or fail;
these modes are described under OPTIONS.
setoutput filename
"", the empty string, output will go to
stdout; if the filename is "nul", no
output will be produced. setoutput is called by
Expand just before each call to begin_hook.
setpasses numberOfPasses
setpasses should only be called in the init_hook;
Otherwise, the results are likely to be unpredictable.
textToID text
textToID is more usually used in command definitions
than directively in an input file.
::expand::webRuleSet
::expand::webRuleSet. In addition, the
"-web" command line option will load the rule set automatically.
dot
link url ?text?
[link foobar.html "A Document"]
[link frobozz.html]
expands to
<a href="foobar.html">A Document</a>
<a href="frobozz.html">frobozz.html</a>
mailto address ?name?
[mailto will@wjduquette.com]
[mailto will@wjduquette.com "Will Duquette"]
expands to
<a href="mailto:will@wjduquette.com">will@wjduquette.com</a>
<a href="mailto:will@wjduquette.com">Will Duquette</a>
tag name args
return "[tag a href foobar.html]foobar.html</a>"
is equivalent to
return "<a href=\"foobar.html\">foobar.html</a>"
today ?format?
cpush and
cpop commands.
Some text [aMacro [anotherMacro]] some more text
In some cases, nested macros would cause Expand 1.X to halt.
::expand::webRuleSet.
Expand was inspired by a cursory study of the M4 macro processor. M4 is an interesting tool, but its syntax and use can become downright weird. I judged it to be an imperfect tool for preprocessing HTML documents, but it gave me some ideas about how to use Tcl in a similar way. After that, well, it just grew. Any similarity to the philosophy of XML/XSL is purely coincidental, as I didn't read about XML until afterwards.
Should you have any questions, comments, or suggestions about Expand, feel free to contact Will at will@wjduquette.com.
Expand is available from the Expand Home Page.
Copyright © 2000, by William H. Duquette. All rights reserved.