%==============================================================================% % Start of Ch1.tex % %==============================================================================% % % Copyright % --------- % Copyright (C) 1992 Ross N. Williams. % This file contains a chapter of the FunnelWeb User's Manual. % See the main TeX file for this manual for further information. % %==============================================================================% \chapter{A Tutorial Introduction} \label{chapintroduction} \x{tutorial} \section{What is Literate Programming?} \x{literate programming} A traditional computer program consists of a text file containing program code. Scattered in amongst the program code are comments which describe the various parts of the code. In \newterm{literate programming} the emphasis is reversed. Instead of writing code containing documentation, the literate programmer writes documentation containing code. No longer does the English commentary injected into a program have to be hidden in comment delimiters at the top of the file, or under procedure headings, or at the end of lines. Instead, it is wrenched into the daylight and made the main focus. The \dq{program} then becomes primarily a document directed at humans, with the code being herded between \dq{code delimiters} from where it can be extracted and shuffled out sideways to the language system by literate programming tools. The effect of this simple shift of emphasis can be so profound as to change one's whole approach to programming. Under the literate programming paradigm, the central activity of programming becomes that of conveying meaning to other intelligent beings rather than merely convincing the computer to behave in a particular way. It is the difference between performing and exposing a magic trick.\x{magic trick} In order to program in a literate style, particular tools are required.\xx{tools}{literate programming} The traditional approach (used in the FunnelWeb system) is to have some sort of text-file-in/text-file-out utility that reads a literate program (containing a program commentary peppered with scraps of program text) and writes out a file containing all the program code and a file containing typesetter commands representing the entire input document, documentation, code, and all (\figarchitecture{}). \begin{figure}[htbp] \begin{verbatim} +-----------------------------------------+ | File containing the program description | | peppered with scraps of program code. | | This is what the programmer works on. | | (e.g. sloth.web) | +-----------------------------------------+ | v o---------------------------o | Literate Programming Tool | o---------------------------o | +-------------------+-------------------+ | | v v +------------------+ +----------------------------------+ | Traditional | | Documentation file suitable for | | Computer Program | | input into a typesetting program | | (e.g. sloth.c) | | (e.g. sloth.tex) | +------------------+ +----------------------------------+ \end{verbatim} \mylabel{\figarchitecture{}: Traditional architecture of literate programming tools.}{% % Literate programming tools could be organized in a number of ways. However, to fit in with current file and command line based environments, most tools conform to the traditional architecture shown here in which the user feeds in a file containing a literate program, and the literate programming utility generates program files and a documentation file. % } \end{figure} Given the coming age of hypertext\x{hypertext} systems, this is probably not the best approach. However, it does mesh beautifully with current text files and command line interfaces, the expectation of linear presentations in the documents we read, and the particular requirements of current programming languages and typesetting systems. It is certainly not a bad approach. With this structure in place, the literate programming system can provide far more than just a reversal of the priority of comments and code. In its full blown form, a good literate programming facility can provide total support for the essential thrust of literate programming, which is that computer programs should be written more for the human reader than for the compiler. In particular, a literate programming system can provide:\xs{literate programming}{facilities} \narrowthing{Re-ordering of code:}{Programming languages often force the programmer to give the various parts of a computer program in a particular order.\xx{program}{ordering} For example, the Pascal\x{Pascal} programming language\paper{BSI82} imposes the ordering: constants, types, variables, procedures, code. Pascal also requires that procedures appear in an order consistent with the partial ordering imposed by the static call graph (but forward declarations allow this to be bypassed). In contrast, the literate style requires that the programmer be free to present the computer program in any order whatsoever. The facility to do this is implemented in literate programming tools by providing text \i{macros} that can be defined and used in any order.} \narrowthing{Typeset code and documentation:}{Traditionally program listings are dull affairs consisting of pages of fan-form paper imprinted with meandering coastlines of structured text in a boring font. In contrast, literate programming systems are capable of producing documentation that is superior in two ways. First, because most of the documentation text is fed straight to the typesetter, the programmer can make use of all the power of the underlying typesetter, resulting in documentation that has the same presentation as an ordinary typeset document. Second, because the literate programming utility sees all the code, it can use its knowledge of the programming language and the features of the typesetting language to typeset the program code as if it were appearing in a technical journal. It is the difference between:} \begin{verbatim} while sloth@{Hello World@+@} \end{verbatim} To \dq{run} this \dq{program}, invoke FunnelWeb using the \dqp{fw} command as follows.\x{invoking FunnelWeb} \begin{verbatim} fw hello \end{verbatim} \i{If this command doesn't work, then chances are that FunnelWeb has not been installed on your machine. Refer to Chapter~\ref{chapinstall} for full details on how to obtain and install a copy of FunnelWeb.} There should be no errors. If there are, have a look at the listing file \p{hello.lis}, which should contain an explanation of the error, and compare the area in the file where the error occurred with the text above. If there are no errors, you will find that the following two files have been created. \begin{verbatim} hello.lis - The LISTING file. hello.txt - The PRODUCT file. \end{verbatim} Take a look at \p{hello.txt}. It should contain a single line with the text \p{Hello World}. Let's take another look at the input file. \begin{verbatim} @O@@{Hello World@+@} \end{verbatim} The whole structure of the input file is controlled by \dqp{@}, called the \newterm{special character}, which introduces \newterm{special sequence}s. A scanner's-eye view of the command line looks like this: \begin{verbatim} @O @< "hello.txt" @> @{ "Hello World" @+ @} \end{verbatim} The \p{@} character controls everything. In this file we have six different special sequences that together form a single macro definition. The \p{@<}\x{@<} and \p{@>}\x{@>} delimit the name of the macro. The \p{@O}\x{@O} signals the start of the macro definition and indicates that the macro is to be connected to a product file with the same name as the macro (This is is why we got a product file when we ran FunnelWeb). The \p{@\{} and \p{@\}}\x{@braces} delimit the body of the macro. Finally, the \p{@+}\x{@+} instructs that an end of line sequence should be inserted at that point in the product file. If you think this syntax looks messy, then you're right. It \i{is} messy. FunnelWeb \i{could} have employed a \dq{simpler} notation in which more of the \p{@} sequences were eliminated. For example: \begin{verbatim} Warning: This example is NOT legal FunnelWeb. #hello.txt{Hello World+} \end{verbatim} However, if such a syntax were used, the user (you!) would have to remember that \p{\#} starts a new macro. You would also have to remember that the characters \p{\}} and \p{+} cannot be used in a macro body without a fuss. And so on. FunnelWeb is messier, but provides one simple rule:\xx{simple}{rule} \i{Nothing special happens unless the special character \p{@} appears.} This means that in FunnelWeb, you can look at large blocks of text in the confidence that (unlike for the C pre-processor\xx{C}{preprocessor}) there are no macro calls hidden in there. If there were, there would be an \p{@} character!\footnote{The only exception to this rule occurs where the user has explicitly changed the special character using the \p{@=}\x{@=} special sequence.} Let's take another look at the hello world program. \begin{verbatim} @O@@{Hello World@+@} \end{verbatim} In its current form, it consists of a single macro definition. This definition, while completely valid on its own, only represents half the power of FunnelWeb. In fact you could say that it is a \dq{Hello Northern Hemisphere Program}.\x{Hello Northern Hemisphere Program} To turn it into a proper FunnelWeb \dq{Hello World} program, we need to add some documentation! A FunnelWeb input file consists of a sequence of macro definitions surrounded by a sea of documentation which is just ordinary text. Modify your hello world document so that it looks like this: \begin{verbatim} This hello world document was created by -insert your name here-. @O@@{Hello World@+@} It writes out a file called hello.txt containing the string ``Hello World''. \end{verbatim} Now run it through FunnelWeb, but this time, add a \p{+t} to the command line. \begin{verbatim} fw hello +t \end{verbatim} If all goes well, you should find that you now have \begin{verbatim} hello.lis - A LISTING file. hello.tex - A DOCUMENTATION file (in TeX format). hello.txt - A PRODUCT file. \end{verbatim} Take a look at \p{hello.txt}. You will find that it is identical to the \p{hello.txt} of the previous run. Only macro definitions affect the product files that FunnelWeb produces (as a result of \p{@O} macro definitions). The surrounding documentation has \i{no} effect. In contrast, the new file, \p{hello.tex} (have a look at it now) which was created as a result of your adding the \p{+t} option contains a fairly full representation of the input file. Whereas \p{hello.txt} is the \i{product file} of FunnelWeb, \p{hello.tex} is the \i{documentation file}. Try typesetting the documentation file now using the \TeX{} typesetting program. Then print it. The following commands are an example of the sort of commands you will have to give to do this. \begin{verbatim} tex hello ! Typeset the documentation. lpr -Pcslw -d hello.dvi ! Print the typeset documentation. \end{verbatim} The result should be a single page containing the two lines of documentation along with a typeset representation of the macro. At this point, you have exercised the two main aspects of FunnelWeb.\xx{FunnelWeb}{two main aspects} Starting with an input file containing macros (or in this case macro) and documentation, you have successfully generated a product file based on the macros, and a documentation file, based on the entire document. Congratulations! The remainder of this tutorial is divided into two parts, which focus on FunnelWeb's macro facilities and its typesetting facilities. By tradition, the generation of program files from a literate text is called \newterm{Tangling}, and the generation of typeset documentation is called \newterm{Weaving}.\footnote{In FunnelWeb, these two functions are aspects of a single computer program. However, in Knuth's WEB\x{WEB} system, the two functions are embodied in two separate computer programs called Tangle and Weave, presumably because, as everyone knows, \dq{it takes two to Tangle}.} \section{Macro Facilities} \xx{tutorial}{macro facilities} The hello world program of the previous section exercised both the macro expansion (product-file) aspect of FunnelWeb, and the typesetting (documentation file) aspect of FunnelWeb. This section contains an exposition of the macro facilities, and totally ignores the documentation side. This is partly to increase the focus of the tutorial, and partly because documentation is usually bulky and would take too long for the reader to type in to make the tutorial effective. \subsection{Simple Macros} \xx{tutorial}{simple macros}\xx{tutorial}{macros simple} The original \dq{Hello World} program consisted of a single macro definition. \begin{verbatim} @O@@{Hello World@+@} \end{verbatim} In fact, this is a rather exceptional macro, as it causes its expansion to be written to a product file. The \p{@O}\x{@O} (for \b{O}utput) signals this. In FunnelWeb, most macros are defined using \p{@\$}.\x{@dollar} This results in a macro that does not generate a product file, but which can be called in other macros (including \p{@O} macros). Let us expand the hello world program to include some other macros. \begin{verbatim} @O@@{@@+@} @$@==@{Hello@} @$@==@{World@} @$@==@{@ @@} \end{verbatim} Type in the file and run it through FunnelWeb using the command: \begin{verbatim} fw hello \end{verbatim} The product file (result.out) should look like this: \begin{verbatim} Hello World \end{verbatim} This short program illustrates some of the features of ordinary macros in FunnelWeb. Consider the \p{@O} macro. Instead of containing straight text (\dq{Hello World}), it now contains the macro call \p{@}. A FunnelWeb macro can be called from within the body of another macro just by giving the macro name delimited in \p{@<} and \p{@>}. At the bottom of the file is the definition of the \p{@} macro. The definition is similar to the definition of \p{hello.txt} except that it starts with \p{@\$} to indicate that no product file is desired from this macro (directly). It also employs the optional \p{==} syntax which has no semantic impact, but can be used to make definitions clearer. The body of the \p{@} macro consists of calls to the \p{H} and \p{W} macros which are defined immediately above. Note that the macros are not constrained to be defined in any particular order. One of the main features of literate programming tools is that they allow the different parts of the text document being developed (usually a computer program) to be layed out in any order.\xx{order}{program}\xx{program}{layout} So long as there is a definition somewhere in the input file for every macro call, FunnelWeb will sort it all out. In fact, FunnelWeb's macro facility is very simple.\xx{macro}{bindings} Unlike many macro preprocessors which allow macros to define other macros, FunnelWeb completely finishes parsing and analysing the macros in the input file before it starts expanding them into product files. Other preprocessors allow macros to be redefined like variables (as in, say, \TeX{}\x{TeX}) taking on many different values as the macro pre-processor travels through the input file. In contrast, FunnelWeb has no concept of \dq{different times} and treats the input as one huge static orderless, timeless, collection of definitions. In FunnelWeb, there is only ever one time, and so there can only ever be one value/definition for each macro. \subsection{Number of Times Called} \x{number of times called}\xx{calls}{number}\xx{invocation}{number} So far we have seen only tiny, degenerate input files. The next example moves up to the level of \dq{trivial}, but starts to convey the flavour of the way FunnelWeb can be used in practice. Normally, there would be documentation text appearing between the macros, but this has been omitted so as to keep the focus on the macros themselves. Although the next example is much longer than the previous example, the only new construct is \p{@-} which can appear only at the end of a line, and suppresses it,\xx{EOL}{suppression} preventing it from appearing in the text. The \p{@-}\x{@-} construct allows the text of a macro to be aligned at the left margin, rather than having the first line hanging at the end of the \p{@\{}. FunnelWeb could have been set up so that this end of line marker was suppressed. However, it would have been a special case that would have broken the very memorable rule \dq{the text of a macro is the text appearing between the \p{@\{} and \p{@\}}}. Type the following text into the file \p{hello.fw} and run it through FunnelWeb. The file contains some intentional errors so be sure to type it in exactly and worry only if FunnelWeb \i{doesn't} generate some errors. \begin{verbatim} @O@==@{@- @ @ @
@} @$@
==@{@- main() { doit(); } @} @$@==@{@- void doit() { int i; for (i=0;i<10;i++) { @ @ } }@} @$@==@{@- printf("Hello World!"); printf("\n");@} @$@==@{scanf@} @$@==@{@- #include #include @} \end{verbatim} What happened? Well, if you haven't typed the file in properly, you may get some miscellaneous syntax errors. Fix these before continuing. If the file has been correctly typed, you should be faced with some error messages to do with the number of times some of the macros are called. By default, FunnelWeb insists that each macro defined is invoked exactly once. However, the file above defines macros that are used more than once and a macro that is not used at all. Let us examine the errors. First, we see that FunnelWeb has alerted us to the fact that the \p{Include Files} macro has been called twice. Once alerted to this, a quick look at the program convinces us that calling the macro twice is a mistake, and that one of the calls should be eliminated. Second, we note that FunnelWeb has alerted us to the fact that the \p{@} macro is never called. Again, a quick look at the program tells us that this is a mistake (and a very common one in the use of FunnelWeb), and that a call to the \p{@} macro should be inserted just above the call to the \p{@
} macro in the definition of \p{@}. These two cases demonstrate why these checks have been placed in FunnelWeb. It is nearly always acceptable for a macro to be called once. However, if a macro is not called at all, or called more than once, this is often a sign that the user has made a mistake. These checks have a dark side too. In addition to the errors mentioned above, FunnelWeb has generated two similar errors that do not help us. First, we are alerted to the fact that the \p{@} macro has been called twice. Clearly, in this case, this is not a problem, and so here FunnelWeb's fussiness is a nuisance. Second, we are alerted to the fact that the \p{@} macro has never been called. Like the \p{@} macro, this macro was defined as a notational convenience, and clearly it does not matter here if it is not used. Again, FunnelWeb is being a nuisance. The four cases above demonstrate the light and dark side of FunnelWeb's insistence that each macro be called exactly once. To resolve the conflict without reducing the strength of the checking, FunnelWeb provides two special sequences \p{@Z} (for \b{Z}ero) and \p{@M} (for \b{M}any) that can be attached to macro definitions. Presence of the \p{@Z}\xx{@Z}{tutorial} tag allows the designated macro to be called zero times. Presence of the \p{@M}\xx{@M}{tutorial} tag allows the designated macro to be called more than once. A single macro may carry both tags. It is always true that all macros are allowed to be called exactly once. Here is the revised program with the errors fixed, by eliminating or adding macro calls, or by adding tags. Try processing the file now. There should be no errors. \begin{verbatim} @O@==@{@- @ @ @
@} @$@
==@{@- main() { doit(); } @} @$@==@{@- void doit() { int i; for (i=0;i<10;i++) { @ @ } }@} @$@@M==@{@- printf("Hello World!"); printf("\n");@} @$@@Z==@{scanf@} @$@==@{@- #include #include @} \end{verbatim} \subsection{Indentation} \xx{indentation}{macro calls} The body of the \p{print} macro of the previous example contains two lines of text. A literal substitution of this macro's body in its context would result in: \begin{verbatim} { printf("Hello World!"); printf("\n"); printf("Hello World!"); printf("\n"); } \end{verbatim} But instead, it comes out as (have a look at this part of \p{hello.c} now): \begin{verbatim} { printf("Hello World!"); printf("\n"); printf("Hello World!"); printf("\n"); } \end{verbatim} The explanation is that FunnelWeb indents each line of multiline macros by the level of indentation at the point of call. This means that, as in the case above, program texts, which are usually highly indented, come out looking \dq{right}. In other circumstances, where the model of the text is one dimensional, FunnelWeb's indentation could become an impediment or even a danger. In these cases, it can be switched off by including the FunnelWeb \newterm{pragma} line \begin{verbatim} @p indentation = none \end{verbatim} anywhere in the input file. One of the design goals of FunnelWeb is to allow the user total control over the product files. This contrasts with the approach of Knuth's WEB\x{WEB} system \paper{Knuth83} (upon which FunnelWeb is based), which mangles the input text at the Pascal\x{Pascal} program syntax level, truncating identifiers, converting the text to upper case, and paragraphing text. Here is an example of part of a Pascal program produced by WEB\xx{output}{WEB} (from page~14 of \paper{Knuth83}): \begin{verbatim} IF R=0 THEN XREF[P]:=XREFPTR ELSE XMEM[R].XLINKFIELD:=XREFPTR;END;{:51} {58:}FUNCTION IDLOOKUP(T:EIGHTBITS):NAMEPOINTER;LABEL 31; VAR I:0..LONGBUFSIZE;H:0..HASHSIZE;K:0..MAXBYTES;W:0..1; L:0..LONGBUFSIZE;P:NAMEPOINTER;BEGIN L:=IDLOC-IDFIRST;{59:} H:=BUFFER[IDFIRST];I=IDFIRST+1; WHILE I==@{@- program adt(input,output); @ @ @ begin startproc; end. @} @!****************************** @$@+=@{@- type buffer_type = record length : integer; buf : array[1..100] of char; end; @} @$@+=@{@- bigbuf : buffer_type; @} @$@+=@{@- procedure buf_init (var b : buffer_type ) {Body of buf_init} procedure buf_add (var b : buffer_type; ch : char) {Body of buf_add} procedure buf_get (var b : buffer_type; var ch : char) {Body of buf_get} @} @!****************************** @$@+=@{@- type complex_type = record r,i : real; end; @} @$@+=@{@- procedure cm_set (var c: complex_type; a,b: real) {Body of cm_set} procedure cm_add (a,b: complex_type; var c: complex_type) {Body of cm_add} {Other procedures and functions} @} @!****************************** {...more pieces of program...} @!****************************** \end{verbatim} It is important to remember that the definition of each macro does not change throughout the input file. FunnelWeb parses the entire input file and assembles all the macro definitions before it even starts to expand macros. As a result, each additive macro can only have one definition, and that definition is the concatenation of all its parts. The example above shows how additive macros can be used to rearrange the presentation of a computer program in the order in which the user wishes to discuss it rather than the order in which the compiler requires that it be consumed. It is easy, however, to abuse the feature of additive macros. In many cases, the same effect can be obtained more clearly by replacing each part of an additive macro in-situ using uniquely named non-additive macros, and then collect them together as a group at the point where the additive macro is called. Doing this is more work, and is more error prone, but can result in a clearer exposition. The following program illustrates this alternative approach. \begin{verbatim} @!****************************** @O@==@{@- program adt(input,output); @ @ @ begin startproc; end. @} @$@==@{@- @ @ @} @$@==@{@- @ @} @$@==@{@- @ @ @} @!****************************** @$@==@{@- type buffer_type = record length : integer; buf : array[1..100] of char; end; @} @$@==@{@- bigbuf : buffer_type; @} @$@==@{@- procedure buf_init(var b : buffer_type) {Body of buf_init} procedure buf_add(var b : buffer_type; ch : char) {Body of buf_add} procedure buf_get(var b : buffer_type; var ch : char) {Body of buf_get} @} @!****************************** @$@==@{@- type complex_type = record r,i : real; end; @} @$@+=@{@- procedure cm_set(var c: complex_type; a,b : real) {Body of cm_set} procedure cm_add(a,b : complex_type; var c: complex_type) {Body of cm_add} {Other procedures and functions} @} @!****************************** {...more pieces of program...} @!****************************** \end{verbatim} One of advantages of FunnelWeb (and literate programming in general) is that (as shown above) it allows the user to lay out the program in whatever order is desired\xx{program}{layout} with near total independence from the ordering requirements of the target programming language. Additive macros are allowed to be tagged with \p{@Z} and \p{@M} just as other macros can, but the tags must appear only on the first definition of the macro. Additive macros cannot be connected directly to product files. \subsection{Parameterized Macros} \xx{parameterized}{macros} No self-respecting macro preprocessor would be complete without some form of macro parameterization, and FunnelWeb is no exception. FunnelWeb allows each macro to have from zero to nine formal parameters\xx{formal}{parameters} named \p{@1}\x{@1...}, \p{@2}, \p{@3}, \p{@4}, \p{@5}, \p{@6}, \p{@7}, \p{@8}, and \p{@9}. To define a macro with one or more parameters, insert a formal parameter list\xx{formal}{parameters} just after the macro name in the macro definition. Because macro parameters have fixed names (\p{@1}$\ldots$\p{@9}), there is no need to specify the names of formal parameters in the formal parameter list. All that need be conveyed is how many parameters the macro has. Here is an example of the definition of a macro having three parameters: \begin{verbatim} @$@@(@3@)@M==@{@- @1 while (@2) { @3 } @} \end{verbatim} To call a parameterized macro, an actual parameter\xx{actual}{parameters} list must be supplied that contains exactly the same number of actual parameters as there are formal parameters in the definition of the macro being called. An actual parameter list is delimited by \p{@(}\x{@(} and \p{@)},\x{@)} and parameters are \i{separated} by \dqp{@,}.\x{@,} The actual parameters themselves are general FunnelWeb expressions (see Chapter~\ref{chapdefinition} for the exact syntax) and can be inserted into the list directly or can be delimited by \p{@"}\x{@"} so as to allow some white space to assist in formatting the actual parameters. Here are some examples of calls of the \p{While loop} macro defined above. \begin{verbatim} @! First form of actual parameters without whitespace and double quotes. @@(x=1;@,x<=10@,printf("X=%u\n",x);@) @! Second form of actual parameters. The double quotes allow non-active @! whitespace that helps to lay out the actual parameters neatly. @! This call is functionally identical to the one above. @@( @"x:=1;@" @, @"x<=10@" @, @"printf("X=%u\n",x);@" @) @! The two forms can be mixed in a single call. @@(x=1;@,x<=10@, @"printf("X=%u\n",x);@" @) \end{verbatim} A few rules about parameterized macros are worth mentioning. Macros that do not have any parameters must have no formal or actual parameter lists.\xs{parameter list}{absent} Additive macros can have parameters, but the formal parameter list must appear in the first definition part only. Here is another example of the use of parameterized macros. This time, parameters and macro calls are used in a FunnelWeb input file that constructs an $O(n)$ representation of a song\x{song}\x{twelve bugs of christmas} whose full size is $O(n^2)$ in the number $n$ of unique lines.\x{rec.humor.funny}\xn{Pat}{Scannel} \begin{verbatim} @O@==@{@- The Twelve Bugs of Christmas ---------------------------- @@(@"first@" @,@<1@>@) @@(@"second@" @,@<2@>@) @@(@"third@" @,@<3@>@) @@(@"fourth@" @,@<4@>@) @@(@"fifth@" @,@<5@>@) @@(@"sixth@" @,@<6@>@) @@(@"seventh@" @,@<7@>@) @@(@"eighth@" @,@<8@>@) @@(@"ninth@" @,@<9@>@) @@(@"tenth@" @,@@) @@(@"eleventh@" @,@@) @@(@"twelfth@" @,@@) This song appeared in the internet newsgroup rec.humor.funny on 24-Dec-1991. It was contributed by Pat Scannell (scannell@@darkstar.ma30.bull.com). @} @$@@(@2@)@M==@{@- For the @1 bug of Christmas, my manager said to me @2 @} @$@<1@>@M==@{See if they can do it again.@} @$@<2@>@M==@{Ask them how they did it and@+@<1@>@} @$@<3@>@M==@{Try to reproduce it@+@<2@>@} @$@<4@>@M==@{Run with the debugger@+@<3@>@} @$@<5@>@M==@{Ask for a dump@+@<4@>@} @$@<6@>@M==@{Reinstall the software@+@<5@>@} @$@<7@>@M==@{Say they need an upgrade@+@<6@>@} @$@<8@>@M==@{Find a way around it@+@<7@>@} @$@<9@>@M==@{Blame it on the hardware@+@<8@>@} @$@@M==@{Change the documentation@+@<9@>@} @$@@M==@{Say it's not supported@+@@} @$@@M==@{Tell them it's a feature@+@@} \end{verbatim} \subsection{Macro Expansion} \xx{macro}{expansion} One of the strengths of FunnelWeb is that, when writing product files, it does not attempt to evaluate any text expression (\eg{}text block, parameter, macro call) in memory\xx{memory}{use of} and then write the result out. Instead, it always writes out what it is expanding dynamically and directly. This means that the user need not fear defining macros that expand to huge amounts of text and then calling those macros in other macros, or passing those huge macros as parameters to other macros. In all cases, FunnelWeb expands directly to the product file, and there can be no danger in running out of memory during expansion (except for running out of stack space and other marginally used resources in pathological cases). The only thing to remember in this regard is that FunnelWeb always stores the entire \i{input} file and all included files, in their entirety in memory, for the duration of the run. Here is an example, that illustrates how robust FunnelWeb is: \begin{verbatim} @! FunnelWeb copes well with the following macro definitions. @! (Providing that it has a little over ten megabytes of memory). @O@==@{@@(@@)@+@} @$@@(@1@)==@{"@1"@} @$@==@{@- ...Ten Megabytes of Text... @} \end{verbatim} \subsection{Include Files} \xx{include}{files} FunnelWeb provides a nested include file facility that can be used for a number of purposes. When FunnelWeb runs into a single line containing the special sequence \p{@i}\x{@i} followed by a blank, followed by a file name, it reads in the designated file and replaces the line containing the command (including the end of line marker at the end of the line) with the entire contents of the designated file. For example, if there was a file called \p{camera.txt} containing the two lines:\xx{poem}{camera}\xx{animal}{poem} \begin{verbatim} 'Cos I shoot with a camera instead of a gun. The animals flock to be petted and fed, \end{verbatim} and another file called \p{poem.fw} containing the following four lines\x{shooting} \begin{verbatim} I like to go shooting, it's a whole lot of fun, @i camera.txt Cos they know my camera isn't loaded with lead. - RNW, 04-Jan-1991. \end{verbatim} Then, if FunnelWeb were to process \p{poem.fw}, the result would be as if FunnelWeb had read in: \begin{verbatim} I like to go shooting, it's a whole lot of fun, 'Cos I shoot with a camera instead of a gun. The animals flock to be petted and fed, 'Cos they know my camera isn't loaded with lead. - RNW, 04-Jan-1991. \end{verbatim} FunnelWeb expands include files before it starts scanning and parsing the included text. The result is that include files can contain anything that can be found in a FunnelWeb file. The following example illustrates the level at which the include mechanism operates. If \p{main.fw} contains \begin{verbatim} @O@==@{@- @i inc.fw This is the text of the sloth macro. @} \end{verbatim} and inc.fw contains \begin{verbatim} @ @} @$@==@{@- \end{verbatim} Then if FunnelWeb were applied to \p{main.fw}, it would see: \begin{verbatim} @O@==@{@- @ @} @$@==@{@- This is the text of the sloth macro. @} \end{verbatim} which it would process in the normal manner. The only special sequence processing that takes place at a level lower than include files is the processing of the \p{=} sequence which changes the special character.\xx{special character}{changing} A few other facts about include files are worth mentioning here. Include files inherit the directory specification supplied using the \p{+I} command line option. The special character is saved at the start of each include file and restored to its previous value at the end of each include file. Include files can be nested up to ten levels. Recursive included files\xx{include files}{recursive} will always cause an infinite recursion as there is no bottoming out mechanism available. Include files must contain an integer number of lines (\ie{}the last line must be terminated with an end of line marker). Once FunnelWeb has seen \dqp{@i\ } at the start of a line, it will grab the rest of the line raw and treat it as a file name. There is no place on the line for things like FunnelWeb comments (see later) or extraneous text. Include files can be used for many purposes, but are particularly useful for hauling in macro libraries.\xx{macro}{libraries} \section{Typesetting Facilities} \xx{tutorial}{typesetting} The first half of this tutorial focuses solely on the macro facilities of FunnelWeb (which it more or less covers completely). As a result, the example documents you have seen so far have been gross distortions of \dq{normal} FunnelWeb documents which often contain as much documentation as code.\x{documentation vs code}\x{code vs documentation}\footnote{As an example, the author used FunnelWeb to develop a largish computer program and found that on average his style of using FunnelWeb resulted in about 30\% documentation and 70\% macros (code) (measured by numbers of lines).} While there are applications where FunnelWeb can be used solely as a macro preprocessor, most applications will use its typesetting facilities as well. This section restores the balance in this tutorial by presenting FunnelWeb's typesetting facilities. \subsection{Overview} \xx{overview}{typesetting} The macro definitions discussed in the macro tutorial completely define the contents of the product files that FunnelWeb will generate. These macro definitions can be arranged in any order and nothing external to them can affect the contents of the product files. The macros can be thought of as a group of self-contained islands. Although FunnelWeb will can process the macros all on their own, the full power of FunnelWeb is realized only when the macros are surrounded by a sea of documentation. This sea can take two forms: directives and free text. Some of the directives control things such as the maximum input line length. However, most of them are typesetting directives that affect the printed documentation. Thus a FunnelWeb document can be viewed as a sequence of \newterm{macro definitions}, \newterm{directives}, and \newterm{free text}. Unlike the product files which consist of unscrambled macro calls, the documentation file is more or less a direct representation of the input file. Each part of the input file appears in the documentation file in the order in which it appears in the input file. However, each different kind of part is typeset\footnote{Here the term \dq{typeset} is used loosely to refer to FunnelWeb's generation of typesetter commands for each construct in the input file. Strictly, the term should be used only to describe the actions of a typesetter program (\eg{}\TeX{}).} in a different manner. Macros are typeset in a particular style, with the macro body appearing in \p{tt font} (see some FunnelWeb printed documentation for an example). Typesetter directives have specific defined effects (more later). Free text is typeset exactly as it is, except that each block of text between blank lines is filled and justified as a paragraph. The following example demonstrates how all this works. Type in the following as \p{example.fw} and run it through FunnelWeb with the command \dqp{fw~example~+t}. The \dqp{+t} instructs FunnelWeb to generate a documentation file called \p{example.tex}. Run the file through \TeX{} and print it. Examine the files \p{example.out} and \p{example.tex}. \begin{verbatim} You are reading some free text before the macro. Free text can consist of any text (not containing the FunnelWeb special character) including typesetter commands such as $, %, #, and \TeX{} which will be typeset to appear exactly as they do in the input file! Look out! Here comes a macro! @O@==@{@- This text is part of a macro definition. @} This is free text following the macro. This sentence contains two @{inline@} typesetter @/directives@/. Now here is a non-inline typesetting directive. @t new_page This sentence will appear on the next page. \end{verbatim} At the top of the \p{example.tex} documentation file will be a set of \TeX{} macro definitions. The \TeX{} code corresponding to the input above appears at the end of the file. It should look something like this. \begin{verbatim} You are reading some free text before the macro. Free text can consist of any text (not containing the FunnelWeb special character) including typesetter commands such as \$, \%, \#, and $\backslash$TeX$\{$$\}$ which will be typeset to appear exactly as they do in the input file! Look out! Here comes a macro! \fwbeginmacro \fwfilename{example.out}{1}\fwequals \fwodef \fwbtx[This text is part of a macro definition. ]fwetx=% \fwcdef \fwbeginmacronotes \fwisafile{This macro is attached to an output file.} \fwendmacronotes \fwendmacro This is free text following the macro. This sentence contains two \fwlit{inline} typesetter \fwemp{directives}. Now here is a non-inline typesetting directive. \fwnewpage This sentence will appear on the next page. \end{verbatim} The following points explain the \p{example.tex} file. \narrowthing{You don't have to know TeX:}{If you don't know \TeX{}, don't pay too much attention to this section. You don't need to know \TeX{} to use FunnelWeb.} \narrowthing{In order:}{FunnelWeb has merely transformed the input. It hasn't rearranged it.} \narrowthing{Free text:}{Most of the free text has been simply copied over. The \TeX{} typesetter justifies and fills all paragraphs fed to it by default, so most of the text has just been copied verbatim.} \narrowthing{TeX codes:}{The characters and sequences which \TeX{} treats as special have been neutralized in the documentation file. For example, \dqp{\$} has become \dqp{\bs\$}. By default, FunnelWeb allows the user to write any text as free text and not have to worry about accidentally invoking typesetter features.} \narrowthing{fw sequences:}{The \p{fw} sequences (\eg{}\p{\bs{}fwbeginmacro}) invoke \TeX{} macros defined earlier in the documentation file (and not shown here).} \narrowthing{The macro:}{The macro is typeset using a set of predefined \TeX{} macros. See the printed documentation to see what this looks like on paper.} \narrowthing{Typesetter directives:}{Unlike the \TeX{} command sequences (which were neutralized), the FunnelWeb typesetter directives turn into \TeX{} macro calls. For example, \dqp{@\{inline@\}} became \dqp{\bs{}fwlit\{inline\}}.} In summary, FunnelWeb produces typeset documentation that transforms, but does not reorder, the input file. Macros are typeset in a specific style. FunnelWeb typesetter directives have particular well-defined effects. Free text is filled and justified, but will otherwise appear in the printed documentation exactly as it appears in the input file. \subsection{Typesetter Independence} \xx{typesetter}{independence} Although the current version of FunnelWeb can only generate documentation files in \TeX{} form, it encourages typesetter independence by neutralizing all \TeX{} control sequences before writing them out. The result is that you don't have worry about upsetting or depending on \TeX{} by accidentally including some special character or sequence. By default your input file is \newterm{typesetter independent}. This scheme differs from other literate programming tools, including all earlier versions of FunnelWeb, which copy their free text directly to the documentation file, the justification being that the programmer can use the full power of the typesetter language to describe the program. The disadvantages of doing this are first that the programmer is required to know the typesetting language and second that the input file becomes typesetter dependent. FunnelWeb avoids these problems by knobbling the free text be default. However, FunnelWeb does provide a trapdoor for those who want their free text to be fed directly to \TeX{}. To open the trapdoor, simply include the following pragma somewhere in your input file. \begin{verbatim} @p typesetter = tex \end{verbatim} See Section~\ref{typesetterpragma} for more information. FunnelWeb leaves the degree to which the user wishes to bind a particular document to a particular typesetter up to the user. In some cases, the extra typesetting power may compensate for the lack of portability. However, as a rule, it is best to avoid typesetter-specific commands, so as to allow your input files to be formatted at a later date for different typesetters. FunnelWeb includes a number of its own typesetter commands so as to support typesetter-independent input files. The following sections describe some of these commands. In particular, the next section describes the most powerful FunnelWeb typesetting directives which allow the user to structure the document hierarchically. \subsection{Hierarchical Structure} \xx{hierarchical}{structure} The tree structure is one of the most effective structuring tools that exists, deriving its power from the principal of divide and conquor. So effective is it that the internal organization of most technical books are tree structures which are concisely summarized in the table of contents. In contrast, computer programs are usually presented as flat sequences of text to be consumed by an anonymous compiler. In order to bring program documentation up to the structural sophistication commonplace in technical books, FunnelWeb provides five levels of section headings\xx{section}{headings} implemented by the five special sequences \p{@A},\x{@A...} \p{@B}, \p{@C}, \p{@D}, and \p{@E}. These must always appear at the start of a line. \p{@A} is the highest level section (\eg{}like \LaTeX{}'s \p{\bs{}chapter}) and \p{@E} is the lowest level section (\eg{}like \LaTeX{}'s \p{\bs{}subsubsubsection}). Section headings can appear anywhere in the free text of a FunnelWeb input file (\ie{}anywhere except inside a macro definition). Each section heading\xx{name}{section} in a FunnelWeb document has an associated name. The name of a section can be provided explicitly by supplying it delimited by \p{@<} and \p{@>} immediately after the section sequence (\eg{}\p{@A}), or implicitly by not providing an explicit name, in which case the section takes the name of the first macro defined between the section header in question and the following section header. An error is generated if a section has not been given an explicit name and does not contain any macro definitions. Here are some example headings: \begin{verbatim} @A@ @B@ @C@ @C@ @B@ @C@ @C@ @C This heading hasn't been given an explicit name, but will inherit the name \p{Save the rest of the world} from the macro definition below. @$@@Z==@{...@} \end{verbatim} The feature of having unnamed sections inherit the name of the first macro defined within their scope is present because a common style of writing in FunnelWeb is to have one section per macro definition. Because, under this style, each section describes a single macro, it usually turns out that the macro name makes a good name for the section too. The inheritance mechanism prevents duplication of the name.\xx{section name}{inheritance} Apart from the requirement that each section have an explicit or implicit name and that its special sequence appear at the start of a line, the only other restriction on section headings is that a section heading at level $n$ cannot appear immediately after a section heading at level $n-1$ or less. In other words, the hierarchy cannot be broken. For example, an \p{@C} cannot appear after an \p{@A} heading unless there is an intervening \p{@B} heading. \begin{verbatim} @A@ @C@ \end{verbatim} This rule extends to the start of the file; if there are any headings at all, the first one must be an \p{@A} heading. The following file, while short, is in error. \begin{verbatim} This FunnelWeb input file is in error because its first section heading is at level C rather than level A. @C@<2@> \end{verbatim} \subsection{Understanding the Printed Documentation} Type in the following file, and use FunnelWeb and \TeX{} to generate the corresponding printed documentation.\xx{programmer's}{cheer}\xx{hacker's}{cheer} \xx{hacker's}{dictionary} \begin{verbatim} @A@ @t table_of_contents @A@ The following macro contain comments that provide moral support in the output code. @$@@M==@{ -- Shift to the left! -- Shift to the right! -- Pop up, push down! -- Byte! Byte! Byte! -- (From "The New Hacker's Dictionary"). @} The next macro is similar but is distributed throughout the program. @$@+=@{@+-- Pointer to the left@+@} @A@ @B@ @$@+=@{-- Pointer to the right@+@} @$@@Z==@{type stack = record ... end;@} @B@ @$@+=@{-- Hack that code@+@} @$@@Z==@{@- procedure push(var b:stack; v:value); @ {...}@} @B@ @$@+=@{-- Tight! Tight! Tight!@+@} @$@@Z==@{@- procedure pop(var b:stack); @ {...}@} @B@ @$@+=@{-- (RNW, 04-Jan-1991).@+@} @$@@Z==@{@- procedure rough(var b:stack); @ {...}@} @O@==@{dummy@+@} \end{verbatim} An examination of the printed documentation reveals a lot about how FunnelWeb's presentation works. First, notice how the \p{@t} typesetter directive at the top of the file has caused a table of contents to appear. This is one of FunnelWeb's typesetting features and is discussed in a later section. The table of contents shows that the sections have been numbered hierarchically. Now take a look at the typeset macro definitions. Most important are the numbers in square brackets that follow each macro name. As well as numbering the headings \i{hierarchically}, FunnelWeb \i{independently} numbers the macro definitions \i{sequentially}. The first macro definition (for \dq{Programmer's Cheer}) is numbered 1. The second (for \dq{Hacker's Cheer}) is numbered 2 and so on. Note that it is not macros that are numbered, but macro definitions. The distinction is necessary because some macros (such as the \dq{Hacker's Cheer} macro) are additive. It is important to realize that there is no relationship between the numbers of the headings and the numbers of the macro definitions. Now take a look at the notes beneath the body of each macro definition. All macro definitions are followed by a note indicating the definitions in which the macro is called. Additive macros have an additional list, listing the definitions in which they are defined. Finally, take a look at the macro \i{call} of \dq{Programmer's Cheer} in section~3.2 of the printed documentation. Macro calls are set in slanted roman (so that they can be distinguished from the \p{tt font} code) and are followed by the number of the defining macro definition. In this case, the macro was defined in definition~1. Further down, the call to the \dq{Hacker's Cheer} macro indicates that the macro was defined in definition~2. In fact the macro is additive and definition~2 is just the first of many definitions. To list all definitions in a call to an additive macro would be unnecessarily messy. \subsection{Literals and Emphasis} \x{literal construct}\x{emphasis construct} When writing about program code, it is often desirable to be able to indicate that a particular word or phrase be typeset in the same manner as the code being discussed. For example, one might talk about the variable \p{topval} or the procedure \p{stack\_pop} and wish for them to be typeset as they are in this sentence. This, of course, is simple to do using \TeX{} macros, but use of the (more general) FunnelWeb typesetting directives to do the same work has the added benefit of keeping the document portable to other typesetters. FunnelWeb provides two in-text type modification constructs: \p{@\{...@\}} and \p{@/...@/}\x{@slash}\x{@braces} where $\ldots$ is raw text. The \p{@\{...@\}} construct sets the enclosed text in the same manner as the text of macro definitions is set. The \p{@/...@/} construct emphasises its enclosed text in some typesetter-dependent fashion. Typically the emphasised text is set in italics. Here is an example of how these constructs might be used: \begin{verbatim} The following procedure @{put_sloth@} writes the @{sloth@} variable to the output file. Note: @/The output file must be opened for writing at this point or the program will crash!@/ \end{verbatim} \subsection{Adding a Header Page} \x{header page} FunnelWeb provides a few typesetter-independent typesetting constructs which are specifically designed for the construction of header pages. These constructs are usually best placed at the top of your input file, but can be placed anywhere the document if desired to create header pages right through. The two main restrictions on these constructs is that the \p{@t} must start at the start of a line (which cannot contain comments), and that the constructs cannot appear inside a macro definition. Here is what the top of an input file might look like: \begin{verbatim} @t vskip 40 mm @t title titlefont centre "Hairy Wombat" @t title titlefont centre "Simulation" @t vskip 10 mm @t title smalltitlefont centre "A Program in Six Parts" @t title smalltitlefont centre "Simulating the Life of Some Hairy Wombats" @t vskip 20 mm @t title normalfont left "By Zqitzypbuswapzra Ypongtatoslrtzz" @t new_page @t table_of_contents @t new_page \end{verbatim} The \p{@t} at the start of each line indicates that each entire line is a typesetter directive. The \p{vskip}\xx{vskip}{directive} directive instructs FunnelWeb to skip some vertical space (measured in millimetres). The \p{title} directive\xx{title}{directive} instructs FunnelWeb to position a string of text on a single line of its own. Options are provided for font and alignment. The first word after \p{title} is the font which can be one of (in decreasing order of size) \p{titlefont}, \p{smalltitlefont}, and \p{normalfont}. The second word after \p{title} is the desired alignment of the text. The options here are \p{left}, \p{right}, and \p{centre}. The \p{new\_page} directive\xx{newpage}{directive} instructs FunnelWeb to skip to a new page. Finally, the \p{table\_of\_contents} directive\xx{table of contents}{directive} instructs FunnelWeb to insert a table of contents at that point in the text. \subsection{Comments} \x{comments} A FunnelWeb comment commences with the \p{@!}\x{@!} sequence and continues up to, but not including, the end of line marker at the end of the line that the comment sequence is on. Comments can be placed on any line except \p{@i} include, \p{@p} pragma, and \p{@t} typesetter directive lines. The text following the FunnelWeb comment sequence \p{@!} will not appear in the product files or the documentation file. It is only for the eyes of those who bother to look at the original \p{.fw} input file. Typically FunnelWeb comments are used to describe the way in which particular FunnelWeb constructs are being used. Example: \begin{verbatim} @! This macro is really revolting. Please forgive me. I had to do it! @$@==@{@- @#X@(@#Y@(@#Z@,@"@#Z@"@)=6@,Teapot@,@"@#Q@(45@)@"@,Tiger@)@} \end{verbatim} \section{A Complete Example} \xx{complete}{example}\xx{FunnelWeb}{Example} To finish off the chapter, a complete example of a FunnelWeb input file is presented. Although unrealistically short, it gives a better idea of what a typical FunnelWeb \p{.fw} file looks like. \begin{verbatim} @!---------------------------------------! @! Start of FunnelWeb Example .fw File ! @!---------------------------------------! @t vskip 40 mm @t title titlefont centre "Powers:" @t title titlefont centre "An Example of" @t title titlefont centre "A Short" @t title titlefont centre "FunnelWeb .fw File" @t vskip 10 mm @t title smalltitlefont centre "by Ross Williams" @t title smalltitlefont centre "26 January 1992" @t vskip 20 mm @t table_of_contents @A@ This program writes out each of the first @{p@} powers of the first @{n@} integers. These constant parameters are located here so that they are easy to change. @$@==@{@- n : constant natural := 10; -- How many numbers? (Ans: [1,n]). p : constant natural := 5; -- How many powers? (Ans: [1,p]).@} @B Here is the outline of the program. This FunnelWeb file generates a single Ada output file called @{Power.ada@}. The main program consists of a loop that iterates once for each number to be written out. @O@==@{@- @ procedure example is @ begin -- example for i in 1..n loop @ end loop; end example; @} @B In this section, we pull in the packages that this program needs to run. In fact, all we need is the IO package so that we can write out the results. To use the IO package, we first of all need to haul it in (@{with text_io@}) and then we need to make all its identifiers visible at the top level (@{use text_io@}). @$@==@{with text_io; use text_io;@} @B Here is the bit that writes out the first @{p@} powers of @{i@}. The power values are calculated incrementally in @{ip@} to avoid the use of the exponentiation operator. @$@==@{@- declare ip : natural := 1; begin for power in 1..p loop ip:=ip*i; put(natural'image(ip) & " "); end loop; new_line; end;@} @!---------------------------------------! @! End of FunnelWeb Example .fw File ! @!---------------------------------------! \end{verbatim} \section{Summary} This chapter has provided an introduction to FunnelWeb and a tutorial that covers most of its features. FunnelWeb's functionality can be split into two parts: a macro preprocessor, and support for typesetting. The reader should be aware that the examples in this chapter, constructed as they were to demonstrate particular features of FunnelWeb, do not present a realistic picture of the best use of the tool. Only the final example of this chapter comes close. The reader should study this last example carefully and then write some real programs using FunnelWeb before proceeding to Chapter~\ref{chapadvanced} which provides more advanced information. At this stage it does not particularly matter exactly how you use Funnelweb, as everyone develops their own style anyway. The important thing is to try it. %==============================================================================% % End of Ch1.tex % %==============================================================================%