From mark@ispc001.demon.co.uk  Thu Oct 25 14:32:05 2001
Received: from mail.actcom.co.il [192.114.47.13]
	by localhost with POP3 (fetchmail-5.5.0)
	for arnold@localhost (single-drop); Thu, 25 Oct 2001 14:32:04 +0200 (IST)
Received: by actcom.co.il (mbox arobbins)
 (with Cubic Circle's cucipop (v1.31 1998/05/13) Thu Oct 25 14:32:53 2001)
X-From_: mark@ispc001.demon.co.uk Thu Oct 25 12:37:22 2001
Received: from lmail.actcom.co.il by actcom.co.il  with ESMTP
	(8.11.6/actcom-0.2) id f9PAb5m18485 for <arobbins@actcom.co.il>;
	Thu, 25 Oct 2001 12:37:11 +0200 (EET)  
	(rfc931-sender: mail.actcom.co.il [192.114.47.13])
Received: from consort.ff0.org (consort.superb.net [209.61.216.22])
	by lmail.actcom.co.il (8.11.6/8.11.6) with ESMTP id f9PA8dh22545
	for <arobbins@actcom.co.il>; Thu, 25 Oct 2001 12:08:39 +0200
Received: from ispc001.demon.co.uk (ispc001.demon.co.uk [158.152.16.99])
	by consort.ff0.org (8.11.6/8.11.6) with SMTP id f9PA82D04356
	for <arnold@skeeve.com>; Thu, 25 Oct 2001 06:08:02 -0400
Date: Thu, 25 Oct 2001 11:04:57 GMT
From: mark@ispc001.demon.co.uk (Mark Katz)
Reply-To: mark@mark-it.co.uk
Message-Id: <66908@ispc001.demon.co.uk>
To: arnold@skeeve.com
Subject: Re: text file cross-reference utility
X-Mailer: PCElm 1.10
Lines: 486
Status: RO

In message <200110250924.f9P9O5114409@skeeve.com> Aharon Robbins writes:
> Would you send me that xref program? I'll put it on my ftp archive eventually.

Listed below is all the files/prog plus a few
other e-mails to do with the topic. I'll let you take
whats important.;-)

Best wishes

Yitz
ps He wrote it 1990 !!!!
--
Path: ispc001.demon.co.uk!news.demon.co.uk!demon!mail2news.demon.co.uk!ispc001.demon.co.uk!mark
From: mark@ispc001.demon.co.uk (Mark Katz)
Newsgroups: comp.lang.awk
Subject: Analysing an awk program
Date: Thu, 30 Apr 98 13:28:21 GMT
Organization: ISPC Ltd
Message-ID: <893942901snz@ispc001.demon.co.uk>
Reply-To: Mark@ispc001.demon.co.uk
X-Mail2News-User: mark@ispc001.demon.co.uk
X-Mail2News-Path: ispc001.demon.co.uk
X-Trace: mail2news.demon.co.uk 893948831 1524 mark ispc001.demon.co.uk
X-Complaints-To: abuse@demon.net
X-Newsreader: Demon Internet Simple News v1.30
Lines: 26
Xref: news.demon.co.uk comp.lang.awk:683

Some 20+ years ago, I was a great fan of Fortran and one of the
nice options available from the 'H' compiler was a cross-reference
listing of all variables and which lines they were on.

Very important for large programs....

I'd like to see a similar facility for my awk programs 
I set up a very crude version (in awk of course) but got bogged
down with implied assignments ++/-- and stuff inside quote "'s

Has anyone written something?

While on the subject of analysing awk programs has anyone written
a program that would either 
- compress (multiple statements on a line subject to a max line width) or 
- expand (1 statement on a line and nicely format 'blocks') 
awk programs

Thanks
Mark
-- 
Mark Katz
ISPC, London - Innovation in data-delivery tools
Tel: (44) 181-455 4665, Fax (44) 181-458 9554
** Visit our website on http://www.efiche.com/efiche **




Newsgroups: comp.lang.awk
Path: ispc001.demon.co.uk!news.demon.co.uk!demon!woodstock.news.demon.net!demon!news.idt.net!netnews.com!news-peer.sprintlink.net!news.sprintlink.net!newsfeed.nacamar.de!nntp.news.xara.net!xara.net!baron.netcom.net.uk!netcom.net.uk!ix.netcom.com!pbewig
From: pbewig@netcom.com (Phil Bewig)
Subject: Re: Analysing an awk program
Message-ID: <pbewigEs90GL.5oK@netcom.com>
Organization: Netcom On-Line Services
X-Newsreader: NN version 6.5.0 CURRENT #9
References: <893942901snz@ispc001.demon.co.uk>
Date: Thu, 30 Apr 1998 22:43:32 GMT
Sender: pbewig@netcom4.netcom.com
Lines: 324
Xref: news.demon.co.uk comp.lang.awk:690


XREF(AWK)                   Philip L. Bewig                   XREF(AWK)
 
 
NAME
 
	xref(awk) - produce a cross reference listing of an awk program
 
 
SYNOPSIS
 
	awk -f xref.awk [ file ... ]
 
 
DESCRIPTION
 
	XREF(AWK) takes as input a valid awk program and produces as out-
	put a cross-reference listing of all variables and function calls
	which appear in the program.
 
	For ordinary variables and array variables, a line of the form
 
		count var(func) lines ...
 
	is produced, where "count" is the number of times the variable is
	used, "var" is the name of the variable, "func" is the function
	name to which the variable is local (a null "func" indicates that
	the variable is global), and "lines" is the number of each line
	where the variable appears.  Appearances of the variable in a
	function's parameter list are ignored.  The number of lines shown
	may differ from "count" if the variable appears more than once on
	the same line.
 
	For functions, a line of the form
 
		count func(define) lines ...
 
	is produced, where "count" is the number of times the function is
	called, "func" is the name of the function, "define" is the lime
	number where the function is defined, and "lines" is the number of
	each line where the function is called.  As for variables, the
	number of lines shown may differ from "count."
 
	Output lines for variables and functions are intermixed and are
	sorted by name.  Though terse, the output is informative, easy to
	read, and amenable to further processing.
 
 
EXAMPLE
 
	The cross-reference listing produced by running xref.awk against
	itself is shown below:
 
		5 NR() 39 45 50 53 68
		8 RLENGTH() 119 120 123 124 127 128 132 133
		10 act() 31 34 36 40 51 54 56 63 65 67
		1 arr(asplit) 90
		2 arr(inarray) 93 94
		1 asplit(89) 6
		3 braces() 55 57 58
		2 flines() 53 79
		1 fs(asplit) 89
		3 funcname() 42 52 61
		16 i() 76 77 78 79 81 82 83 84 90
		3 inarray(92) 37 43 48
		6 j() 82 83 84
		3 j(inarray) 93 94
		3 keywords() 10 125 129
		1 lex(97) 29
		31 line() 103 104 107 108 109 110 112 113 114 115 116 117 118
			119 120 122 123 124 126 127 128 131 132 133
		6 lines() 39 45 50 83 84
		4 local() 41 59 60 64
		3 machine() 17 30 31
		2 n(asplit) 89 90
		15 names() 37 38 43 44 48 49 77 78 79 81 82 83 84
		3 nextstate() 30 62 73
		4 nnames() 38 44 49 76
		4 state() 23 30 31 73
		1 str(asplit) 89
		3 symb() 29 30 31
		2 temp() 59 60
		2 temp_asplit() 89 90
		31 tok() 37 38 39 41 42 43 44 45 47 48 49 50 52 53 64 101 105
			113 115 117 119 123 125 127 129 132
		1 val(inarray) 94
		5 xnames() 39 45 50 77 82
 
	For readability, some lines have been folded.
 
 
SOURCE CODE
 
	# xref.awk - cross reference an awk program
	
	BEGIN {
	
		# create array of keywords to be ignored by lexer
		asplit("BEGIN:END:atan2:break:close:continue:cos:delete:" \
			"do:else:exit:exp:for:getline:gsub:if:in:index:int:"  \
			"length:log:match:next:print:printf:rand:return:sin:" \
			"split:sprintf:sqrt:srand:sub:substr:system:while",
			keywords,":")
	
		# build the symbol-state table
		split("00:00:00:00:00:00:00:00:00:00:" \
			  "20:10:10:12:12:11:07:00:00:00:" \
			  "08:08:08:08:08:33:08:00:00:00:" \
			  "08:44:08:36:08:08:08:00:00:00:" \
			  "08:44:45:42:42:41:08",machine,":")
	
		# parse the input and store an intermediate representation
		# of the cross-reference information
	
		# set up the machine
		state = 1
	
		# run the machine
		for (;;) {
	
			# get next symbol
			symb = lex()
			nextstate = substr(machine[state symb],1,1)
			act = substr(machine[state symb],2,1)
	
			# perform required action
			if ( act == "0" )
				; # do nothing
			else if ( act == "1" ) {
				if ( ! inarray(tok,names) )
					names[++nnames] = tok
				lines[tok,++xnames[tok]] = NR }
			else if ( act == "2" ) {
				if ( tok in local ) {
					tok = tok "(" funcname ")"
					if ( ! inarray(tok,names) )
						names[++nnames] = tok
					lines[tok,++xnames[tok]] = NR }
				else {
					tok = tok "()"
					if ( ! inarray(tok,names) )
						names[++nnames] = tok
					lines[tok,++xnames[tok]] = NR } }
			else if ( act == "3" ) {
				funcname = tok
				flines[tok] = NR }
			else if ( act == "4" )
				braces++
			else if ( act == "5" ) {
				braces--
				if ( braces == 0 ) {
					for ( temp in local )
						delete local[temp]
					funcname = ""
					nextstate = 1 } }
			else if ( act == "6" ) {
				local[tok] = 1 }
			else if ( act == "7" )
				break
			else if ( act == "8" ) {
				print "error: xref.awk: line " NR ": aborting" \
					> "/dev/con"
				exit 1 }
	
			# finished with current token
			state = nextstate }
	
		# finished parsing, now ready to print output
		for ( i = 1; i <= nnames; i++ ) {
			printf "%d ", xnames[names[i]] |"sort +1"
			if ( index(names[i],"(") == 0 )
				printf "%s(%d)", names[i], flines[names[i]] |"sort +1"
			else
				printf "%s", names[i] |"sort +1"
			for ( j = 1; j <= xnames[names[i]]; j++ )
				if ( lines[names[i],j] != lines[names[i],j-1] )
					printf " %d", lines[names[i],j] |"sort +1"
			printf "\n" |"sort +1" }
	
		} # END OF PROGRAM
	
	function asplit(str,arr,fs,  n) { n = split(str,temp_asplit,fs)
		for ( i = 1; i <= n; i++ ) arr[temp_asplit[i]]++ }
	
	function inarray(val,arr,  j) {
	    for ( j in arr )
	        if ( arr[j] == val ) return j
	    return "" }
	
	function lex() {
	
		for (;;) {
	
			if ( tok == "(eof)" ) return 7
	
			while ( length(line) == 0 )
				if ( getline line == 0 ) {
					tok = "(eof)"; return 7 }
	
			sub(/^[ \t]+/,"",line)				# remove white space,
			sub(/^"([^"]|\\")*"/,"",line)		# quoted strings,
			sub(/^\/([^\/]|\\\/)+\//,"",line)	# regular expressions,
			sub(/^#.*/,"",line)					# and comments
	
			if ( line ~ /^function/ ) {
				tok = "function"; line = substr(line,9); return 1 }
			else if ( line ~ /^{/ ) {
				tok = "{"; line = substr(line,2); return 2 }
			else if ( line ~ /^}/ ) {
				tok = "}"; line = substr(line,2); return 3 }
			else if ( match(line,/^[A-Za-z_][A-Za-z_0-9]*\[/) ) {
				tok = substr(line,1,RLENGTH-1)
				line = substr(line,RLENGTH+1)
				return 5 }
			else if ( match(line,/^[A-Za-z_][A-Za-z_0-9]*\(/) ) {
				tok = substr(line,1,RLENGTH-1)
				line = substr(line,RLENGTH+1)
				if ( ! ( tok in keywords ) ) return 6 }
			else if ( match(line,/^[A-Za-z_][A-Za-z_0-9]*/) ) {
				tok = substr(line,1,RLENGTH)
				line = substr(line,RLENGTH+1)
				if ( ! ( tok in keywords ) ) return 4 }
			else {
				match(line,/^[^A-Za-z_{}]/)
				tok = substr(line,1,RLENGTH)
				line = substr(line,RLENGTH+1) } } }
 
 
TECHNICAL DISCUSSION
 
	Broadly, XREF(AWK) parses an awk program using a symbol-state
	table, in much the same way as a yacc-generated parser.  The
	lexical analyzer recognizes seven distinct symbols:  the word
	"function", the left brace, the right brace, identifiers used
	as variables, identifiers used as arrays, identifiers used as
	functions, and end of file.  The type of symbol is returned to
	the parser as the value of the "lex" function, and the global
	variable "tok" is set to the text of the current token.
 
	The symbol-state table is stored in the "machine" array.  The
	table can be represented as follows:
 
	               symbol |     1       2  3   4     5     6     7
	                      |
	state                 | "function"  {  }  var  array  func  eof
	-- -- -- -- -- -- -- -+- -- -- -- -- -- -- -- -- -- -- -- -- --
	1 any                 |     20     10  10  12    12    11   07
	2 "function"          |     08     08  08  08    08    33   08
	3 "function" name     |     08     44  08  36    08    08   08
	4 "function" name "{" |     08     44  45  42    42    41   08
 
	where the first digit is the state to be entered after process-
	ing the current token and the second digit is an action to be
	performed.  The actions are listed below:
 
		1	found a function call
		2	found a variable or array
		3	found a function definition
		4	found a left brace
		5	found a right brace
		6	found a local variable declaration
		7	found end of file
		8	found an error
 
	Each of the first six actions causes some information about the
	target program to be stored for later processing; the structures
	used will be discussed below.  The seventh action causes the
	parser to exit.  The eighth action causes errors to be reported
	to standard error and the program to abort.
 
	Before describing the intermediate data structures, we will
	discuss some of the more interesting points in the action calls.
	The "braces" variable keeps track of whether we are currently
	within a functions; it is positive within a function and zero
	without.  When the right brace which causes the value of "braces"
	to go from one to zero is found, the value of "nextstate" is
	changed from four (scanning a function) to one (any) and the
	names of local variables are forgotten.  The "local" array is
	accumulated from the variables found after the function name but
	before the opening left brace of the function; action two care-
	fully checks whether a variable is global or local before writing
	to the intermediate data structure.  The variable "funcname" is
	the name of the current function when within a function and null
	without.
 
	The following arrays store an intermediate representation of the
	variable and function identifiers of the target program:
 
		names[1..nnames] = list of all identifiers, both variable and
			function names; for variables, the name has the form
			var(func), but for functions, there are no parentheses
 
		xnames[names[i]] = number of times names[i] is used
 
		lines[names[i],1..xnames[names[i]]] = list of line numbers
			where names[i] is used
 
		flines[names[i]] = line number where function names[i] is
			defined
 
	These arrays are created as the parser reads the input; when the
	parser is finished, the arrays are output in user-readable form.
 
 
PORTABILITY
 
	XREF(AWK) will work with any implementation of nawk.  The MKS
	ToolKit implementation requires the large-model version of awk.
 
 
HISTORY
 
	Written by Phil Bewig on February 10, 1990.  Inspired by
	Exercise 3-16 of the book "The Awk Programming Language" by
	Alfred V. Aho, Brian W. Kernighan and Peter J. Weinberger
	(Addison-Wesley:  1988).
 
 
COPYRIGHT
 
	This program is placed in the public domain.  However, the
	author requests credit when distributed.
-- 
Phil Bewig ... pbewig@netcom.COM



Newsgroups: comp.lang.awk
Path: ispc001.demon.co.uk!news.demon.co.uk!demon!dispose.news.demon.net!demon!woodstock.news.demon.net!demon!feed2.news.erols.com!erols!howland.erols.net!ix.netcom.com!pbewig
From: pbewig@netcom.com (Phil Bewig)
Subject: Re: Analysing an awk program
Message-ID: <pbewigEs93xL.LI4@netcom.com>
Organization: Netcom On-Line Services
X-Newsreader: NN version 6.5.0 CURRENT #9
References: <893942901snz@ispc001.demon.co.uk>
Date: Thu, 30 Apr 1998 23:58:32 GMT
Sender: pbewig@netcom13.netcom.com
Lines: 36
Xref: news.demon.co.uk comp.lang.awk:693

mark@ispc001.demon.co.uk (Mark Katz) writes:

>Some 20+ years ago, I was a great fan of Fortran and one of the
>nice options available from the 'H' compiler was a cross-reference
>listing of all variables and which lines they were on.

>Very important for large programs....

>I'd like to see a similar facility for my awk programs 
>I set up a very crude version (in awk of course) but got bogged
>down with implied assignments ++/-- and stuff inside quote "'s

>Has anyone written something?

Yes.  See the companion posting to this one.

>While on the subject of analysing awk programs has anyone written
>a program that would either 
>- compress (multiple statements on a line subject to a max line width) or 
>- expand (1 statement on a line and nicely format 'blocks') 
>awk programs

Gnu indent should be able to handle your "expand" request.
I know of no solution to your "compress" request, although
it doesn't seem like it should be too hard to do.

>Thanks
>Mark
>-- 
>Mark Katz
>ISPC, London - Innovation in data-delivery tools
>Tel: (44) 181-455 4665, Fax (44) 181-458 9554
>** Visit our website on http://www.efiche.com/efiche **

-- 
Phil Bewig ... pbewig@netcom.COM



Path: ispc001.demon.co.uk!news.demon.co.uk!demon!dispose.news.demon.net!demon!nntp.news.xara.net!xara.net!news-feed.inet.tele.dk!bofh.vszbr.cz!pln-e!spln!extra.newsguy.com!newsp.newsguy.com!enews3
From: pjfarley@dorsai.org (Peter J. Farley III)
Newsgroups: comp.lang.awk
Subject: Re: Analysing an awk program
Date: Fri, 01 May 1998 03:51:14 GMT
Organization: http://extra.newsguy.com
Message-ID: <3549462f.9462022@news.dorsai.org>
References: <893942901snz@ispc001.demon.co.uk> <pbewigEs93xL.LI4@netcom.com>
NNTP-Posting-Host: p-327.newsdawg.com
X-Newsreader: Forte Free Agent 1.1/32.230
Lines: 16
Xref: news.demon.co.uk comp.lang.awk:695

pbewig@netcom.com (Phil Bewig) wrote:

>mark@ispc001.demon.co.uk (Mark Katz) writes:
<Snipped>
>>Has anyone written something?
>
>Yes.  See the companion posting to this one.

Phil,

That is a *very* nice piece of work.  Thank you for posting it, and
for placing it in the public domain.  An excellent example of
netizenship, IMHO.

----------------------------------------------------
Peter J. Farley III (pjfarley@dorsai.org)




-- 
Mark Katz
Mark-it, London. Delivering MR-IT/Internet solutions
Tel: (44) 20-8731 7516, Fax: (44) 20-8458 9554; http://www.mark-it.co.uk

