DSN Radio Astronomy
Languages for Quick Astronomy Software

by Tom Kuiper
kuiper@jpl.nasa.gov
1997 Aug 11

In keeping with a philosophy of using publicly available languages and libraries with access to source code, Tcl/Tk has been selected as the scripting language for DSN Radio Astronomy monitor and control and data acquisition applications.

While Perl seems a good candidate for data reduction and analysis purposes, no software has yet been written so the choice is still open.


Contents

The Need for Quick Programming Languages

At the forefront of astronomical discovery are those observers and observatories who can adapt rapidly to new ideas, new technologies, and new methods. The observatories which are designed to support this mode of work are like laboratories in which instruments and software packages can be re-assembled by the observers themselves, avoiding the long cycle of requesting an upgrade, having a committee assign a priority, developing an implementation plan, etc., etc. My moment of glory in this regard was the day we took off from Moffett Field for the first mm-wave observations ever with NASA KAO. As we crossed the beach heading out over the Pacific, I was pounding away at my keyboard, making some essential last-minute changes to our data analysis package.

The traditional software implementation cycle of

edit -> compile -> link -> test -> edit -> ...
is also somewhat frustrating when you need quick results, especially when the program fails because of one mis-typed character. This led to the early development of GOTRAN and BASIC, and to increasingly power UNIX shells. The common feature of all of these is that you immediately execute the source file. (Strictly speaking, the file is interpreted rather than executed.) Typically, in a multi- window environment, you keep the source file open, so that the implementation cycle is simplified to
edit <-> test

The UNIX approach to satisfying this need is to develop simple self-contained programs which are linked together by a powerful shell into a software environment. Shells are highly programmable, but they have an illusive syntax, probably because of the ad hoc in which they evolved.

One lamentable result is the continuing popularity of that crippled excuse for an operating system DOS which nevertheless supports some very sophisticated descendants of BASIC. The interrupt and memory handling of DOS makes it prone to "crashing", from which a system re-boot is the only possible recovery -- not a desirable feature in an operational environment. Our experience with OS/2 and the Macintosh operating systems is that they are better, but still not reliable.

DSN Need for Quick Programming Languages

The DSN has unique capabilities in radio astronomy. Because it is primarily a deep space communications instrument, and only about 5% of the antenna time is devoted to astronomy, it is not cost effective to implement it as a national or even major university observatory. Consequently, astronomers are expected either to fit their work within the framework of DSN procedures or to "fend for themselves". VLBI astronomy has largely taken the former approach. Other kinds of astronomy have depended largely on their own equipment and software. This includes key software written in BASIC, either on DOS platforms or under HP-UX on HP 9000 series 300 and 700 platforms.

A control and monitoring system designed to support the non-standard users of the DSN is now under development. The Equipment Activity Controller is designed to be responsive to these users needs to control non-DSN equipment, and to be readily re-configurable so as to support an experimental mode of operation.

In keeping with this spirit, it is essential that experimenters have access to tools which allow them to adapt their software on the spot (or at least very quickly) in response to new ideas or new discoveries.

UNIX Quick Programming Languages

To my knowledge, the only BASIC-like languages that ever made it into UNIX were HP BASIC and COMAL. COMAL is a cross between BASIC and PASCAL, but its UNIX implementation was only as a limited version of the more powerful DOS one. COMAL has since fallen into obscurity and is probably not supported anymore. HP BASIC continues to have strong support, but is platform specific, and has a somewhat limiting interface with C and the underlying operating system. Also, its built-in editor will drive you bananas! However, the main objection to HP-BASIC is that if you don't like some feature, or lack of feature, your only recourse is to petition Hewlett Packard for a software enhancement.

In UNIX, the starting point or model for powerful interpreters was the shell. The two most popular interpreters are perl and Tcl. Both have a substantial following, which means that many useful packages have been developed, are covered by a public license, which means that continued support is not tied to a specific vendor, and are freely available for essentially all UNIX platforms, and some others as well. All the source code is available, so you can tailor them to your specific needs.

The remainder of this section describes the languages and gives enough examples to provide an idea of how to use them. As you will see, each is fully capable of doing what the other can do. So, to first order, the choice of which to use is one of taste. At present, Tcl/Tk is more easily integrated into existing applications which are oriented around a flexible user interface, either command line parser, or GUI, or both. perl is more fully developed as a stand-alone application proto-typing language, recognizing that some applications never leave the proto-typing stage.

Tcl differs from other programming languages is that the basic data type is a character string. This means that every variable or array constains ASCII data, and the conversion to and from numbers for internal calculations is done transparently. Perl stores numbers as C doubles and strings as strings. Conversion is only upon demand, e.g. number interpolated in to a string. If numbers are only used numerically there is no conversion and full precision is maintained. This is one of the reasons Perl is about 100x faster than TCL. Both languages borrow some of the concepts of lisp for handling lists, which are special kinds of strings.

In summary, I think that Tcl/Tk is probably the best choice for developing data acquisition software, while perl is more suited to data analysis tasks.

perl

perl Overview

perl stands for Practical Extraction and Report Language, which may be how it was conceived, but it is now a fully developed programming language. It has a syntax which falls between csh and C, with conceptual borrowings from LISP, Pascal and BASIC. perl can be extended with modules written in C. Packages can also be written in perl. There is provision for data structures, object oriented programming, debugging, inter-process communication based on Berkely sockets, and more. Two final points whose importance will be clearer after the discussion on Tcl are that perl is being extended to allow perl to be called from C. There also is an X-Windows interface using the Tk toolkit.

One entirely valid use for perl is to develop software by a rapid-prototyping method, and then later to convert the code either entirely or in part into C.

perl Example

The best way to make that point is to show an example of a program that scans through a log file, extracts records that have power level information and plots the data as a function of time. Appropriately, it is called strip_chart:
#!/usr/bin/perl
#
# plot total power from DAVOS43 log files
#
The first line invokes the perl interpreter, similar to the way a shell script starts.
use PGPLOT;
This line attaches an independently developed package, PGperl, which provides an interface to the PGPLOT routines popular in astronomy. The code below extracts the data from the data file named on the command line.
if ($#ARGV < 0){
   print "The syntax is:\n\nstrip_chart file_name [device [lval rval]]\n\n";
   print "device = \"?\" will cause a choice of devices to be presented\n";
   print "         /XWINDOW will put the plot on the screen (default)\n";
   print "and lval and rval are optional left and right values for the 
X-axis.\n";
   exit(1);
}

open (FILE,$ARGV[0]) || die "Data file not found";

$min_power = 1e99;
$max_power =-1e99;
while($text_line = ){
   if ($text_line =~ /level/){
      chop($text_line);
      @power_data= split(/[ =]+/,$text_line);
      if (!$first_ut){
	 $first_ut=$power_data[0];
      }
      $min_power = &MIN($min_power,$power_data[2]);
      $max_power = &MAX($max_power,$power_data[2]);
      $x[$i] = &decimal_angle($power_data[0]);
      $y[$i] = $power_data[2];
      $i++;
   }
}
$last_ut=$power_data[0];
if ($#ARGV < 2){
   $begin = &decimal_angle($first_ut);
   $finish = &decimal_angle($last_ut);
} else {
   $begin  = $ARGV[2];
   $finish = $ARGV[3];
}
print("The X-axis goes from ",$begin," to ",$finish,"\n");
$min_p = $min_power/1.;
$max_p = $max_power/1.;
print("The Y-axis goes from ",$min_p," to ",$max_p);

if ($#ARGV > 0){
   $device = $ARGV[1];
} else {
   $device = "/XWINDOW";
}
The following code should look familiar to anyone who has used PGPLOT.
&pgbegin(0,$device,1,1);
&pgscf(2);
&pgslw(4);
&pgsch(1.6);
&pgenv($begin,$finish,$min_p,$max_p,0,0);
&pglabel("UT","Power (dB)","Strip Chart");
&pgsci(5);
&pgpoint($i,*x,*y,1);
&pgend;
And here are some locally defined subroutines. These are generally useful enough that they will eventually go into a separate file, so that they may be called from other scripts.
sub MIN {
   if ($_[0] <= $_[1]){
      return $_[0];
   } else {
      return $_[1];
   }
} 

sub MAX {
   if ($_[1] >= $_[0]){
      return $_[1];
   } else {
      return $_[0];
   }
} 

sub decimal_angle {
   ($ang,$min,$sec) = split(/:/,$_[0]);
   $ang += ($sec/60 + $min)/60;
}

perl Resources

Tcl - Tool Command Language

Tcl Overview

Tcl, which stands for Tool Command Language, was designed as a simple scripting language to be embedded into larger applications. Continued development of TCL as a free "product" will be supported by SunScript, a team at Sun Labs.

Its basic format will be familiar to many who have written simple parsers for entering commands into an instrument control program:

command parameter1 parameter2 ...
This was done so cleverly that Tcl has the full power of a programming language. This includes all the usual control structures, subroutines, file access, an interface to UNIX, etc. Tcl is also extendable, which means that modules (called "toolkits") can be written in C to add more commands to the basic set provided by core Tcl. The most commonly used module is Tk which provides an interface to the X-Window system.

While Tcl/Tk seem to be designed to do exactly what we need in the DSN for controlling instrumentation in a flexible way, it does require the programmer to shift mental gears. Though the syntax is designed to have much in common with shell scripts and C, it is subtly different. The programmer must get used to the syntactical structure, which is much simpler than that of shells or C. Everything, even control structures, are implemented with the command syntax. Once this is understood, the programming rules for Tcl become clear.

Although Tcl was designed to be embedded into larger applications, standalone use is possible through a minimal C program that does nothing more than provide a user interface to Tcl. The program is called tclsh, which stands for "Tcl shell". Like a shell, it can be run interactively, or from a script. The equivalent program for Tcl/Tk is called wish, which stands for "Windows shell". These shells make it very easy (and reportedly not uncommon) to develop entire X-Window applications in Tcl/Tk.

Tcl/Tk Example

The first example shows a simple X-Windows RPN calculator program written entirely in Tcl/Tk using wish.
#!/usr/local/bin/wish -f

# initialize the registers
set t 0
set z 0
set y 0
This well illustrates the over-riding syntactical rule. t = 0 is not a valid statement. The command is set and the remaining items on the line are parameters to set. Notice also that Tcl adheres more closely to the shell convention of naming variables without a leading $ unless their contents are being used (as will be seen below). perl, on the other hand, always prefaces a variable name with $, no matter what the context.

# the main widget is divided into series of horizontal frames
#  the frames and their contents are defined below

frame .t
label .t_label -width 1 -text "T"
label .t_value -width 16 -textvariable t -relief sunken

frame .z
label .z_label -width 1 -text "Z"
label .z_value -width 16 -textvariable z -relief sunken

frame .y
label .y_label -width 1 -text "Y"
label .y_value -width 16 -textvariable y -relief sunken

frame .x
label .x_label -width 1 -text "X"
entry .x_value -width 16 -textvariable x -relief sunken -background white
bind .x_value <Return> {
set t $z
set z $y
set y $x
}
The above code defines a bunch of widgets called frames and their contents. It binds the <Enter> key action for the entry widget .x_value to some Tcl code which pushes values up the RPN stack. Here we see the use of $ when the contents of a variable is being used. This also illustrates how the basic syntax can be so powerful. The section of code between { and } is treated by the parser as one parameter to the command bind, and its interpretation is deferred until later, when the pressing of the <Enter> key in the .x_value widget causes the contents of the fourth parameter to be interpreted. Continuing with the definitions of the widgets...
frame .math
button .plus -text + -command {
set x [expr $x + $y]
set y $z
set z $t
}
button .minus -text - -command {
set x [expr $y - $x]
set y $z
set z $t
}
button .times -text x -command {
set x [expr $x * $y]
set y $z
set z $t
}
button .divide -text / -command {
set x [expr $y / $x]
set y $z
set z $t
}
button .clx -text clrX -command {
set x 0
}
Having defined all the widgets, they must now be arranged in the window.
# arrange the frames vertically (default)

pack .t .z .y .x .math

# arrange the contents of each of the frames

pack .t_label .t_value -in .t \
-side left -padx 2m -pady 1m
pack .z_label .z_value -in .z \
-side left -padx 2m -pady 1m
pack .y_label .y_value -in .y \
-side left -padx 2m -pady 1m
pack .x_label .x_value -in .x \
-side left -padx 2m -pady 1m
pack .plus .minus .times .divide .clx -in .math \
-side left -padx 2m -pady 2m
Like shell scripts, and unlike perl, commands are terminated by <newline> instead of a semi-colon. Lines can be extended by means of a \ at the end, as with shell scripts. However, anything inside braces is actually one parameter, and so is seen by the parser as being on one line. A later invocation of the parser on the material inside the braces will cause the <newline>s there to be interpreted.

Readers who have not previously programmed for the X-Window environment will now have been introduced to a different programming style, which consists mostly declarations of the properties of widgets. The algorithmic coding is used only to describe what should be done in response to some user action on a widget. The term callback is used to describe such code, because the X-Window "calls it back" later. It is a lot easier to see and understand the programming style in this script than in C code which accomplishes the same thing.

Example: extending wish

The following is an example of a simple way of extending the program wish to add a user-supplied function. In this case, the function
char *out_angle(float Angle, int Length)
provides a way of converting a decimal Angle into a string of a specified Length. It is pretty smart about when to use decimal and when to use sexagesimal notation. The interfacing is done here in the main code module for simplicity. The more usual thing, especially when a whole library of C routines is being called, is to make a separate package and to interface the main module to the package.
/*  myWish.c  */

#include 

extern char	*out_angle();

/* The following variable is a special hack that is needed in order for
 * Sun shared libraries to be used for Tcl.  */
extern int matherr();
int *tclDummyMathPtr = (int *) matherr;

/* ----------------------------------------------------------------------
 * main --
 *	This is the main program for the application.
 *
 * Results:
 *	None: Tk_Main never returns here, so this procedure never
 *	returns either.
 *
 *----------------------------------------------------------------------
 */

int main(argc, argv)
    int argc;			/* Number of command-line arguments. */
    char **argv;		/* Values of command-line arguments. */
{
    Tk_Main(argc, argv, Tcl_AppInit);
    return 0;			/* Needed only to prevent compiler warning. */
}

int OutangleProc(ClientData clientData,
                 Tcl_Interp *interp,
                 int argc,
                 char *argv[]          ){
   /* this calls the external routine 'outangle' which formats a decimal
      angle into hexagesimal                                    */

   float angle;
   int   length;

   if (argc !=3) {
      interp->result = "need  and  as arguments";
      return TCL_ERROR;
   }
   /* the arguments passed from the Tcl command are strings, which
      need to be converted to numbers                              */
   sscanf(argv[1],"%f",&angle);
   sscanf(argv[2],"%d",&length);
   interp->result = out_angle(angle,length);
   return TCL_OK;
}
/* ----------------------------------------------------------------------
 * Tcl_AppInit --
 *
 *	This procedure performs application-specific initialization.
 *	Most applications, especially those that incorporate additional
 *	packages, will have their own version of this procedure.
 *
 * Results:
 *	Returns a standard Tcl completion code, and leaves an error
 *	message in interp->result if an error occurs.
 *
 * Side effects:
 *	Depends on the startup script.
 *
 *----------------------------------------------------------------------
 */

int Tcl_AppInit(interp)
    Tcl_Interp *interp;		/* Interpreter for this application. */
{
    Tk_Window main;

    if (Tcl_Init(interp) == TCL_ERROR) {
	return TCL_ERROR;
    }
    if (Tk_Init(interp) == TCL_ERROR) {
	return TCL_ERROR;
    }

    /*
     * Call Tcl_CreateCommand for application-specific commands, if
     * they weren't already created by the init procedures called above.
     */
    Tcl_CreateCommand(interp, "outangle", OutangleProc,
                     (ClientData) NULL,
                     (Tcl_CmdDeleteProc *) NULL       );


    /*
     * Specify a user-specific startup file to invoke if the application
     * is run interactively.  Typically the startup file is "~/.apprc"
     * where "app" is the name of the application.  If this line is deleted
     * then no user-specific startup file will be run under any conditions.
     */

    tcl_RcFileName = "~/.wishrc";
    return TCL_OK;
}
This has been tested with Tcl 7.4 and Tk 4.0. The relevant Makefile entries are
myWish:	myWish.o
	cc myWish.o /usr/local/src/io/outangle.o \
	-L/usr/X11/lib -ltk -ltcl -lX11 -lm -o myWish

.c.o:
	cc -c -I/usr/X11/include $*.c
This new version of wish, called myWish allows the Tcl/Tk script above to be extended by adding a button which converts the number in the X-register to a formatted string with a length which is given by the contents of the Y-register. The changed or extra lines in rpn are
#!./myWish -f

   . . . 

frame .functions1
button .to_sexagesimal -text "-> d:m:s" -command {
   set x [outangle $x $y]
}
   . . . 

pack .t .z .y .x .math .functions1

   . . .

pack .to_sexagesimal -in .functions1 \
      	-side left -padx 2m -pady 1m 	

Example: Embedding Tcl

In order to embed tcl in another program, one simply calls on the parser/interpreter directly, instead of invoking the library user interface Tk_Main or Tcl_Main. The following is a simple example of a user program which calls on Tcl to interpret the contents of a file:
/* simple.c -- Tcl application to evaluate script file.  */

#include 
#include 
#include 

main(int argc, char *argv[]) {
   Tcl_Interp *interp;
   int        code;

   /* check that there are two items only on the command line */
   if (argc != 2) {
      fprintf(stderr, "Wrong # arguments: ");
      fprintf(stderr, "should be \"%s filename\"\n", argv[0]);
      exit(1);
   }

   /* create a new interpreter and return the token for it */
   interp = Tcl_CreateInterp();

   /* evaluate the script in argv[1] with interpreter 'interp' */
   code = Tcl_EvalFile(interp, argv[1]);

   /* print the completion code if there is an error */
   if (*interp->result != 0) {
      printf("interpreter error: %s\n", interp->result);
   }
   if (code != TCL_OK) {
      exit(1);
   }
   exit(0);
}
To compile this program do
cc simple.c -ltcl -lm -o simple

Tcl/Tk Resources