Subversion Repositories or1k

This file is the NEWS file from the Expect distribution.

======================================================================

======================================================================

Date: 8/18/96

Expect now works with Tcl 8.0.

No changes were made to take advantage of 8.0 features such as

namespaces.  (If you want to put the Expect commands in a namespace,

declare a namespace before loading them in.)

Even thought Tcl allows embedded nulls in commands, Expect still does

not.  Tcl still doesn't support embedded in patterns and regexps.

I'll wait til Tcl supports that before rewriting Expect's null

support.

======================================================================

======================================================================

Date: 9/28/96

There is now an Expect FAQ and home page.  Feedback is encouraged.

You can get to the FAQ from the home page and vice versa, so you only

need to remember one of the links, but here are both for completeness:

home page: http://expect.nist.gov

FAQ:       http://expect.nist.gov/FAQ.html

======================================================================

======================================================================

This section describes the changes to Expect 5, many due to changes

from Tcl 7.4 to 7.5.

Improvements

==============================

You can now use expect_background from Tcl along with all the

Tcl-based event stuff such as "after".  You can also do fun things

such as have "after" actions run while expect is waiting for input.

It's worth comparing the difference between commands such as expect

(which waits in the event loop) and gets (which doesn't).

Incompatibilities

==============================

libexpectk is gone.  Because of the event loop was moved into Tcl,

this is no longer necessary.  Just use the regular Expect library.

This only affects you if are hand-linking.

The name of the static C library now has the extension on the end -

just like Tcl.  This only affects you if are hand-linking.

==============================

Some obvious questions and answers

Nothing in the user interface has changed.  All your old scripts

should run.  Tcl now has commands that replace Expect functionality in

the area of version control and date handling.  I encourage you to use

Tcl's support rather than Expect's for any new scripts that you write.

However, I won't be removing Expect's commands, so don't worry about

having to convert old scripts.

It is my understand that shared/dl libs work.  (I say "it is my

understanding", because my own environment doesn't handle it, sigh.

My system admins tell me that they're working on it.)  So I've had to

guess on some things - in fact, I've talked to a number of people and

I get the feeling that a lot of people are guessing on shared/dl libs.

I have yet to talk to anyone that REALLY understands this stuff (by

"understand", I mean "can write the configure/Makefile portably and

correctly".  So much for my griping.  In theory, the shared/dl support

is pretty much a freebie because Tcl itself provides all the support

for this.  (There is some reorganization that could be done to improve

shared library memory use - I'll get to it eventually - it shouldn't

affect most people.)  Don't send me complaints about shared/dl libs

unless you are *positive* it is something that I am responsible for.

Even if Tcl works and Expect fails, it is likely to be a Tcl error (or

more precisely, a configuration problem that is more appropriately

fixed from the Tcl distribution).

For Tcl-package purposes, Expect is called "Expect".  (Duh...)

Expect's multiple interpreter support is minimal.  It should work for

most things, serendipitously.  I haven't spent any time making this

perfect.

No, this release isn't portable to Windows or Mac.  Let me know if

you're seriously interested in a lot of work.  I'm not saying it's not

possible.  It's definitely possible and the porting working at Sun has

made it easier than before.  But it's still not a weekend hack.

Industrial support for the NT port, would be very helpful.  If you are

interested, either as a student or an industrial sponsor, let me know.

==============================

Building the code

==============================

Expect builds as usual.  (See the INSTALL file for details.)

The only significant change is that Expect now has to find the

tclConfig.sh file (and tkConfig.sh if you want).  So if you like to

store Tcl where Expect can't find it, you'll need to use even more

configure flags than you used to.

Shared/dl Expect libraries are built if you configured Tcl for

shared/dl libraries.

All support for earlier versions of Tcl and Tk have been removed from

Expect.  For example, if you're still using Tcl 7.4 (or earlier),

don't bother to install this release.

======================================================================

======================================================================

This section describes the changes from Expect 4 to Expect 5.

The changes that people will find most interesting or annoying are as

follows.  Some of them may seem rather arbitrary but fix inconsistencies

leading to much cleaner coding both internally and externally.

-- Expect version 5.20 and above is designed to work with Tcl 7.5 and

Tk 4.1.  Expect 5.20 and above will not work with earlier versions.

-- Glob-style patterns do longest-possible matches (from the earliest

possible position) just like regexps.  Previously, they matched the

shortest-possible strings.  However, the documentation didn't actually

say this (it didn't say anything)

-- Exact patterns are now supported from expect.  Use the "-ex" flag.

Exact patterns work just like those in interact.  No special handling

is made of *, ^, etc.

-- The new command "expect_background" registers patterns that are to

be tested against spawned process output whenever it appears (i.e.,

asynchronously).  This only works in the Tk environment.  The

arguments are the same as the expect command.

-- expect_before and expect_after now handle their arguments like

expect_background.  Previously, a command such as "expect_before"

with no arguments deleted patterns for all spawn ids.  Now, it only

deletes patterns for the current spawn id.  Similarly with the "-i"

form.

-- expect_background/before/after support an -info flag to query what

the current patterns are.  The results are returned in such a way that

they can be re-used by a new expect command.

The -info flag must be the first flag in the command.  With no other

arguments, it returns the setting for the current spawn id.  With a -i

descriptor, information is returned for that spawn id.  The argument

-noindirect may be used to suppress indirects which also match a

direct spawn id.  Only a single -i specification may be given with

-info.  With the argument "-all", all spawn id specifications are

reported.

-- There is now a sleep command.  It understands decimal values such as

        sleep .5

Interrupts and other asynchronous events are processed while Expect sleeps.

-- Traps now use Tcl's "Async" support.  This has advantages and

disadvantages.  One advantage is that traps have no chance of screwing

up the Tcl internals.  One disadvantage is that trap handlers are

delayed at certain specific times and places.  For example, a handler

cannot occur inside another handler.  While one handler is running,

all other handlers are blocked.  This is probably the most noticable

place where handlers are blocked.  Others are generally small windows,

so you shouldn't notice the delay in executing the handlers.

Several traps are initially defined:

        trap exit {SIGINT SIGTERM}

If you use the -D flag to start the debugger, the following trap is

defined:

        trap {exp_debug 1} SIGINT

You can, of course, override these.  In particular, if you have your

own "trap exit SIGINT", this will override the debugger trap.  Because

SIGINT is tied to exit (see above) by default anyway, you should

remove your own "trap exit SIGINT" unless you specifically do not want

to be able to get to the debugger by ^C.

If you want to define your own trap on SIGINT but still trap to the

debugger when it is running, use:

        if ![exp_debug] {trap mystuff SIGINT}

Alternatively, you can trap to the debugger using some other signal.

The ONEXIT trap is no longer available.  Instead, say "exit -onexit ..."

Traps are now deleted by using the empty ({}) handler.  The current

handler is returned if no action is supplied.  With no arguments, trap

returns the signal number of the trap command currently being executed.

-- The wait command now returns a four element list if a valid child

was waited on.

Element 1: pid

Element 2: spawn id

Element 3: 0 (or -1 if there was an OS error)

Element 4: status (or errno if element 3 == -1)

-- expect and interact notes:

The timeout and eof patterns were initially named "-timeout" and

"-eof" but have been renamed "timeout" and "eof" to match those of

expect.  The ability to define default timeout/eof actions has been

removed.  (You can do this more easily by grouping spawn ids.)

expect and interact now support a "null" keyword to match an ASCII 0.

send supports -null and -break keywords.

Since a large number of special keywords have been added to interact,

a new keyword "-ex" for "exact" was added descriptive of its default

treatment of patterns.  This protects the next token from being

misinterpreted as a keyword.  The expect command provides "-gl" for

"glob" for analogous reasons.

Any string starting with "-" should be protected by the "-ex" or "-gl"

flag, even those that are not keywords currently.  (All strings

starting with "-" are reserved for future options.)

String start/end indices are no longer written to expect_out and

interact_out unless the -indices flag is given.

expect_out(spawn_id) is set to the spawn id associated with the spawn

id that produced the last output in an expect command.  For example,

you can use this to delete files that have closed, by removing this

element from an indirect spawn ids spec.  The same effect is

reproducable with interact (and interact_out(spawn_id)) but for

efficiency reasons, it requires the -iwrite flag before each pattern.

Expect's -i and interact's -i, -u, -input, and -output flags can now

describe a list of spawn ids.  So you can say things like:

        interact -input "$id1 $id2 $id3" .... -output "$id1 $id2" ...

In this case, id1, 2, 3 would be sent to id1, and 2.

The spawn id may be given as a global variable name (called an

"indirect spawn id specification"), in which case, the variable

contains the list of spawn ids.  Whenever the variable is changed, the

new list of spawn ids is automatically used.  This is particularly

useful with any long running expect command such as expect_before,

expect_after, expect_background, and interact.

The -update flag was removed.  Use indirect spawn ids (see previous

paragraph).

-- interact notes:

Interact now support -input and -output flags that provide very

flexible means of moving data from/to multiple spawn ids in complex

ways (but very quickly).  It is possible to write most expect loops

using a simple interact statement.  For instance, the three way

interaction inside kibitz (between two users and a process) is written

this way:

        interact {

                -output $shell

                -input $userin eof { . . . } -output $shell

                -input $shell -output "$user_spawn_id $userout"

        }

-- send command notes:

It is possible to send a break by using the "-break" flag.

Any string starting with "-" should be protected by preceding it with

the "--" flag, even those that are not keywords currently.  (All

strings starting "-" are reserved for future options.)

-- The spawn command now takes an "-open" flag which in turns takes a

Tcl file as an argument.  This lets you treat raw devices, files, and

pipelines as spawned processes without using a pty.

This was actually in Expect 4, but I forgot to document it.  Oops!

-- The old "debug" command (which describes what Expect is doing

internally) was renamed "exp_internal".  "debug" (and "exp_debug") now

invoke the interactive debugger.

-- The new command "stty" now takes over the job of "system stty".  It

works much better, allowing POSIX-style redirection to affect other

ttys.  It otherwise takes arguments as "system stty" did.

-- The "-tcl" option to "return" has gone away.  (This was dangerous

to anyone that actually happened to return the value "-tcl".)

Instead, use inter_return.

-- Added exp_timestamp command to produce very fast timestamps.

-- Added exp_pid command to return pid of given spawn id.

-- The close command now takes an argument of "-onexec" with a following

current spawn id from being closed when another process is exec'd or

spawn'd.

        close -onexec 0

While "-onexec 1" returns it to the default condition where it will be

closed upon exec or spawn.

-- log_user now returns previous value.  It is acceptable to call now,

without arguments just to get the value.

-- The following forms are deprecated.  They will be allowed

indefinitely but not advertised or supported if they break.

        -eof, -timeout in interact (reason: didn't match expect.

                Instead, use eof or timeout.)

        -- in expect or interact (reason: no easier to read.

                Instead, use -gl in expect or -ex in interact.)

        continue -expect (reason: when mixing in extensions, you have

                to use exp_continue, so -expect becomes irrelevant.

                Instead, use exp_continue.)

        getpid (reason: Tcl now supplies same functionality as "pid".

                Instead, use pid.)

        expect_version and expect_library (reason: the name implies

                they have something to do with the expect command,

                which they doesn't.

                Instead, use exp_version and exp_library.)

        -timestamp for obtaining tm and ctime in expect and interact

                (reason: separate command now exists for this purpose.

                Instead, use exp_timestamp.)

        system stty (reason: bad interaction with redirection.

                Instead, use stty.)

-- New examples have been added:

"dislocate" lets you disconnect and reconnect to processes.

"tkpasswd" illustrates passwd embedded in a GUI.

They may not be overwhelmingly useful, but run them once to see what

they do.  If you ever need to do anything similar, you can look back

at them.

"tknewsbiff" pops up a window or plays a audio clip when you have

unread news.

-- Changes to the Expect libraries:

The expect-tcl library (libexpectcl.a) has been integrated with the

expect library (libexpect.a).  So references to libexpectcl.a should

be removed.

The Expect C library now supports buffering, multiplexing, null

matching, full buffer matching.  Basically, all of the features in

Expect are now in the library.

Buffering and multiplexing has caused the biggest change to the

library.  Previously, exp_match contained the entire buffer that

matched.  Now exp_match just points to where in the buffer the match

started.  exp_buffer points to the beginning of the buffer.

Previously, the previous buffer was thrown away at the beginning of

each expect function call.  Now, previously unmatched characters are

eligible for matching.

To match on different file descriptors, exp_match, exp_match_end,

exp_buffer_end must be restored to their previous values.  Initially,

they should be zero.

The meaning of timeout == 0 in the Expect library has been changed.

See the man page for more info.

======================================================================

======================================================================

This file describes the changes from Expect 3 to Expect 4.

The improvements that people will find most interesting are:

1) Expect version 4 is designed to work with Tcl 6.7 and Tk 3.2.

   (Earlier versions of Expect will not work with Tcl 6.7)

   Expect can now be layered in with any Tcl program.

   Note that in Tk, Expect's send command is called "exp_send".

   (It used to be called "send_spawn" but this bit the dust.)

2) A debugger is provided.

3) The interact command has been totally rewritten and supports regular

   expressions, timeout/eof patterns, and a number of other new things.

4) The default behavior of ^C (SIGINT) is exit whether or not you are in

   a read.

5) Expect uses "sane" terminal parameters by default, allowing scripts

   to work the same whether inside emacs shell mode or not.  (See man

   page on "spawn" for more info.)

6) All the hard parts of the installation process are automated.  This

   was done primarily by Rob Savoye at Cygnus.  Thank you, Rob!

7) It is now possible to buy a support contract for Expect from Cygnus.

The changes that people will find most annoying are:

1) send now only sends a single string.  (It used to send any number of

   strings with spaces jammed in between.)

2) getpid was renamed pid.

3) interact's -flush was renamed -nobuffer (much more descriptive).

4) interact now runs all actions in raw mode unless the flag -reset

   is used.  -f and -F are ignored.  send automatically understands

   how to do the right thing.  The most likely thing to watch out for

   are actions like "exec kill -STOP 0" which almost certainly need

   the -reset flag.

5) argv0 is initialized to script name.  argv no longer contains it.

   argc is initialized [llength $argv].  This follows new Tcl style.

All differences are described in the man page.  Some of the less

significant differences are described in the HISTORY file.  The

debugger is described in a separate document (see the README).

This version also introduces one incompatibility that may require

changes to scripts.  While this may initially annoy you, the final

result will simplify the process of writing scripts.  Namely:

In version 3, the expect command accepted lists of glob-style patterns

such as:

        expect "a\ b c" action

where "a b" or "c" would cause action to be executed.  The problem

with this is that the pattern list is hard to write and hard to read.

Patterns with control-characters, backslashes and dollar signs were

very difficult to deal with.

Regular-expression patterns provide a much simpler solution.  Via the

alternation feature (using a "|") the above pattern can be written as:

        expect -re "a b|c" action

I was concerned about people having a significant investment in code

that depended on the old syntax but responders to a comp.lang.tcl poll

about such a change in pattern handling were 100% in favor of it.  (I

even proposed hooks for backward compatibility, but there was no

interest in it.)

Fortunately, most simple things will work as before including:

        expect foobar

        expect {foobar}

        expect "foobar"

        expect "foo\ bar"

However, some things won't work as before.  For example, the following

will behave differently - now the braces will be considered as part of

the pattern.

        expect "{foo bar}"

Here are examples of patterns in my own code that I had to change:

        was                             changed to

        Version 3 pattern list          Version 4 pattern

        {Whois:\ }                      "Whois: "

        {No\ match}                     "No match"

        {250*ftp>* 200*ftp>*}           -re "2(5|0)0.*ftp>.*"

        {{Press Return to continue*}}   "Press Return to continue*"

        {*\r\n*\\\\\r\n}                "\r\n*\\\r\n"

Future Change Alert

John Ousterhout has pre-announced a future change in Tcl that may

affect you.  In particular, backslash sequences other than those

explicitly listed in the Tcl documentation will be handled as if the

backslash was not present.

The likely place this arises is when quoting characters that are

special to the pattern matcher but not to Tcl.

For example in Tcl 6.7, the following command matches a period.

        expect -re "\."

In Tcl 7.0, it will match any character, because Tcl will throw away

the backslash.  If you want to match a period, you will have to say:

        expect -re "\\."

or

        expect -re {\.}

The following command will find occurrences of this.  (It may find

other things, but it will at least find the problem cases.)

        egrep '(\\$)|(\\[^][bfnrtv\0-9{}$ ;"])' *.exp

======================================================================

======================================================================

This section describes the changes from Expect 2 to Expect 3.

If you used to use Expect version 2 (any version written before

September '91) you will find that the current version of Expect (3)

introduced minor but significant incompatibilities.

The HISTORY file describes these briefly.  They are described at

length in the man page.

I'm sorry if you feel annoyed at the incompatibilities, but Expect has

been out for a year and a half, Tcl even longer.  Both Tcl and Expect

are using this as a last chance to make significant changes, so that

we will not disturb even more users in the future.

There is no automated conversion procedure (although see note below)

for Expect or even raw Tcl.  For now, I suggest that you not bother

fixing things that already work - just keep the old Expect around.

The binary isn't very big after all.  If you do write a translation

script, let me know.  Thanks.

Of course, I felt obligated to convert the examples distributed with

expect.  I did this by hand while writing the new version itself,

partly as an aid but mostly to test lots of variations.  In 90% of the

scripts, all I had to do was change:

(changes due to Tcl)

        'index' to 'lindex'

        'range' to 'lrange'

        'length' to 'llength'

        'print' to 'send_user' or 'puts' depending on how you use it

        'function .... c' with '[join [function [split string ""]] ""]'

(changes due to Expect)

        'expect_match' to 'expect_out(buffer)'

        'set match_max' to 'match_max' (perhaps with -d flag)

        '*' to '-re .+'

If anyone wants to write a script to do this, note the pitfalls:

1) functions and variables do not share the same namespace, so it is a

inappropriate to just globally rename things.

A number of optimizations can be made:

1) If you are doing multiple split/joins, you should probably cache the

split string.

2) Virtually all uses of scan are unnecessary now, due to exec's automatic

stripping of terminating newlines, and expect's support of regexps.

3) String manipulation can be reduced or avoided entirely if you use

expect -re.

4) exec is no longer necessary to retrieve environment variables, since

they can now be retrieved from $env.

5) If you have been really anal about testing for timeout and eof, you

can dramatically reduce the size of your scripts by using expect_before

and expect_after.  This is more efficient, as well, since those actions

are only parsed once.