RC & ES ======= Review of Bourne shell syntax (similar to lab 1) command | stdout-pipe command < stdin-file command > stdout-file command >> stdout-append-file command 2> stderr-file command 2>> stderr-append-file command 3< input-for-fd-3 command 2>&1 # fd 2 output to fd 1 exec REDIRECTION # with no command, set redirection for current shell ; # separate commands & # separate commands, run last in background &&, || # short-circuit boolean operators var=value # assign variable var=value cmd # assign environment variable just for execution of cmd $var, ${var} # use variable "blah $var" # quote with variable expansion (blah value) 'blah $var' # quote without variable expansion (blah $var) \' # \ quotes next char except within single quotes (') << # "here document" `command` # substitute command output for argument (echo `ls`) ?, *, [...] # globbing $* # List of all arguments $@ # Same as $@, except "$@" expands to multiple arguments $n # access nth element of $* $# # size of $* shift # pops front of Simple parg Bourne shell test program: #!/bin/sh echo "Received $# arguments" i=1 while test "$#" -gt 0; do echo "ARG$i: $1" i=`expr $i + 1` shift done What's wrong with the Bourne shell syntax? Grammar very complicated Attempted specifications did not admit "who | wc" Recursive descent parser, but with flags that subtle change parsing Bourne shell re-scans input, which is dangerous? Example: foo='echo 1 2 3' $foo First scans '$foo' Then expands into 'echo 1 2 3' Then uses IFS to split into 'echo', '1', '2', '3' Why is this bad? Makes it extremely error-prone to pass around arguments with spaces touch x touch 'x y' foo=x\ y rm $foo rm: y: No such file or directory # oops, just deleted x Quoting confusion 4 different types of quoting Recursive use of backtick (`) very confusing Must use # of \ escapes exponential in depth of nesting rc paper cites security hole in popen. What is this? set IFS=/ in environment; place program bin in path if setuid program runs popen ("/bin/sh", "r"), gets "bin" instead of "ls" Signal handlers are strings trap 'echo hi' 2 # ok trap 'exti 0' 2 # syntax error, won't be caught until parsed No support for non-traditional pipelines E.g., how do you compare output of two programs What are the design principles behind rc? Not a "macro processor" -- what does this mean? Never scan input more than once (except for eval) Why/how does this lead to a change in the types of variables? Still want to pass around commands, just want them in parsed form So variables are lists of strings, not just strings rc lists Uninitialized variables all consist of the empty list () How to specify a list? foo=(a b c); ./parg $foo foo='a b c'; ./parg $foo # sets foo to be single-element list How to access elements of list? $foo(1) - first element of list $foo(2 1 2) - get multiple elements out of list $#foo - number of elements in list What does '^' operator do? Concatenates lists $foo^$foo - concatenates lists pairwise -> aa bb cc $foo^.c - concatenates string .c to each element -> a.c b.c c.c (very useful for things like makefiles) ^ is implicit between certain character classes Note: can use whatis to examine variables How does rc solve other problems with Bourne shell? Backtick quoting problem? Uses `{ ... } - asymmetric open/close operators Quoting mess? Only one type of quoting (') Non-traditional pipelines? Can redirect from a command: cmp <{command one} <{command 2} How is this implemented? Must use named pipes or file descriptors What would you expect from: echo <{echo one} <{echo two} /dev/fd/3 /dev/fd/4 - Uses /dev/fd on OpenBSD (plan9-inspired) Signals? Define handlers as real functions What is ~ command? Allows matching against patterns ~ subject pattern1 [pattern2 ...] True if subject matches any of the patterns Note that subject can be string, or can be list (in which case any match ok): ~ $foo c && echo yes What would you expect for this? bar='*' ~ xxx $bar && echo yes (empty) eval ~ xxx $bar && echo yes (echos yes) Remember, rc doesn't re-scan input, and this goes for glob chars, too Go over man example on page 13 rc sounds good--why do we need es? Incorporate real programming language ideas First class functions, lexically scoped variables Implement most of shell with itself Small number primitives hard-coded in shell (echo <=$&primitives) Most shell functions desugared into overridable builtin functions true && echo yes var fn-%and fn %and { echo you just executed the and command } true && echo yes As a general principal, implementing system within itself is very powerful Means users have as much power to customize as you had when building shell We will revisit this principle in 3 lectures when we talk about exokernel What is the difference between a function return value and output? Consider: fn hw { return 'hello world' }; hw What is expected output? (nothing) How to see output? echo <={hw} (note paper said <>, but now is <=) Why make distinction? Why not echo hello and use `{...}? Point is you can return objects other than strings... like functions First note that @ introduces an "anonymous function" (lambda expression) var fn-hw fn-hw = '@ * {return ''hello world''}' Can run hw, or can run $fn-hw -- same thing General syntax is @ arg1 arg2 ... { function body } If called with more arguments, last arg is a list Note by default, all arguments go into $* list which is why hw starts '@ *' even though declared with no args Look at cons example on p. 56. What's going on here? cons A D returns function that takes a function f, runs f A B car A D returns A; cdr A D returns D Lexically vs. Dynamically scoped variables See binding example on p. 55 Explain let example on pp. 59-60 Why do you need %closure construct? What would happen without this? Look at figure 2 (path caching) on p. 58. How does this work? How is es mismatched to Unix interface? ES allows you to return any type from a function, including another function Unix only allows a return value that is a number from 0-255 Exceptions also cannot propagate across processes Might want to throw an exception from a sub shell; can only exit 0-255 Lexically scoped variables don't work right across processes Again, consequence of Unix fork. How might plan9 fix this? (flags to rfork could allow you to keep same environment) Lists are not hierarchical (because can't pass list to exec)