Unix Shell Built-in Commands(3)

1. set:
script_1:

 #! /bin/bash  

 #without any option, set will output all shell variables in current environment  
 set  
 #output:  
 #BASH=/bin/bash  
 #BASHOPTS=cmdhist:complete_fullquote:extquote:force_fignore:hostcomplete:interactive_comments:progcomp:promptvars:sourcepath  
 #BASH_ALIASES=()  
 #BASH_ARGC=()  
 #BASH_ARGV=()  
 #......  

 #with only -o option, it will output all shell option's settings in current environment  
 set -o  
 #output:  
 #allexport       off  
 #braceexpand      on  
 #emacs         off  
 #errexit        off  
 #errtrace        off  
 #functrace       off  
 #hashall        on  
 #histexpand       off  
 #history        off  
 #ignoreeof       off  
 #interactive-comments    on  
 #keyword        off  
 #monitor        off  
 #noclobber       off  
 #noexec         off  
 #noglob         off  
 #nolog         off  
 #notify         off  
 #nounset        off  
 #onecmd         off  
 #physical        off  
 #pipefail        off  
 #posix         off  
 #privileged       off  
 #verbose        off  
 #vi           off  
 #xtrace         off  
 #with only +o option, it will output all shell option's settings in current environment  
 #in a different compared to "-o" option.  
 #-o means "enable", +o means disable  

 set +o  
 #output:  
 #set +o allexport  
 #set -o braceexpand  
 #set +o emacs  
 #set +o errexit  
 #set +o errtrace  
 #set +o functrace  
 #set -o hashall  
 #set +o histexpand  
 #set +o history  
 #set +o ignoreeof  
 #set -o interactive-comments  
 #set +o keyword  
 #set +o monitor  
 #set +o noclobber  
 #set +o noexec  
 #set +o noglob  
 #set +o nolog  
 #set +o notify  
 #set +o nounset  
 #set +o onecmd  
 #set +o physical  
 #set +o pipefail  
 #set +o posix  
 #set +o privileged  
 #set +o verbose  
 #set +o vi  
 #set +o xtrace  

2. Enable or Disable shell option
terminal:
-o will enable emacs shell option
+o will disable emacs shell option

 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ set -o emacs  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ set -o | grep emacs  
 emacs         on  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ set +o emacs  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ set -o | grep emacs  
 emacs         off  

3. $- : currently enabled shell options
terminal:
After disabling "C" option, $- lack C inside.

 
aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ set -C
aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ echo $-  
 himBCH  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ set +C  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ echo $-  
 himBH  

-C is equal to -o noclobber:

 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ set -o noclobber  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ echo $-  
 himBCH  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ set +o noclobber  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ echo $-  
 himBH  

each long form shell option(allexport, braceexpand, emacs, etc.) has a corresponding short form shell option(noclobber mapped to C)

4. Change positional parameters
./script_1:

 #! /bin/bash  

 echo $#  
 echo $1 $2  

 # use "--" to end option then change the positional parameter  
 set -- Hello amazing world  

 echo $#  
 echo $1 $2 $3  

terminal:

 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ ./script_1 my god  
 2  
 my god  
 3  
 Hello amazing world  

Unix Shell Built-in Commands(2)

1. alias unalias
./script_1:

 #! /bin/bash  
 func()  
 {  
   echo Hello world!  
   alias l='echo Hello world!'  
   return 0  
 }  

 func  
 l # it will output the "l: command not found"  
 #seems like we are not allowed to use alias at shell script  
 #and only usable at the shell  

terminal:

 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ ./script_1  
 Hello world!  
 ./script_1: line 11: l: command not found  

==================================
Use alias at the terminal:
We are setting up l to represent "ls -lrt" in this case.
But note: such alias only works in curr shell, which is current process. If you open a new process(new shell), it won't work!

 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ alias l="ls -lrt"  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ l  
 total 40  
 -rw-rw-r-- 1 aubinxia aubinxia 8453 May 17 11:52 backup~  
 -rw-rw-r-- 1 aubinxia aubinxia  53 May 18 18:55 text~  
 -rw-rw-r-- 1 aubinxia aubinxia  0 May 20 22:33 backup  
 -rw-rw-r-- 1 aubinxia aubinxia  35 May 24 10:34 text  
 -rw-rw-r-- 1 aubinxia aubinxia  12 May 24 11:16 script?  
 -rw-rw-r-- 1 aubinxia aubinxia  12 May 24 11:16 script2  
 -rw-rw-r-- 1 aubinxia aubinxia  34 May 24 16:02 out  
 -rwxrwxr-x 1 aubinxia aubinxia  34 May 25 12:15 script_1~  
 -rwxrwxr-x 1 aubinxia aubinxia 234 May 25 12:23 script_1  

==================================
source file's function to set up alias:
./script_1 file is same as above
terminal:
1) First command run ./script_1, but unfortunately, alias doesn't work in script
2) Second command is trying to import ./script_1 to current shell, which make:
func() becomes one shell function!
And run "func" and "l" which output "Hello world!" separately!
3) After importing script_1, alias is already set up, we can use "l" to print out "Hello world!"
This example indicate that alias could only work in shell, and source just import everything from the ./script_1 and make them run at the current shell.

 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ ./script_1  
 Hello world!  
 ./script_1: line 11: l: command not found  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ source ./script_1  
 Hello world!  
 Hello world!  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ l  
 Hello world!  

==================================
Use unalias to remove all alias:

unalias -a: can remove all alias in current process's environment
terminal:

 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ alias l="echo hello world"  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ l  
 hello world  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ alias m="echo amazing world"  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ m  
 amazing world  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ unalias -a  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ l  
 ml: command not found  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ m  
 m: command not found  

===================================
Use unalias to remove single specified alias:

 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ alias l="echo Hello world!"  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ alias m="echo amazing world!"  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ l  
 Hello world!  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ m  
 amazing world!  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ unalias l  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ l  
 l: command not found  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ m  
 amazing world!  

2. umask
umask affect the default permission when creating file or folder in unix
terminal:

 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ umask 0  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ echo Hello >t0  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ umask 777  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ echo Hello >t777  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ umask 1  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ echo Hello >t1  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ umask 2  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ echo Hello >t2  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ umask 3  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ echo Hello >t3  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ umask 4  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ echo Hello >t4  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ umask 8  
 bash: umask: 8: octal number out of range  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ umask 10  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ echo Hello >t10  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ ls -lrt t*  
 -rw-rw-rw- 1 aubinxia aubinxia 6 May 25 13:42 t0  
 ---------- 1 aubinxia aubinxia 6 May 25 13:42 t777  
 -rw-rw-rw- 1 aubinxia aubinxia 6 May 25 13:42 t1  
 -rw-rw-r-- 1 aubinxia aubinxia 6 May 25 13:42 t2  
 -rw-rw-r-- 1 aubinxia aubinxia 6 May 25 13:42 t3  
 -rw-rw--w- 1 aubinxia aubinxia 6 May 25 13:42 t4  
 -rw-rw-rw- 1 aubinxia aubinxia 6 May 25 13:43 t10  

From above examples, we can see that the number specified by umask will turn off the corresponding bit. But the execution permission is one exception, it is always off when creating the file.
Note: umask has to provide one octal number, so "umask 8" is wrong, the correct way is "umask 10".

Unix Shell Built-in Commands(1)

1. Command look-up Order:
Special Built-In Command: break, eval, continue, exit export, readonly, set etc.
Shell Functions: function defined by user at the shell
Regular Built-In Command: read cd umask command jobs etc.
External Commands listed in $PATH

2. Define one Shell function:
terminal:

 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ test() {  
 > echo Hello world  
 > }  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ test  
 Hello world  

3.Shell function can be re-defined and override
terminal:
We re-define chdir after the first definition, then run chdir, chdir will follow the 2nd definition.

 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ chdir() {  
 > echo "$@"  
 > }  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ chdir ..  
 ..  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ chdir ../..  
 ../..  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ chdir ~/Desktop/xxdev  
 /home/aubinxia/Desktop/xxdev  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ chdir() {  
 > echo "$@"  
 > cd "$@"  
 > }  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ chdir ../..  
 ../..  

4. A more complicated shell function
terminal:
In this shell function, we firstly use "cd" to enter the provided path, and PS1 controls the "look" of shell's prompt. "PS1="${x##*/}\$" means,  it will look for the longest pattern "*/" from the beginning of $x. If found, then remove that pattern and return the remaining part of the variable. \$ means treat $ literally.
So the entire statement means, remove front part of $x until the /, only leave the last path name, add a $. This is the prompt we want!

 aubinxia@aubinxia-fastdev:~$ chdir() {  
 > echo "$@"  
 > cd "$@"  
 > x=$(pwd)  
 > PS1="${x##*/}\$"  
 > }  
 aubinxia$chdir ./Desktop  
 ./Desktop  
 Desktop$chdir ./xxdev  
 ./xxdev  
 xxdev$  

5. Override regular built-in function
Since the command look-up order is: special built-in command, shell function, regular built-in command, and external commands. We can define a shell function with the same name from one regular built-in command, then we can override that regular built-in command!

terminal:

 aubinxia@aubinxia-fastdev:~$ cd() { echo "$@"; cd "$@"; x=$(pwd); PS1="${x##*/}\$ "; }  
 aubinxia@aubinxia-fastdev:~$ cd Desktop  

In above example, we override the regular built-in function "cd". And the last command "cd Desktop" will call our own cd function. But the problem is: inside the definition of cd function, we call "cd "$@"", which makes it call own own cd function recursively! So after calling the last command, we are entering one infinite loop.

6. Use "command" to call built-in function
"command" forces the shell to look up the command with the order : special  built-in command, regular built-in command and external function in the list of $PATH

The correct override cd function:
terminal:
The only difference here is using "command" to prefix "cd "$@"". Then shell will look up the "cd" function from the built-in command list.

 aubinxia@aubinxia-fastdev:~$ cd() { echo "$@"; command cd "$@"; x=$(pwd); PS1="${x##*/}\$ "; }  
 aubinxia@aubinxia-fastdev:~$ cd Desktop  
 Desktop  
 Desktop$ cd xxdev  
 xxdev  
 xxdev$   

7. "command" with -p
./script_1:

 #! /bin/bash  
 echo "Hello world! script_1 is running!"  

terminal:
As mentioned above, command will use look-up order: special built-in command, regular built-in command, and external commands listed inside the $PATH. We can use -p to provide an override $PATH value, then command will tell shell to look up command at the provided $PATH place.

1) First command, it recognize script_1 because we provided the path
2) Second command, it doesn't recognize script_1 because we don't provide the path
3) Third command could recognize script_1, because we use -p to override the $PATH, and let command to search command from here.

 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ command ./script_1  
 Hello world! script_1 is running!  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ command script_1  
 script_1: command not found  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ command -p . script_1  
 Hello world! script_1 is running!  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$   

8. Special built-in command feature:
./script_1:

 #! /bin/bash  
 var="Hello world!"  
 echo $var  

 #Although we change the variable value at following lines  
 #But right after the command running, including special built-in  
 #command(shift) and regular built-in command(echo), $var is still  
 #not changed. This indicates that variable assignment along with   
 #command is not working!  
 var=2 shift  
 var=0 echo $var #still output "Hello world!"  

 echo $var #output "Hello world!"  

Unix Shell Evaluation Order: eval

1. Unix Shell evaluation order

 #! /bin/bash  

 num=10  

 if [ $num -ge 2 ]  
 # 1. The shell will separate the statement into different tokens by  
 #  using the fixed set of metacharacters(tab, space etc.)  
 # 2. Check the first word "if", whether it is keyword or not, if yes,  
 #  Then it will setup the internal configuration for "compound command"  
 #  and then ready to read next command  

 then  
 #it is exactly same as above, "then" is shell keyword, the shell will setup  
 #internally for "compound command"  

   f=Desktop  
   y="hello world"  
   echo ~/${f} $y $(echo my world) $((3+2))   
 #1. Similar, separate the statement into different tokens by using the  
 #  fixed set of metacharacters.   
 #2. Check the first word "echo", it is not keyword in this case, so proceed  
 #  to next step  
 #3. Check first word whether it is aliases if yes, substitute it with the   
 #  alias then go back to step 1(need to re-check if it is keyword)  
 #4. Scan all words for tilde expansion, in this case, ~ will be substituted,  
 #  so, the statement will be like:  
 #  echo /home/aubinxia/${f} $y $(echo my world) $((3+2))  
 #5. Scan all words for variable substitution. In this case, ${f} will be substituted  
 #  with Desktop, $y will be replaced with "hello world", and right now the  
 #  statement is like:  
 #  echo /home/aubinxia/Desktop hello world $(echo my world) $((3+2))  
 #6. Scan all words for command substitution. In this case, $(echo my world) will be  
 #  run and substituted with "my word". Note at this step, it will recursively go  
 #  through all steps from step 1. Right now the statement is like:  
 #  echo /home/aubinxia/Desktop hello world my world $((3+2))  
 #7. Scan all words for arithmetic substitution. In this case, $((3+2)) will be replaced  
 #  with 5. And right now the statement is like:  
 #  echo /home/aubinxia/Desktop hello world my world 5  
 #8. Go through all words again and use IFS as separator to get all words again:  
 #  Before this step, statement has 5 words:  
 #  1) echo  
 #  2) /home/aubinxia/Desktop  
 #  3) hello world(because it is from original word: $y)  
 #  4) my world(because it is from original word: $(echo my world))  
 #  5) 5  
 #  
 #  after this step, statement has 7 words:  
 #  1) echo 2) /home/aubinxia/Desktop 3) hello 4) world 5) my 6) world 7) 5  
 #9. Scan all words for wildcard expansion.  
 #10. Run the final command, and start to look up echo in shell.  

 fi  

2. eval statement:
eval statement force shell to re-go-through all steps needed in evaluation again.
script:

 #! /bin/bash  

 comm="ls -lrt | sort"  
 $comm  
 #output:  
 #ls: cannot access |: No such file or directory  
 #ls: cannot access sort: No such file or directory  
 #  
 #The reason:  
 #when running $comm, it already comes to the evaluation step:  
 #variable substitution. The remaining steps are just for  
 #word separation and run the command.  
 #  
 #So "-lrt | sort" are just taken as the parameter of ls!  
 #  
 #If we want to to separate ls and sort to different commands  
 #we have to go through the step 1 in evaluation process.  
 #Here is the better way:  
 #eval will let $comm go through the evaluation process again  

 eval $comm  
 #output:  
 #-rw-rw-r-- 1 aubinxia aubinxia  0 May 20 22:33 backup  
 #-rw-rw-r-- 1 aubinxia aubinxia  12 May 24 11:16 script?  
 #-rw-rw-r-- 1 aubinxia aubinxia  12 May 24 11:16 script2  
 #-rw-rw-r-- 1 aubinxia aubinxia  34 May 24 16:02 out  
 #-rw-rw-r-- 1 aubinxia aubinxia  35 May 24 10:34 text  
 #-rw-rw-r-- 1 aubinxia aubinxia  53 May 18 18:55 text~  
 #-rw-rw-r-- 1 aubinxia aubinxia 8453 May 17 11:52 backup~  
 #-rwxrwxr-x 1 aubinxia aubinxia 539 May 24 16:52 script_1  
 #total 36  

3. subshell & code block
subshell will start the new process, which is not related with current process
./script_1:
inside the subshell, we go to the another folder, and pwd is already changed.
But right leaving the subshell, we come back to the current  process, pwd doesn't change, since the change in another process will not affect the current process.

 #! /bin/bash  
 (cd ../..;pwd)  
 echo =====  
 pwd  

terminal:

 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ ./script_1  
 /home/aubinxia  
 =====  
 /home/aubinxia/Desktop/xxdev  

====================================
./script_1:
code block using braces have to use format like following code. It can coexist with other codes in the same line like sub shell.
Two occurances of pwd show the same path, meaning that code block share the same state with parent process. For the shell, it doesn't start a new process when entering a a code block.

 #! /bin/bash  
 {   
   cd ../..  
   pwd   
 }  
 echo =====  
 pwd  

terminal:

 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ ./script_1  
 /home/aubinxia  
 =====  
 /home/aubinxia  

====================================
./script_1:
Following example indicate that even if we use the subshell to start a new process, it still "inherits" all information from the parent process and pwd show the same path as the parent's one.

 #! /bin/bash  
 cd ../..  
 pwd  
 ( pwd )  

terminal:

 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ ./script_1  
 /home/aubinxia  
 /home/aubinxia  

Unix Shell Role of Quoting

1. Backslash escaping
Backslash escaping tells the shell to treat the character literally.

terminal:
First command is trying to list all files whose name starts with "script" and end up with one ANY character.
Second command is trying to list all files whose name starts with "script" and end up with just question mark

 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ ls script?  
 script? script2  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ ls script\?  
 script?  

2. Single quote
Single quote enforce the shell to treat all characters literally.

terminal:
1) First command list all files whose name starts with "script"
2) Second command list all files whose name starts with "script", and plus ONE ANY character
3) Third command list all files whose name is just "script?". Single quote treat question mark literally
4) Fourth command list all files whose name is just "script*". Unfortunately, no file has such a name
5) Fifth command is tricky. We intends to nest one question mark inside the single quote.('?' is nested in outsider quote). But the shell treat it in different way, it will use first 2 single quote to tackle "script", then question mark is outside any single quotes, represents any single character here, the last two single quotes combine together to represent empty string. So the entire statement means, list all files whose name starts with script, plus any single character and plus empty string.

 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ ls script*  
 script? script2  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ ls script?  
 script? script2  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ ls 'script?'  
 script?  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ ls 'script*'  
 ls: cannot access script*: No such file or directory  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ ls 'script'?''  
 script? script2  

3. Single quotes along with double quotes
terminal:
1) First command, double quote is nested inside the single quotes, and every thing in single quotes is treated literally. So in this case, double quote just get outputted
2) Second command, we intend to escape the last single quote, but unfortunately, the backslash is treated literally inside single quotes
3) Third command, the last escaped single quote is not belonging to any single quotes pair, it is just tell echo to literally treat the single quote, and we just want to output one single quote here. The 2nd single quote in the statement is pairing with the first single quote.
4) Based on third command, we just append 's it going"', which output everything inside these single quotes.

 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ echo 'He said:"'  
 He said:"  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ echo 'He said:"How\'  
 He said:"How\  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ echo 'He said:"How'\'  
 He said:"How'  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ echo 'He said:"How'\''s it going"'  
 He said:"How's it going"  

4. Double quotes
Double quotes will not treat every thing inside quotes literally, it will escape the special character like $, `, ", \.

terminal:
1) escape $ to let shell treat $ literally
2) otherwise, $n means empty variable here
3) without backslash to escape `, echo world will be taken as command substitution.
4) using backslash to treat ` literally
5) using backslash to treat " literally

 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ echo "Hello \$n"  
 Hello $n  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ echo "Hello $n"  
 Hello   
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ echo "Hello `echo world`"  
 Hello world  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ echo "Hello \`echo world\`"  
 Hello `echo world`  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ echo "Hello \"world\""  
 Hello "world"  

Unix Shell Command Substitution(2)

1. Self-made head command
./script_1:

 #! /bin/bash  
 #remove the "-" in the very beginning and leave the number here  
 number=$(echo $1 | sed -e 's/^-//')  

 #remove the first positional parameter  
 shift  

 #option q means "exit code", sed will read first "number" lines of   
 #text, do nothing, output, and quit (if with -n, suppressing the   
 #output of each pattern space, nothing get output and then quit)  
 #So following command means, output first number lines of text of  
 #file $1  
 sed ${number}q $1  

terminal:

 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ ./script_1 -2 ./text  
 Hello  
 Hello world  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ ./script_1 -3 ./text  
 Hello  
 Hello world  
 Hello amazing world!  

2. Example of Command Substitution
./text:

 xx:/California/Los Angeles  
 yy:  
 zz:  

./script_1:

 #! /bin/bash  

 #note: how while condition works:  
 #read user address and detect if there is more string to read  
 #if yes, then while condition is true, it will proceed to run  
 #the loop body, if no, quit.  
 #So the point is: if the last line doesn't end up with new line  
 #operator, after reading user and address, it will detect that   
 #there is no more string to read, then it quit, so the last line  
 #doesn't get processed!  
 #So, for the input file, the last line must end up with new line  
 #operator.  

 #firstly remove all mail_list file to clear up all existing records  
 rm *.mail_list  

 #the shell read user and address from the ./text file, and use command  
 #substitution to transform the address to part of file name  
 #lastly append the user name to generated file  
 cat ./text | \  
 while IFS=":" read user address  
 do  
   path=${address:-/New York/New York City}  
   file=./$(echo $path | sed -e 's/^\///' -e 's/ /_/g' -e 's/\//-/g').mail_list  
   echo $user >> $file  
 done  

terminal:

 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ ./script_1  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ cat ./California-Los_Angeles.mail_list  
 xx  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ cat ./New_York-New_York_City.mail_list  
 yy  
 zz  

3. expr command:
Mainly used on arithmetic calculations:
The reason to escape '*' is: shell may take it as wildcard
If using double quote(last command), then it will be taken as string
expr will output the result

 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ expr 5 + 2  
 7  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ expr 5 - 2  
 3  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ expr 5 * 2  
 expr: syntax error  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ expr 5 \* 2  
 10  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ expr 5 / 2  
 2  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ expr "5 + 2"  
 5 + 2  

Unix Shell Command Substitution(1)

1. Use ` ` for command substitution
./script:
`ls script*` represent the output of the command here, which is a list of all file names. And for loop will use space as default separator to assign each file name to variable i.

 #! /bin/bash  

 for i in `ls script*`  
 do  
   echo $i  
 done  

terminal:

 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ ./script_1  
 script~  
 script!  
 script_1  
 script_1~  
 script_2  
 script_2~  

2. Nested Command Substitution
./script:

 #! /bin/bash  

 #echo [] is nested inside another echo  
 echo { `echo []` }  
 #output:  
 #{ [] }  

 #trying to insert another "echo ()" inside the 2nd level  
 #echo, but this is wrong!  
 echo { `echo [ `echo ()` ]` }  

 #correct way:  
 #we need to escape the "`" and "()". The reason to escape "`"  
 #is: it is nested in another command substitution  
 #The reason to escape () is: we need them to be interpreted   
 #literally by the shell  
 echo { `echo [ \`echo \(\)\` ]` }  
 #output:  
 #{ [ () ] }  

 #trying to insert another "echo hello" inside. Following three  
 #ways are all wrong. First one, "echo hello" is taken as string  
 #instead of command substitution  
 #Second one, syntax error  
 #Third one, still syntax error. The point is: the second "\`" is   
 #combined with first "\`" to form a command, which totally messed up  
 #the command.  
 echo { `echo [ \`echo \( echo hello \)\` ]` }  
 #output: { [ ( echo hello ) ] }  
 echo { `echo [ \`echo \( `echo hello` \)\` ]` } #error  
 echo { `echo [ \`echo \( \`echo hello \` \)\` ]` } #error  

3. Nested Command Substitution and double quote
./script_1:

 #! /bin/bash  
 echo "outer outer"  
 #output: outer outer  

 echo "outer --`echo []`-- outer"  
 #output:  
 #outer --[]-- outer  

 #Explanation:  
 #\`echo \"Hello world\"\` is the inner level, which is executed firstly  
 #double quote is used to interpret string "Hello world"  
 #then: `echo [\`echo \"Hello world\"\`]` is run secondly, but at this time  
 #it is already transformed into: `echo [Hello world]`  
 #Finally, the outside echo is run.  
 echo "outer --`echo [\`echo \"Hello world\"\`]`-- outer"  
 #output:  
 #outer --[Hello world]-- outer  

4. Use $(...) for command substitution

 #! /bin/bash  

 echo "outer outer"  
 #output: outer outer  

 echo "outer --$(echo [])-- outer"  
 #output:  
 #outer --[]-- outer  
 #the reason to use double quote "()" is: otherwise () will be taken as  
 #part of command subsitution structure "$(...)"  

 echo "outer --$(echo [$(echo "()")])-- outer"  
 #output:  
 #outer --[()]-- outer  

 echo "outer --$(echo [$(echo "($(echo Hello world))")])-- outer"  
 #output:  
 #outer --[(Hello world)]-- outer  

Unix Shell Tilde Expansion and Wildcards

1. Tilde Expansion:
"cd ~" means going to the home directory of the current user
"cd ~aubinxia" means  searching the "aubinxia" from /etc/passwd, and get the home directory there, and then go to that directory
"cd ~pulse" means similar thing as above.

 aubinxia@aubinxia-fastdev:~$ grep aubinxia /etc/passwd  
 aubinxia:x:1000:1000:AubinXia,,,:/home/aubinxia:/bin/bash  
 aubinxia@aubinxia-fastdev:~$ grep pulse /etc/passwd  
 pulse:x:115:122:PulseAudio daemon,,,:/var/run/pulse:/bin/false  
 aubinxia@aubinxia-fastdev:~$ cd ~  
 aubinxia@aubinxia-fastdev:~$ cd ~aubinxia  
 aubinxia@aubinxia-fastdev:~$ cd ~pulse  
 bash: cd: /var/run/pulse: No such file or directory  

2. Wildcarding
? means any single character
* means any string of characters
so "ls -lrt script*" means any files whose name starts with "script".
[12] means any character who is 1 or 2. so "ls -lrt script?[12]" means any file whose name starts with "script" plus one any character, and end up with 1 or 2.

[!12] , "!" means negate, this means, any character which is not 1 or 2. so "ls script[!12]" means any file whose name starts with script, and end up with one character who is not 1 or 2.

\! in set means ! literally. So "ls -lrt script[12\!]" means any file whose starts with "script, and end up with character who is 1 or 2 or !.

[0-9] means range 0 to 9, "ls -lrt script?[0-9]" means any file whose name starts with script , and plus one any character, and end up with one any number(from 0 to 9) as specified. Note: range expression is not portable, since different system defines code differently.

 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ ls -lrt script*  
 -rwxrwxr-x 1 aubinxia aubinxia 405 May 10 16:37 script~  
 -rwxrwxr-x 1 aubinxia aubinxia 962 May 17 19:38 script_2~  
 -rwxrwxr-x 1 aubinxia aubinxia 1715 May 18 22:13 script_2  
 -rwxrwxr-x 1 aubinxia aubinxia 155 May 21 21:34 script_1~  
 -rwxrwxr-x 1 aubinxia aubinxia  15 May 22 21:09 script_1  
 -rw-rw-r-- 1 aubinxia aubinxia  11 May 22 21:42 script!  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ ls -lrt script?[12]  
 -rwxrwxr-x 1 aubinxia aubinxia 1715 May 18 22:13 script_2  
 -rwxrwxr-x 1 aubinxia aubinxia  15 May 22 21:09 script_1  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ ls -lrt script[!12]  
 -rwxrwxr-x 1 aubinxia aubinxia 405 May 10 16:37 script~  
 -rw-rw-r-- 1 aubinxia aubinxia 11 May 22 21:42 script!  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ ls -lrt script[12\!]  
 -rw-rw-r-- 1 aubinxia aubinxia 11 May 22 21:42 script!  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ ls -lrt script?[0-9]  
 -rwxrwxr-x 1 aubinxia aubinxia 1715 May 18 22:13 script_2  
 -rwxrwxr-x 1 aubinxia aubinxia  15 May 22 21:09 script_1  

Unix Shell Standard Output: printf

1. Escape sequences

 #! /bin/bash  

 #\n means newline operator  
 printf "Hello \n world\n"  
 #output:  
 #Hello  
 # world!  

 #\t means horizontal tab  
 printf "Hello\tworld\n"  
 #output: Hello  world  

 #\v means vertical tab  
 printf "Hello\vworld\n"  
 #output:  
 #Hello  
 #   world
  
 #\ddd represents character with ascii code "ddd", which  
 #is octal value. 141 is "97"(decimal), which is the code  
 #of 'a'  
 printf "\141\n"  
 #output: a  

 #\\ represents a literal \
 printf "\\ \n"
 #output: \

2. format specifiers

 #! /bin/bash  

 #%s means replacing with string, and '\n' and other escape sequences  
 #will just be taken as normal string  
 printf "<%s>\n" "Hello\nworld!"  
 #output:  
 #<Hello\world!>"  

 #%b will take "\n" as the newline operator in a string, instead of  
 #part of string  
 printf "<%b>\n" "Hello\nworld!"  
 #output:  
 #<Hello\  
 #world!>  

 #%c means ascii character, if provided parameter is a string, then  
 #only first character of string will be taken.  
 printf "<%c>\n" "Hello"  
 #output:  
 #<H>  

 #%d %i means integer, the provided parameter must be integer  
 printf "<%d %i>\n" 19 20  
 #output:  
 #<19 20>  

 #%e means float, with format ###e[+-]###  
 printf "<%e>\n" 0.99999999999999  
 #output:  
 #<1.000000e+00>  

 #%E means float, with format ###E[+-]###  
 #the only difference from %e is, it is uppercase  
 printf "<%E>\n" 0.99999999999999  
 #output:  
 #<1.000000E+00>  

 #%f means float, with format ###.####  
 printf "<%f>\n" 0.99999999999999  
 #output:  
 #<1.000000>  

 #%g means: %f or %e, which one is shorter get output  
 printf "<%g>\n" 0.88888888888888  
 #output:  
 #<0.888889>  

 #%G means: %f or %E, which one is greater get output  
 printf "<%G>\n" 0.88888888888888  
 #output:  
 #<0.888889>  

 #%o means unsigned octal value, if provided parameter is minus  
 #number, the value is undefined  
 printf "<%o>\n" 97  
 #output:  
 #<141>  (octal value of 97)  

 #%x means unsigned hexadecimal number, use a-f. If provided parameter  
 #is minus number, the value is undefined  
 printf "<%x>\n" 15  
 #output:  
 #<f>  

 #%X means unsigned hexadecimal number, use A-F. If provided parameter  
 #is minus number, the value is undefined. The only difference from %x  
 #is: it used A-F(uppercase) instead of a-f(lowercase)  
 printf "<%X>\n" 15  
 #output:  
 #<F>  

 #%% is a literal %  
 printf "<%%>\n"  
 #output:  
 #<%>  

3. printf flags

 #! /bin/bash  
 #"10" means the width is 10, "-10" means width is 10 while   
 # left-justify it, without "-"(minus) shell will right-justify  
 #the element  
 printf "|%10s|\n" Hello  
 #output:  
 #|   Hello|  

 printf "|%-10s|\n" Hello  
 #output:  
 #|Hello   |  

 printf "|%10d|\n" 100  
 #output:  
 #|    100|  

 printf "|%-10d|\n" 100  
 #output:  
 #|100    |  

 #space before d means, prefix positive number with space,   
 #for negative number, prefix a space and minus sign  
 printf "|% d|| %d|\n" 100 -100  
 #output:  
 #| 100|| -100|  

 printf "|% f|| %f|\n" 0.88888 -0.88888  
 #output:  
 #| 0.888880|| -0.888880|  

 printf "|% e|| %e|\n" 0.888888888 -0.88888888  
 #output:  
 #| 8.888889e-01|| -8.888889e-01|  

 #"+" sign means prefix "+" for positive number, prefix "-" for  
 #negative number  
 printf "|%+d||%+d|\n" 100 -100  
 #output:  
 #|+100||-100|  

 #"#" will prefix octal number with a 0  
 #"#" will prefix hex number with a "0x"  
 #"#" will make %f %e %E %g %G always have decimal point  
 printf "|%#o||%o||%#x||%x||%#g||%g||%#G||%G|\n" 8 8 15 15 15 15 15 15  
 #output:  
 #|010||10||0xf||f||15.0000||15||15.0000||15|  
 #"0" will pad output with 0s(not effictive on %s)  

 printf "|%010s||%10s||%05d||%5d||%010f||%10f|\n" Hello Hello 5 5 5 5  
 #output:  
 #|   Hello||   Hello||00005||  5||005.000000|| 5.000000|  

4. Precision

 #! /bin/bash  
 #precision applying on %s means: maximum number of characters  
 #that could be shown. If provided string doesn't have enough  
 #characters, then it has no effect.  
 printf "%.3s %.5s %.10s\n" Hello Hello Hello  
 #output:  
 #Hel Hello Hello  

 #precision applying on %d means: specified number of digits must be  
 #shown, if not enough, pad with 0, if number itself has more  
 #digits, then it has no effect  
 printf "%.2d %.5d %.5d\n" 100 100 -100  
 #output:  
 #100 00100 -00100  

 #%.2d ask show 2 digits but 100 already has 3 digits, then it has no  
 #effect in this case  
 #%i same as %d  
 printf "%.2i %.5i %.5i\n" 100 100 -100  
 #output:   
 #100 00100 -00100  

 #precision applying on %o means: specified number of digits must be shown  
 #if number doesn't have enough digits, it will be padded with 0  
 #if number has more digits, then it has no effect. %o doesn't support  
 #negative number, it would be undefined.  
 printf "%.2o %.5o %.5o\n" 100 100 -100  
 #output:  
 #144 00144 1777777777777777777634  

 #precision applying on %u(unsigned decimal): specified number of digits must  
 #be shown. If number doesn't have enough digits, it will be padded with 0  
 #if number has more digits, then it has not effect.  
 printf "%.2u %.5u\n" 100 100  
 #output:  
 #100 00100  

 #precision applying on %x: specified number of digits must be shown. If number  
 #doesn't have enough digits, it will be padded with 0. If number has more digits,  
 #then it has no effect.  
 printf "%.1x %.2x %.5x\n" 15 15 15  
 #output:  
 #f 0f 0000f  

 #precision applying on %e: specified number of digits must be shown after the  
 #decimal point. If number has more digits after decimal point, then it will  
 #be rounded. If number doesn't have enough digits after decimal point, then  
 #it will be padded with 0 afterwards.  
 printf "%.2e %.5e %.10e\n" 10.88888 10.88888 10.88888  
 #output:  
 #1.09e+01 1.08889e+01 1.0888880000e+01  

 #precision applying on %f: specified number of digits must be shown after the  
 #decimal point. If number has more digits after decimal point, then it will  
 #be rounded. If number doesn't have enough digits after decimal point, then  
 #it will be padded with 0 afterwords.  
 printf "%.2f %.5f %.10f\n" 10.88888 10.88888 10.88888  
 #output:  
 #10.89 10.88888 10.8888800000  

 #precision on g means: the specified number of significant digit  
 #If number doesn't have enough digits, then it won't be padded with 0, it  
 #just show its original number of digits  
 #If number has more digits, it will be rounded accordingly.  
 #Note the %.1g result, it is transformed to scientific format to satisfy   
 #"1" siginificant digit requirement.  
 printf "%.1g %.2g %.5g %.10g\n" 10.88888 10.88888 10.88888 10.88888  
 #output:  
 #1e+01 11 10.889 10.88888  

5. ASCII Code Character

 #! /bin/bash  
 #if using "'"(single quote) to prefix string character,  
 #it means we are trying to get the ASCII code  
 printf "%c %d\n" a "'a"  
 #output:  
 #a 97  

Unix Shell Redirection(2)

1. exec to change the file descriptor
./script_2:

 #! /bin/bash  

 exec 5>&1 #save the current standard output to fd 5  
 exec 6>&2 #save the current standard error to fd 6  

 #redirect standard output to ./text_output  
 #redirect standard error to ./text_error  
 exec 1>./text_output  
 exec 2>./text_error  

 if [ echo "h" ] #error here, output to standard error(text_error)  
 then :  
 fi  

 echo "Hello world!"  
 # output to standard output(text_output)  

 exec 1>&5 #restore standard output to terminal  
 exec 2>&6 #restore standard error to terminal  

 if [ echo "h" ]  
 then :  
 fi  
 #now output error information to terminal  
 #output(terminal): ./script_2: line 21: [: echo: unary operator expected

 echo "Hello world"  
 #now output standard output to terminal  
 #output(terminal): Hello world

terminal:

 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ ./script_2  
 ./script_2: line 21: [: echo: unary operator expected  
 Hello world  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ cat ./text_output  
 Hello world!  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ cat ./text_error  
 ./script_2: line 11: [: echo: unary operator expected  

===================================
script_2:

 #! /bin/bash  

 exec 5>&1  
 exec 6>&2  

 #">>" make shell append string to the file while doing the redirection  
 exec 1>>./text_output  
 exec 2>>./text_error  

 if [ echo "h" ] #error here, append error to text_error  
 then :  
 fi  
 echo "Hello world from script!" #append string to text_output  

 exec 1>&5  
 exec 2>&6  

text_output: already exists, and contain "Hello world!"
text_error: already exists, and contain "Hello world!"

terminal:
It append string to the end of both files.

 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ ./script_2  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ cat ./text_error  
 Hello world!  
 ./script_2: line 14: [: echo: unary operator expected  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ cat ./text_output  
 Hello world!  
 Hello world from script!  

====================================
text_output already exists and contain some string there.

script_2:

 #! /bin/bash  
 exec 5>&1  
 #">" will truncate the "text_output" file and make it empty here  
 exec 1>./text_output  
 echo Hello world!  
 echo Amazing world!  

 cat ./text_output  
 #text_output content:  
 #Hello world!  
 #Amazing world!  
 #  
 #text_output's original content is gone.
 #This indicates that "truncate" operation only happens at the beginning  
 #of this script's process, for all remaining print out, it will be appended   
 #to the end of text_output file  
 #If using "exec 1>>..", then it will never truncate, just append all the time.  
 # and text_output's original content is still there.
 exec 1>&5  

2. exec to execute command
./script_1:

 #! /bin/bash  
 echo "script_1 is running!"  
 echo $1 $2  

 ./script_2:

 #! /bin/bash  
 echo "script_2 is running"  
 exec ./script_1 Hello world  
 echo "back to script_2!"  

terminal:
Note: the last echo command in script_2 is never run.
Because exec is the one-way command, it will launch the ./script_1, and then it never come back.

 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ ./script_2  
 script_2 is running  
 script_1 is running!  
 Hello world  

Unix Shell Redirection(1)

1. no clobber:
./script_2:

 #! /bin/bash  

 echo "Hello world" > text  
 set -C  
 echo "Amazing world!" >| text  
 echo "Hello world!" > text  

terminal:
set -C will force the shell check if the output file is already existed, if yes, it will not let user to output content in there. >| will explicitly "ignore" such check and output content inside.
So, first echo command, truncate the file text and output "Hello world" there.
Second echo command, still truncate the file text and output string there.
Third echo command failed, since "set -C" already enable the redirection check, and it doesn't use the ">!" to avoid the check.

 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ ./script_2  
 ./script_2: line 7: text: cannot overwrite existing file  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ cat ./text  
 Amazing world!  

2. Provide input with "<<" "<<-"
terminal:
2nd command "<<-" will make shell ignore all tab characters in the input.

 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ cat<<EOF  
 Hello world!  
 This is a multi-line input!  
 EOF  
 Hello world!  
 This is a multi-line input!  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ cat<<-EOF  
 Hello world!  
 This is a multi-line input!  
 EOF  
 Hello world!  
 This is a multi-line input!  

3. "<" ">" "<>" to open file
text:

 My world!  
 Your world!  
 Hello world!  

script_2:

 #! /bin/bash  

 #"<>" means the file after it can be used to be write or read,   
 #and it really depends on implementation of the command. for grep,   
 #it will just use it for "read"  
 grep world! <>text  

 #"<" means the file after it could only be used to read, but  
 #based on implementation of "echo command", this is useless  
 echo "Amazing world!" <text #output "Amazing world!" to standard output  
 #text doesn't chagne here.  

 echo "Amazing world!" <>text #output "Amazing world!" to standard output  
 #text doesn't change here.  

 #">" means the file after it could only be used to write, and it  
 #will truncate the file if it is already existed.  
 echo "Amazing world!" >text #clear up the text file, and output "Amazing world!"  

 cat ./text  
 #output "Amazing world!"  

 #">>" make output "appended" to the file, instead of removing original content  
 echo "Amazing world!" >>text  
 cat ./text  
 #output:  
 #Amazing world!  
 #Amazing world!  

4. File descriptor manipulation

 #! /bin/bash  

 #unix shell file descriptor:  
 #0: standard input  
 #1: standard output  
 #2: standard error output  

 # "0<text" means making the text file as source of standard input  
 # In real life, "0" can be ignored  
 read var 0<text  
 echo $var #output "Hello world!"  

 # "1>text" means making the text file as the standard output  
 # In real life, "1" can be ignored  
 echo Hello world 1>text  
 # output nothing  

 # "2>text" means making the text file as the standard error output  
 # But following statement just output the "Hello world!" at the   
 # standard output: terminal. It doesn't output error information  
 # so "2>text" is useless here.  
 echo "Hello world!" 2>text  
 #output "Hello world!"  

 echo Hello world 1>text | echo amazing world >&1  
 #output: "amazing world"  
 #this indicates that: "1>text" only applies on first command, after  
 #going through the pipeline, file descriptor 1 still represents the  
 #terminal  

========================================
script_2:

 #! /bin/bash  

 if [ echo "H" ]  
 then  
   :  
 fi  
 echo $?  

terminal:
1) First "script_2" command is trying to "truncate" file ./text, and then redirect the standard error to ./text. So ./text contain one line of error information. "0" at the terminal belongs to information of standard output.
2) Second  "script_2" command is trying to "append" standard error information to file ./text, so ./text just contain 2 lines of error information here.
3) Third "script_2" command firstly redirect the standard error information to ./text(2>./text), and redirect standard output information to standard error, which is ./text now.(1>&2). ./text contains both information from standard output and standard error.
4) Fourth "script_2" command firstly redirect the standard output to standard error(terminal) now, then redirect standard error to ./text. And the result is: standard output is terminal, standard error is ./text. This indicates that: order of redirection is very important!

 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ ./script_2 2>./text  
 0  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ cat ./text  
 ./script_2: line 3: [: echo: unary operator expected  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ ./script_2 2>>./text  
 0  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ cat ./text  
 ./script_2: line 3: [: echo: unary operator expected  
 ./script_2: line 3: [: echo: unary operator expected  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ ./script_2 2>./text 1>&2  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ cat ./text  
 ./script_2: line 3: [: echo: unary operator expected  
 0  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ ./script_2 1>&2 2>./text  
 0  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ cat ./text  
 ./script_2: line 3: [: echo: unary operator expected  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$   

Unix Shell Readling Lines: read

1. Basic Usage
./script_2

 #! /bin/bash  

 printf "Please input two strings:"  
 read var1 var2  
 echo $var1, $var2  

terminal:
There are 2 ways to put in string from standard input:
1) "Hello world", the shell will use variable IFS as the separator to read in different strings into different varaibles
2) "Hello \
world", the backslash and new line operator make shell think, it should go to the next line to read in next string.

 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ ./script_2  
 Please input two strings:Hello world  
 Hello, world  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ ./script_2  
 Please input two strings:Hello \  
 world  
 Hello, world  

=====================================
./script_2:

 #! /bin/bash  

 printf "Please input two strings:"  
 read -r var1 var2  
 echo $var1, $var2  

terminal:
The only difference of above script is using "-r" option for read command. It make shell think: backslash is not the indicator to skip and read variable from the next line, it will read it into the next variable. -r means "raw read"

 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ ./script_2  
 Please input two strings:Hello \  
 Hello, \  

2. Change Default Separator
./script_2:

 #! /bin/bash  

 IFS=':'  
 printf "Please input 3 strings separated by colon:"  
 read var1 var2 var3  
 echo $var1:$var2:$var3  

terminal:
We changed the default separator to ":". And the first command although use backslash in the end, since it only recognize ":" as the default separator, so the first variable is still "34"!

 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ ./script_2  
 Please input 3 strings separated by colon:3\  
 4:5:6  
 34:5:6  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ ./script_2  
 Please input 3 strings separated by colon:4:5:6  
 4:5:6  

3. read with inconsistent number of variables
text:

 Hello, New York  

script_2:

 #! /bin/bash  

 IFS=' '  
 read var1 var2<./text  
 echo $?  
 echo $var1  
 echo $var2  

 read var1 var2 var3<./text  
 echo $?  
 echo $var1  
 echo $var2  
 echo $var3  

 read var1 var2 var3 var4<./text  
 echo $?  
 echo $var1  
 echo $var2  
 echo $var3  
 echo $var4  

terminal:

 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ ./script_2  
 1  
 Hello,  
 New York  
 1  
 Hello,  
 New  
 York  
 1  
 Hello,  
 New  
 York  

Above example demonstrates that what happens if the number of provided arguments at input is different from the number of "read variables" in script.
If there are fewer read variables, all the remaining strings at input will be put into the last variable.
If there are more read variables, then remaining variables will be assigned empty string. That's why the last read command(4 variables), when printing out echo4, in terminal it is just one empty line.

Note: in text file, there is no newline operator in the end. So read return status 1,
if the line of input text end up with newline operator, read will return status 0, since it means there is more thing waiting to be read. Following example can demonstrate this:

text  file is almost same as above, the only difference is: we add a new line operator at the end of string.

script_2:

 #! /bin/bash  

 IFS=' '  
 read var1<./text  
 echo $?  
 echo $var1  

terminal:

 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ ./script_2  
 0  
 Hello, New York  

3. read in pipeline and loop
./script_2

 #! /bin/bash  

 var1=Hello  
 cat ./text | read var1  
 echo $?  
 echo $var1  
 echo "================================"  
 cat ./text | \  
 while read var1  
 do  
   echo $?  
   echo $var1  
 done 
 echo $var1 

terminal:
following result is very surprising. The reason is: when read in the pipeline, the shell normally starts a separate process to handle it. Separate process means, it has its own variable space, which is not related with parent script's at all!.

So the reason the first "read" command doesn't change the value of var1 is, read command change the var1 of its own process's variable space.

And the reason the 2nd "read" command does change the value fo var1 "inside the while scope" is: read command and while loop are in the same process! And they share the same variable space. So leaving the "while loop space", and try to print var1 again, we figured that it is still "Hello", since we return back to the script's process space, and it never change at all.

 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ ./script_2  
 0  
 Hello  
 ================================  
 0  
 Hello, New York  
 Hello

Unix Shell function

1. Function in the same file

 #! /bin/bash  

 testFunc_1()  
 {  
   echo $1  
   echo Hello world  
   return 0  
 }  
 testFunc_2()  
 {  
   return 10  
 }  

 ##################Normal Call############################  

 testFunc_1 wow!   
 #output:  
 #wow!  
 #Hello world!  

 echo $? #output 0, since the "testFunc_1" just return 0 as the status  

 testFunc_2  
 echo $? #output 10, since the "testFunc_2" just return 10 as the status  

 ##################Command Substitution###################  
 echo "==================separator=================="  
 var=$(testFunc_1 wow!)  
 #this will output nothing, but variable "var" just contain all content  
 #outputted from the testFunc_1  
 #note: var doesn't save return status  

 echo $? #output 0, since testFunc_1 return status 0  
 echo $var #output: wow! Hello world  
 printf "%s\n" $var   
 #output:  
 #wow!  
 #Hello  
 #world  

 var=$(testFunc_2) #since testFunc_2 doesn't output anything there, so  
 #$var will be empty. Note again: $var doesn't save return status in  
 #command substitution  

 echo $? #output 10, since testFunc_2 return the status 10  
 #This indicate that command substitution also output the command's return status  
 # to $?  
 echo $var #output: empty line  
 printf "%s\n" $var #output: empty line  

 #############conditional expression###################  

 echo "==================separator=================="  
 var=$(testFunc_1 wow!)  
 if [ -n "$var" ] #double quote is necessary here  
 then  
   echo "return string"  
 else  
   echo "return empty string"  
 fi  
 #$var is taken as the string value here, content is: wow! Hello world  
 #double quote is needed, otherwise, it will intepreted with:  
 # -n wow! Hello world, which make system complain too many arguments  
 #for test command.  

 if $var #error, this will complain, "wow!: command not found"  
 then  
   echo "return success status"  
 else  
   echo "return failure status"  
 fi  
 #again, $var just contain string from the function, and it is just interpreted  
 #as string here, making system think "command not found"  

 if testFunc_1 wow!  
 then  
   echo "return success status"  
 else  
   echo "return failure status"  
 fi  
 #output:  
 #wow!  
 #Hello world  
 #return success status  
 #  
 #first two lines are from the testFunc_1  
 #last line is from caller, since testFunc_1 return 0 as the status code  

 if testFunc_2  
 then  
   echo "return success status"  
 else  
   echo "return failure status"  
 fi  
 #output: return failure status  

 #Above examples indicate that: after command substitution to save function  
 #result into a variable, the meaning totally changed. The variable doesn't  
 #save return code of function, instead, it save the output from the function  
 #as the string, and whenever the user used that variable at other statements,  
 #it is just interpreted as string.  

2. Function and Variables:
./script_1:

 #! /bin/bash  

 testFunc_3()  
 {  
   var="Hello amazing world!"  
   return 0  
 }  
 testFunc_1()  
 {  
   echo "script_1's testFunc_1()!"  
   return 0  
 }  

./script_2:

 #! /bin/bash  

 testFunc_1()  
 {  
   echo $#  
   echo $0 $1 $2  
   return 0  
 }  

 testFunc_1 Hello world!  
 #output:  
 #2  
 #./script_2 Hello world!  

 echo $#  
 echo $0 $1 $2  
 #output:  
 #2  
 #./script_2 good morning!  

 #above examples indicated that: when entering the function scope  
 # "$#", $0, $1, $2...all these variables content are replaced  
 #temporarily, when leaving scope, it is restored to script's ones  

 ##################Other Variables#########################  

 var="Hello world!"  
 testFunc_2()  
 {  
   var="Amazing world!"  
   return 0  
 }  

 echo $var  
 #output "Hello world!"  

 testFunc_2  
 echo $var  
 #output "Amazing world!"  
 #  
 #This indicate that: in script, there is only one variable entity   
 #for variables with same name, there is no concept about global variables  
 #or local ones. So we should be careful with hanlding variables by functions  

 ######################Functions in other files##########################  
 source ./script_1  
 testFunc_3  
 echo $var  
 #output "Hello amazing world!"  
 #  
 #we can use "source" command to "inject" another script file here, and then   
 #use other script's function as local function  

 testFunc_1  
 #output "script_1's testFunc_1()!"  
 #  
 #This indicate that if there is another function with same name as previous one  
 #it will just replace the previous one. And any calling this function will be  
 #directed to the new function   

Unix Shell Option Processing: getopts

1. Process parameter with loop and case
./script_1:

 #! /bin/bash  

 file=  
 verbose=  
 quiet=  
 while [ $# -ne 0 ]  
 do  
   case $1 in  
   -f) file=$2  
     echo "file:$file"  
     shift 2 #shift 2 parameters  
     ;;  
   -v) verbose=true  
     quiet=  
     echo "verbose flag is on"  
     shift  
     ;;  
   -q) quiet=true  
     verbose=  
     echo "quiet flag is on"  
     shift  
     ;;  
   *) echo unrecognized option  
     shift  
     ;;  
   esac  
 done  

terminal:

 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ ./script_1 -f text  
 file:text  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ ./script_1 -v  
 verbose flag is on  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ ./script_1 -q  
 quiet flag is on  
 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ ./script_1 ff ff  
 unrecognized option  
 unrecognized option  

2. Process Parameters with getopt
./script_1:

 #! /bin/bash  

 file=  
 verbose=  
 quiet=  

 #":" right after 'f' means, for option f, user must supply the argument, otherwise  
 # system will complain  

 #OPTARG is the "argument" right after the "option", if we need to supply an argument here  

 #OPTIND is the index of next option waiting to be processed.  

 while getopts f:vq opt  
 do  
   echo $OPTARG  
   case $opt in  
   f) file=$OPTARG  
     echo "file:$file"  
     ;;  
   v) verbose=true  
     quiet=  
     echo "verbose flag is on"  
     ;;  
   q) quiet=true  
     verbose=  
     echo "quiet flag is on"  
     ;;  
   '?') echo "Input is invalid option"  
      ;;  
   esac  
   echo "index of next argument:"$OPTIND  
 done  

terminal:
Following command doesn't provide the argument for option -f, so getopts:
1) provide the error information on output 2) make $opt variable a questino mark, so we see output: "input is invalid option"

 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ ./script_1 -f  
 ./script_1: option requires an argument -- f  
 Input is invalid option  
 index of next argument:2  

Following command provides the filename "fff" as the argument, then getopts:
1) assign "fff" to $OPTARG
2) assign "f" to $opt, so we can see the output: "file:fff"
3) assign 3 to $OPTIND, since after "-f fff", the next option's index is 3

 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ ./script_1 -f fff  
 fff  
 file:fff  
 index of next argument:3  

Following command firstly process option "v", getopts:
1) assign empty space to $OPTARG, since there is no argument needed
2) assign "v" to $opt
3) assign 1 to $OPTIND, since the next option's index is still 1, because vq is at the same place

Then it process option "q", getopts is similar as above: 1) assign empty space to $OPTARG
2) assign "q" to $opt
3) assign 2 to $OPTIND, since the next option's index is indeed 2(option -f)

Finally it process option "f", getopts:
1) assign "fff" to $OPTARG
2) assign "f" to $opt
3) assign 4 to $OPTIND, since -f has one argument, which occupied one place

 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ ./script_1 -vq -f fff  

 verbose flag is on  
 index of next argument:1  

 quiet flag is on  
 index of next argument:2  
 fff  
 file:fff  
 index of next argument:4  

3. Make getopts silent
script_1:

 #! /bin/bash  

 file=  
 verbose=  
 quiet=  

 #":" at the first place of entire "option string" of getopts means:  
 #1) it won't print any error message  
 #2) assign '?' to $opt  
 #3) assign invalid option to $OPTARG  
 while getopts :f: opt  
 do  
   echo $OPTARG  
   case $opt in  
   f) file=$OPTARG  
     echo "file:$file"  
     ;;  
   '?') echo "Input is invalid option"  
      ;;  
   esac  
   echo "index of next argument:"$OPTIND  
 done  

terminal:
$OPTARG is assigned with 'd', which is the problematic option, and $opt is assigned with '?'

 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ ./script_1 -d  
 d  
 Input is invalid option  
 index of next argument:2  

===================================================
./script_1:

 #! /bin/bash  

 file=  
 verbose=  
 quiet=  

 #without ":" at the first place of entire option string of "getopts", it will:  
 #1) print out the error message  
 #2) assign "?" to $opt  
 #3) assign empty space to $OPTARG  
 while getopts f: opt  
 do  
   echo $OPTARG  
   case $opt in  
   f) file=$OPTARG  
     echo "file:$file"  
     ;;  
   '?') echo "Input is invalid option"  
      ;;  
   esac  
   echo "index of next argument:"$OPTIND  
 done  

terminal:

 aubinxia@aubinxia-fastdev:~/Desktop/xxdev$ ./script_1 -d  
 ./script_1: illegal option -- d  

 Input is invalid option  
 index of next argument:2