dokucama

User Tools

Site Tools

Trace: • cicd • netconf • gitlabimport • raspberry • bash
scripting:bash

BASH CONDITIONS: 

# run ssh agent in /etc/bashrc
if [ ! -S ~/.ssh/ssh_auth_sock ]; then # if file does('nt) exist and is socket
  eval `ssh-agent`
  ln -sf "$SSH_AUTH_SOCK" ~/.ssh/ssh_auth_sock
fi
ssh-add -l > /dev/null || ssh-add
#run ssh agent
ssh-add ~/.ssh/* >/dev/null 2>&1

EXAMPLE OF ssh/config FILE:

$ cat .ssh/config 
Host *
  AddKeysToAgent yes
  #UseKeychain yes
  ServerAliveCountMax 6
  ServerAliveInterval 10
  TCPKeepAlive yes
  ForwardAgent yes
  Compression yes
  GatewayPorts yes
  StrictHostKeyChecking no
  User jaime.santos.amandi

## This is saying for anything for those IP ranges or for domains  myorgdatacloud.com OR myorgdatacloud.com EXCEPT anything starting with bastion-62
## use socks proxy over that host: bastion-lhr.myorgdatacloud.com
Host 10.32.* 10.33.* *.myorgdatacloud.com *.bigshot.co.uk !bastion-62-*
  ProxyCommand ssh bastion-lhr.myorgdatacloud.com -W %h:%p

BASH CONDITIONS:

The tests below are test conditions provided by the shell:

  • A file = True if the file exists.
  • b file = True if the file exists and is block special file.
  • c file = True if the file exists and is character special file.
  • d file = True if the file exists and is a directory.
  • e file = True if the file exists.
  • f file = True if the file exists and is a regular file
  • g file = True if the file exists and the set-group-id bit is set.
  • k file = True if the files' “sticky” bit is set.
  • L file = True if the file exists and is a symbolic link.
  • p file = True if the file exists and is a named pipe.
  • r file = True if the file exists and is readable.
  • s file = True if the file exists and its size is greater than zero.
  • s file = True if the file exists and is a socket.
  • t fd = True if the file descriptor is opened on a terminal.
  • u file = True if the file exists and its set-user-id bit is set.
  • w file = True if the file exists and is writable.
  • x file = True if the file exists and is executable.
  • O file = True if the file exists and is owned by the effective user id.
  • G file = True if the file exists and is owned by the effective group id.


  • file1 –nt file2 = True if file1 is newer, by modification date, than file2.
  • file1 ot file2 = True if file1 is older than file2.
  • file1 ef file2 = True if file1 and file2 have the same device and inode numbers.
  • z string = True if the length of the string is 0.
  • n string = True if the length of the string is non-zero.
  • string1 = string2 = True if the strings are equal.
  • string1 != string2 = True if the strings are not equal.
  • !expr = True if the expr evaluates to false.
  • expr1 –a expr2 = True if both expr1 and expr2 are true.
  • expr1 –o expr2 = True is either expr1 or expr2 is true.

This is an example of a very simple script: https://docs.google.com/document/d/1uNUGXkHmb9jvHA4Y12nTXEDSOjkLzzVoAcHkSVwxNmw/edit?usp=sharing

http://www.freeos.com/guides/lsst/while

Flow control: http://linuxcommand.org/wss0090.php

http://www.ieor.berkeley.edu/~faridani/bash.htm ←- notes

$# represents the number of arguments:

OR & AND OPERATORS FOR PROCESS FLOW “&& and || “
The right side of && will only be evaluated if the exit status of the left side is zero. || is the opposite: it will evaluate the right side only if the left side exit status is nonzero. You can consider [ … ] to be a program with a return value. If the test inside evaluates to true, it returns zero; it returns nonzero otherwise. 

(ip route add 10.0.0.0/8 via 10.20.3.5 dev eth0 ; echo 'successful'; sleep 30 && ip route del 10.0.0.0/8 via 10.20.3.5 dev eth0)
ping -c 1 $router_ip &>/dev/null || exit 0


IF Comparisons: The following flags (preceded by minus), represent

  • eq equal to
  • ne not equal to
  • lt less than
  • le less than or equal to
  • gt greater than
  • ge greater than or equal to

File Operations: The following flags (preceded by minus), represent :

  • s file exists and is not empty-f file exists and is not a directory
  • d directory exists
  • x file is executable
  • w file is writable
  • r file is readable

CONDITIONAL CLAUSES
IF
test operators, see this link 

#!/bin/bash
if [ "foo" = "foo" ]; then
   echo expression evaluated as true
fi


IN CLI (SINGLE LINE) - One-liner 

for s in ALL_SERVERS ; do ssh $s '/usr/whatever' ; done

Apply to multiple hosts with no need for a script: 

for i in admin-worker00.dc.mycompany1.co.uk \
centosGUI.dc.mycompany1.co.uk \
dbtest.dc.mycompany1.co.uk \
redis-admin.dc.mycompany1.co.uk \
security-scanner01.dc.mycompany1.co.uk \
test.dc.mycompany1.co.uk \
test01.dc.mycompany1.co.uk ; \
do ssh root@"$i" "ping -c 1 10.33.1.4"  ; done 
conns=`ps aux|grep java|egrep '(IM|IDS)'|awk '{print $2}'`;for i in $conns;do netstat -tulpan|grep $i|grep "ESTABLISHED"|egrep '(10.23.54.5|10.23.47.196)';done
if [ "$2" != "" ]; then diff output$1 expected$1; fi

Run whatever command in a set of nodes defined in file list.txt 

for in $i(cat list.txt); do ssh user@$i 'bash command'; done

EXECUTE SSH COMMAND IN A SET OF DEVICES FOR LOOP:

for host in $(seq --format='cc%02.0f' 01 17); do ssh $host "/sbin/ip link show|egrep '(em|eth|bond)[0-9]:' | cut -d: -f 2"| xargs -n 1 ssh $host ip link set mtu 9000 dev ; done

FOR
Examples: 

for i in {1..5}; do COMMAND-HERE; done
for((i=1;i<=10;i+=2)); do echo "Welcome $i times"; done
if [ "$2" != "" ]; then diff output$1 expected$1; fi
if [ -s data/temp ]; then echo "EXISTS"; else echo "doesnt"; fi


To automatically generate configurations (configuration generator) from a single line in the CLI. See this example: 

for i in {1..6}; do echo -e "conf t \n int fa0/0 \n ipv6 address 2001:150:1:$i::$i/128 \n no shut \n exit \n exit \n wr mem"; done
for f in *.HEIC; do echo "converting file $f"; heif-convert $f $f.jpg; done  # multiple commads (convert HEIC)
#!/bin/bash
while IFS=" " read -r val1 val2 rem 
do 
echo -e "
Host $val1
     HostName $val1.dc.mycompany1.co.uk
     User jaime_santos
     Port 22";
done < ./input


This one applies actions to all file in a folder one-liner: 

for f in *.mp3; do ffmpeg -i "$f" -vn -acodec libmp3lame -ac 2 -ab 160k -ar 48000 "encoded/${f%.avi}.mp4"; done

Small script with counter in for loop: 

#!/bin/bash
t=1
for i in {189..204}
do echo "
${t}
set services nat rule r2 match-direction input
set services nat rule r2 term t$t from source-address 10.8.14.11/32
set services nat rule r2 term t$t then translated source-prefix 148.64.56.$i/32
set services nat rule r2 term t$t then translated translation-type basic-nat44";
t=$[t+1];
done


The counter variable is not initialized in the for but the whole range is defined beforehand. 

for i in counter; do xxxx; done

http://www.cyberciti.biz/faq/bash-for-loop/
FOR cycle to generate repetitive text configuration file: Observe that variable is simply defined inside script and FOR iterates the text substituting variable per each of the lines: 

#!/bin/bash
var='
one
two
three'
for i in $var
do echo 'This is the configuration I want to follow: '$i';
done

And I will repeat '$i' many times' done

CASE http://www.freeos.com/guides/lsst/ch03sec08.html

BASH IDIOMS:
Redirection ( stdout , stderr ). All files have an id (descriptor) but also! stdout1 , stderr2 habe ids!. 

cat foo.txt 1> output.txt  same as cat foo.txt 1> output.txt
cat foo.txt 2> output.txt   # only the error messages 
ls foo > /dev/null 2>&1     # 2>&1 you are basically saying “Redirect the stderr to the same place we are redirecting the stdout”.   & is like the ind
foo.txt > output.txt 2>&1   # out and err both to file and and  screen
 > /dev/null 2>&1           # for the cron jobs. redirecting standard output into /dev/null, which is a place you can dump anything you don�t want (often called the bit-bucket), then redirecting standard error into standard output (you have to put an & in front of the destination when you do this).
 > /dev/null 2>&1          # This sends BOTH stdout and stderr to dev null
 
cat << EOF > file.txt
[text]   
EOF

This is an interesting way of generating a json file with a bash script using EOF (inside an ansible facts file): 

#!/bin/bash
user=$(who | cut -f 1 -d " " | tail -1 )
ipuser=$(who -u | cut -f 2 -d "(" | cut -f 1 -d ")" | head -1) 

cat << EOF 
{
  "user" : "$user",
  "ipuser" : "$ipuser"
}


Increment variable 

$ x=$[x+1]


Add to numeric variables 

num=`expr $num1 + $num2`


REGULAR EXPRESSIONS http://www.zytrax.com/tech/web/regex.htm
Caret ^ : Beginning of the string
Dollar $: End of the string

READ ALL LINES IN A FILE: 

value=0;


while read line
do
value=`expr $value + 1`;
echo $line;
done < "myfile"
echo $value;


 while read UNDOFILE ; do FILE=$( echo "$UNDOFILE" | sed -r -e 's/.un~$//' -e 's&/\.([^/]*)&/\1&' ) ; [[ -e "$FILE" ]] || rm "$UNDOFILE" ; done


while read ADD ; do ping -c 3 $ADD -i 0.2 ; done < ./adds

—- 

tester.sh
value=""
while read line
do
value="$value | grep -v $line ";
done < testex

value="grep -i erro * $value > ~/errors_cleaned"
# eval $value
echo $value
cd vars/$folder
grep -i erro * > ~/errors.txt
grep -i excep * > ~/exceptions.txt
grep -i warnin * > ~/warnings.txt

Test if the result of a serch is not empty: 

RESULT=$(find . -type f -mtime +1 | grep -v '/\.' | grep -v '.gz')
echo $RESULT
if [ ! -z "$RESULT" ] ; then
gzip $RESULT
else
echo "nothing to do"
fi

Read lines from two files and concatenate interleaved results (see 1.tar attached) 

START='tcpdump -ni eth1 -s 0 -C 100 -W 100 -w /home/jaime.santos/test/test vlan 4001 and (host 94.142.191.6 or (vlan and host 94.142.191.6))'ps -ef|grep -v grep|grep -F "$START"  >/dev/null 2>&1

case “$?” in 

0)
$NOP
;;
1)
$START >/dev/null 2>&1 &
#NOTICE=/tmp/watchdog.txt
#echo "$NAME was not running and was started on `$DATE`" > $NOTICE
#$MAIL -n -s "watchdog notice" -c $NOTIFYCC $NOTIFY < $NOTICE
#$RM -f $NOTICE
;;
esac
exit

FIND TRADES (FIELDS 150=D) IN TODAY'S TRADES 

find . -mtime -1 -exec cat -A {} \; | egrep "150=F"

CONFIGURATION SCRAPING

See the script named “bash_config_scraping1.tar.gz” attached

TO EVALUATE COMMANDS REMOTELY (SSH) 

ssh root@jaguar27.dc.mycompany1.co.uk "/usr/bin/sed -i '/^IPADDR=/d' /etc/sysconfig/network-scripts/ifcfg-bond0"

SEND FILE IN RUNTIME - HEREDOC
With substitution: 

#!/bin/bash
fruit='ONION'
cat <<EOF | ssh -p 5201 192.168.0.5 'cat - > /tmp/my_remote_file.txt'
apple
$fruit
green
tree
EOF

Without substitution: 

#!/bin/bash
fruit='ONION'
cat <<'EOF' | ssh -p 5201 192.168.0.5 'cat - > /tmp/my_remote_file.txt'
apple
$fruit
green
tree
EOF
scripting/bash.txt · Last modified: by 127.0.0.1

Page Tools

Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
CC Attribution-Share Alike 4.0 International Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki