Pages

Thursday, December 31, 2015

Where to get help (i.e. man pages)

Home

Man was originated from the linux programmer’s manual and its large enough to be multiple printed books
There are various sections in manual, below would be list of those section with basic purpose of each
1 – User commands – executable and shell programs
2 – System calls – kernel routines invoked from the user space
3 – Library functions – functions provided by program libraries
4 – Special files – example of this can be device files
5 – File formats – configuration files and structures
6 – Games – amusing programs
7 – Conventions, standards and miscellaneous – protocols, filesystems
8 – System administration and privileged commands – maintenance tasks
9 – Linux kernel APIs – internal kernel calls

A keyword search of man pages is performed using man –k keyword, below can be a nice example,

[root@serveros73 ~]# man -k passwd -l
chpasswd (8)         - update passwords in batch mode
gpasswd (1)          - administer /etc/group and /etc/gshadow
grub2-mkpasswd-pbkdf2 (1) - Generate a PBKDF2 password hash.
lpasswd (1)          - Change group or user password
pam_localuser (8)    - require users to be listed in /etc/passwd
passwd (1)           - update user's authentication tokens
sslpasswd (1ssl)     - compute password hashes
pwhistory_helper (8) - Helper binary that transfers password hashes from passwd or shadow to opasswd
[root@serveros73 ~]#

Popular system administration topics are in section 1 i.e user commands, section 5 i.e. file formats, section 8 administrative commands, section 2 i.e. system calls.

Some examples can be seen as below,
  • View su(1) man page – man su
  • Find command for tuning ext4 – man -k ext4
Back To Top
Home

What is bash/shell programming?

Home

The shell is a command interpreter. More than just the insulating layer between the operating system kernel and the user, it's also a fairly powerful programming language. 

Shell scripts are especially well suited for administrative system tasks and other routine repetitive tasks not requiring a full-blown tightly structured programming language.

Person who is working in or eager to work on Linux System administration domain can attend this course. Basic knowledge of Linux is must. 

Ability to think logically would help a person to learn things related to shell scripting faster.

A working knowledge of shell scripting is essential to anyone wishing to become reasonably proficient at system administration even if individual do not anticipate ever having to actually write a script.

This basic knowledge of bash scripting can be useful in various scripts used in process that is executed boot up process of Linux machine.

In Redhat/CentoOS 6, first process i.e. “init” executes the shell scripts in /etc/rc.d to restore the system configuration and set up services. A detailed understanding of these startup scripts is important for analyzing the behavior of a system and possibly modifying it for business or security related implementations.

Understanding special characters

Home

What makes a character special?  If it has a meaning beyond its literal meaning, a meta-meaning, then we refer to it as a special character.

Along with commands and keywords, special characters are building blocks of Bash scripts.
Comments or Hash (#)

This Special character (#) is used for adding lines to the script which will not be executed, a recent example can be Sha-Bang
Below are various scenarios of using Comment with different meaning for each one,
# This line is a comment.
Comment at start of line, making whole line not executable. 
echo "A comment will follow" # Comment here.
Comment at end part of line, making part of line not executable.
echo "The # here does not begin a comment."
Comment used as Text for printing in output, within double quotes
echo The \# here does not begin a comment.
Comment used as Text for printing in output, by escaping it
echo The # here begins a comment.
Comment used without double quotes
echo ${PATH#*:}      
Comment used as parameter substitution
echo $(( 2#101011 )); echo $(( 6#101 )) 
Comment used for conversion

Command separator or Semicolon (;)
Semicolon permits two commands on same line, it’s a command separator to be used in command line as well as inside a script.
echo hello; echo there

Terminator or double Semicolon (;;)
This is mainly used in case statement to show that commands to be executed as a part of a case criteria are finished. i.e. end of logic to be executed when a case is matched as script execution goes in sequence line by line.
case "$variable" in
  abc)  echo "\$variable = abc" ;;
  xyz)  echo "\$variable = xyz" ;;
esac

Dot (.)
When working with filenames, a leading dot is the prefix of a "hidden" file, a file that an ls will not normally show.

bash$ touch .hidden-file
bash$ ls -l      
total 10
 -rw-r--r--    1 sam  sam      4034 Jul 18 22:04 file1
 -rw-r--r--    1 sam  sam      4602 May 25 13:58 file2
 -rw-r--r--    1 sam  sam       877 Dec 17  2000 file3
bash$ ls -al      
total 14
 drwxrwxr-x    2 sam  sam      1024 Aug 29 20:54 ./
 drwx------   52 sam  sam      3072 Aug 29 20:51 ../
 -rw-r--r--    1 sam  sam      4034 Jul 18 22:04 file1
 -rw-r--r--    1 sam  sam      4602 May 25 13:58 file2
 -rw-r--r--    1 sam  sam       877 Dec 17  2000 file3
 -rw-rw-r--    1 sam  sam         0 Aug 29 20:54 .hidden-file

When considering directory names, a single dot represents the current working directory, and two dots denote the parent directory.

bash$ pwd
/home/sam/projects
bash$ cd .
bash$ pwd
/home/sam/projects
bash$ cd ..
bash$ pwd
/home/sam/

The dot often appears as the destination (directory) of a file movement command, in this context meaning current directory. Copy all the "junk" files to $PWD.
$ cp /home/sam/current_work/junk/* .

File name path separator or forward slash (/)
Separates the components of a filename (as in /home/sam/projects/Makefile).
This is also the division arithmetic operator.

Command substitution (`)
The `command` construct makes available the output of command for assignment to a variable. This is also known as backquotes or backticks.
$ echo $value   # NULL
$ pwd
$ value=`pwd`
$ echo $value   # Path obtained in pwd command output

Null command or colon (:)
This is the shell equivalent of a "NOP" (no op, a do-nothing operation).
The ":" command is itself a Bash built-in and its exit status is true (0).

$ :
$ echo $?   # 0
Endless loop using “:” can be constructed as below,

while : ; do echo `date`; sleep 20; done;

Same as that of traditional approach of writing loop as below,

while true ; do echo `date`; sleep 20; done;

Placeholder in if/then test:
if condition
then :   # Do nothing and branch ahead
else     # Or else ...
   some-action
fi

The ":" serves as a field separator in some OS level config files, an example can be /etc/passwd, and in the $PATH from environment variables.
$ echo $PATH
/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin:/sbin:/usr/sbin:/usr/games

Reverse (or negate) the sense of a test or exit status or exclamation (!)
The (!) operator inverts the exit status of the command to which it is applied.
It also inverts the meaning of a test operator.
For example, change the sense of equal ( = ) to not-equal ( != ).

Wild card or asterisk (*)
The * character serves as a "wild card" for filename expansion. By itself, it matches every filename in a given directory.

$ echo *   # gives name of files and folders in a particular directory
$ ls /*    # gives list of all folders and files within / along with listing of content of subdirectories to one level of /

In the context of arithmetic operations, the * denotes multiplication.
Dollar ($)
Character ($) is used for variable substitution or showing contents of a variable
var1=5
var2=23skidoo
echo $var1     # 5

A $ prefixing a variable name indicates the value the variable holds.

echo $var2     # 23skidoo

$ also indicates end-of-line. In a regular expression, a "$" addresses the end of a line of text.

find   /  -ls  |awk '{print $3, $NF}'  |grep Z$  # files engins with letter Z

${} can be used for parameter substitution.

$' ... ' i.e. Quoted string expansion. This construct expands single or multiple escaped octal or hex values
into ASCII [3] or Unicode characters.

$*, $@ are used as positional parameters.

$? provides exit status variable. The $? Variable holds the exit status of a command, a function, or of the script itself.

$ :
$ echo $?   # 0
$$ is a process ID variable. The $$ variable holds the process ID of the script in which it appears. below steps can be tried to understand its usage inside a script.
Create a file test.sh with below content.
#!/bin.bash
while :
do
echo $$  >> test.out.log
sleep 10
done
Execute file in background and observe the PID obtained when you get it ran in background
# sh test.sh >> test.out.log 2>&1
Once executed in background check output file test.out.log, it will keep on capturing a PID which should match with PID that you observed while getting job added to background for execution.
# cat test.out.log


Redirection (> &> >& >> < <>)
# command > out.log
Redirects the output of command to file out.log. Overwrite filename if it already exists. As error file not specified it will be shown on console.

# command > out.log 2> error.log
Redirects the output of command to file out.log and error of command to file error.log. Overwrite filename if it already exists.

command &> out.log
Redirects both the stdout and the stderr of command to file out.log.

command > out.log 2>&1
Another way to redirect both the stdout and the stderr of command to file out.log.

command >&2  
Redirects stdout of command to stderr. As error file not specified it will be shown on console.

command 2> error.log >&2 
Redirects stdout of command to stderr. As error file not specified it will be shown on console.

command >> out.log
Appends the output of scriptname to file filename. If filename does not already exist, it is created.

[i]<>in.file
Opens file in.file for reading and writing, and assigns file descriptor i to it. If filename does not exist, it is created.

Pipe (|)
This is a very frequently special character. Pipe passes the output (stdout) of a previous command to the input (stdin) of the next one, or to the shell.

echo ls -l | sh

Passes the output of "echo ls -l" to the shell, with the same result as a simple "ls -l".
cat *.lst | sort | uniq
Merges and sorts all ".lst" files, then deletes duplicate lines.
OR (||)
OR logical operator. In a test construct, the || operator causes a return of 0 (success) if either of the linked test conditions is true.
Single and (&)
Run job in background. A command followed by an & will run in the background.
$ sleep 10 &
[1] 850
[1]+ Done sleep 10

Double and (&&)
AND logical operator. In a test construct, the && operator causes a return of 0 (success) only if both the linked test conditions are true.

Option, prefix (-)

Option flag for a command or filter. Prefix for an operator. Prefix for a default parameter in parameter substitution.
COMMAND -[Option1][Option2][...]
ls -al
sort -dfu $filename

Home directory or tilde (~)
This corresponds to the $HOME internal variable. ~sam is sam's home directory, and ls ~sam lists the contents of it. ~/ is the current user's home directory, and ls ~/ lists the contents of it.
$ echo ~bozo
/home/bozo
$ echo ~
/home/bozo
$ echo ~/
/home/bozo/
$ echo ~:
/home/sam:

Escape or backslash (\)
One of the most important special character. In order to use other special characters well in either on command line or within script combined use of Escape and other special character is required. This concept it termed as escaping your special character.

Basic example can be below to escape # and print it in output.
$ echo The \# here does not begin a comment.

Back To Top
Home

Understanding type of files

Home

File types in a long list

Symbol
Meaning
-
Regular file
d
Directory
l
Link
c
Special file
s
Socket
p
Named pipe
b
Block device

Regular file

The regular file is a most common file type found on the Linux system. It governs all different files such us text files, images, binary files, shared libraries, etc. You can create a regular file with the touch command:

$ touch testfile.out
$ ls -ld testfile.out
-rw-rw-r-- 1 samual samual 0 Jan 10 12:52 testfile.out

The first character of the ls command, in this case "-", denotes the identification code for the regular file. Below command can be used to find out these type of files on system

$ find   /  -ls  |awk '{print $3, $NF}'  |grep ^-

To remove a regular file you can use the rm command:
$ rm testfile.out

Directory file

Directory is second most common file type found in Linux. Directory can be created with the mkdir command:
$ mkdir FileTypes
$ ls -ld FileTypes/
drwxrwxr-x 2 lubos lubos 4096 Jan 10 13:14 FileTypes/

Directory file can be identified by "d" symbol from the ls command output. Below command can be used to find out these type of files on system.

$ find   /  -ls  |awk '{print $3, $NF}'  |grep ^d

To remove empty  directory use the rmdir command.

$ rmdir FileTypes

When trying to remove directory with the rmdir command, which contains additional files you will get an error message:

rmdir: failed to remove `FileTypes/': Directory not empty

In this case you need to use a command:

$ rm -r FileTypes/

Character device file

Character device files allow users and programs to communicate with hardware peripheral devices. Terminal files are very easy example of such files.

The first character of the ls command, in this case "c", denotes the identification code for the character device file. Below command can be used to find out these type of files on system

$ find   /  -ls  |awk '{print $3, $NF}'  |grep ^c

Block device file

Block devices are similar to character devices. They mostly govern hardware as hard drives, memory, etc.

$ ls -ld /dev/sda
brw-rw---- 1 root disk 8, 0 Jan  4 10:12 /dev/sda

The first character of the ls command, in this case "b", denotes the identification code for the block device file. Below command can be used to find out these type of files on system

$ find / -ls |awk '{print $3, $NF}'  |grep ^b


Socket Files

Local domain sockets are used for communication between processes. Generally, they are used by services such as X windows, syslog and etc.

$ ls -ld /dev/log
srw-rw-rw- 1 root root 0 Jan  4 10:13 /dev/log

The first character of the ls command, in this case "b", denotes the identification code for the socket file. Below command can be used to find out these type of files on system

$ find   /  -ls  |awk '{print $3, $NF}'  |grep ^s

Sockets can be created by socket system call and removed by the unlink or rm commands.

Named pipe file

Similarly as Local sockets, named pipes allow communication between two local processes. They can be created by the mknod command and removed with the rm command.

The first character of the ls command, in this case "p", denotes the identification code for the named pipe file. Below command can be used to find out these type of files on system

$ find   /  -ls  |awk '{print $3, $NF}'  |grep ^l

Symbolic link file

With symbolic links an administrator can assign a file or directory multiple identities. Symbolic link can be a pointer to an original file. There are two types of symbolic links:

-          hard links
-          soft links

The difference between hard and soft links is that soft links use file name as reference and hard links use direct reference to the original file.  Furthermore, hard links cannot cross file systems and partitions. To create symbolic soft link we can use ln -s command:

$ echo file1 > file1
$ ln -s file1 file2
$ cat file2
file1
$ ls -ld file2
lrwxrwxrwx 1 lubos lubos 5 Jan 10 14:42 file2 -> file1

The first character of the ls command, in this case "l", denotes the identification code for the symbolic link file. Below command can be used to find out these type of files on system

$ find   /  -ls  |awk '{print $3, $NF}'  |grep ^l


To remove symbolic link we can use unlink or rm command.

Back To Top
Home

Wednesday, December 30, 2015

Starting off with a shebang

Home

Sha-Bang is the character sequence consisting of the characters number sign and exclamation mark (that is, "#!") at the beginning of a script.

When a script with a She-Bang is run as a program, the “program loader” parses the rest of the script's initial line as an “interpreter directive” Lets understand a bit what exactly these terms mean.
#!interpreter [optional-arg]

  • The interpreter must be an absolute path to an executable program 
  • The optional‑arg should either not be included or it should be a string that is meant to be a single argument 

Some typical shebang lines:

  • #!/bin/sh — Execute the file using sh, the Bourne shell, or a compatible shell
  • #!/bin/csh -f — Execute the file using csh, the C shell, or a compatible shell, and suppress the execution of the user’s .cshrc file on startup
  • #!/usr/bin/perl -T — Execute using Perl with the option for taint checks
  • #!/usr/bin/env python2 — Execute using Python by looking up the path to the Python interpreter automatically via env


Invoking a script with three versions

Considering that you have a script ready with you, you can invoke it by
sh scriptname
./scriptname ( This needs to make the script itself directly executable with a chmod.)
Either:
chmod 555 scriptname (gives everyone read/execute permission)
or
chmod +rx scriptname (gives everyone read/execute permission)
or
chmod u+rx scriptname (gives only the script owner read/execute permission)

Extra tricks with Sha-Bang
Try setting other values for Sha-Bang interpreter
- /bin/rm
- Multiple Sha-Bang in single script

Friday, December 18, 2015

Working with Network on CentOS/RHEL 07 - Part 12

Home

Configure DHCP IP address using configuration files

- We will simply see a basic example of changing IP address to DHCP mode using “ifcfg-*” files present in network-script directory of NetworkManager Daemon service.
- First check for the existing content in the configuration file

[root@myserver2 network-scripts]# cat ifcfg-Wired_connection_3
TYPE=Ethernet
BOOTPROTO=none
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=no
NAME=Wired_connection_3
UUID=46a40de1-e50a-4b1f-90e4-413cc53a34e9
DEVICE=enp0s9
ONBOOT=yes
HWADDR=08:00:27:B6:50:F3
IPADDR=192.168.248.100
PREFIX=24

[root@myserver2 network-scripts]# ip a sh dev enp0s9
4: enp0s9: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 08:00:27:b6:50:f3 brd ff:ff:ff:ff:ff:ff
    inet 192.168.248.100/24 brd 192.168.248.255 scope global enp0s9
       valid_lft forever preferred_lft forever
    inet6 fe80::a00:27ff:feb6:50f3/64 scope link
       valid_lft forever preferred_lft forever

- Now time to get some changes done to the configuration file, in our example we will change the IP address so new IP

[root@myserver2 network-scripts]# vi ifcfg-Wired_connection_3
[root@myserver2 network-scripts]# cat ifcfg-Wired_connection_3
TYPE=Ethernet
BOOTPROTO=dhcp
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=no
NAME=Wired_connection_3
UUID=46a40de1-e50a-4b1f-90e4-413cc53a34e9
DEVICE=enp0s9
ONBOOT=yes
HWADDR=08:00:27:B6:50:F3
#IPADDR=192.168.248.100
#PREFIX=24

- Post changes in the configuration file, to get those reflected get the interface down and up using ifdown and ifup commands.

[root@myserver2 network-scripts]# ifdown enp0s9
[root@myserver2 network-scripts]# nmcli dev status
DEVICE   TYPE      STATE         CONNECTION
enp0s3   ethernet  connected     Wired_connection_1
enp0s8   ethernet  connected     Wired_connection_2
enp0s10  ethernet  disconnected  --
enp0s9   ethernet  disconnected  --
lo       loopback  unmanaged     --
[root@myserver2 network-scripts]# ifup enp0s9
Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/7)
[root@myserver2 network-scripts]# nmcli dev status
DEVICE   TYPE      STATE         CONNECTION
enp0s3   ethernet  connected     Wired_connection_1
enp0s8   ethernet  connected     Wired_connection_2
enp0s9   ethernet  connected     Wired_connection_3
enp0s10  ethernet  disconnected  --
lo       loopback  unmanaged     --
[root@myserver2 network-scripts]# ip a sh dev enp0s9
4: enp0s9: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 08:00:27:b6:50:f3 brd ff:ff:ff:ff:ff:ff
    inet 192.168.248.203/24 brd 192.168.248.255 scope global dynamic enp0s9
       valid_lft 1174sec preferred_lft 1174sec
    inet6 fe80::a00:27ff:feb6:50f3/64 scope link
       valid_lft forever preferred_lft forever

- Permanent changes has been made successfully to the interface using configuration files.