Nagios Plugin Apache - "check_apachestatus_auto.pl"

1. DESCRIPTION

- Plugin Name :-  check_apachestatus_auto.pl
- Plugin Purpose :- To check for performance related parameters for Apache Server

2. PRE-REQUISITS

- Perl modules needed are strict;  Getopt::Long;  LWP::UserAgent;  Time::HiRes qw(gettimeofday tv_interval);  Digest::MD5 qw(md5 md5_hex);
- Nagios plugins repository
- Mod_status enabled on Remote Apache Server.

3. PLUGIN DETAILS

- Plugin is written in Perl
- Plugin simply checks for performance parameter details and provides a summary of it for apache server
- Based on configured threshold for the specifed parameter alert is thrown

4. INPUT

- IP address of Cassandra Database or Hostname of Cassandra Database (Note:- Hostname is getting resolved from Nagios Server)

5. OUTPUT

Possible outputs are as below,
- “UNKNOWN 500 Can't connect to XX.XXX.XX.XXX:80 (Bad hostname 'XX.XXX.XX.XXX')”
Whenever plugin execution is not able to reach the specified apache host XX.XXX.XX.XXX

- “APACHE OK - 0.018 sec. response time, Busy/Idle 1/10, open 245/256, ReqPerSec 1.2, BytesPerReq 17612, BytesPerSec 22016|Idle=10 Busy=1 OpenSlots=245 Slots=256 Starting=0 Reading=0 Sending=1 Keepalive=0 DNS=0 Closing=0 Logging=0 Finishing=0 ReqPerSec=1.250000 BytesPerReq=17612 BytesPerSec=22016.000000 Accesses=5”
Successful execution of plugin gives above summary information about remote apache server.

6. EXAMPLE

/usr/lib64/nagios/plugins/check_apachestatus_auto.pl -H [ IP Address/Hostname ] -w [ Warning Threshold ] -c [ Critical Threshold ]

7. Plugin can be downloaded from Link

Nagios Plugin Mysql DB - "check_mysql_health"

1. DESCRIPTION

- Plugin Name :-  check_mysql_health
- Plugin Purpose :- To check MySQL Database Server for various parameters on performance and health level

2. PRE-REQUISITS

- MySQL Client packages i.e. “mysql”
- Database level access username and password

3. PLUGIN DETAILS

- Plugin simply checks for MySQL server information on the basis of available options for monitoring.
- Based on configured threshold for various available options alert is thrown

4. INPUT

--hostname
IP address of MySQL Database Server or Hostname of MySQL Database Server (Note:- Hostname is getting resolved from Nagios Server)
--port
Port number (default: 3306) for MySQL Database Server
--username
Username to connect to MySQL Database Server
--password
Password to connect to MySQL Database Server
--database
Database name to be checked for in MySQL Database Server
--warning
WARNING status if Value is more than INTEGER value for specified parameter/mode of plugin output for MySQL Database Server
--critical
CRITICAL status if Value is more than INTEGER value for specified parameter/mode of plugin output for MySQL Database Server
--mode
• connection-time - Time to connect to MySQL Database server
• uptime - Time MySQL Database server is running
• threads-connected - Number of currently open connections on MySQL Database Server
• threadcache-hitrate - Hit rate of the thread-cache on MySQL Database Server
• threads-created - Number of threads created per sec on MySQL Database Server
• threads-running - Number of currently running threads on MySQL Database Server
• threads-cached - Number of currently cached threads on MySQL Database Server
• connects-aborted - Number of aborted connections per sec on MySQL Database Server
• clients-aborted - Number of aborted connections (because the client died) per sec on MySQL Database Server
• slave-lag - MySQL Database Server is X seconds behind master
• slave-io-running - Slave io running on MySQL Database Server: Yes
• slave-sql-running - Slave sql running on MySQL Database Server: Yes
• qcache-hitrate - Query cache hitrate on MySQL Database Server
• keycache-hitrate - MyISAM key cache hitrate on MySQL Database Server
• bufferpool-hitrate - InnoDB buffer pool hitrate MySQL Database Server
• bufferpool-wait-free - InnoDB buffer pool waits for clean page available on MySQL Database Server
• log-waits - InnoDB log waits because of a too small log buffer on MySQL Database Server
• tablecache-hitrate - Table cache hitrate on MySQL Database Server
• table-lock-contention - Table lock contention on MySQL Database Server
• index-usage - Usage of indices on MySQL Database Server
• tmp-disk-tables - Percent of temp tables created on disk on MySQL Database Server
• table-fragmentation - Show tables which should be optimized on MySQL Database Server
• open-files - Percent of opened files on MySQL Database Server
• slow-queries - Slow queries running on MySQL Database Server
• long-running-procs- long running processes on MySQL Database Server
• sql - any sql command to be executed on MySQL Database Server
--name
The name of something that needs to be further specified, currently only used for sql statements
--name2
if name is a sql statement, this statement would appear in the output and the performance data. This can be ugly, so name2 can be used to appear instead.

5. OUTPUT

Possible outputs are as below,
- “table CHARACTER_SETS is 0.00% fragmented, table COLLATIONS is 0.00% fragmented, table COLLATION_CHARACTER_SET_APPLICABILITY is 0.00% fragmented, table COLUMNS is 0.00% fragmented, table COLUMN_PRIVILEGES is 0.00% fragmented, table KEY_COLU”
Fragmentation related health check output.

- “index_usage=0.01%;60;80 index_usage_now=0.00%”
Index usage on MySQL Database Server

- “pct_open_files=0.299%;40.000;50.000 open_files=299;40000;50000”
Open files in use on MySQL Database Server

- “qps=0;50;100”
Query Per Seconds running on MySQL Database Server

- “slave_lag=0;1;2”
Slave LAG for an MySQL Database Server

- “slow_queries_rate=0.00%;5;10”
Number of slow running queries on MySQL Database Server

- “connection_time=0.0157s;0;0”
Connection time to MySQL Database Server

6. EXAMPLE

/usr/lib64/nagios/plugins/check_mysql_health --hostname [ HOSTADDRESS ] --port [ PORT ]  --username [ DBUser ] --password [ DBPass ] --database [ Database ] --warning [ Warning ] --critical [ Critical ] --mode connection-time

/usr/lib64/nagios/plugins/check_mysql_health --hostname [ HOSTADDRESS ] --port [ PORT ]  --username [ DBUser ] --password [ DBPass ] --database [ Database ] --warning [ Warning ] --critical [ Critical ] --mode uptime

/usr/lib64/nagios/plugins/check_mysql_health --hostname [ HOSTADDRESS ] --port [ PORT ]  --username [ DBUser ] --password [ DBPass ] --database [ Database ] --warning [ Warning ] --critical [ Critical ] --mode threads-connected

/usr/lib64/nagios/plugins/check_mysql_health --hostname [ HOSTADDRESS ] --port [ PORT ]  --username [ DBUser ] --password [ DBPass ] --database [ Database ] --warning [ Warning ] --critical [ Critical ] --mode threadcache-hitrate

/usr/lib64/nagios/plugins/check_mysql_health --hostname [ HOSTADDRESS ] --port [ PORT ]  --username [ DBUser ] --password [ DBPass ] --database [ Database ] --warning [ Warning ] --critical [ Critical ] --mode threads-created

/usr/lib64/nagios/plugins/check_mysql_health --hostname [ HOSTADDRESS ] --port [ PORT ]  --username [ DBUser ] --password [ DBPass ] --database [ Database ] --warning [ Warning ] --critical [ Critical ] --mode threads-running

/usr/lib64/nagios/plugins/check_mysql_health --hostname [ HOSTADDRESS ] --port [ PORT ]  --username [ DBUser ] --password [ DBPass ] --database [ Database ] --warning [ Warning ] --critical [ Critical ] --mode threads-cached

/usr/lib64/nagios/plugins/check_mysql_health --hostname [ HOSTADDRESS ] --port [ PORT ]  --username [ DBUser ] --password [ DBPass ] --database [ Database ] --warning [ Warning ] --critical [ Critical ] --mode connects-aborted

/usr/lib64/nagios/plugins/check_mysql_health --hostname [ HOSTADDRESS ] --port [ PORT ]  --username [ DBUser ] --password [ DBPass ] --database [ Database ] --warning [ Warning ] --critical [ Critical ] --mode clients-aborted

/usr/lib64/nagios/plugins/check_mysql_health --hostname [ HOSTADDRESS ] --port [ PORT ]  --username [ DBUser ] --password [ DBPass ] --database [ Database ] --warning [ Warning ] --critical [ Critical ] --mode slave-lag

/usr/lib64/nagios/plugins/check_mysql_health --hostname [ HOSTADDRESS ] --port [ PORT ]  --username [ DBUser ] --password [ DBPass ] --database [ Database ] --warning [ Warning ] --critical [ Critical ] --mode slave-io-running

/usr/lib64/nagios/plugins/check_mysql_health --hostname [ HOSTADDRESS ] --port [ PORT ]  --username [ DBUser ] --password [ DBPass ] --database [ Database ] --warning [ Warning ] --critical [ Critical ] --mode slave-sql-running

/usr/lib64/nagios/plugins/check_mysql_health --hostname [ HOSTADDRESS ] --port [ PORT ]  --username [ DBUser ] --password [ DBPass ] --database [ Database ] --warning [ Warning ] --critical [ Critical ] --mode qcache-hitrate

/usr/lib64/nagios/plugins/check_mysql_health --hostname [ HOSTADDRESS ] --port [ PORT ]  --username [ DBUser ] --password [ DBPass ] --database [ Database ] --warning [ Warning ] --critical [ Critical ] --mode bufferpool-hitrate

/usr/lib64/nagios/plugins/check_mysql_health --hostname [ HOSTADDRESS ] --port [ PORT ]  --username [ DBUser ] --password [ DBPass ] --database [ Database ] --warning [ Warning ] --critical [ Critical ] --mode keycache-hitrate

/usr/lib64/nagios/plugins/check_mysql_health --hostname [ HOSTADDRESS ] --port [ PORT ]  --username [ DBUser ] --password [ DBPass ] --database [ Database ] --warning [ Warning ] --critical [ Critical ] --mode log-waits

/usr/lib64/nagios/plugins/check_mysql_health --hostname [ HOSTADDRESS ] --port [ PORT ]  --username [ DBUser ] --password [ DBPass ] --database [ Database ] --warning [ Warning ] --critical [ Critical ] --mode tablecache-hitrate

/usr/lib64/nagios/plugins/check_mysql_health --hostname [ HOSTADDRESS ] --port [ PORT ]  --username [ DBUser ] --password [ DBPass ] --database [ Database ] --warning [ Warning ] --critical [ Critical ] --mode table-lock-contention

/usr/lib64/nagios/plugins/check_mysql_health --hostname [ HOSTADDRESS ] --port [ PORT ]  --username [ DBUser ] --password [ DBPass ] --database [ Database ] --warning [ Warning ] --critical [ Critical ] --mode index-usage

/usr/lib64/nagios/plugins/check_mysql_health --hostname [ HOSTADDRESS ] --port [ PORT ]  --username [ DBUser ] --password [ DBPass ] --database [ Database ] --warning [ Warning ] --critical [ Critical ] --mode tmp-disk-tables

/usr/lib64/nagios/plugins/check_mysql_health --hostname [ HOSTADDRESS ] --port [ PORT ]  --username [ DBUser ] --password [ DBPass ] --database [ Database ] --warning [ Warning ] --critical [ Critical ] --mode table-fragmentation

/usr/lib64/nagios/plugins/check_mysql_health --hostname [ HOSTADDRESS ] --port [ PORT ]  --username [ DBUser ] --password [ DBPass ] --database [ Database ] --warning [ Warning ] --critical [ Critical ] --mode open-files

/usr/lib64/nagios/plugins/check_mysql_health --hostname [ HOSTADDRESS ] --port [ PORT ]  --username [ DBUser ] --password [ DBPass ] --database [ Database ] --warning [ Warning ] --critical [ Critical ] --mode slow-queries

/usr/lib64/nagios/plugins/check_mysql_health --hostname [ HOSTADDRESS ] --port [ PORT ]  --username [ DBUser ] --password [ DBPass ] --database [ Database ] --warning [ Warning ] --critical [ Critical ] --mode [ SQL QUERY ] --name [ **** ] --name2 [ **** ]

7. Plugin can be downloaded from Link

Nagios Plugin Mysql DB - "check_mysql"

1. DESCRIPTION

- Plugin Name :-  check_mysql
- Plugin Purpose :- To check health summary of MySQL Server

2. PRE-REQUISITS

- MySQL Client packages i.e. “mysql”

3. PLUGIN DETAILS

- Plugin simply checks for statitics of MySQL Server
- Based on “Excetpions” variable value alert is thrown

4. INPUT

-H, --hostname=ADDRESS
IP address or Hostname of MySQL Database Server (Note:- Hostname is getting resolved from Nagios Server)
-P, --port=INTEGER
Port number (default: 3306) for MySQL Database Server
-d, --database=STRING
Database name to be checked for in MySQL Database Server
-u, --username=STRING
Username to connect to MySQL Database Server
-p, --password=STRING
Password to connect to MySQL Database Server
-S, --check-slave
Check slave’s running status on MySQL Database Server
-w, --warning
WARNING status if slave server is more than INTEGER seconds behind master of MySQL Database Server
-c, --critical
CRITICAL status if slave server is more then INTEGER seconds behind master MySQL Database Server

5. OUTPUT

Possible outputs are as below,
- “ Uptime: 1398528  Threads: 10  Questions: 350332599  Slow queries: 236  Opens: 19348  Flush tables: 1  Open tables: 64  Queries per second avg: 250.501 “
Whenever plugin execution completes it gets the statitics of MySQL node as specified above.

6. EXAMPLE

/usr/lib64/nagios/plugins/check_mysql  -H [ HOSTADDRESS ] -P [ Port ]-d [ Database ] -u [ DBUser ] -p [ DBPass ]

7. Plugin can be found in default installation of Nagios Core or Nagios XI

Nagios Plugin for Nginx - "check_nginx_status.pl"

1. DESCRIPTION

- Plugin Name :-  check_nginx_status.pl
- Plugin Purpose :- To check NGINX Server performance stats

2. PRE-REQUISITS

- Status Page defined on Remote NGINX Server to be monitored.
- Access available to Status Page for Server where this plugin will be installed and fired from.
- Perl modules required,
use strict; use Getopt::Long; use LWP::UserAgent; use Time::HiRes qw(gettimeofday tv_interval); use Digest::MD5 qw(md5 md5_hex); use FindBin;
- Nagios Default Plugin repository is mandetory

3. PLUGIN DETAILS

- Plugin is written in Bash
- Plugin simpley checked for the node information of cassandra and checked for value of variable “Exceptions”
- Based on “Excetpions” variable value alert is thrown

4. INPUT

-H, --hostname=HOST
IP address of NGINX Server or Hostname of NGINX Server (Note:- Hostname is getting resolved from Nagios Server)
-p, --port=PORT
Http port being used by NGINX Server
-u, --url=URL
Context URL to i.e. /nginx_status instead of the default "http://<hostname or IP>/nginx_status"
-S, --ssl
Wether Remote server is using HTTPS instead of HTTP this argument is compulsary
-U, --user=user
Username for basic authentication supported on Remote NGINX Server
-P, --pass=PASS
Password for basic authentication supported on Remote NGINX Server
-r, --realm=REALM
Realm for basic authentication supported on Remote NGINX Server
-m, --maxreach=MAX
Number of max processes reached (since last check) that should trigger an alert for NGINX server
-t, --timeout=INTEGER
Timeout in seconds (Default: 15) for making connection with NGINX server
-w, --warn=ACTIVE_CONN,REQ_PER_SEC,CONN_PER_SEC
Number of active connections, ReqPerSec or ConnPerSec that will cause a WARNING if set to -1 then it means no warning
-c, --critical=ACTIVE_CONN,REQ_PER_SEC,CONN_PER_SEC
Number of active connections, ReqPerSec or ConnPerSec that will cause a CRITICAL if set to -1 then it means no CRITICAL

5. OUTPUT

Possible outputs are as below,
- “NGINX CRITICAL - 500 Can't connect to XX.XXX.XX.XX:YY (Bad hostname 'XX.XXX.XX.XX')”
Whenever plugin execution is not able to reach the specified NGINX Server i.e. XX.XXX.XX.XX for port YY

- “NGINX CRITICAL - 404 Not Found”
In case the Context URL passed to Plugin is not present on Remote NGINX server or may be miss spelled can lead to above output.

- “NGINX OK -  0.018 sec. response time, Active: 1 (Writing: 1 Reading: 0 Waiting: 0) ReqPerSec: 0.000 ConnPerSec: 0.000 ReqPerConn: 0.186|Writing=1;;;; Reading=0;;;; Waiting=0;;;; Active=1;;;; ReqPerSec=0.000000;;;; ConnPerSec=0.000000;;;; ReqPerConn=0.186081;;;;”
Post successful execution of plugin remote NGINX stats are captured in above format.

- Sample output of status page for NGINX
--------------------------------------------
Active connections: 2
server accepts handled requests
 151994 151995 50163
Reading: 0 Writing: 1 Waiting: 1
--------------------------------------------

- Capture values from the Status page as below,
• Active Connection as "Active connections: 2"
• Connection in waiting state as "Waiting: 1"
• Connection in reading state as "Reading: 0"
• Connection in writing state as "Writing: 1"
• Accepted connections count as 151994
• Handled connections count as 151995
• Number of requests count as 50163
• "Connection Per Second"= ("Last Accepted connections count" - "Current Accepted connections count")/"Difference between two checks"
• "Request Per Second" = ("Last Number of Request count" - "Current Number of Request count")/"Difference between two checks";
• "Request Per Connection" = ("Request Per Second")/("Connection Per Second")

6. EXAMPLE

/usr/lib64/nagios/plugins/check_nginx_status.pl -H [ IP Address/Hostname ] -p [ Port ] -u [ Context URL ] -w [ Active Connection Warning Threshold ],[Request Per Seconds Warning Threshold],[ Connection Per Seconds Warning Threshold] -c [ Active Connection Critical Threshold ],[Request Per Seconds Critical Threshold],[ Connection Per Seconds Critical Threshold]

7. Plugin can be downloaded from Link

How To - Ansible adhoc commands (One Liners)

Make a note of below assumptions for one liners explained on this page are as below,

User root - the superhero, superuser
User normaluser - user that has root sudo privilege and all ansible required connectivity in place User user1 - user that has root sudo privilege

Understanding basic differences in commands using various users with various rights

To run command on all servers to check reach ability from ansible point of view with 25 parallel forks with current user (i.e. root or normaluser)

# ansible all -m ping -f 25

To run command on all servers to check reach ability from ansible point of view with 25 parallel forks when you logged in as normal user (user1) and using ansible privilege of another normal user (normaluser)

# sudo -u normaluser ansible all -m ping -f 25

To run a command on list of servers (IP addresses) specified in a text file with 25 parallel forks with current user (i.e. root or normaluser)

# ansible all -i /tmp/my-inventory -m ping -f 25

To run a command on list of servers (IP addresses) specified in a text file with 25 parallel forks when you logged in as normal user (user1) and using ansible privilege of another normal user (normaluser)

# sudo -u normaluser ansible all -i /tmp/my-inventory -m ping -f 25

To run command on all servers in a group with 25 parallel forks with current user (i.e. root or normaluser)

# ansible all -a "df -h" -f 25

To run command on all servers in a group with 25 parallel forks when you logged in as normal user (user1) and using ansible privilege of another normal user (normaluser)

# sudo -u normaluser ansible all -a "df -h" -f 25

To run command which requires super user rights on all servers in a group with 25 parallel forks with current user (i.e. root)

# ansible all -a "/etc/init.d/snmpd restart" -f 25

To run command which requires super user rights on all servers in a group with 25 parallel forks with current user (i.e. normaluser)

# ansible all -a "/etc/init.d/snmpd restart" -f 25 --sudo 

To run command which requires super user rights on all servers in a group with 25 parallel forks when you logged in as normal user (user1) and using ansible privilege of another normal user (normaluser)

# sudo -u normaluser ansible all -a "/etc/init.d/snmpd restart " -f 25 --sudo 

Ping Module

To limit ping for random first 10 servers from a group of hosts defined in /etc/ansible/hosts file

# sudo -u normaluser all dev -m ping --limit allvmhosts[0-10]

To limit Ping some servers from "api" group of hosts defined in /etc/ansible/hosts file i.e. [11 to 20]

# sudo -u normaluser ansible all -m ping --limit allvmhosts[11-20]

In case of JSON Error "Error: ansible requires a json module, none found!" Try your command in raw mode as below

# sudo -u normaluser ansible all -m ping  -m raw

Service Module

To install a service on all servers in a group using service module

# ansible all -m service -a "name=snmpd state=installed"

To start a service on all servers in a group using service module

# ansible all -m service -a "name=snmpd state=started"

To stop a service on all servers in a group using service module

# ansible all -m service -a "name=snmpd state=stopped"

To restart a service on all servers in a group using service module

# ansible all -m service -a "name=snmpd state=restarted"

To reload a service on all servers in a group using service module

# ansible all -m service -a "name=snmpd state=reloaded"

To enable a service on all servers in a group using service module to get started in OS startup/bootup

# ansible all -m service -a "name=snmpd enabled=yes"

To disable a service on all servers in a group using service module to get started in OS startup/bootup

# ansible all -m service -a "name=snmpd enabled=no"

User Module

To add a user 'sam' and assign shell as /bin/bash, secondary group as wheel to it on a group on all servers in a group

# sudo -u normaluser ansible all -m user -a "name=sam shell=/bin/bash groups=wheel"

To add the user 'sam' with a specific uid and a primary group of 'admin' on all servers in a group

# sudo -u normaluser ansible all -m user -a "name=sam comment="Samual Pol" uid=1040 group=admin"

To add the user 'sam' with a bash shell, appending the group 'admins' and 'developers' to the user's groups on all servers in a group

# sudo -u normaluser ansible all -m user -a "name=sam shell=/bin/bash groups=admins,developers append=yes"

To Remove the user 'sam' from all servers in a group

# sudo -u normaluser ansible all -m user -a "name=sam state=absent remove=yes"

Shell Module

To create a backup file from command line with sudo on a group

# sudo -u normaluser ansible all -m shell -a "cp /etc/sudoers /etc/sudoer_DDMMYYYY" --sudo

To replace content of a file with special characters with sudo on a group

# sudo -u normaluser ansible all -m shell -a "sed -i 's|\# %wheel\tALL=(ALL)\tALL|%wheel\tALL=(ALL)\tALL|g' /etc/sudoers" --sudo

To change primary group of a user on a group

# sudo -u normaluser ansible all -m shell -a "usermod -aG wheel idcadm"

To execute the command in remote shell by specifying stdout going to the specified file on all remote servers in a group

# sudo -u normaluser ansible all -m shell -a "somescript.sh >> somelog.txt"

To change the working directory to somedir/ before executing the command

# sudo -u normaluser ansible all -m shell -a "somescript.sh >> somelog.txt chdir=somedir/"

File Module =

To change permissions of a file on all servers in a group

# sudo -u normaluser ansible all -m file  -a "path=/usr/bin/sudo  state=touch mode='4755' " --sudo 

To change file ownership, group and mode using octal numbers (Note: first digit should always be 0 whenever octal used)

# sudo -u normaluser ansible all -m file  -a "path=/etc/ntp.conf owner=ntp group=ntp mode=0644"

To create symbolic link

# sudo -u normaluser ansible all -m file  -a "src=/file/to/link/to dest=/path/to/symlink owner=sam group=sam state=link"

To touch a file, using symbolic modes to set the permissions (equivalent to 0644)

# sudo -u normaluser ansible all -m file  -a "path=/etc/test.conf state=touch mode='u=rw,g=r,o=r'"

Replace Module

To replace a particular line in a file on a group

# sudo -u normaluser ansible all -m replace -a "dest=/root/test.sh regexp='^(\sread -p \"gateway recover after nodes \" gateway_recover_after_nodes)$' replace='XXXXXXXXX'"

lineinfile Module

To append a line in file use lineinfilemodule on a group

# sudo -u normaluser ansible all -m lineinfile  -a "dest=/etc/sudoers_07May2015 line='# Sudo access for running passwd command'"

Yum Module

TODO

Apt Module

TODO

Group Module

TODO

Mail Module

TODO

Selinux Module

TODO