Linux-Some handy commands

How to check which process is using up the most disk resources
iotop -Use the “input output top” command to investigate which process is doing the most IO in real time.

#iotop

Total DISK READ: 0.00 B/s | Total DISK WRITE: 15.57 K/s
TID PRIO USER DISK READ DISK WRITE SWAPIN IO> COMMAND
946 be/3 root 0.00 B/s 3.89 K/s 0.00 % 0.07 % [jbd2/dm-4-8]
27526 be/4 root 0.00 B/s 3.89 K/s 0.00 % 0.00 % rotatelogs /opt/www/logs/webserver/apache/access_log-%Y-%m-%d-%H 86400
27527 be/4 root 0.00 B/s 3.89 K/s 0.00 % 0.00 % rotatelogs /opt/www/logs/webserver/apache/mod_jk.log-%Y-%m-%d-%H 86400

Capture OS level data to show CPU and Memory usage Thread wise

#top -b -n 1 -H

Or for a specific Process

#top -b -n 1 -H <Process_PID>

Top 10 CPU consuming Process

#ps -eo pcpu,pid,user,args | sort -k 1 -r | head -10

Comparison of CPU utilization

# sar -u 5 10 ( 5 sec of interval & 10 times )

Apache Web Server – How Name based Virtual host works

The ServerName and ServerAlias is more like a company’s internal phone list. Your webserver is the switchboard; it will accept all incoming connections to the server. Then the client/caller will tell them what name they’re looking for, and it will look in the Apache configuration for how to handle that name. If the name isn’t listed as a ServerName/ServerAlias in the apache configuration, apache will always give them the first VirtualHost listed. Or, if there’s no VirtualHost at all, it will give the same content no matter what hostname is given in the request.

So, step by step for a normal connection:
1.  You type “http://www.example.com” into your browser.
2.  Your computer asks its DNS resolver which IP address it should use when it wants to talk to www.example.com.
3.  Your computer connects to that IP address, and says that it wants to talk to www.example.com (that’s the Host:header in HTTP).
4.  The webserver looks at its configuration to figure out what to do with a request for content from www.example.com.

Any one of the following may happen:

  • www.example.com is listed as a ServerName or ServerAlias for a VirtualHost – if so, then it will use the configuration for that VirtualHost to deliver the content.
  • The server doesn’t have any VirtualHosts at all – if so, then it will use the configuration in its httpd.conf to deliver the content.
  • The server has VirtualHosts but www.example.com isn’t listed in any of them – if so, the first Virtualhost in the list will be used to deliver the content.

<VirtualHost *:80>
# Admin email, Server Name (domain name), and any aliases
ServerAdmin [email protected]
ServerName abc.com
ServerAlias example.com
…

Apache Virtual Hosting

There are two types Apache Virtual Hosting

Name Based Virtual Hosting

With the name based virtual hosting you can host several domains/websites on a single machine with a single IP. All domains on that server will be sharing a single IP. It’s easier to configure than IP based virtual hosting, you only need to configure DNS of the domain to map it with its correct IP address and then configure Apache to recognize it with the domain names.
A technical prerequisite needed for name-based virtual hosts is a web browser with HTTP/1.1 support (commonplace today) to include the target hostname in the request. This allows a server hosting multiple sites behind one IP address to deliver the correct site’s content. More specifically it means setting the Host HTTP header.

Name

IP Based Virtual Hosting

With the IP based virtual hosting, you can assign a separate IP for each domain on a single server, these IP’s can be attached to the server with single NIC cards and as well as multiple NICs.

IP

JDK VS JRE

JDK-JRE

JRE: Java Runtime Environment– It is basically the Java Virtual Machine where your Java programs run on.If you want to run Java programs, but not develop them, download the Java Runtime Environment.

JDK: Java Development Kit -If you want to develop applications for Java, download the Java Development Kit or JDK. It’s the full featured Software Development Kit for Java, including JRE, and the compilers and tools (like JavaDoc, and Java Debugger) to create and compile programs.

Usually, when you only care about running Java programs on your browser or computer you will only install JRE. It’s all you need. On the other hand, if you are planning to do some Java programming, you will also need JDK.

Sometimes, even though you are not planning to do any Java Development on a computer, you still need the JDK installed. For example, if you are deploying a WebApp with JSP, you are technically just running Java Programs inside the application server. Why would you need JDK then? Because application server will convert JSP into Servlets and use JDK to compile the servlets.

mod_cluster VS mod_jk and mod_proxy

mod_cluster advantages over other httpd-based load balancers mod_jk and mod_proxy

Server-side load balance factor calculation

In contrast with traditional httpd-based load balancers, mod_cluster uses load balance factors calculated and provided by the application servers, rather than computing these in the proxy. Consequently, mod_cluster offers a more robust and accurate set of load metrics than is available from the proxy.

Dynamic configuration of httpd workers

Traditional httpd-based load balancers require explicit configuration of the workers available to a proxy.
Like mod_jk maintain the static worker list in worker.properties but mod-cluster allow dynamic worker list means whenever the user load increases you can add Application server nodes dynamically to balance the load. In mod_cluster, the bulk of the proxy’s configuration resides on the application servers. The set of proxies to which an application server will communicate is determined either by a static list or using dynamic discovery via the advertise mechanism.
The application server relays life-cycle events (e.g. server startup/shutdown) to the proxies allowing them to effectively auto-configure themselves. Notably, the graceful shutdown of a server will not result in a fail-over response by a proxy, as is the case with traditional httpd-based load balancers.

Fine grained web-app lifecycle control

Traditional httpd-based load balancers do not handle web application undeployments particularly well.
From the proxy’s perspective requests to an un-deployed web application are indistinguishable from a request for an non-existent resource, and will result in 404 errors. In mod_cluster, each server forwards any web application context lifecycle events (e.g. web-app deploy/un-deploy) to the proxy informing it to start/stop routing requests for a given context to that server.

mod_cluster advantages over mod_jk

AJP is optional

Unlike mod_jk, mod_cluster does not require AJP. httpd connections to application server nodes can use HTTP, HTTPS, or AJP.

Install local rpm files using yum

We can use the command yum localinstall /path/to/file.rpm file. This command will install the local rpm file and search for required rpms (dependencies) on RHN or other repositories that are configured and install it for the user.

#yum localinstall /path/to/RPM_FILE.rpm

Some awk & sed

Awk is very useful for field like cut and stripping in a more powerful way. One-liners for awk for some cool examples.

awk ‘{ print $2, $1 }’ file               # Print and inverse first two columns
awk ‘{printf(“%5d : %s\n”, NR,$0)}’ file  # Add line number left aligned
awk ‘{print FNR “\t” $0}’ files           # Add line number right aligned
awk NF test.txt                           # remove blank lines (same as grep ‘.’)
awk ‘length > 50’                         # print line longer than 50 char

Some sed commands

sed ‘s/string1/string2/g’                 # Replace string1 with string2
sed -i ‘s/wroong/wrong/g’ *.txt           # Replace a recurring word with g
sed ‘s/\(.*\)1/\12/g’                     # Modify anystring1 to anystring2
sed ‘/ *#/d; /^ *$/d’                     # Remove comments and blank lines
sed ‘s/[ \t]*$//’                         # Remove trailing spaces (use tab as \t)
sed ‘s/^[ \t]*//;s/[ \t]*$//’             # Remove leading and trailing spaces

Redirects-Pipes and Special Variables

Redirects and Pipes for bash and sh:

# cmd 1> file   # Redirect stdout to file.
# cmd 2> file   # Redirect stderr to file.
# cmd 1>> file  # Redirect and append stdout to file.
# cmd &> file   # Redirect both stdout and stderr to file.
# cmd >file 2>&1   # Redirects stderr to stdout and then to file.
# cmd1 | cmd2      # pipe stdout to cmd2
# cmd1 2>&1 | cmd2 # pipe stdout and stderr to cmd2

The command line arguments are

$0, $1, $2, … # $0 is the command itself
$# # The number of arguments
$* # All arguments (also [email protected])

Special Variables
$$ # The current process ID
$? # exit status of last command

Difference between /dev/null and /dev/zero files

/dev/zero and /dev/null are two pseudo files which are useful for creating empty files.

/dev/zero: This file is used to create a file with no data but with required size(A file with all zero’s).
In other words this will create a data file with all zeros in the file which will give the size to a file.

Create a file with /dev/zero file

#dd if=/dev/zero of=/opt/file.txt bs=4096 count=1000

So we conclude that /dev/zero is a file use full for creating a file with some required size without any meaning to the data.

/dev/null: This is one more Pseudo file which is useful in many places like redirecting unwanted output/error etc to this file.
This file acts as a black hole(Which eat up everything and do not show any output).
So whenever you feed some data to this file, you can not retrieve the data which is fed to it. This file even useful for creating files with zero size.