MSSQL error logs in *nix boxes

Today we had to parse one SQL error log file in UNIX box and I saw a kind diff file format -

#grep "Database mirroring is inactive" ERRORLOG.1

And i saw no output. :-(

I thought its dos2unix which can help me. BUT not!!

#file ERRORLOG.4


ERRORLOG.4: Little-endian UTF-16 Unicode English character data, with very long lines, with CRLF line terminators


grep command couldn’t parse the UTF-16 Unicode file. Hence, the file had to be converted to the format which ‘grep’ can parse.

#iconv -f UTF-16 -t UTF-8 ERRORLOG.1 | grep "Database mirroring is inactive"

And cheers! I am done.

-DK

Our new little angel..

We've added another color to our family's rainbow... we have named her 'Srishti' . She may be small when it comes to size, but she's the queen of the cutie pies! And very naughty alrerady... :-)

"A Precious one,
So small,
and So cute..

Dancing in
her angel feet
Straight from Heaven's
brightest star..

What a miracle
you are! "



- Delighted Parents
Debajit & Nandita

Recover deleted /var/log/messages

I had to handle a recent escalation where /var reportedly got filled up - Someone with 'dbuser' userid deleted /var/log/message file on this server

[root@vapdb01 ~]# last

dkataki pts/0 10.152.208.19 Sat Aug 6 20:46 still logged in

dbuser pts/0 10.152.208.19 Sat Aug 6 20:14 - 20:22 (00:08)

dbuser pts/0 10.152.208.19 Fri Aug 5 04:17 - 10:08 (05:51)

This is how I would tackle this situation

[root@vapdb01 ]# df -h | grep /var

/dev/xvda3 3.0G 2.5G 282M 91% /var

[root@vapdb01 ]# cd /var

[root@vapdb01 var]# du -csh .

347M .

347M total

Looks like some file got deleted but some pid is still holding this file --

[root@vapdb01 var]# lsof | grep -i deleted | grep "/var/"

syslogd 1404 root 2w REG 202,3 2230328164 503883 /var/log/messages (deleted)

gpnpd.bin 4948 oracle 17uw REG 202,3 0 31522 /var/tmp/.oracle/ora_gipc_GPNPD_vapdb01_lock (deleted)

ocssd.bin 5013 oracle 17uw REG 202,3 0

That means someone tried to clear /var partition and deleted unknowingly or accidentally a crucial file from the server - and size of the file was - 2.1G+ , good catch but wrong measure by the user i would say!!!

Anyway - time to avert this situation now -

[root@vapdb01 var]# cat /proc/1404/fd/7 > /root/messages.1

[root@vapdb01 var]# /etc/init.d/syslog restart

[root@vapdb01 var]# cd /root/

[root@vapdb01 var]# gzip messages.1

[root@vapdb01 ~]# mv messages.1.gz /var/log/

[root@vapdb01 ~]# df -h

Filesystem Size Used Avail Use% Mounted on

/dev/xvda2 14G 4.5G 8.2G 36% /

/dev/xvda3 3.0G 434M 2.4G 16% /var

..and a happy Ending -

This only can someone do who has little knowledge on how UNIX works. Shame!! Is not that why we SA folks are very careful while giving root privilege to normal users?

-DK

history

Its always handy to capture the timestamp in your server's  history for audit purpose.

This is how it can be enabled.

Open ~/.bashrc file.

export HISTTIMEFORMAT="%h/%d - %H:%M:%S "



Some of my Favorite key on shell is :

Ctrl + r

Ctrl + p

history -c (Dangerous! It will wipe off entire history) , useful sometime for some victors ^_^ though..!!

Enjoy!
DK

Linux send mail attachements

Whats are the possible ways to send and email attachment - say needed to embedded in a script or something. May be attachment of the error/debug logs, that your script generated.

Well i can think of -

• mutt -s "demo email attachement" -a cvs6bld.php debajitkataki@gmail.com < body.txt
• uuencode bgraph.php | mail -s "demo email attachement" debajitkataki@gmail.com < body.txt

Note: uuencode is a utility which come with sharutils rpm package.

• mailx -s "demo email attachement" debajitkataki@gmail.com < cvs6bld.php

However the one with mailx command, is NOT a mime attachment - it's just your file content mixed in to the mail message body. Some e-mail clients will see it and might treat it as a real attachment, so we might be good.

Comments below are welcome- if you have tried some other raw way! :-)



Enjoy!
DK

capture debug log of my bash script

#!/bin/sh

export PS4='$0.$LINENO+ '
exec > /tmp/TTpull.log
exec 2>&1
set -x


This is how I like to capture - now what those above means?

The PS4 i already explained in my previous post. The $0 variable holds the name of the script. $LINENO displays the current line no. within the script. The exec command redirects I/O streams. to a file /tmp/TTpull.log. 2>&1 redirects stderr and stdout. and finally - 'set -x' enables debugging.


Cheers!
DK

>/dev/null 2>&1" and "&> /dev/null does the same job

We all know that /dev/null is the 'bit bucker' or the 'black hole' (nothing but a special file
that discards all data written to it)

Background:

We also know the standard streaming conventions: STDIN(0) STDOUT(1) and STDERR(2)

&>filename =

Redirect both stdout and stderr to file "filename."

2>&1
# Redirects stderr to stdout.
# Error messages get sent to same place as standard output.

Now lets analyze:

>/dev/null 2>&1"
: This means that standard output goes to /dev/null and
standard error(2) should get redirected to where your standard output
is going(in our case it is /dev/null)

&> /dev/null
: As explained above, redirect standard output(1) and standard error(2)
to /dev/null


So coders and SA's this is actaully same and dont get(urself) and try
make other confused with this please :P

More you can read here

-DEBA

logsave

Sometime I feel it is still gonna take years to explore all the available commands to leverage for what they were made for. For ex. yesterday I came to know the existence of a command called 'logsave' which made so many of tasks real easy.

Logsave command is used to save the output of a commands in a log file.

Syntax: logsave


I used it in my below script -

#!/bin/bash
# SCRIPT NAME: PreHealthCheck.sh
# DESCRIPTION: It does a precheck of all our Servers servers on disk and connectivity
# AUTHOR: D E B A
# OWNER: Fundoo Team
#########################################################
ADMIN="debajitkataki@gmail.com"
ALERT=80
flag=0
rm -f /tmp/PreChkRprt.txt
rm -f /tmp/report.txt
tmpFileName=/tmp/PreChkRprt.txt
touch ${tmpFileName}


email__msg(){
MailSubject="Prod Build Precheck report"
SuccessMessage=" Please check the below sections for any precheck concern:\n\n"

sed -i '1i ---------------------------------------------------------' ${tmpFileName}
sed -i '2i Please check the below sections for any precheck concern:' ${tmpFileName}
sed -i '3i ---------------------------------------------------------' ${tmpFileName}
echo -e "\n\n Script Signature: `hostname`:${0}" >> ${tmpFileName}
mail -s "Alert: Prod Build Precheck report" $ADMIN < ${tmpFileName}



diskSpace(){

echo -e "\n\n ##### -- High disk utilization Report --#####\n" >> ${tmpFileName}
for i in `cat wslist.txt`
do
usepar=0
output=0
ssh -oConnectTimeout=5 -xq $i df -Ph > /tmp/df.out
output=`cat /tmp/df.out | grep -vE '^Filesystem|tmpfs|cdrom' | awk '{ print $5 " " $6 }'`

usepar=$(echo $output | awk '{ print $1}' | cut -d'%' -f1 )
partition=$(echo $output | awk '{ print $2 }' )
if [ "$usepar" -ge 80 ]
then
echo " $i is running out of space \"$partition = $usepar% \" " >> ${tmpFileName}
flag=1
fi
done

}

connctivity_check(){

echo -e "\n\n ##### ---- Checking connectivity ----#####" >> ${tmpFileName}
echo -e "--------------------------------------------------------------" >> ${tmpFileName}
echo -e"\n"
for i in `cat wslist.txt`; do logsave -a /tmp/report.txt nc -v -w 1 -z $i 22 ; done
grep "Connection timed out" /tmp/report.txt >> ${tmpFileName}
echo -e "\n ....rest all connecting fine on port 22!!" >> ${tmpFileName}

}

diskSpace
connctivity_check
email__msg



Thanks/-
D E B U

Apache: Track how much time a request took

To track the time taken to server a request by apache is just a matter of adding one extra parameter in your config file and this is how you do that -

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %T/%D" combined

So if you see we have added only two additional parameters -

%T = The time taken to serve the request, in seconds.
%D = The time taken to serve the request, in microseconds.

Once you make the config changes live by restarting your apache server, tail the access logs and you will now see the the time spent to serve the request at the end of the line, along with other access logs parameters.


172.28.90.21 - - [31/Jan/2011:21:29:15 -0800] "GET /repos/ HTTP/1.1" 401 401 "-" "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13" 0/446

Hope it helps someone. Thanks/-

-Debu

Linux way of selctive backups

Sometimes we have a choice of only backing up some of the selected files from an
application or some other very crucial files. Below is how I do it in a real quick way:

cat >> /etc/bkup.conf
/usr/local/apache/conf/httpd.conf
/usr/local/apache/conf/httpd.global.conf
/usr/local/apache/conf/httpd.mod_jk.conf
/usr/local/apache/conf/vhosts/*
/usr/local/apache/conf/httpd.vhosts.include.conf
/usr/local/apache/conf/httpd.ssl.conf
EOF

Then I run tar command with -T flag pointing to the file which contains
the list. In my case it is /etc/bkup.conf

tar -cjf bkup-WebConfig-`date +%Y-%m-%d`.tar.bz2 -T /etc/bkup.conf

And you have your selective backup ready!

Thanks/-
DEBAJIT