Decompress multiple .gz files from one directory to a different target directory

Lets say you have a directory full of .gz files and you want to decompress all of them at the same time using one statement.

find . -name "*.gz" | while read filename; do gzip -cdv $filename > ../decompressed/$filename; done;

 The above statement uses 'find' command to get list of .gz files in the current (.) directory. Then it pipes it to while loop which executes gzip -cdv for each file and redirect the output to ./decompressed/ directory.

Send result from cron script to email using mail and inside crontab

Sometime you want to send yourself an email with the output of certain crontab script execution.

Here is the statement I use to get this done:

/usr/bin/php /path/to/my_script.php | mail -s "My Results" yourname@email.com

Nice, simple and effective.

How to add / insert date using Linux CLI bash shell in cron jobs

Knowing how to add / insert date using Linux CLI bash shell is useful when creating crontab jobs that executes daily. For example you run a script daily and would like to keep the output logged each time it runs and you want to keep those output logs for the past 30 days.

What has worked for me is adding the following behind whatever script you are executing:

`date +\%Y_\%m_\%d-\%H_\%M`

When combined with 'tee' and &2>1 it will send all output to a particular file, here is an example:

some_script.sh 2>&1 | tee /log/`date +\%Y_\%m_\%d-\%H_\%M`.log 

The above statement will store the output of some_script.sh into a log file with the file name:

/log/2015_03_29.log
/log/2015_03_30.log
/log/2015_03_31.log

and so forth...

Basically the example above uses the execution character ` ... `  to execute date command.

Here are some options in date command that you can use:

  %%   a literal %
  %a   locale's abbreviated weekday name (e.g., Sun)
  %A   locale's full weekday name (e.g., Sunday)
  %b   locale's abbreviated month name (e.g., Jan)
  %B   locale's full month name (e.g., January)
  %c   locale's date and time (e.g., Thu Mar  3 23:05:25 2005)
  %C   century; like %Y, except omit last two digits (e.g., 20)
  %d   day of month (e.g., 01)
  %D   date; same as %m/%d/%y
  %e   day of month, space padded; same as %_d
  %F   full date; same as %Y-%m-%d
  %g   last two digits of year of ISO week number (see %G)
  %G   year of ISO week number (see %V); normally useful only with %V
  %h   same as %b
  %H   hour (00..23)
  %I   hour (01..12)
  %j   day of year (001..366)
  %k   hour, space padded ( 0..23); same as %_H
  %l   hour, space padded ( 1..12); same as %_I
  %m   month (01..12)
  %M   minute (00..59)
  %n   a newline
  %N   nanoseconds (000000000..999999999)
  %p   locale's equivalent of either AM or PM; blank if not known
  %P   like %p, but lower case
  %r   locale's 12-hour clock time (e.g., 11:11:04 PM)
  %R   24-hour hour and minute; same as %H:%M
  %s   seconds since 1970-01-01 00:00:00 UTC
  %S   second (00..60)
  %t   a tab
  %T   time; same as %H:%M:%S
  %u   day of week (1..7); 1 is Monday
  %U   week number of year, with Sunday as first day of week (00..53)
  %V   ISO week number, with Monday as first day of week (01..53)
  %w   day of week (0..6); 0 is Sunday
  %W   week number of year, with Monday as first day of week (00..53)
  %x   locale's date representation (e.g., 12/31/99)
  %X   locale's time representation (e.g., 23:13:48)
  %y   last two digits of year (00..99)
  %Y   year
  %z   +hhmm numeric time zone (e.g., -0400)
  %:z  +hh:mm numeric time zone (e.g., -04:00)
  %::z  +hh:mm:ss numeric time zone (e.g., -04:00:00)
  %:::z  numeric time zone with : to necessary precision (e.g., -04, +05:30)
  %Z   alphabetic time zone abbreviation (e.g., EDT)

Tail ending log rows and create new aggregated log file (rotate truncate shorten rotating log file)

These examples are very useful ways to quickly truncate and rotate your log files.  This is a very basic necessity for any server administration. If you do not rotate your logs, you will run out of disk space and one day your server will stop functioning properly.


Example for NGINX web server logs:

cp /log/nginx/error.log /data_local/tmp_nginx_log/nginx_error_`date +"%Y_%m_%d"`.log; truncate -s0 /log/nginx/error.log; kill -USR1 $( cat /var/run/nginx.pid )

kill -HUP $( cat /var/run/nginx.pid ) 

Example for APACHE web server logs:

tail -n 1000 error_php_apache.log > error_php_apache.log.tmp && mv -f error_php_apache.log.tmp error_php_apache.log && /etc/init.d/httpd graceful

or simpler way:

tail -n 1000 apache-error.log > apache-error.1.log && truncate -s 1M apache-error.log && service apache2 graceful

Get first 1000 lines from all *.log files and store them all in /log/all_logs

find /log -name "*.log" -type f | xargs tail -n 1000 2>&1 | tee /log/all_logs