Posts

Showing posts with the label Linux Commands

Deleting files with SIZE range

The  -a  in an explicit  AND  operator that allows you to join two primaries. In this case creating a range using  -size . rm -rf `find . -size +300c -a -size -400c`; The above command deletes the files which size are in between 300kb to 400kb. Note the size is a numeric argument that can optionally be appended with  +  and  - .  Numeric arguments can be specified as     +n      for  greater than n,    -n      for  less than n,     n      for  exactly n.

[SOLVED] - RSYNC not executing via CRON

The below error was faced and tried for online answers, made my head to heat for couple of days and finally i was able to crack the solution for the below issue. umasarath@ubuntu:~$ tail -50 cron_alc.log Cronjob started for back-up files + rsync -v umasarath@xx.xx.xx.xx:/tmp/compressed_logfiles/*.zip /home/umasarath/archive/ Permission denied, please try again. Permission denied, please try again. Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password). rsync: connection unexpectedly closed (0 bytes received so far) [Receiver] rsync error: unexplained error (code 255) at io.c(601) [Receiver=3.0.7] umasarath@ubuntu:~$ Please follow the below steps inorder to avoid the above error. My script contains the below code. umasarath@ubuntu:~$ cat transfer_files.sh #!/bin/sh set -xv echo "Cronjob started for back-up files" `date` /usr/bin/rsync -vv umasarath@xx.xx.xx.xx:/tmp/compressed_logfiles/*.zip /home/umasarath/archive/ echo "Cronjob ended for back-u

CHMOD command and strange numbers

You might come across the below command like  chmod 755 filename and of course you will be wondering what this is.  The thing is, that you can change the entire permission pattern of a file in one go using one number like the one in this example. Every mode has a corresponding code number, and as we shall see there is a very simple way to figure out what number corresponds to any mode.  Every one of the three digits on the mode number corresponds to one of the three permission triplets. (u, g and o) Every permission bit in a triplet corresponds to a value:  4 for r,  2 for w,  1 for x.  If the permission bit you add this value to the number of the permission triplet. If it is cleared, then you add nothing. So if a file has rwxr-xr-x permissions we do the following calculation:  Triplet for u: rwx => 4 + 2 + 1 = 7 Triplet for g: r-x => 4 + 0 + 1 = 5 Triplet for o: r-x => 4 + 0 + 1 = 5   Which makes : 755  Now i guess,

UNIX permissions

Permissions Every file on the system has associated with it a set of permissions. Permissions tell UNIX what can be done with that file and by whom. There are three things you can (or can't) do with a given file:   read it,   write (modify) it and   execute it. For any given ownership relation, we need three letters to specify access permissions: the first denotes read (r) access, the second denotes write (w) access and the third denotes execute (x) access. We have three ownership relations: 'owner', 'group' and 'all' so we need a triplet for each, resulting in nine letters. Lets try something in our command prompt with ls -l command. umasarath@ubuntu:~$ ls -l total 53780 -rw-r--r-- 1 umasarath group 27455157 2012-11-16 10:28 sample.xml drwxr-xr-x 2 umasarath group      4096 2013-01-23 10:17 files -rw-r--r-- 1 umasarath group      1338 2013-09-14 08:32 keystore drwxr-xr-x 2 umasarath group      4096 2013-09-16 10:

Running "ssh-agent" to run automatically

The following code can run ssh-agent automatically when you open bash by adding the following to ~/.profile or ~/.bashrc file: Please note after inserting the code in your profile or bashrc, passphrase will be asked for the first time. From the next login, you can enjoy password less login. # Note: ~/.ssh/environment should not be used, as it # already has a different purpose in SSH. env=~/.ssh/agent.env # Note: Don't bother checking SSH_AGENT_PID. It's not used # by SSH itself, and it might even be incorrect # (for example, when using agent-forwarding over SSH). agent_is_running() { if [ "$SSH_AUTH_SOCK" ]; then # ssh-add returns: # 0 = agent running, has keys # 1 = agent running, no keys # 2 = agent not running ssh-add -l >/dev/null 2>&1 || [ $? -eq 1 ] else false fi } agent_has_keys() { ssh-add -l >/dev/null 2>&1 } agent_load_env() { . "

TAR Command with examples

TAR is archiving utility in Linux. With tar command, multiple files and directories can be archived into a single file, with extension of that file is ".tar". "gzip" and "bzip" compression techniques can be used with tar command. Syntax: $ tar [options] [desired tar file name] [list of file and/or directories] Examples for your reference I need to create a tar file named umasarath from the directory_name folder. Below is the basic command to create a tar archive. $ tar cvf umasarath.tar directory_name/ Here cvf stands for  c - create new archive file v - verbosely list files f - following is archive file name. The above example tar cvf option, does not provide any compression. Inorder to use a gzip compression on the tar archive, use the z option as shown below. $ tar cvzf umasarath.tar directory_name/ Here z stands for  z - filter the archive through gzip Please Note: .tgz is same as .tar.gz

Checking Disk Space with "df" or "du"

A simple way to get used disk space on your Linux system and summary of the available space, just type "df" or "du" command in a terminal. The command "df" stands for disk filesystem. The "du" command on the other hand shows the disk space used by the files and directories in the current directory. All all commands in linux, we also have options for this. $ df [options] $ du [options] Examples for your reference: $ df -h Filesystem Size Used Avail Use% Mounted on /dev/mapper/root 909G 728G 135G 85% / none 9.9G 268K 9.9G 1% /dev shmfs 20G 7.6G 13G 38% /dev/shm none 9.9G 356K 9.9G 1% /var/run none 9.9G 4.0K 9.9G 1% /var/lock none 9.9G 0 9.9G 0% /lib/init/rw /dev/sda1 228M 196M 20M 91% /boot /dev/mapper/oracle 2.7T 661G 2.0T 26% /u01 Option "-h" represents human readable

GREP command with examples

GREP  command is used to search a pattern specified by user in the given text or files or block passed from first command to it through the pipe symbol (symbolic representation |) Syntax: $ grep [options] pattern [files] The complete list of options can be accessed with grep --help command. If you want to match more than one word pass  within double quotes” “ $ grep "Google is search engine" test1 test2 test3 It will list all lines of these three file containing “Google is search engine” in it in separate line, each line will be preceded with name of the file in which they appear. If you do not want the file name you can use -h option as below $ grep -h "Google is search engine" test1 test2 test3 It will not precede the matched line with the file in which they are found $ grep -i "Google is search engine" test1 test2 test3 The above command with option -i will tell to ignore the case i,e “google is SEARCH engine” will also match $ grep "Google is s

Touch Command with examples

Touch command will create a file if the given file is not present and if it is already present will modify its modification or access time depending on argument you pass to it, but not overwrite its content. Syntax :   touch [options] file_name $ touch -help will show you all available option. If you do not pass any option you can create multiple file with single touch command. Examples for your reference : $ touch google.txt The above command will create a file google.txt  $ touch google.txt yahoo.txt umasarath.txt The above command will create files yahoo and umasarath. Since google was already present, it will modify its access time. $ touch -a google.txt The above command will modify the access time of google file. $ touch -m yahoo.txt The above command will modify the modification time of yahoo file. $ touch -am umasarath.txt The above command will modify both access and modification time of umasarath file. $ touch -r yahoo.txt uma

Listing all the files with desired word - GREP command

One should get a grip on the Linux grep command.  The  grep   command allows you to search one file or multiple files for lines that contain a pattern. Exit status is 0 if matches were found, 1 if no matches were found, and 2 if errors occurred. $ grep [options] pattern [files] Option Description -b Display the block number at the beginning of each line. -c Display the number of matched lines. -h Display the matched lines, but do not display the filenames. -i Ignore case sensitivity. -l Display the filenames, but do not display the matched lines. -n Display the matched lines and their line numbers. -s Silent mode. -v Display all lines that do NOT match. -w Match whole word.

RSYNC command without authentication - 8 simple steps

Initially, when i try to sync the files from remotehost to localhost, i have to execute manually by executing in command prompt. I thought of executing the command by shell script and then call this script from cronjob. When i do this, the script was asking for authentication which is seems no difference between manual effort and an automated one. In order to avoid this, the following steps will explain how to perform RSYNC from localhost to remote host without entering password. umasarath@localhost:~$ rsync -v umasarath@remotehost:/home/umasarath/test/* test/ umasarath@remotehost's password: test11.txt umasarath.txt sent 61 bytes received 144 bytes 82.00 bytes/sec total size is 18 speedup is 0.09 umasarath@localhost:~$ Verify that localhost and remotehost is running openSSH umasarath@local:~$ ssh -V OpenSSH_5.3p1 Debian-3ubuntu7, OpenSSL 0.9.8k 25 Mar 2009 umasarath@remote:~$ ssh -V OpenSSH_5.3p1 Debian-3ubuntu7, OpenSSL 0.9.8k 25 Mar 2009 Generate key-pair on the lo

Transfer of files from one server to existing server using SCP and its performance

SCP (Secure Copy) scp allows files to be copied to, from, or between different hosts. It uses ssh for data transfer and provides the same authentication and same level of security as ssh. $ scp username@remotehost:sample.txt /some/local/directory SCP Performance improvement: By default scp uses the Triple-DES cipher to encrypt the data being sent. Using the Blowfish cipher has been shown to increase speed. This can be done by using option -c blowfish in the command line. $ scp -c blowfish <<filename>> username@remotehost:~ It is often suggested that the  -C  option for compression should also be used to increase speed. The effect of compression, however, will only increase speed if your connection is very slow. Otherwise it may just be adding extra burden to the CPU.  An example of using blowfish and compression: $ scp -c blowfish -C <<filename>> username@remotehost:~ Before executing the above command, make sure about folder permissions.

Archiving files older than 7 days using Cronjob

Most of us know how to zip files manually from the command prompt. Before going how to automate this zipping of files in a particular directory, lets brush up how do we zip from command prompt. If the files in one directory to be zipped, the command is as follows: zip <<desired file name>>.zip <<files to be zipped>> For Ex:- zip compressed_file.zip *.log From the above command, compressed_file.zip is the desired filename and *.log represents all the log files in the current directory. In order to zip files, we have to login to Unix and run this command whenever there is a need for this operation. It would be really handy if this job is automated by cron. Here comes the pseudo idea for making this operation automated. Let’s start to create a Shell script and then incorporate this in cron. SHELL SCRIPT FOR ZIPPING Open command prompt and navigate to the directory where you want to create the script. Create a file using the below c

List all the users in the system - LINUX/UNIX

What is the command to list users under Linux operating systems? /etc/passwd  file contains one line for each user account, with seven fields delimited by colons. This is a text file. You can easily list users using the cat command  as follows  $ cat /etc/passwd Sample Output: root:x:0:0:root:/root:/bin/bash daemon:x:1:1:daemon:/usr/sbin:/bin/sh bin:x:2:2:bin:/bin:/bin/sh