Sorting and Grep

Read a big file line by line and do some grep

cat files.txt | xargs grep keyword

Appending a list of files to one file using xargs command

cat filelist | xargs cat >> bigfile.dat

Appending the content of a list of files to one file using xargs command
cat /tmp/filelist.txt | xargs cat |grep " ERROR] Failed path" >> /tmp/ed_search.err

Cut extract multiple columns
e.g Get the first column

/etc/passwd
root:x:0:0:root:/root:/bin/bash

cut -d":" -f1 /etc/passwd
root

Find out and sort the folder according to size

du --max-depth=1 /home/ | sort -nr

To sort the file on the third field (serial number) in reverse order and save the results in sorted.data, use this command:

sort -r +2 -3 company.data > sorted.data
Mel Ancholie 636496 Research
Jan Itorre 406378 Sales
Ed Jucacion 396082 Sales
Jim Nasium 031762 Marketing


Sort by field 5 numerically
$ ls -al | sort -n -k5



Cut out column 9 where the fields are separated by one or more spaces

[root@localhost locallog]# ll -r ed_search.*.log | awk -F" " '{print "/path/"$9}' >> /tmp/filelist.txt
[root@localhost locallog]# cat /tmp/filelist.txt
/path/ed_search.8.log
/path/ed_search.7.log
/path/ed_search.6.log
/path/ed_search.5.log
/path/ed_search.4.log
/path/ed_search.3.log
/path/ed_search.2.log
/path/ed_search.1.log



Eliminlate duplicate lines, sort before uniq
sort /tmp/ed_search_err_list.txt | uniq | wc -l

grep line below and above
grep -A 1 word file.txt --> Line and one line after
grep -B 1 word file.txt --> Line and one line before
grep -C 1 word file.txt --> One line before and after


grep "string" file , how to get the line number also?

grep -n "string" file

Range of characters from a string

The first parameter after the variable describes the starting position from zero inclusively.
The second parameter describes the number of character from the starting position.
[root@localhost scripts]# d=12345678
[root@localhost scripts]# echo ${d:3:2}
45
[root@localhost scripts]# echo ${d:0:2}
12

Return the number of character in a string

[root@localhost scripts]# d=12345678
[root@localhost scripts]# echo ${#d}
8