sed command in unix with practical examples

sed command

All about sed command in unix with Practical examples and tips and tricks


sed – Stream EDitor – works as a filter processing input line by line.

Sed reads one line at a time, chops off the terminating newline, puts what is left into the pattern space (buffer) where the sed script can process it (often using regular expressions), then if there anything to print – sed appends a newline and prints out the result (usually to stdout).

Therefore, changes are not made to the edit file itself, instead the input file, along with any changes, is written to standard output.

If you want to make the changes from sed permanent, they must be redirected from standard output to a file.
sed -f scriptfile editfile >outputfile

sed goes through the file a line at a time, so if no specific address is provided for a command, it operates on all lines.

How sed works( what are basics of sed)

sed commands have the general form
[address[, address]][!]command [arguments]
1) The sed utility works by sequentially reading a file, line by line, into memory. sed copies each input line into a pattern space

2) It then performs all actions specified for the line and places the line back in memory to dump to the terminal with the requested changes made. After all actions have taken place to this one line

-If the address of the command matches the line in the pattern space, the command is applied to that line
-If the command has no address, it is applied to each line as it enters pattern space
-If a command changes the line in pattern space, subsequent commands operate on the modified line
-An address can be either a line number or a pattern, enclosed in slashes ( /pattern/ )
-A pattern is described using regular expressions
-Additionally a NEWLINE can be specified using the “\n” character pair
-If no pattern is specified, the command will be applied to all lines of the input file

3) it reads the next line of the file and repeats the process until it is finished with the file.

4) When all commands have been read, the line in pattern space is written to standard output and a new line is read into pattern space
The output can be redirected to another file to save the changes; second, the original file, by default, is left unchanged. The default is for sed to read the entire file and make changes to each line within it. It can, however, be restricted to specified lines as needed.

What is the difference between awk and sed?
The sed command is designed to edit lines of an input stream. The awk command is used to select lines out of an input stream and do some kind of processing on those lines. There is overlap between these commands, either one can do many functions, but one usually lends itself better to your solution. The sed command works best when you merely want to make a frequent modification to many lines in the input. On the other hand, awk works best if you will be rearranging the order of the input lines or the information on the lines (e.g. swapping columns).Related : All about awk command


How address are given in sed?
Addresses in sed are given in following forms

1) Line number : This is line number of the files

sed ‘1,2d’: This delete the lines 1 and 2

2) /regexp/ : This is regular expression which is checked on each line

Regular expression description
/./ Will match any line that contains at least one character
/../ Will match any line that contains at least two characters
/^#/ Will match any line that begins with a ‘#’
/^$/ Will match all blank lines
/[abc]/ Will match any line that contains a lowercase ‘a’, ‘b’, or ‘c’
/^[abc]/ Will match any line that begins with an ‘a’, ‘b’, or ‘c’

Whole line oriented functions:

delete the addressed line(s) from the pattern space; line(s) not passed to standard output.

Delete Examples:

1) d deletes the current line
2)  8d deletes line 8
3) /^$/d deletes all blank lines
4) 1,100d deletes lines 1 through 100
5) 1,/^$/d deletes from line 1 through the first blank line
6) /^$/,/$/d deletes from the first blank line through the last line of the file
7) /^$/,10d deletes from the first blank line through line 10
8) /^go*t/,/[0-9]$/d deletes from the first line that begins with got, goot, gooot, etc through
the first line that ends with a digit

Line-oriented Patterns with practical examples:

1) Suppose we wanted to delete 8 lines starting from first line to 8 line in the file, the application of sed would be to delete the first 8 lines of stdin and echo the rest to stdout:

sed -e ‘1,8d’ trace.log

2) Commands in sed programs are separated by new lines. So Suppose if we wanted to delete the lines 1 to 11 and 20 to 39, in the file ,then we could use

sed -e ‘1,11d

Another possibility is to use the -e option more than once:

sed -e ‘1,11d’ -e ‘20,39d’

3) Suppose we wanted to delete all the lines which contains the work tom ,then we can use sed like this

sed -e ‘/tom/d’ < i.log

4) If you don’t want se to print each line by default, you can give sed the -n option. Then only lines that you print explicitly (with the “p” action) appear on stdout.

Another way to get only the first 10 lines is to use the -n option:

sed -n -e ‘1,10p’

5) Delete lines with the word IAS, but we only want lines that contain “ORA”. The traditional way to handle this would be:

grep ‘ORA’ < log | grep -v IAS

Note that this spawns two grep processes. The sed equivalent would be:

Related: Unix tutorial: Compress,Uncompress command,gzip and gunzip command

sed -n -e ‘/IAS/d’ -e ‘/ORA/p’

Here -n option inhibits printing, first pattern deletes all the lines with /debug/, and the second command forces printing of some of the remaining lines (which match /foo/)
Append, insert, and change line with practical examples:

6) To insert , we use option i

The given line will be inserted before third line

sed ‘3i\
> This is new line\
> the sed’ filename

This line will be inserted before each line where GEMS is present

sed ‘/GEMS/ i\

Add this line after every line with GEMS’

7) The append command works similarly, but will insert a line or lines after the current line in the pattern space. It’s used as follows:
To apppend ,we use option a

sed ‘3a\
> This is new line\
> the sed’ filename

Here the text ” this is new line the sed’ is append at the end of line 3
This example will add a line after every line with “GEMS:”
sed ‘/GEMS/ a\
Add this line after every line with GEMS’

8) Change a line with ‘c’:
You can change the current line with a new line.
sed ‘/GEMS/ c\
Replace the current line with the line’

The above command will replace all lines having GEMS with the new line

9) Reading in a file with the ‘r’ command:
sed ‘/INCLUDE/ r filename’ <in >out
Above will insert a filename after the line with the word “INCLUDE”

Substitute Command in sed with practical examples:

Syntax: [address(es)]s/pattern/replacement/[flags]

pattern – search pattern
replacement – replacement string for pattern
flags – optionally any of the following

n a number from 1 to 512 indicating which occurrence of pattern should be replaced
g global, replace all occurrences of pattern in pattern space
p print contents of pattern space
w file write the contents of pattern space to file

Character Description
^ Matches the beginning of the line
$ Matches the end of the line
. Matches any single character
* Will match zero or more occurrences of the previous character
[ ] Matches all the characters inside the [ ]

Regular expression description
/./ Will match any line that contains at least one character
/../ Will match any line that contains at least two characters
/^#/ Will match any line that begins with a ‘#’
/^$/ Will match all blank lines
/[abc]/ Will match any line that contains a lowercase ‘a’, ‘b’, or ‘c’
/^[abc]/ Will match any line that begins with an ‘a’, ‘b’, or ‘c’
sed substitute “s” Command Practical Examples:

1) Substitute dick for the first occurrence of tom in pattern space


2)Substitutes Dick for the second occurrence of Tom in the pattern space


3)  Substitutes plastic for the first occurrence of wood and outputs (prints) pattern space


4)   Substitutes Dr for every occurrence of Mr in pattern space


5)sed ‘s/^/ /’

6) it removes comments from all the lines

sed -e ‘s/#.*//’ myfile.txt

7) replaces only 1st instance in a line

sed ‘s/foo/bar/’

8)  replaces only 4th instance in a line

sed ‘s/foot/feet/4’

9)  replaces ALL instances in a line

sed ‘s/foot/feet/g’

10)  substitute “foo” with “bar” EXCEPT for lines which contain “bat” here ! is used to negate the presence of bat

sed ‘/bat/!s/foo/bar/g’

11)  change “scarlet” or “ruby” or “puce” to “red”. This is multiple substitution command

sed ‘s/scarlet/red/g;s/ruby/red/g;s/puce/red/g’

Related: Unix tutorial :Find command decoded
12)   This substitue the first occurence of A to a from lines 101 to end lines

sed ‘101,$ s/A/a/’

13) it will remove comments from the beginning of the file until it finds the keyword “start”

sed -e ‘1,/start/ s/#.*//’

14)it will remove comments everywhere except the lines between the two keywords

sed -e ‘1,/start/ s/#.*//’ -e ‘/stop/,$ s/#.*//’

15) it will swap ‘hills’ for ‘mountains’, but only on blocks of text beginning with a blank line, and ending with a line beginning with the three characters ‘END’, inclusive.

sed -e ‘/^$/,/^END/s/hills/mountains/g’ myfile3.txt

16) it will match a phrase beginning with ‘<‘ and ending with ‘>’, and containing any number of characters in between. This phrase will be deleted (replaced with an empty string)

sed -e ‘s/<.*>//g’ myfile.html

17) To remove all alpha-numeric characters present in every line

sed ‘s/[a-zA-Z0-9]//g’ filename

How to apply Multiple commands for one address using sed

Braces { } can be used to apply multiple commands to an address:
Ptracticak Examples

1) The below example will apply four substitution commands to lines 10 through 25, inclusive



Related : Useful Unix command for DBA
2) You can also use regular expression addresses, or a combination of the two:


This example will apply all the commands between ‘{ }’ to the lines starting at 1 and up to a line beginning with the letters
“END”, or the end of file if “END” is not found in the source file.

How to use sed command as scriptsed -f scriptname
If you have a large number of sed commands, you can put then into a file and use
sed -f sedscript <old >newwhere sedscript could look like this:
# sed comment – This script changes lower case vowels to upper case

Some more general practical examples

1) To remove the first letter in all the lines in the file

sed ‘s/^.//’ filename

2) To remove the last letter in all the lines in the file

sed ‘s/.$//’ file

3) To remove all digits present in every line of a file:

sed ‘s/[0-9]//g’ file

4) sed ‘/start/,/stop/ s/#.*//’ # The first pattern turns on a flag that tells sed to perform the substitute command on every line. The second pattern turns off the flag. If the “start” and “stop” pattern occurs twice,the substitution is done both times. If the “stop” pattern is missing, the flag is never turned off, and the substitution will be performed on every line until the end of the file.