Uniq Command in Linux with examples

uniq is a command used to detect and filter the duplicate lines in text files or strings. Basically, this command will filter out the adjacent repeated lines from INPUT and write to OUTPUT.

Here at LinuxAPT, as part of our Server Management Services, we regularly help our Customers to perform related Linux System commands queries.

In this context, we shall look into how to use the uniq command in Linux.

What is the syntax of Uniq Command ?

The structure of uniq command takes the following format:

$ uniq [option] [input[output]]

The [option] of this command are described below:

• -c (count) display the number of times a line was duplicated.
• -d (duplicate) just only print the duplicate lines.
• -f N (skip-fields) skip N fields of a line then determine the uniqueness of a line.
• -i (ignore case) by default, the command is case identification but when using this option, it doesn't do that.
• -s N (skip-chars) skip N special characters.
• -u (unique) print only unique lines.

Now, lets us review uniq command with examples.

1. -c (count)

Now we will try creating a file named count.txt by cat command then type the following lines:

$ cat > count.txt
ubuntu
ubuntu
linux
linux
mac
mac
mac

Then we use uniq with -c option to count lines was duplicated:

$ uniq -c count.txt

The numbers before each line are the number of repetitions of that line.

2. -d (duplicate)

Now we will try creating a file named dup.txt by cat command then type the following lines:

$ cat > dup.txt
UNIX
UNIX
Linux
IOS
IOS
Andriod
Android

Then we use uniq with -d option:

$ uniq -d dup.txt

Now, you will see that only duplicate lines are printed out.

3. -f N (skip-fields)

Now we will try creating a file named fn.txt by cat command then type the following lines:

$ cat > fn.txt
1. limit
2. limit
3. limit of function
4. limit of function

Then we use uniq with -f N option:

$ uniq -f 2 fn.txt

This option is useful with the lines are numbered. N = 2 means the command will compare from the 2nd line onwards.

3. -i (ignore case)

Now we will try creating a file named ignore.txt by cat command then type the following lines:

$ cat > ignore.txt
linux
LINUX
mac

Then we use uniq with -i option:

$ uniq -i ignore.txt

By default, the command is case identification but when using this option, it doesn't do that.

4. -s N (skip-chars)

Now we will try creating a file named skip.txt by cat command then type the following lines:

$ cat > skip.txt
$%music
&^%music
@#$%live
&^%$live

Then we use uniq with -s N option:

$ uniq -s 3 skip.txt

Here, N = 3 means the command omitted the first 3 characters and start filter out.

5.  -u (unique)

Now we will try creating a file named unique.txt by cat command then type the following lines:

$ cat > unique.txt
MUSIC
band
band
album
album

Then we use uniq with -u option:

$ uniq -u unique.txt

The line "MUSIC" is unique and it will be printed out.

[Need to Install Open Source Software on your Linux System ? We can help you. ]