uniq is a command used to detect and filter the duplicate lines in text files or strings. Basically, this command will filter out the adjacent repeated lines from INPUT and write to OUTPUT.
Here at LinuxAPT, as part of our Server Management Services, we regularly help our Customers to perform related Linux System commands queries.
In this context, we shall look into how to use the uniq command in Linux.
The structure of uniq command takes the following format:
$ uniq [option] [input[output]]
The [option] of this command are described below:
Now, lets us review uniq command with examples.
1. -c (count)
Now we will try creating a file named count.txt by cat command then type the following lines:
$ cat > count.txt
ubuntu
ubuntu
linux
linux
mac
mac
mac
Then we use uniq with -c option to count lines was duplicated:
$ uniq -c count.txt
The numbers before each line are the number of repetitions of that line.
2. -d (duplicate)
Now we will try creating a file named dup.txt by cat command then type the following lines:
$ cat > dup.txt
UNIX
UNIX
Linux
IOS
IOS
Andriod
Android
Then we use uniq with -d option:
$ uniq -d dup.txt
Now, you will see that only duplicate lines are printed out.
3. -f N (skip-fields)
Now we will try creating a file named fn.txt by cat command then type the following lines:
$ cat > fn.txt
1. limit
2. limit
3. limit of function
4. limit of function
Then we use uniq with -f N option:
$ uniq -f 2 fn.txt
This option is useful with the lines are numbered. N = 2 means the command will compare from the 2nd line onwards.
3. -i (ignore case)
Now we will try creating a file named ignore.txt by cat command then type the following lines:
$ cat > ignore.txt
linux
LINUX
mac
Then we use uniq with -i option:
$ uniq -i ignore.txt
By default, the command is case identification but when using this option, it doesn't do that.
4. -s N (skip-chars)
Now we will try creating a file named skip.txt by cat command then type the following lines:
$ cat > skip.txt
$%music
&^%music
@#$%live
&^%$live
Then we use uniq with -s N option:
$ uniq -s 3 skip.txt
Here, N = 3 means the command omitted the first 3 characters and start filter out.
5. -u (unique)
Now we will try creating a file named unique.txt by cat command then type the following lines:
$ cat > unique.txt
MUSIC
band
band
album
album
Then we use uniq with -u option:
$ uniq -u unique.txt
The line "MUSIC" is unique and it will be printed out.
This article covers how to use the uniq command in Linux. In fact, the uniq command reports or filters out repeated lines in a file. uniq filters out adjacent, matching lines from input file INPUT, writing the filtered data to output file OUTPUT. A matching line is "adjacent" if it's immediately before or after another matching line.
Related Linux commands: