Remove Duplicates From Array In Java

When we deal with bulky data, one of the efficient ways to manage it is by storing it in the array data structure. With its pros, it has some cons as well. The most common problem that arises while storing data in arrays is that we get duplicate elements. There are multiple ways to remove duplicates from arrays in Java. In this article, we will get to know about each way in detail one by one. Read till the end to completely learn about how to remove duplicates from an array in Java.

Before digging into the main topic of this article, let's just quickly remember the concept of arrays. An array is a data structure that stores elements in consecutive memory spaces. We assign this memory space while declaring an array. Each memory space is important to us. So getting duplicates in an array is of no use and we must remove duplicates from the array.

There are multiple methods to remove duplicates from an array in Java. Some of them are listed below.

Using a temporary array
Using separate index
Using arrays.sort() method
Using Hashmap
Using a linked hash set
Using streams

Now we will discuss every method one by one in detail.

Temporary Array to remove duplicates from array in java

In this method, we will declare another array that will be a temporary array. In this temporary array, we will store our original array but without duplicate elements. After that, we will store the values of the temporary array in the original array.

Code example - Temporary array to remove duplicates from array in java

public class removeDuplicatesFromArray{
    public static int removeDuplicates(int array[], int n){
        int[] temp = new int[n];
        int j = 0;
        for (int i=0; i<n-1; i++){
            if (array[i] != array[i+1]){
                temp[j++] = array[i];
            }
         }
        temp[j++] = array[n-1];
        for (int i=0; i<j; i++){
            array[i] = temp[i];
        }
        return j;
    }
    public static void main (String[] args) {
        int array[] = {1, 2, 3, 3, 4, 6, 6, 7, 9, 9, 9, 10, 11, 11, 11};
        int length = array.length;
        System.out.println("Array Before Removing Duplicates:");
        for (int i=0; i<length; i++)
           System.out.print(array[i]+" ");
        length = removeDuplicates(array, length);
        System.out.println("\nArray After Removing Duplicates:");
        for (int i=0; i<length; i++)
           System.out.print(array[i]+" ");
    }
}

Output

Array Before Removing Duplicates:
1 2 3 3 4 6 6 7 9 9 9 10 11 11 11 

Array After Removing Duplicates:
1 2 3 4 6 7 9 10 11

A prerequisite for using this method is that our array must be sorted. If the array is not sorted, then you can use the arrays.sort() method to sort the array. We have used a user-defined method for removing duplicates from an array in java. The main part where the whole logic lies in the if condition.

if (array[i] != array[i+1]){  
    temp[j++] = array[i];

We have removed the duplicates by comparing the elements and adding unequal elements to the temporary array.

Separate index to remove duplicates from array in java

This method is quite similar to the method described above. The only difference is that instead of using a new array i.e. temporary array, we will make changes in the original array. We will introduce a new index to make changes in the existing array. There is no need of declaring a new array after introducing a new pointer.

Code example - Separate index to remove duplicates from array in java

public class removeDuplicatesFromArray{
    public static int removeDuplicates(int array[], int n){
        int j = 0;
        for (int i=0; i<n-1; i++){
            if (array[i] != array[i+1]){
                array[j++] = array[i];
            }
         }
        array[j++] = array[n-1];
        return j;
    }
    public static void main (String[] args) {
        int array[] = {1, 2, 3, 3, 4, 6, 6, 7, 9, 9, 9, 10, 11, 11, 11};
        int length = array.length;
        System.out.println("Array Before Removing Duplicates:");
        for (int i=0; i<length; i++)
           System.out.print(array[i]+" ");
        length = removeDuplicates(array, length);
        System.out.println("\nArray After Removing Duplicates:");
        for (int i=0; i<length; i++)
           System.out.print(array[i]+" ");
    }
}

Output

Array Before Removing Duplicates:
1 2 3 3 4 6 6 7 9 9 9 10 11 11 11 

Array After Removing Duplicates:
1 2 3 4 6 7 9 10 11

The new index j is removing the duplicates from the array.

Arrays.sort() method to remove duplicates from array in java

In the above methods, our array must be sorted. But what if our array is not sorted and we have to remove duplicates from it. We can use arrays.sort() method to sort our array first and then remove the duplicates from the array in java.

Code example - Arrays.sort() method to remove duplicates from array in java

import java.util.*;
public class removeDuplicatesFromArray{
   public static int removeDuplicates(int array[], int n){
        int[] temp = new int[n];
        int j = 0;
        for (int i=0; i<n-1; i++){
            if (array[i] != array[i+1]){
                temp[j++] = array[i];
            }
        }
        temp[j++] = array[n-1];
        for (int i=0; i<j; i++){
            array[i] = temp[i];
        }
        return j;
    }
    public static void main (String[] args) {
        int array[] = {1, 3, 4, 5, 1, 3, 4, 5, 4, 2, 6, 5, 7};
        int length = array.length;
        System.out.println("Unsorted Array:");
        for (int i=0; i<length; i++)
           System.out.print(array[i]+" ");
        Arrays.sort(array);
        System.out.println("\nSorted Array:");
        for (int i=0; i<length; i++)
           System.out.print(array[i]+" ");
        System.out.println("\nArray Before Removing Duplicates:");
        for (int i=0; i<length; i++)
           System.out.print(array[i]+" ");
        length = removeDuplicates(array, length);
        System.out.println("\nArray After Removing Duplicates:");
        for (int i=0; i<length; i++)
           System.out.print(array[i]+" ");
    }
}

Output

Unsorted Array:
1 3 4 5 1 3 4 5 4 2 6 5 7 

Sorted Array:
1 1 2 3 3 4 4 4 5 5 5 6 7 

Array Before Removing Duplicates:
1 1 2 3 3 4 4 4 5 5 5 6 7 

Array After Removing Duplicates:
1 2 3 4 5 6 7

Hashmap to remove duplicates from array in java

It is one of the simplest methods to remove duplicates from an array. We will use a hashmap to achieve this. We will traverse our array in such a way that we will check if our element is present in the hashmap or not. If our element is not in the hashmap, then we will print its value.

Code example - Hashmap to remove duplicates from array in java

import java.util.HashMap;
public class removeDuplicatesFromArray{
    static void removeDuplicates(int[] array, int n)
    {
        HashMap<Integer, Boolean> map = new HashMap<>();
        for (int i = 0; i < n; ++i)
        {
            if (map.get(array[i]) == null)
                System.out.print(array[i] + " ");
            map.put(array[i], true);
        }
    }
    public static void main(String[] args)
    {
        int[] array = { 1, 3, 4, 5, 1, 3, 4, 5, 4, 2, 6, 5, 7};
        int length = array.length;
        System.out.println("Array Before Removing Duplicates:");
        for (int i=0; i<length; i++)  
           System.out.print(array[i]+" "); 
        System.out.println("\nArray After Removing Duplicates:");
        removeDuplicates(array, length);
    }
}

Output

Array Before Removing Duplicates:
1 3 4 5 1 3 4 5 4 2 6 5 7 

Array After Removing Duplicates:
1 3 4 5 2 6 7

Linked Hashset to remove duplicates from array in java

Sets in Java store unique elements only. We can easily remove duplicates from an array by storing the array elements in a linked hashset. The duplicates will get removed. We will then store the elements of the linked hashset in the original array.

Code example - Linked Hashset to remove duplicates from array in java

import java.util.*;
public class removeDuplicatesFromArray{
    static int[] removeDuplicates(int[] array)
    {
        List<Integer> list = new ArrayList<Integer>();
        for(int i : array){
        list.add(i);
    }
    Set<Integer> set = new LinkedHashSet<Integer>(list);
    int[] temparray = new int[set.size()];
    int j =0;
    for(int num:set){
        temparray[j++] = num;
    }
    return temparray;
    }
    public static void main(String[] args)
    {
        int[] array = { 1, 3, 4, 5, 1, 3, 4, 5, 4, 2, 6, 5, 7};
        int length = array.length;
        System.out.println("Array Before Removing Duplicates:");
        for (int i=0; i<length; i++)  
           System.out.print(array[i]+" "); 
        array = removeDuplicates(array);
        System.out.println("\nArray After Removing Duplicates:");
        int length2 = array.length;
        for (int i=0; i<length2; i++)  
           System.out.print(array[i]+" "); 
    }
}

Output

Array Before Removing Duplicates:
1 3 4 5 1 3 4 5 4 2 6 5 7 

Array After Removing Duplicates:
1 3 4 5 2 6 7

Stream to remove duplicates from array in java

We can use Java 8 stream 's distinct() method to remove duplicates from the array in java. We will convert our array into a list first and then call stream.distinct() method to remove the duplicate elements.

Code example - Stream to remove duplicates from array in java

import java.util.*;
import java.util.stream.Collectors;
public class removeDuplicatesFromArray{
    static int[] removeDuplicates(int[] array)
    {
        List<Integer> list = new ArrayList<Integer>();
        for(int i : array){
            list.add(i);
        }
        List<Integer> withoutDuplicates = list.stream().distinct().collect(Collectors.toList());
        int[] myArray = new int[withoutDuplicates.size()];
        for(int i = 0;i < myArray.length;i++)
          myArray[i] = withoutDuplicates.get(i);
        return myArray;
    }
    public static void main(String[] args)
    {
        int[] array = { 1, 3, 4, 5, 1, 3, 4, 5, 4, 2, 6, 5, 7};
        int length = array.length;
        System.out.println("Array Before Removing Duplicates:");
        for (int i=0; i<length; i++)  
           System.out.print(array[i]+" "); 
        array = removeDuplicates(array); 
        int length2 = array.length;
        System.out.println("\nArray After Removing Duplicates:");
        for (int i=0; i<length2; i++)  
           System.out.print(array[i]+" ");
    }
}

Output

Array Before Removing Duplicates:
1 3 4 5 1 3 4 5 4 2 6 5 7 

Array After Removing Duplicates:
1 3 4 5 2 6 7