Remove duplicate values from integer array in java

Removing duplicate integers from an array is a task that most of us have faced at work. It is also a commonly asked question in technical interview process. There are many ways in java to do this. All are listed below.

Before diving into the methods, let’s define the array which we will be using in all the methods. The array will contain following elements :

1. Comparing individual elements

This method involves iterating over the array, comparing each element with other elements for equality and taking appropriate action to remove one of the duplicate elements.

Let’s see the code and then get into its detail :

static void eleminateDuplicates(int[] duplicates){
  int length = duplicates.length;
  int size = duplicates.length;
  //Comparing each element with all other elements
  for (int i = 0; i < size; i++) {
    // iterate over elements ahead of the current loop element
    for (int j = i+1; j < size; j++) {
      //check for equality
      if(duplicates[i] == duplicates[j]) {
        duplicates[j] = duplicates[size-1];
        size--;
        j--;
        length--;  
      }
     }
  }
  //Copy unique elements to other array
  int[] unique = Arrays.copyOf(duplicates, length);

  System.out.println("Array Without Duplicates : ");
  for (int i = 0; i < unique.length; i++){
    System.out.print(unique[i]);
    System.out.print(" ");
  }
}

Output will be :

Explanation

This method sets up two loops :
A. Outer loop for iterating entire array, and
B. Inner loop starting at one index greater than the outer loop index.

So at every iteration of the outer loop, an element at the current location of the array is compared with all the elements ahead it in the array.

Now as soon as two elements match, means a duplicity is detected following actions are performed. Note that at this point i=0, j=5,size=8 :

1. The element at the position of the value of sizevariable (which will be equal to the length of the array when first duplicate is detected) is swapped with the element at the current index of the inner loop.
   
   Thus, if we consider our sample array { 1, 2, 34, 10, 2, 1, 5, 34 }, then after this step it will become {1, 2, 34, 10, 2, 34, 5, 34} since element at position 6 (j=5), which is 1, matches the first element (i=0) and gets replaced by 34 whose position is at index size-1.
2. Since the element (34) at last position is already included in the array, the last position should be decreased. Hence, size--.
3. Now the element at position j is new and has not been compared with the other elements so j is reduced by one.

After both the iterations complete, all the duplicate array elements would be placed at the end of the array with the size variable containing the number of unique elements in the array.

Finally, this array is copied to a new array with its length equal  to the number of unique elements in the original array (that is, value of size) using copyOfmethod of java.util.Arraysclass.

Listed below are the contents of the array at every point when a duplicate element is detected during iteration along with the values of i, j and size variables :

array	i	j	size
{1, 2, 34, 10, 2, 34, 5, 34}	0	5	8
{1, 2, 34, 10, 5, 34, 5, 34}	1	4	7
{1, 2, 34, 10, 5, 34, 5, 34}	2	5	6

2. Using Set Collection

This method utilizes fundamental property of a set that it only holds unique elements and discards duplicate elements.

Initialize a java.util.Set. Iterate over the array and in each iteration add its elements to the set.

The set will only add unique elements thus removing all duplicate elements.
Convert this set to an array using its toArray() method.

static void eliminateDuplicateUsingSet(int[] duplicates) {
  Set set = new HashSet();
  for (int counter = 0; counter < duplicates.length; counter++) {
    set.add(duplicates[counter]);
  }
  Integer[] newArr = set.toArray(new Integer[0]);
  System.out.println("Array without duplicates");
  // print the unique array
  for (int counter = 0; counter < newArr.length; counter++) {
    System.out.print(newArr[counter]);
    System.out.print(" "); 
  }
}

Output will be :

3. Using List Collection

Initialize an ArrayList object.
Iterate over the array and in each iteration check it the current array element exists in the list.

If it does not, then add the element to it.
After iteration completes, the array list contains only unique elements of the array.
Convert it to a integer array using toArray() method of java.util.ArrayList.

static void eliminateDuplicateUsingArrayList(int[] duplicates) {
  ArrayList list = new ArrayList();
  for (int counter = 0; counter < duplicates.length; counter++) {
  if (!list.contains(arr[counter]))
    list.add(duplicates[counter]);
  }
  System.out.println("Array without duplicates");
  Integer[] unique = list.toArray(new Integer[0]);
    // print the unique array
  for (int counter = 0; counter < unique.length; counter++) {
    System.out.println(unique[counter]);
    System.out.print(" ");
  }
}

Output :

4. Using streams in Java 8

This is the smallest and easiest method to remove duplicates from an array which uses Stream API introduced in java 8.
Get a java.util.Streamobject by calling stream()method of java.util.Arraysclass.

Calling distinct() method on this stream again returns the stream object. distinct()method returns a stream of unique elements.

Again calling toArray()method on this stream object returns the array representation of this stream.

static void eliminateUsingJava8(int[] duplicates) {
  int[] unique = Arrays.
                                 stream(duplicates).
                                 distinct().
                                 toArray();
  System.out.println("Array without duplicates");
  // print the unique array
  for (int counter = 0; counter < unique .length; counter++) {
    System.out.println(unique[counter]);
    System.out.print(" ");   
  }
}

Output :

Hope this article was useful.