javascript - Efficient way to find duplicate object in array - Stack Overflow

admin管理员组

文章数量:1356902

I am working with angular4. I need to display a list with unique values.

When accessing an API I get an array, from that array I need to display only unique data. I will be accessing that api at certain time interval and I need to update list only if new data is present.

response= [{"id":"0DRDCH03DR51GGJGJNP80F7XZ8","value":"36af1784bec4_566601260"},{"id":"0DRDCFYGM2CAHAXYK96BPT9RHV","value":"36af1784bec4_566601140"},...]

listData = [];
for(let data of response) {
    let tempValue = {id: '', time: ''};
    let value = data.value.split('_')
    if (value.length==2) {
        if(value[value.length-1].length==2) {
            tempValue.id = value[0];
            tempValue.time = value[1];
        }
        let isPresent = false;
        if(this.listData.length>0){
            for(let value of this.listData){
                if(value.time===tempValue.time){
                    isPresent = true;
                }
            }
        }
        if(!isPresent) {
            this.listData.push(tempValue);
        }
    }

    }

the final listData

listData = [{id:'36af1784bec4', time: '566601140'},...]

The above function does give me a unique listData array. I tried using array.filter and new Set but could not achieve the desired result.

I would like to know if there is an efficient way to do this.

I am working with angular4. I need to display a list with unique values.

When accessing an API I get an array, from that array I need to display only unique data. I will be accessing that api at certain time interval and I need to update list only if new data is present.

response= [{"id":"0DRDCH03DR51GGJGJNP80F7XZ8","value":"36af1784bec4_566601260"},{"id":"0DRDCFYGM2CAHAXYK96BPT9RHV","value":"36af1784bec4_566601140"},...]

listData = [];
for(let data of response) {
    let tempValue = {id: '', time: ''};
    let value = data.value.split('_')
    if (value.length==2) {
        if(value[value.length-1].length==2) {
            tempValue.id = value[0];
            tempValue.time = value[1];
        }
        let isPresent = false;
        if(this.listData.length>0){
            for(let value of this.listData){
                if(value.time===tempValue.time){
                    isPresent = true;
                }
            }
        }
        if(!isPresent) {
            this.listData.push(tempValue);
        }
    }

    }

the final listData

listData = [{id:'36af1784bec4', time: '566601140'},...]

The above function does give me a unique listData array. I tried using array.filter and new Set but could not achieve the desired result.

I would like to know if there is an efficient way to do this.

• javascript
• arrays
• angular
• typescript
• unique

Share Improve this question Follow edited Jan 3, 2018 at 8:57 Tony Roczz asked Jan 3, 2018 at 8:26 Tony RoczzTony Roczz 2,39866 gold badges3333 silver badges6262 bronze badges 6

• 1 If your definition of new data is new id and value, then finding the existing of data may be more costly than updating the list regardless. – gurvinder372 Commented Jan 3, 2018 at 8:30
• You can convert array to Set – bigless Commented Jan 3, 2018 at 8:33
• @bigless I tried converting to Set and tried using .has but it always returned true even when the listData was empty – Tony Roczz Commented Jan 3, 2018 at 8:34
• @gurvinder372 the response that I am getting has id and value, but new listData is value.split('_') and value[0] is id and value[1] is time. listData = [{id: '36af1784bec4', time: '566601260'},...] – Tony Roczz Commented Jan 3, 2018 at 8:37
• @Tony Roczz yea I was naive. its probably based on same reference... – bigless Commented Jan 3, 2018 at 8:38

 |  Show 1 more ment

6 Answers 6

Sorted by: Reset to default 2

You can remove duplicate by the following code.

var obj = {};

for ( var i=0, len=response.length; i < len; i++ )
    obj[response[i]['id']] = response[i]; // if id Duplicate otherwise use value if the check on value.

response = new Array();
for ( var key in obj )
    response.push(obj[key]);

console.log(response); // listData 

Edit:

and if you want to use a plete check of your prop then you can go for this one

function removeDuplicates(myArr, prop) {
    return myArr.filter((obj, pos, arr) => {
        return arr.map(mapObj => mapObj[prop]).indexOf(obj[prop]) === pos;
    });
}


console.log(removeDuplicates(response,['id','value'])); // for id or value use only 'id' or 'value' in second parameter.

I think this will do

for(let data of response) {
    if(!this.listData.find((ldata) => data.value.substring(data.value.lastIndexOf('_')+1) === ldata.time)) {
        let tempValue = {id: '', time: ''};
        let value = data.value.split('_')
        if (value.length==2) {
            if(value[value.length-1].length==2) {
                tempValue.id = value[0];
                tempValue.time = value[1];
            }
            this.listData.push(tempValue);
        }
    }
}

For an Optimised solution (loop only once i.e. O(n)) use object property to check if property already added, by this way you will avoid looping to find if property already added.

Below is the sample solution

let response = [{
    "id": "0DRDCH03DR51GGJGJNP80F7XZ8",
    "value": "36af1784bec4_566601260"
  }, {
    "id": "0DRDCH03DR51GGJGJNP80F7XZ8",
    "value": "36af1784bec4_566601260"
  }, {
    "id": "0DRDCFYGM2CAHAXYK96BPT9RHV",
    "value": "36af1784bec4_566601140"
  }],
  listData = [],
  tempObj = {};
for (let data of response) {
    let value = data.value.split('_');
  if (!tempObj.hasOwnProperty(value[1])) {
    tempObj[data.value[1]] = "";
    listData.push({
      id: value[0],
      time: value[1]
    });
  }
}

console.log(listData);

Update: based on ments, checking only for duplicate time value

You can also do this by JSON.stringify the objects like following :

let response = [{
  "id": "0DRDCH03DR51GGJGJNP80F7XZ8",
  "value": "36af1784bec4_566601260"
}, {
  "id": "0DRDCFYGM2CAHAXYK96BPT9RHV",
  "value": "36af1784bec4_566601140"
}, {
  "id": "0DRDCH03DR51GGJGJNP80F7XZ8",
  "value": "36af1784bec4_566601260"
}]

let o = response.reduce((acc, cv) => {
  if (!acc[JSON.stringify(cv)]) {
    acc[JSON.stringify(cv)] = true; //something non-falsy
  }
  return acc;
}, {});

let res = Object.keys(o).map(x => JSON.parse(x));
console.log(res);

Edit As user3297291 pointed out

Might be helpful to note that JSON.stringify({ a: 1, b: 2}) !== JSON.stringify({ b: 2, a: 1 })

So the above will fail in that case.

A more optimised solution would be

let response = [{
  "id": "0DRDCH03DR51GGJGJNP80F7XZ8",
  "value": "36af1784bec4_566601260"
}, {
  "id": "0DRDCFYGM2CAHAXYK96BPT9RHV",
  "value": "36af1784bec4_566601140"
}, {
  "id": "0DRDCH03DR51GGJGJNP80F7XZ8",
  "value": "36af1784bec4_566601260"
}];

let o = response.reduce((acc, cv) => {
  if (!acc[cv.value]) {
    acc[cv.value] = true; //something non-falsy
  }
  return acc;
}, {});

let res = Object.keys(o).map(x => {
  let t = x.split('_');
  return {
    id: t[0],
    time: t[1]
  };
});
console.log(res);

You can do this with a map to collect the ids and then with a filter to remove the duplicates :

const response= [{"id":"0DRDCH03DR51GGJGJNP80F7XZ8","value":"36af1784bec4_566601260"},{"id":"0DRDCFYGM2CAHAXYK96BPT9RHV","value":"36af1784bec4_566601140"}];

const ids = response.map(it => it.value.split('_')[0]).filter((it, i, arr) => i === arr.indexOf(it))

console.log(ids);

Or with a reduce :

const response= [{"id":"0DRDCH03DR51GGJGJNP80F7XZ8","value":"36af1784bec4_566601260"},{"id":"0DRDCFYGM2CAHAXYK96BPT9RHV","value":"36af1784bec4_566601140"}];

const ids = response.reduce((acc, it) => {
  const id = it.value.split('_')[0];
  return acc.includes(id) ? acc : [...acc, id];
}, []);

console.log(ids);

Recent versions of JavaScript allow for reliance on object structure to stay the same... Therefore we can now rely on stringifying and paring the two strings:

JSON.stringify({id:0, name:foo}) === JSON.stringify({id:0, name:foo}) // true

So with an array of objects:

1. stringifying an object to test
2. looping through a collection of objects stringifying each
3. paring the two

is helpful because you don't need to know key names. It just works on any object types...

arr = [{object}, {object2}, {etc...}]
test = JSON.stringify(obj);

for ( let i = 0, j = arr.length; i < j; i++ ) {
  let testAgainst = JSON.stringify(arr[i]);
  if (test === testAgainst) { 
    console.log('matches');
  } else {
    console.log('no match');
  }
}

Now you'd be matching two strings. If I'm missing something please ment.

本文标签：