The Cache: Technology Expert's Forum
 
*
Welcome, Guest. Please login or register. September 16, 2019, 05:19:17 PM

Login with username, password and session length


Pages: [1]
  Print  
Author Topic: Remove Duplicates from CSV file  (Read 6135 times)
hvshah69
Rookie
**
Offline Offline

Posts: 21


View Profile
« on: October 20, 2008, 10:28:42 AM »

I have a csv file with following format

field1,field2,field3\n
field1,field2,field3\n
and so on...

I would like to remove duplicate entries only in field1 by removing the entire row (all 3 fields)
Again, I am only concerned about duplicate entries ONLY in field1. However, the whole row has to be deleted whenever a duplicate is encountered in field1.

If it was only one field per line, I could have easily achieved the desired result by using array_unique() function but the presence of additional comma separated fields make things a little complicated.

Is there a clever way of using array_unique() or another php function to do this? Huh?

Thanks.
Logged
vsloathe
vim ftw!
Global Moderator
Lifer
*****
Offline Offline

Posts: 1669



View Profile
« Reply #1 on: October 20, 2008, 10:43:40 AM »

Should be fairly straightforward...

Check out the function array_keys

You can create a 2-dimensional array with the Field1 value as the key. Then do array_keys and remove any dupes.
Logged

hai
perkiset
Olde World Hacker
Administrator
Lifer
*****
Offline Offline

Posts: 10096



View Profile
« Reply #2 on: October 20, 2008, 11:20:56 AM »

Code:
<?php

// $inputBuff = your original file

$arr explode(chr(10), $inputBuff);
$finalArr = array();
foreach(
$arr as $line)
{
$parts explode(','$line);
if (!$finalArr[$parts[0]])
$finalArr[$parts[0]] = $line;
}
$outBuff implode(chr(10), $finalArr);
?>


This simple routine explodes the input file based on a LF, then explodes each line on the comma. If there is no entry in the array with the key $parts[0], which is the part to the left of the first comma, then it adds it to the output array. If it already exists, the loop simply continues on. At the end the array is returned to a single string.
Logged

It is now believed, that after having lived in one compound with 3 wives and never leaving the house for 5 years, Bin Laden called the U.S. Navy Seals himself.
vsloathe
vim ftw!
Global Moderator
Lifer
*****
Offline Offline

Posts: 1669



View Profile
« Reply #3 on: October 20, 2008, 01:19:24 PM »

If this day were less busy, I'd have taken the time to type something like that out as well. Thanks Perks Smiley
Logged

hai
perkiset
Olde World Hacker
Administrator
Lifer
*****
Offline Offline

Posts: 10096



View Profile
« Reply #4 on: October 20, 2008, 01:45:36 PM »

NW mate, nw! Appreciate you sticking around, particularly since I am still swimming hard trying to catch up with the world Smiley
Logged

It is now believed, that after having lived in one compound with 3 wives and never leaving the house for 5 years, Bin Laden called the U.S. Navy Seals himself.
hvshah69
Rookie
**
Offline Offline

Posts: 21


View Profile
« Reply #5 on: October 21, 2008, 02:42:33 PM »

You guys are amazing Grin

I have asked questions twice in this particular forum and both times I have got A+ quality reply.

I can't thank you enough. Praise
Logged
perkiset
Olde World Hacker
Administrator
Lifer
*****
Offline Offline

Posts: 10096



View Profile
« Reply #6 on: October 21, 2008, 06:24:52 PM »

No problem HV. And I appreciate you paying the invoice you'll be receiving on time as well  Don't make me...

ROFLMAO
Logged

It is now believed, that after having lived in one compound with 3 wives and never leaving the house for 5 years, Bin Laden called the U.S. Navy Seals himself.
Pages: [1]
  Print  
 
Jump to:  

Perkiset's Place Home   Best of The Cache   phpMyIDE: MySQL Stored Procedures, Functions & Triggers
Politics @ Perkiset's   Pinkhat's Perspective   
cache
mart
coder
programmers
ajax
php
javascript
Powered by MySQL Powered by PHP Powered by SMF 1.1.2 | SMF © 2006-2007, Simple Machines LLC
Seo4Smf v0.2 © Webmaster's Talks


Valid XHTML 1.0! Valid CSS!