I have a JSON like text file of records with duplicate ids that look like:{ID:93 , ST:[42,77,51]}{ID:21 , ST:[43,4]}{ID:94 , ST:[65,11,4]}{ID:93 , ST:[42,77,51,29,63]}{ID:73 , ST:[21,20]}{ID:94 , ST:[65,11,4]}{ID:77 , ST:[87]}I am trying to filter the duplicates and always keep the first occurrence of such a match. The field ST could be the same but also different for the records with the same id.The output would look like:{ID:93 , ST:[42,77,51]}{ID:21 , ST:[43,4]}{ID:94 , ST:[65,11,4]}{ID:73 , ST:[21,20]}{ID:77 , ST:[87]}A similar question has already been asked here, but for this case the data file being edited was a comma separated file. Here we are dealing with a JSON data and the goal will be to find the lines that have the same id values(may be a regex match) and keep the latest one. Anyone has an idea how to tackle that with awk, sed or pure command line tools?
Obtaining unique lines with matching strings
shell script;awk;grep;filter