title: cleaning my dataframe (similar lines and \xc3\x28 in the field)
body: <p>I am working on dataframe with python.</p>
  <p>in my first dataframe df1 i have :</p>
  <pre><code>+------+---------+-------------+-------------------------------+  
  |  ID  |  PUBLICATION TITLE  |    DATE      |      JOURNAL     |
  +------+---------------------+--------------+------------------+
  |   1            &quot;a&quot;           &quot;01/10/2000&quot;        &quot;book1&quot;     |
  |   2            &quot;b&quot;           &quot;09/03/2005&quot;          NaN       |
  |  NaN           &quot;b&quot;           &quot;09/03/2005&quot;        &quot;book2      |
  |   5            &quot;z&quot;           &quot;21/08/1995&quot;        &quot;book4&quot;     |
  |   6            &quot;n&quot;           &quot;15/04/1993&quot;   &quot;book9\xc3\x28&quot;  |
  +--------------------------------------------------------------+
  </code></pre>
  <p>Here I would like to clean my dataframe but I don't know how to do it in this case.
  Indeed there are two points which block me.</p>
  <p>The first one is that lines 2 and 3 seems to be the same line because the title of the publication is the same and because I think that the title of the publication is unique to a newspaper</p>
  <p>The second point is for the last line one to \xc3\x28.</p>
  <p>How can I clean my dataframe in a smart way, to be able to use this code for other daataframe if possible?</p>