@hnf (Ibrahima Keita) posted a new topic, subject "Logic Error after ingesting number". It read:

   I have an xlsx file that I convert in CSV file using ssconvert tool. In the new generated file, there's one field in floating point type. But after ingestion, the values taken from the file is no more in float.
    
    For exemple: This value `99,8923` become `"99,8923"` in the generated file but other values in the same field are correctly parsed, like `54` is `54`.
    
    I suspect that the probleme may come from `"` added around the number. After ingesting this number with **Logstash** , it becomes `998923` even if I specify in logstash that it is a `float` and I can see clearly that the field is typed with `float` in Kibana.
    
    Here is the version I'm currently using:  
    **Elasticsearch : 8.3**  
    **Kibana : 8.3**  
    **Operating Sys : Red Hat Enterprise Linux 8.5 (Ootpa)**
    
    Thanks for your help.

So @cheshirecat (Krzysztof Smaga) replied

    Hello There!
    
    hnf:
    
    > `"99,8923"`
    
    How about changing "," to "." as a decimal separator in `ssconvert`?

@hnf (Ibrahima Keita) replied

    Hello @cheshirecat ,
    
    I didn't find a way for changing this with `ssconvert`. Do you know how to do it with it? or do you know a way to change `","` to `"."`?

To which @stephenb (Stephen Brown) replied

    Perhaps look at mutate convert here
    
    Also make sure you create a mapping ahead of time for that type
    
    >   * `float`:
    > 
    >     * integers are converted to floats
    >     * strings are parsed; comma-separators and dot-decimals are supported (e.g., `"1,000.5"` produces a float with value of one thousand and one half)
    >     * boolean true and boolean false are converted to `1.0` and `0.0` respectively
    >   * `float_eu`:
    > 
    >     * same as `float`, except string values support dot-separators and comma-decimals (e.g., `"1.000,5"` produces a float with value of one thousand and one half)
    >

Then @hnf (Ibrahima Keita) said

    Hello @stephenb ,
    
    I already used `convert`, and there is others that I convert perfectly in `floating point` type with logstash.
    
    The field with the issue is in type float, but values are not coherent...
    
    I'm actually working on a way to treat this before sending to logstash...

After which, @stephenb (Stephen Brown) added

    OK if you showed us a full source line... pretty sure we could fix it in logstash... but at the source is even better.

To which @stephenb (Stephen Brown) replied

    I just ran this pipeline
    [code] 
        input { stdin {  ecs_compatibility => "v8"} }
        
        filter {
           mutate {
                convert => {
                  "message" => "float_eu"
                }
            }
        }
        
        output { stdout {} }
        
    [/code]
    
    stdin
    [code] 
        > 99,8923
        {
            "@timestamp" => 2022-09-14T21:12:13.177705Z,
                  "host" => {
                "hostname" => "hyperion"
            },
               "message" => 99.8923,
              "@version" => "1",
                 "event" => {
                "original" => "99,8923"
            }
        }
        > 1.123,45     
        {
            "@timestamp" => 2022-09-14T21:14:21.722249Z,
                  "host" => {
                "hostname" => "hyperion"
            },
               "message" => 1123.45,
              "@version" => "1",
                 "event" => {
                "original" => "1.123,45"
            }
        }
        
    [/code]
    
    Worked Fine Note it is a normal `float` on the output not `text/keword`

After which, @hnf (Ibrahima Keita) added

    Hello @stephenb ,
    
    I really appreciate your help and it's working. Sometimes, I feel like "Why I didn't see that" ... That's why the community is around , so your help was a light...
    
    Thanks so much...

To which @system (system) replied

    This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.


{'source': 'discourse', 'url': 'https://discuss.elastic.co/t/logic-error-after-ingesting-number/314331', 'title': 'Logic Error after ingesting number', 'views': 80, 'has_accepted_answer': True, 'word_count': 506, 'like_count': 1, 'reply_count': 6, 'participant_count': 3, 'posts': 9, 'tags': '', 'author': '@hnf (Ibrahima Keita)', 'participants': '@hnf (Ibrahima Keita), @cheshirecat (Krzysztof Smaga), @stephenb (Stephen Brown), @system (system)', 'image_url': [None], 'created_at': '2022-09-13T18:54:00.040Z', 'last_posted_at': '2022-10-13T08:32:53.095Z'}