Loading and converting data from CSV My application imports a CSV file and parses each row, turning it into an object in my code.

The speed for a 500,000 row x 68 column takes 2+ minutes to read. I've narrowed down the bottleneck to the part where the strings are being converted into their respective types using the converter I created. I need some pointers on how I can optimize my code.

This is the code I use to load:

public static void Load(object target, string[] fields, bool supressErrors)
{
    Type targetType = target.GetType();
    PropertyInfo[] properties = targetType.GetProperties();

    foreach (PropertyInfo property in properties)
    {
        object[] attributes = property.GetCustomAttributes(typeof(CSVPositionAttribute), false);

        if (attributes.Length &gt; 0)
        {
            CSVPositionAttribute positionAttr = (CSVPositionAttribute)attributes[0];

            int position = positionAttr.Position;
            try
            {
                object data = fields[position];

                if (positionAttr.DataTransform != string.Empty)
                {
                    MethodInfo method = targetType.GetMethod(positionAttr.DataTransform);
                    data = method.Invoke(target, new object[] { data });
                }
                var prop = property.PropertyType;

                //bottleneck here:
                if (prop == typeof(string))
                {
                    property.SetValue(target, CheckAndConvert.CheckString(data), null);
                }
                else if (prop == typeof(int))
                {
                    property.SetValue(target, CheckAndConvert.CheckInt(data.ToString()), null);
                }
                else if (prop == typeof(double))
                {
                    property.SetValue(target, CheckAndConvert.CheckDouble(data.ToString()), null);
                }
                else if (prop == typeof(long))
                {
                    property.SetValue(target, CheckAndConvert.CheckLong(data.ToString()), null);
                }
                else if (prop == typeof(bool))
                {
                    property.SetValue(target, CheckAndConvert.CheckBool(data.ToString()), null);
                }

            }
            catch(Exception e)
            {
                if (!supressErrors)
                    throw;
            }
        }
    }
}


This is part of the code I use to convert each cell:

public class CheckAndConvert
{
    public static int CheckInt(string input)
    {
        int convertedInput = 0;
        double convertedInput2 = 0;

        if (int.TryParse(input, out convertedInput))
        {
            return convertedInput;
        }
        else if (double.TryParse(input, out convertedInput2))
        {
            return (int)convertedInput2;
        }
        else
        {
            return 0;
        }
    }
    public static double CheckDouble(string input)
    {
        double convertedInput = 0;
        int convertedInput2 = 0;
        if (double.TryParse(input, out convertedInput))
        {
            return convertedInput;
        }
        else if (int.TryParse(input, out convertedInput2))
        {
            return convertedInput2;
        }
        else
        {
            return 0;
        }
    }
    public static bool CheckBool(string input)
    {
        bool convertedInput = true;
        if (bool.TryParse(input, out convertedInput))
        {
            return convertedInput;
        }
        else
        {
            return true;
        }
    }
    public static string CheckString(object input)
    {
        if (input == null)
        {
            return "";
        }
        else
        {
            return input.ToString();
        }
    }

}
There's a few ways you could improve your CheckAndConvert class and functions. 

Design Suggestions


Fluency - It's not a requirement by any means but fluency helps readability. CheckAndConvert.CheckInt is almost a sentence, perhaps something shorter Parse.Int or Parse.Bool.
Naming Simplicity - On the lines of the previous suggestion, your class/function names imply that it's doing more than simply parsing a given value. You're checking if the parse succeeds but then returning a default value if it fails. The user of this function needs not to know about the check part at all. And the word check is repeated (class and function name). Programmers don't typically like redundancy.
Naming Accuracy - CheckInt and CheckDouble they check each other if their primary data type TryParse fails. In the case of CheckDouble I don't think it's ever going to be true for the second if-statement. In the case of CheckInt since int.TryParse can fail on decimals that you want to be integers I would just always use double.TryParse with a cast. (Remember: Cast truncates and Convert.ToInt32 rounds).


I would be surprised if any of the above suggestions other than maybe the removal of int.TryParse would actually improve performance but it would still make for some cleaner simpler code.

Performance

If you've narrowed down that the parsing of values into their respective properties is the time consumer research alternative ways to parse numbers (they're ugly but claim they're faster) additionally this might be helpful for possibly speeding up your SetValue calls (be warned it's a long read).

Link Information


The parse numbers link decribes parsing the string yourself as .Parse uses CultureInfo and stuff that can slows it down.
The SetValue link describes research into using delegates for method calls in reflection and shows some promising performance boosts that I've used myself.