Improve grouping in LINQ
There's quite a lot of logic in there, so I doubt you will get there completely by using LINQ, and, more importantly, if it would actually be more elegant.

I would rewrite as follows, and be done with it:

        foreach (AddressInfo addressInfo in subDetails)
        {
            grpInfo = lstGroupInfo.SingleOrDefault(x =&gt; x.Address1 == addressInfo.Address1);
            if (grpInfo != null)
            {
                grpInfo.Rural = addressInfo.Rural;
                grpInfo.Urban = addressInfo.Urban;
                grpInfo.SubDetails.Add(addressInfo);
            }
            else
            {
                grpInfo = new GroupInfo();
                grpInfo.AddressID = addressInfo.AddressID;
                grpInfo.LocationID = addressInfo.NamedLocationID;
                grpInfo.Rural = addressInfo.Rural;
                grpInfo.Urban = addressInfo.Urban;
            }
        }


In addition, it seems to me that Rural and Urban are mutually exclusive, so why not define an enumeration that contains those two (or more) values, or declare one boolean property IsRural to indicate if it's "Rural", and if not, it's urban. That would bring the code down to:

        foreach (AddressInfo addressInfo in subDetails)
        {
            grpInfo = lstGroupInfo.SingleOrDefault(x =&gt; x.Address1 == addressInfo.Address1);
            if (grpInfo != null)
            {
                grpInfo.IsRural = addressInfo.IsRural;
                grpInfo.SubDetails.Add(addressInfo);
            }
            else
            {
                grpInfo = new GroupInfo();
                grpInfo.AddressID = addressInfo.AddressID;
                grpInfo.LocationID = addressInfo.NamedLocationID;
                grpInfo.IsRural = addressInfo.IsRural;
            }
        }