Connection String Decorating
What struck me most are these lines:


p.Name.ToUpper() != name.ToUpper()



I find it's always better to use the Equals overload that accepts a StringComparer and set it as a readonly field for the entire class rather then hoping you don't forget ToUpper everywhere.



This is a minor thing but I find that Excerpt should be called Without because it does not actually create an excerpt where you can say what you want to have but rather the other way around. The parameter says what should be excluded from the string. I also think it should take a params string[] argument. 




new Regex(regex ?? throw new ArgumentNullException(nameof(regex)))



I find newing a Regex should be reserved for initializing a field. In a situation like here the static Regex.Matches would be more suitable - unless there is a reason why you chose new over the static method.




public string ToString(string format) =&gt;



This one I don't like. It's not a standard .net practice. If you want to use a formattable ToString then the class should implement the IFormattable interface correctly... or you should make this one private.

Besides having this helper method to just do


ToString("{0}={1};")



is a little bit overengineered. How many of such formats do you have? A single ToString() would do too.



ConnectionParameter should be ConnectionStringField (or Property) to clearly communicate it belongs to a connection string.



I'm not sure if you know that there is already a SqlConnectionStringBuider and most providers have their own implementations. 

I prefer to write extensions against this type rather than parsing everything myself. It also validates the keys you use so you cannot use anything that isn't supported... well, unless it was your intention and a reason for the custom builder.