title: Compiling a Delegate with Expression.Lambda() - Parameter Out Of Scope, but is it really?
body: <p>I came across an interesting problem today whilst implementing a feature into a dynamic expression building library. More specifically, but irrelevantly, a feature to define operator precedence in an expression.</p>
  
  <p>When the LINQ engine was compiling the final expression, I was encountering a <code>InvalidOperationException</code> declaring <code>Lambda parameter out of scope</code>.</p>
  
  <p>The problem manifests itself after assigning the relevant <code>ParameterExpression</code> objects.</p>
  
  <p>Working with a complete and well formed lambda expression tree, I discovered that reassigning the Lambda's <code>ParameterExpression</code> objects to valid references was invalid when compiling the Lambda. </p>
  
  <p>This is a short description of the behaviour I initially employed before I applied a fix: </p>
  
  <ul>
  <li>Build the expression tree, destined for use with <code>Queryable.Where</code>, the root expression being a <code>LambdaExpression</code>, constructed using <code>Expression.Lambda(<em>expression</em>, Expression.Parameter(GetType(<em>type</em>), "<em>name</em>"))</code></li>
  <li>Visit the expression tree (using LinqKit), build hash table of parameters encountered</li>
  <li>Subsequent parameters of the same name are substituted with the first parameter encountered of identical name</li>
  </ul>
  
  <p>The result was an expression tree whereby all the <code>ParameterExpression</code> references of the same name were all pointing to the same object- but the <code>InvalidOperationException</code> was encountered when compiling.</p>
  
  <p>The fix I applied employed the following behaviour:</p>
  
  <ul>
  <li>Build the parameters as an array of <code>ParameterExpression</code></li>
  <li>Construct the root Lambda, using <code>Expression.Lambda(<em>expression</em>, <em>parameterArray</em>)</code></li>
  <li>Visit the expression tree (using LinqKit), substitute parameters encountered with parameters from <code><em>parameterArray</em></code></li>
  </ul>
  
  <p>The end result compiles fine, even though the Lambda expression structure <em>is conceptually the same</em> as the output from the former behaviour.</p>
  
  <p>The question is: <strong>Why does the first fail, and the second succeed</strong>?</p>
  
  <p>Below is a test fixture class to reproduce (excuse the vb), with the test cases and a couple of supporting classes (depends on nUnit, LinqKit):</p>
  
  <p><em>note: TestFixture &amp; Test attribute declarations are missing- how to do in markdown???</em></p>
  
  <pre><code>
  
  Imports LinqKit
  Imports NUnit.Framework
  Imports System.Linq.Expressions
  
   _
  Public Class ParameterOutOfScopeTests
  
      Public Class TestObject
          Public Name As String
          Public DateOfBirth As DateTime = DateTime.Now
          Public DateOfDeath As DateTime?
      End Class
  
      Public Class ParameterNormalisation
          Inherits ExpressionVisitor
  
          Public Sub New(ByVal expression As Expression)
              _expression = expression
          End Sub
  
          Private _expression As expression
          Private _parameter As ParameterExpression
          Private _name As String
  
          Public Function Normalise(ByVal parameter As ParameterExpression) As Expression
              _parameter = parameter
              _name = parameter.Name
              _expression = Me.Visit(_expression)
              Return _expression
          End Function
  
          Public Function Normalise(ByVal name As String) As Expression
              _name = name
              _expression = Me.Visit(_expression)
              Return _expression
          End Function
  
          Protected Overrides Function VisitParameter(ByVal p As System.Linq.Expressions.ParameterExpression) As System.Linq.Expressions.Expression
  
              Debug.WriteLine("ClientExpressionParameterNormalisation.VisitParameter:: Parameter visited: " & p.Name & " " & p.GetHashCode)
              If p.Name.Equals(_name) Then
  
                  If _parameter Is Nothing Then
                      _parameter = p
                      Debug.WriteLine("ClientExpressionParameterNormalisation.VisitParameter:: Primary parameter identified: " & p.GetHashCode)
                  ElseIf Not p Is _parameter Then
                      Debug.WriteLine("ClientExpressionParameterNormalisation.VisitParameter:: Secondary parameter substituted: " & p.GetHashCode & " with " & _parameter.GetHashCode)
                      Return MyBase.VisitParameter(_parameter)
                  Else
                      Debug.WriteLine("ClientExpressionParameterNormalisation.VisitParameter:: Parameter already common: " & p.GetHashCode & " with " & _parameter.GetHashCode)
                  End If
  
              End If
  
              Return MyBase.VisitParameter(p)
  
          End Function
  
  
      End Class
  
       _
      Public Sub Lambda_Parameter_Out_Of_Scope_As_Expected()
  
          Dim treeOne As Expression(Of Func(Of TestObject, Boolean)) = Function(test As TestObject) test.DateOfBirth > Now And test.Name.Contains("name")
          Dim treeTwo As Expression(Of Func(Of TestObject, Boolean)) = Function(test As TestObject) Not test.DateOfDeath.HasValue
  
          Dim treeThree As Expression = Expression.And(treeOne.Body, treeTwo.Body)
  
          Dim realParameter As ParameterExpression = Expression.Parameter(GetType(TestObject), "test")
  
          Dim lambdaOne As LambdaExpression = Expression.Lambda(treeThree, realParameter)
          Dim delegateOne As [Delegate] = lambdaOne.Compile
  
      End Sub
  
       _
      Public Sub Lambda_Compiles()
  
          Dim treeOne As Expression(Of Func(Of TestObject, Boolean)) = Function(test As TestObject) test.DateOfBirth > Now And test.Name.Contains("name")
          Dim treeTwo As Expression(Of Func(Of TestObject, Boolean)) = Function(test As TestObject) Not test.DateOfDeath.HasValue
  
          Dim treeThree As Expression = Expression.And(treeOne.Body, treeTwo.Body)
  
          Dim normaliser As New ParameterNormalisation(treeThree)
          Dim realParameter As ParameterExpression = Expression.Parameter(GetType(TestObject), "test")
          treeThree = normaliser.Normalise(realParameter)
  
          Dim lambdaOne As LambdaExpression = Expression.Lambda(treeThree, realParameter)
          Dim delegateOne As [Delegate] = lambdaOne.Compile
  
      End Sub
  
       _
      Public Sub Lambda_Fails_But_Is__Conceptually__Sound()
  
          Dim treeOne As Expression(Of Func(Of TestObject, Boolean)) = Function(test As TestObject) test.DateOfBirth > Now And test.Name.Contains("name")
          Dim treeTwo As Expression(Of Func(Of TestObject, Boolean)) = Function(test As TestObject) Not test.DateOfDeath.HasValue
  
          Dim treeThree As Expression = Expression.And(treeOne.Body, treeTwo.Body)
  
          Dim realParameter As ParameterExpression = Expression.Parameter(GetType(TestObject), "test")
          Dim lambdaOne As LambdaExpression = Expression.Lambda(treeThree, realParameter)
  
          Dim normaliser As New ParameterNormalisation(lambdaOne)
          lambdaOne = DirectCast(normaliser.Normalise("test"), LambdaExpression)
  
          Dim delegateOne As [Delegate] = lambdaOne.Compile
  
      End Sub
  
  End Class
  
  </code></pre>