Binary search tree insertion in Racket
As you suspected, you don't need append for this problem. The trick is to notice that if, for example, your goal is to create the list '(1 2 3), then writing (list 1 2 3) is more straight-forward and more efficient than writing (append '(1) '(2) '(3)).

With that in mind, consider the following insertion function:

(define (insert BST n)
  (cond
    ((null? BST)
     (list empty n empty))
    ((&lt; n (cadr BST))
     (list (insert (car BST) n) (cadr BST) (caddr BST)))
    ((&gt; n (cadr BST))
     (list (car BST) (cadr BST) (insert (caddr BST) n)))
    (else BST)))


To run a small test, suppose you have the list '((() 3 ()) 5 (() 8 (() 10 ()))), which looks as follows:


then inserting 7 to it, ie. (insert '((() 3 ()) 5 (() 8 (() 10 ()))) 7) will produce '((() 3 ()) 5 ((() 7 ()) 8 (() 10 ()))), which looks as follows: