Title: How can I compare two source code files/ ast trees?
Tags: parsing;go;compare;abstract-syntax-tree
Question: I'm generating some source code using the templates package( is there a better method? )and part of the testing I need to check if the output matches the expected source code.
I tried a string comparison but it fails due the extra spaces / new lines generated by the templates package. I've also tried format.Source with not success. ( FAIL)
I tried to parse the ast of the both sources (see bellow) but the ast doesn't match either even if the code is basically same except the new lines / spaces. (FAIL)
package main
import (
"fmt"
"go/parser"
"go/token"
"reflect"
func main() {
stub1 := `package main
func myfunc(s string) error {
return nil
}`
stub2 := `package main
func myfunc(s string) error {
return nil
}`
fset := token.NewFileSet()
r1, err := parser.ParseFile(fset, "", stub1, parser.AllErrors)
if err != nil {
panic(err)
fset = token.NewFileSet()
r2, err := parser.ParseFile(fset, "", stub2, parser.AllErrors)
if err != nil {
panic(err)
if !reflect.DeepEqual(r1, r2) {
fmt.Printf("e %v, r %s, ", r1, r2)
Playground
Comment: Do you want to compare arbitrary trees, or just go trees that you have parsed?

Comment: Just go source code/trees, thus the go tag

Comment: https://www.diffchecker.com/

Here is the accepted answer: This was easier than I thought. All I had to do was to remove the empty new lines(after formatting). Below is the code.
package main
import (
"fmt"
"go/format"
"strings"
func main() {
a, err := fmtSource(stub1)
if err != nil {
panic(err)
}
b, err := fmtSource(stub2)
if err != nil {
panic(err)
}
if a != b {
fmt.Printf("a %v, \n b %v", a, b)
}
func fmtSource(source string) (string, error) {
if !strings.Contains(source, "package") {
source = "package main\n" + source
b, err := format.Source([]byte(source))
if err != nil {
return "", err
// cleanLine replaces double space with one space
cleanLine := func(s string)string{
sa := strings.Fields(s)
return strings.Join(sa, " ")
lines := strings.Split(string(b), "\n")
n := 0
var startLn *int
for _, line := range lines {
if line != "" {
line = cleanLine(line)
lines[n] = line
if startLn == nil {
x := n
startLn = &amp;x
}
n++
}
lines = lines[*startLn:n]
// Add final "" entry to get trailing newline from Join.
if n &gt; 0 &amp;&amp; lines[n-1] != "" {
lines = append(lines, "")
// Make it pretty
b, err = format.Source([]byte(strings.Join(lines, "\n")))
if err != nil {
return "", err
return string(b), nil
Comment for this answer: Honestly, i think a better approach would've been to add a `hashCode` method to every AST node, then to match them you just check if both AST have the same hashCode

Here is another answer: Well, one simple way to achieve this is to use the go/printer library, that gives you better control of output formatting, and is basically like running gofmt on the source, normalizing both trees:
package main
import (
"fmt"
"go/parser"
"go/token"
"go/printer"
//"reflect"
"bytes"
func main() {
stub1 := `package main
func myfunc(s string) error {
return nil
}`
stub2 := `package main
func myfunc(s string) error {
return nil
}`
fset1 := token.NewFileSet()
r1, err := parser.ParseFile(fset1, "", stub1, parser.AllErrors)
if err != nil {
panic(err)
fset2 := token.NewFileSet()
r2, err := parser.ParseFile(fset1, "", stub2, parser.AllErrors)
if err != nil {
panic(err)
// we create two output buffers for each source tree
out1 := bytes.NewBuffer(nil)
out2 := bytes.NewBuffer(nil)
// we use the same printer config for both
conf := &amp;printer.Config{Mode: printer.TabIndent | printer.UseSpaces, Tabwidth: 8}
// print to both outputs
if err := conf.Fprint(out1, fset1, r1); err != nil {
panic(err)
if err := conf.Fprint(out2, fset2, r2); err != nil {
panic(err)
// they should be identical!
if string(out1.Bytes()) != string(out2.Bytes()) {
panic(string(out1.Bytes()) +"\n" + string(out2.Bytes()))
} else {
fmt.Println("A-OKAY!")
Of course this code needs to be refactored to not look as stupid. Another approach is instead of using DeepEqual, create a tree comparison function yourself, that skips irrelevant nodes.
Comment for this answer: The comparison function may not be trivial so I'm trying to avoid that. printer seems to fail on more "complex" structures http://play.golang.org/p/I9cAVEYLAm
Comment for this answer: Any idea how exactly can I filter the irrelevant nodes? I've tried to remove the nil nodes assuming they represent the new lines / spaces but it doesn't seem to be the case. play.golang.org/p/JVVpKIzela
Comment for this answer: A nit: `bytes.Buffer` has a `String()` method, so you don't need all those conversions.
Comment for this answer: @mihai maybe traverse the tree and filter it, then use DeepEqual?

{'source': 'StackExchangeDataset', 'subset': 'stackoverflow.com', 'metadata': {'AnswerCount': 2, 'CommentCount': 3, 'ContentLicense': 'CC BY-SA 3.0', 'CreationDate': '2015:00:05/12/15', 'FavoriteCount': 2.0, 'PostTypeId': 1, 'Tags': '<parsing><go><compare><abstract-syntax-tree>', 'Title': 'How can I compare two source code files/ ast trees?', 'ViewCount': 1867, 'id': 30196236, 'user_AboutMe': '', 'user_AccountId': 304121.0, 'user_DisplayName': 'themihai', 'user_Reputation': 7464.0, 'user_Views': 523.0, 'user_id': 613453.0}}