Are there ways to speed up this string fuzzy matching in Golang?
Review I - package go-fuzzywuzzy



After a quick read of your Go code, the Go code for package go-fuzzywuzzy, and the Go documentation, it is reasonable to hope for at least a 60% to 95% improvement in Go performance.



For example, 

$ go build fuzzy.go &amp;&amp; time ./fuzzy
real    0m55.183s
user    0m58.858s
sys     0m0.944s
$


After moving one line in package go-fuzzywuzzy,

$ go build fuzzy.go &amp;&amp; time ./fuzzy
real    0m6.321s
user    0m7.211s
sys     0m0.188s
$


The line to move:

r := regexp.MustCompile("[^\\p{L}\\p{N}]")


A diff of package go-fuzzywuzzy:

diff --git a/stringutility.go b/stringutility.go
index 935f6e1..95441cc 100644
--- a/stringutility.go
+++ b/stringutility.go
@@ -5,6 +5,8 @@ import (
        "strings"
 )

+var rxCleanse = regexp.MustCompile("[^\\p{L}\\p{N}]")
+
 func Cleanse(s string, forceAscii bool) string {
        s = strings.TrimSpace(s)
        s = strings.ToLower(s)
@@ -12,8 +14,7 @@ func Cleanse(s string, forceAscii bool) string {
                s = ASCIIOnly(s)
        }

-       r := regexp.MustCompile("[^\\p{L}\\p{N}]")
-       s = r.ReplaceAllString(s, " ")
+       s = rxCleanse.ReplaceAllString(s, " ")
        return s
 }





  Package regexp

 import "regexp"

  
  func MustCompile

func MustCompile(str string) *Regexp

  
  MustCompile is like Compile but panics if the expression cannot be
  parsed. It simplifies safe initialization of global variables holding
  compiled regular expressions.


The documentation clearly says global variables.



If I spent more time reading the code, I would expect further performance improvements.





After more reading, package go-fuzzywuzzy is written in "PyGo." Writing the package in Go yields efficiencies.

For example, completely rewriting the go-fuzzywuzzy file stringutility.go in Go.

Before:

$ go build fuzzy.go &amp;&amp; time ./fuzzy
real    0m55.183s
user    0m58.858s
sys     0m0.944s
$


After:

$ go build fuzzy.go &amp;&amp; time ./fuzzy
real    0m5.735s
user    0m6.601s
sys     0m0.193s
$


stringutility.go:

package fuzzy

import (
    "strings"
    "unicode"
)

func Cleanse(s string, forceAscii bool) string {
    if forceAscii {
        s = ASCIIOnly(s)
    }
    s = strings.TrimSpace(s)
    rs := make([]rune, 0, len(s))
    for _, r := range s {
        if !unicode.IsLetter(r) &amp;&amp; !unicode.IsNumber(r) {
            r = ' '
        }
        rs = append(rs, r)
    }
    return strings.ToLower(string(rs))
}

func ASCIIOnly(s string) string {
    b := make([]byte, 0, len(s))
    for _, r := range s {
        if r &lt;= unicode.MaxASCII {
            b = append(b, byte(r))
        }
    }
    return string(b)
}






UPDATE:

The package go-fuzzywuzzy author adopted the suggested stringutility.go changes:

[Fixes #4] Stop reinitializing costly regex in stringutility
https://github.com/paul-mannino/go-fuzzywuzzy/commit/f14294bf5858c8a7fa51b026a9ee9a2802c816bf

Was using regexp.MustCompile within a frequently invoked
method Cleanse. Since go does not cache these calls, it was
incurring a costly regex compilation each time Cleanse was
called. The other changes made in the stackoverflow post that
caught this issue seem to further improve performance by
about 10% on top of the massive gains from fixing this issue,
so incorporating those as well.