Building an Interactive Shell in Golang
Building an interactive shell in Golang
Go is great for building command-line applications. We built one: Dolt, the
world's first version-controlled SQL database. We wrote our own command line parser for handling all
of Dolt's subcommands and arguments, but maybe we shouldn't have. There are lots of great ones out
there that if we might have used instead if we were starting the project today:

spf13/cobra has great support for code generation from a simple
text command format, and will generate you zsh and other shell completions for free.
charmbracelet/gum is a golang tool for generating very
stylish command line prompts you can compose into shell scripts.
alecthomas/kingpin is a great all-around library for
building command-line apps, and is probably the closest to what we built ourselves.

So there's lots of great tooling for addressing this common use case which Go is great at.
But what if you want to build an interactive shell in Go? What do you use? There aren't nearly as
many options out there.
This blog will teach you how to use the best option we know of,
abiosoft/ishell, and discuss how to get the most out of
it. You'll learn how to configure the interactive shell with the commands you want to handle, how to
quit the shell, and how to use the built-in quality-of-life features of the package. We'll also show
how we used it to build Dolt's built-in SQL shell capabilities.
Demo
When you fire up Dolt's SQL shell, this is what you see:
% dolt sql
# Welcome to the DoltSQL shell.
# Statements must be terminated with ';'.
# "exit" or "quit" (or Ctrl-D) to exit.
last_insert/main*> show tables;
+-----------------------+
| Tables_in_last_insert |
+-----------------------+
| test                  |
+-----------------------+
1 row in set (0.00 sec)

last_insert/main*> select * from test;
+------+----+
| name | id |
+------+----+
| one  | 1  |
+------+----+
1 row in set (0.00 sec)

last_insert/main*> call dolt_checkout('-b', 'newBranch');
+--------+--------------------------------+
| status | message                        |
+--------+--------------------------------+
| 0      | Switched to branch 'newBranch' |
+--------+--------------------------------+
1 row in set (0.01 sec)

last_insert/newBranch> call dolt_checkout('main');
+--------+---------------------------+
| status | message                   |
+--------+---------------------------+
| 0      | Switched to branch 'main' |
+--------+---------------------------+
1 row in set (0.00 sec)

last_insert/main*> exit
Bye
There's a couple things to pay attention to here:

The shell begins with a greeting message that tells users how to use it.
Each line where the user can enter input has a prompt, which can be altered depending on the
state of the shell. In our SQL shell, it shows you which database and branch you're connected to.
You exit the shell with a particular input, either quit or exit in our case.

We also use color in the shell, both for the prompts and the output. Here's how it appears in my
terminal:

      
    
  
  
    
So that's what a shell does, and how it differs from normal command-line applications: you have a
loop where you accept user input over and over, giving answers or doing other work, until the user
decides to exit with a predefined command.
Pre-defined commands or free-form?
The original abiosoft/ishell package is built to process
predefined commands, where each command dispatches to a different handler. In action, this looks
like:
shell.AddCmd(&ishell.Cmd{
    Name: "login",
    Help: "simulate a login",
    Func: func(c *ishell.Context) {
        // disable the '>>>' for cleaner same line input.
        c.ShowPrompt(false)
        defer c.ShowPrompt(true) // yes, revert after login.

        // get username
        c.Print("Username: ")
        username := c.ReadLine()

        // get password.
        c.Print("Password: ")
        password := c.ReadPassword()

        ... // do something with username and password

        c.Println("Authentication Successful.")
    },
})
Then at runtime, you would see:
>>> login
Username: someusername
Password:
Authentication Successful.
This is great and has a ton of uses, but we wanted something slightly different. Rather than have
pre-defined commands with their own handlers, we wanted something more like a
REPL, where we just read input
until we find a delimiter, then process it the same way every time. For Dolt's SQL shell, this means
reading a query until we see a ; character, then sending that query to the database and printing
results, over and over. This wasn't easy to do with the original package, so we forked our own
copy to add this capability. That's how we're handling the
free-form SQL query capability above. If that's what you're trying to do, feel free to use our fork
instead of the original package. We'll demonstrate how to configure the shell in free-form mode in
the following sections.
Launching the shell for pre-defined commands
To launch your shell, first choose some configuration options and create a new shell:
	rlConf := readline.Config{
		Prompt:                 initialPrompt,
		Stdout:                 cli.CliOut,
		Stderr:                 cli.CliOut,
		HistoryFile:            historyFile,
		HistoryLimit:           500,
		HistorySearchFold:      true,
		DisableAutoSaveHistory: true,
	}

    shell := ishell.NewWithConfig(&rlConf)
Then add your commands. Each command is a *ishell.Cmd.
    shell.AddCmd(&ishell.Cmd{
        Name: "login",
        Help: "simulate a login",
        Func: func(c *ishell.Context) {
            ...
        },
    })
    shell.AddCmd(...)
    shell.AddCmd(...)
Finally, run it:
    // blocks until shell.Stop() is called by some command
    shell.Run()
Launching an uninterpreted (free-form) shell
If you want your shell to be free-form, your setup is different. What you do depends on your
configuration, but you'll need to create your shell with additional configuration to control line
terminators and how to quit the shell:
	shellConf := ishell.UninterpretedConfig{
		ReadlineConfig: &rlConf,
		QuitKeywords: []string{
			"quit", "exit", "quit()", "exit()",
		},
		LineTerminator: ";",
	}

	shell := ishell.NewUninterpreted(&shellConf)
Then start the shell in uninterpreted (free-form) mode, with a single function to handle parsing all
input. Here's what ours does:
    shell.Uninterpreted(func(c *ishell.Context) {
        // The entire input line is provided as the single element in c.Args
		query := c.Args[0]
		if len(strings.TrimSpace(query)) == 0 {
			return
		}

		singleLine := strings.ReplaceAll(query, "\n", " ")

        // Add this query to our command history
        if err := shell.AddHistory(singleLine); err != nil {
			shell.Println(color.RedString(err.Error()))
		}

		query = strings.TrimSuffix(query, shell.LineTerminator())

        var nextPrompt string
		var multiPrompt string
		var sqlSch sql.Schema
		var rowIter sql.RowIter

        // Execute the query on the database, then either print the query results or an error if there was one
	    func() {
            // We start a new context here so the user can interrupt a long-running query
			subCtx, stop := signal.NotifyContext(initialCtx, os.Interrupt, syscall.SIGTERM)
			defer stop()

			sqlCtx := sql.NewContext(subCtx, sql.WithSession(sqlCtx.Session))

            // Execute the query and print the results or the error
			if sqlSch, rowIter, err = processQuery(sqlCtx, query, qryist); err != nil {
				verr := formatQueryError("", err)
				shell.Println(verr.Verbose())
			} else if rowIter != nil {
				switch closureFormat {
				case engine.FormatTabular, engine.FormatVertical:
					err = engine.PrettyPrintResultsExtended(sqlCtx, closureFormat, sqlSch, rowIter)
				default:
					err = engine.PrettyPrintResults(sqlCtx, closureFormat, sqlSch, rowIter)
				}

				if err != nil {
					shell.Println(color.RedString(err.Error()))
				}
			}

			nextPrompt, multiPrompt = formattedPrompts(db, branch, dirty)
		}()

        // Update the prompts with the current database and branch name
		shell.SetPrompt(nextPrompt)
		shell.SetMultiPrompt(multiPrompt)
	})

    // Run the shell. This blocks until the user exits.
    shell.Run()
This is long but what it's doing is actually pretty simple: get a query, process it, and either
print the results or an error message. Then change the prompts to the shell as necessary. We add
color where necessary with the fatih/color package.
Interrupting execution
To stop execution of the shell, there are a few different options.
EOF handler
First, you can install an end-of-file handler to decide what to do if input runs out:
	shell.EOF(func(c *ishell.Context) {
		c.Stop()
	})
This handler gets called if the user pipes a file into the program and it ends, or when the user
sends the special EOF character (Ctrl-D on Unix systems) from their keyboard. You can do whatever
you want here, but the simplest option is to stop the shell like we do above.
Interrupt handler
Next, you can install an interrupt handler. This controls what happens when the process gets a
SIGINT signal, like when the user presses Ctrl-C.
	shell.Interrupt(func(c *ishell.Context, count int, input string) {
		if count > 1 {
			c.Stop()
		} else {
			c.Println("Received SIGINT. Interrupt again to exit, or use ^D, quit, or exit")
		}
	})
Again, you can do anything you want here. The shell keeps track of how many times in a row the
handler was invoked. We chose to only quit on the second interrupt signal.
Quit keywords
Finally, for free-form shells, the QuitKeywords field of the shell configuration will
automatically cause the shell to exit if a quit keyword is encountered verbatim.
	shellConf := ishell.UninterpretedConfig{
		ReadlineConfig: &rlConf,
		QuitKeywords: []string{
			"quit", "exit", "quit()", "exit()",
		},
		LineTerminator: ";",
	}
Getting fancy: adding shell history and tab completion
One of the reasons we were initially so impressed with the ishell package is its support of two
great quality-of-life features: shell history and auto-complete.
History is built into pretty much every shell you use for Unix or Mac systems. It's what causes
your previous commands to cycle through when you press the up arrow. Some shells also allow you to
search through your history with a hotkey, usually Ctrl-R. This is a must-have feature for a SQL
shell, where you often want to run the same query over and over with slight variations. ishell has
great history support, including search.
To enable history, just provide a filepath to the history file at config time, then make sure to
update the history on every command:
shell.AddHistory(inputLine)
To enable auto-complete is a bit more work. In standard mode (pre-defined commands), names of
commands will auto-complete. But if you want to do something fancier, you'll need to implement a
custom completer. Ours is complicated by the fact that we want to offer different completions if we
think the thing being completed is a column name in a SQL query. Here it is for reference:
func (c *sqlCompleter) Do(line []rune, pos int) (newLine [][]rune, length int) {
	var words []string
	if w, err := shlex.Split(string(line)); err == nil {
		words = w
	} else {
		// fall back
		words = strings.Fields(string(line))
	}

	var cWords []string
	prefix := ""
	lastWord := ""
	if len(words) > 0 && pos > 0 && line[pos-1] != ' ' {
		lastWord = words[len(words)-1]
		prefix = strings.ToLower(lastWord)
	} else if len(words) > 0 {
		lastWord = words[len(words)-1]
	}

	cWords = c.getWords(lastWord)

	var suggestions [][]rune
	for _, w := range cWords {
		lowered := strings.ToLower(w)
		if strings.HasPrefix(lowered, prefix) {
			suggestions = append(suggestions, []rune(strings.TrimPrefix(lowered, prefix)))
		}
	}
	if len(suggestions) == 1 && prefix != "" && string(suggestions[0]) == "" {
		suggestions = [][]rune{[]rune(" ")}
	}

	return suggestions, len(prefix)
}

// Simple suggestion function. Returns column name suggestions if the last word in the input has exactly one '.' in it,
// otherwise returns all tables, columns, and reserved words.
func (c *sqlCompleter) getWords(lastWord string) (s []string) {
	lastDot := strings.LastIndex(lastWord, ".")
	if lastDot > 0 && strings.Count(lastWord, ".") == 1 {
		alias := lastWord[:lastDot]
		return prepend(alias+".", c.columnNames)
	}

	return c.allWords
}
You install a custom completer with the CustomCompleter() method:
shell.CustomCompleter(completer)
We're doing this to to auto-complete SQL keywords and schema elements. Another common use case would
be to complete the names of files in arguments.
Conclusion
https://github.com/abiosoft/ishell is a great package for
building interactive shells in Go, and we hope it gets more love. It's not in very active
development anymore, but it works, is stable, and is the best option for building interactive shells
that we know of.
Have questions about Dolt, or building interactive shells in Go? Have a suggestion on how to improve
this tutorial? Join us on Discord to talk to our engineering team
and meet other Dolt users.