Go 编译器介绍
Go 编译
`cmd/compile` 包含构成 Go 编译器主要的包。编译器在逻辑上可以被分为四个阶段，我们将简要介绍这几个阶段以及包含相应代码的包的列表。


![](/data/attachment/album/201809/04/110843j9uhgeq9yh88hy8e.jpg)


在谈到编译器时，有时可能会听到<ruby> 前端 <rt>  front-end </rt></ruby>和<ruby> 后端 <rt>  back-end </rt></ruby>这两个术语。粗略地说，这些对应于我们将在此列出的前两个和后两个阶段。第三个术语<ruby> 中间端 <rt>  middle-end </rt></ruby>通常指的是第二阶段执行的大部分工作。


请注意，`go/parser` 和 `go/types` 等 `go/*` 系列的包与编译器无关。由于编译器最初是用 C 编写的，所以这些 `go/*` 包被开发出来以便于能够写出和 `Go` 代码一起工作的工具，例如 `gofmt` 和 `vet`。


需要澄清的是，名称 “gc” 代表 “<ruby> Go 编译器 <rt>  Go compiler </rt></ruby>”，与大写 GC 无关，后者代表<ruby> 垃圾收集 <rt>  garbage collection </rt></ruby>。


### 1、解析


* `cmd/compile/internal/syntax`（<ruby> 词法分析器 <rt>  lexer </rt></ruby>、<ruby> 解析器 <rt>  parser </rt></ruby>、<ruby> 语法树 <rt>  syntax tree </rt></ruby>）


在编译的第一阶段，源代码被标记化（词法分析）、解析（语法分析），并为每个源文件构造语法树（LCTT 译注：这里标记指 token，它是一组预定义的、能够识别的字符串，通常由名字和值构成，其中名字一般是词法的类别，如标识符、关键字、分隔符、操作符、文字和注释等；语法树，以及下文提到的<ruby> 抽象语法树 <rt>  Abstract Syntax Tree </rt></ruby>（AST），是指用树来表达程序设计语言的语法结构，通常叶子节点是操作数，其它节点是操作码）。


每个语法树都是相应源文件的确切表示，其中节点对应于源文件的各种元素，例如表达式、声明和语句。语法树还包括位置信息，用于错误报告和创建调试信息。


### 2、类型检查和 AST 变换


* `cmd/compile/internal/gc`（创建编译器 AST，<ruby> 类型检查 <rt>  type-checking </rt></ruby>，<ruby> AST 变换 <rt>  AST transformation </rt></ruby>）


gc 包中包含一个继承自（早期）C 语言实现的版本的 AST 定义。所有代码都是基于它编写的，所以 gc 包必须做的第一件事就是将 syntax 包（定义）的语法树转换为编译器的 AST 表示法。这个额外步骤可能会在将来重构。


然后对 AST 进行类型检查。第一步是名字解析和类型推断，它们确定哪个对象属于哪个标识符，以及每个表达式具有的类型。类型检查包括特定的额外检查，例如“声明但未使用”以及确定函数是否会终止。


特定变换也基于 AST 完成。一些节点被基于类型信息而细化，例如把字符串加法从算术加法的节点类型中拆分出来。其它一些例子是<ruby> 死代码消除 <rt>  dead code elimination </rt></ruby>，<ruby> 函数调用内联 <rt>  function call inlining </rt></ruby>和<ruby> 逃逸分析 <rt>  escape analysis </rt></ruby>（LCTT 译注：逃逸分析是一种分析指针有效范围的方法）。


### 3、通用 SSA


* `cmd/compile/internal/gc`（转换成 SSA）
* `cmd/compile/internal/ssa`（SSA 相关的<ruby> 环节 <rt>  pass </rt></ruby>和规则）


（LCTT 译注：许多常见高级语言的编译器无法通过一次扫描源代码或 AST 就完成所有编译工作，取而代之的做法是多次扫描，每次完成一部分工作，并将输出结果作为下次扫描的输入，直到最终产生目标代码。这里每次扫描称作一个<ruby> 环节 <rt>  pass </rt></ruby>；最后一个环节之前所有的环节得到的结果都可称作中间表示法，本文中 AST、SSA 等都属于中间表示法。SSA，静态单赋值形式，是中间表示法的一种性质，它要求每个变量只被赋值一次且在使用前被定义）。


在此阶段，AST 将被转换为<ruby> 静态单赋值 <rt>  Static Single Assignment </rt></ruby>（SSA）形式，这是一种具有特定属性的低级<ruby> 中间表示法 <rt>  intermediate representation </rt></ruby>，可以更轻松地实现优化并最终从它生成机器码。


在这个转换过程中，将完成<ruby> 内置函数 <rt>  function intrinsics </rt></ruby>的处理。这些是特殊的函数，编译器被告知逐个分析这些函数并决定是否用深度优化的代码替换它们（LCTT 译注：内置函数指由语言本身定义的函数，通常编译器的处理方式是使用相应实现函数的指令序列代替对函数的调用指令，有点类似内联函数）。


在 AST 转化成 SSA 的过程中，特定节点也被低级化为更简单的组件，以便于剩余的编译阶段可以基于它们工作。例如，内建的拷贝被替换为内存移动，`range` 循环被改写为 `for` 循环。由于历史原因，目前这里面有些在转化到 SSA 之前发生，但长期计划则是把它们都移到这里（转化 SSA）。


然后，一系列机器无关的规则和编译环节会被执行。这些并不考虑特定计算机体系结构，因此对所有 `GOARCH` 变量的值都会运行。


这类通用的编译环节的一些例子包括，死代码消除、移除不必要的空值检查，以及移除无用的分支等。通用改写规则主要考虑表达式，例如将一些表达式替换为常量，优化乘法和浮点操作。


### 4、生成机器码


* `cmd/compile/internal/ssa`（SSA 低级化和架构特定的环节）
* `cmd/internal/obj`（机器码生成）


编译器中机器相关的阶段开始于“低级”的编译环节，该阶段将通用变量改写为它们的特定的机器码形式。例如，在 amd64 架构中操作数可以在内存中操作，这样许多<ruby> 加载-存储 <rt>  load-store </rt></ruby>操作就可以被合并。


注意低级的编译环节运行所有机器特定的重写规则，因此当前它也应用了大量优化。


一旦 SSA 被“低级化”并且更具体地针对目标体系结构，就要运行最终代码优化的编译环节了。这包含了另外一个死代码消除的环节，它将变量移动到更靠近它们使用的地方，移除从来没有被读过的局部变量，以及<ruby> 寄存器 <rt>  register </rt></ruby>分配。


本步骤中完成的其它重要工作包括<ruby> 堆栈布局 <rt>  stack frame layout </rt></ruby>，它将堆栈偏移位置分配给局部变量，以及<ruby> 指针活性分析 <rt>  pointer liveness analysis </rt></ruby>，后者计算每个垃圾收集安全点上的哪些堆栈上的指针仍然是活动的。


在 SSA 生成阶段结束时，Go 函数已被转换为一系列 `obj.Prog` 指令。它们被传递给汇编程序（`cmd/internal/obj`），后者将它们转换为机器码并输出最终的目标文件。目标文件还将包含反射数据，导出数据和调试信息。


### 扩展阅读


要深入了解 SSA 包的工作方式，包括它的环节和规则，请转到 [cmd/compile/internal/ssa/README.md](https://github.com/golang/go/blob/master/src/cmd/compile/internal/ssa/README.md)。




---


via: <https://github.com/golang/go/blob/master/src/cmd/compile/README.md>


作者：[mvdan](https://github.com/mvdan) 译者：[stephenxs](https://github.com/stephenxs) 校对：[pityonline](https://github.com/pityonline), [wxy](https://github.com/wxy)


本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出


OK
`cmd/compile`

contains the main packages that form the Go compiler. The compiler
may be logically split in four phases, which we will briefly describe alongside
the list of packages that contain their code.

You may sometimes hear the terms "front-end" and "back-end" when referring to the compiler. Roughly speaking, these translate to the first two and last two phases we are going to list here. A third term, "middle-end", often refers to much of the work that happens in the second phase.

Note that the `go/*`

family of packages, such as `go/parser`

and
`go/types`

, are mostly unused by the compiler. Since the compiler was
initially written in C, the `go/*`

packages were developed to enable
writing tools working with Go code, such as `gofmt`

and `vet`

.
However, over time the compiler's internal APIs have slowly evolved to
be more familiar to users of the `go/*`

packages.

It should be clarified that the name "gc" stands for "Go compiler", and has little to do with uppercase "GC", which stands for garbage collection.

`cmd/compile/internal/syntax`

(lexer, parser, syntax tree)

In the first phase of compilation, source code is tokenized (lexical analysis), parsed (syntax analysis), and a syntax tree is constructed for each source file.

Each syntax tree is an exact representation of the respective source file, with nodes corresponding to the various elements of the source such as expressions, declarations, and statements. The syntax tree also includes position information which is used for error reporting and the creation of debugging information.

`cmd/compile/internal/types2`

(type checking)

The types2 package is a port of `go/types`

to use the syntax package's
AST instead of `go/ast`

.

`cmd/compile/internal/types`

(compiler types)`cmd/compile/internal/ir`

(compiler AST)`cmd/compile/internal/noder`

(create compiler AST)

The compiler middle end uses its own AST definition and representation of Go types carried over from when it was written in C. All of its code is written in terms of these, so the next step after type checking is to convert the syntax and types2 representations to ir and types. This process is referred to as "noding."

Noding using a process called Unified IR, which builds a node representation using a serialized version of the typechecked code from step 2. Unified IR is also involved in import/export of packages and inlining.

`cmd/compile/internal/inline`

(function call inlining)`cmd/compile/internal/devirtualize`

(devirtualization of known interface method calls)`cmd/compile/internal/escape`

(escape analysis)

Several optimization passes are performed on the IR representation: dead code elimination, (early) devirtualization, function call inlining, and escape analysis.

The early dead code elimination pass is integrated into the unified IR writer phase.

`cmd/compile/internal/walk`

(order of evaluation, desugaring)

The final pass over the IR representation is "walk," which serves two purposes:

-
It decomposes complex statements into individual, simpler statements, introducing temporary variables and respecting order of evaluation. This step is also referred to as "order."

-
It desugars higher-level Go constructs into more primitive ones. For example,

`switch`

statements are turned into binary search or jump tables, and operations on maps and channels are replaced with runtime calls.

`cmd/compile/internal/ssa`

(SSA passes and rules)`cmd/compile/internal/ssagen`

(converting IR to SSA)

In this phase, IR is converted into Static Single Assignment (SSA) form, a lower-level intermediate representation with specific properties that make it easier to implement optimizations and to eventually generate machine code from it.

During this conversion, function intrinsics are applied. These are special functions that the compiler has been taught to replace with heavily optimized code on a case-by-case basis.

Certain nodes are also lowered into simpler components during the AST to SSA conversion, so that the rest of the compiler can work with them. For instance, the copy builtin is replaced by memory moves, and range loops are rewritten into for loops. Some of these currently happen before the conversion to SSA due to historical reasons, but the long-term plan is to move all of them here.

Then, a series of machine-independent passes and rules are applied. These do not
concern any single computer architecture, and thus run on all `GOARCH`

variants.
These passes include dead code elimination, removal of
unneeded nil checks, and removal of unused branches. The generic rewrite rules
mainly concern expressions, such as replacing some expressions with constant
values, and optimizing multiplications and float operations.

`cmd/compile/internal/ssa`

(SSA lowering and arch-specific passes)`cmd/internal/obj`

(machine code generation)

The machine-dependent phase of the compiler begins with the "lower" pass, which rewrites generic values into their machine-specific variants. For example, on amd64 memory operands are possible, so many load-store operations may be combined.

Note that the lower pass runs all machine-specific rewrite rules, and thus it currently applies lots of optimizations too.

Once the SSA has been "lowered" and is more specific to the target architecture, the final code optimization passes are run. This includes yet another dead code elimination pass, moving values closer to their uses, the removal of local variables that are never read from, and register allocation.

Other important pieces of work done as part of this step include stack frame layout, which assigns stack offsets to local variables, and pointer liveness analysis, which computes which on-stack pointers are live at each GC safe point.

At the end of the SSA generation phase, Go functions have been transformed into
a series of obj.Prog instructions. These are passed to the assembler
(`cmd/internal/obj`

), which turns them into machine code and writes out the
final object file. The object file will also contain reflect data, export data,
and debugging information.

In addition to writing a file of object code for the linker, the compiler also writes a file of "export data" for downstream compilation units. The export data file holds all the information computed during compilation of package P that may be needed when compiling a package Q that directly imports P. It includes type information for all exported declarations, IR for bodies of functions that are candidates for inlining, IR for bodies of generic functions that may be instantiated in another package, and a summary of the findings of escape analysis on function parameters.

The format of the export data file has gone through a number of iterations. Its current form is called "unified", and it is a serialized representation of an object graph, with an index allowing lazy decoding of parts of the whole (since most imports are used to provide only a handful of symbols).

The GOROOT repository contains a reader and a writer for the unified format; it encodes from/decodes to the compiler's IR. The golang.org/x/tools repository also provides a public API for an export data reader (using the go/types representation) that always supports the compiler's current file format and a small number of historic versions. (It is used by x/tools/go/packages in modes that require type information but not type-annotated syntax.)

The x/tools repository also provides public APIs for reading and writing exported type information (but nothing more) using the older "indexed" format. (For example, gopls uses this version for its database of workspace information, which includes types.)

Export data usually provides a "deep" summary, so that compilation of package Q can read the export data files only for each direct import, and be assured that these provide all necessary information about declarations in indirect imports, such as the methods and struct fields of types referred to in P's public API. Deep export data is simpler for build systems, since only one file is needed per direct dependency. However, it does have a tendency to grow as one gets higher up the import graph of a big repository: if there is a set of very commonly used types with a large API, nearly every package's export data will include a copy. This problem motivated the "indexed" design, which allowed partial loading on demand. (gopls does less work than the compiler for each import and is thus more sensitive to export data overheads. For this reason, it uses "shallow" export data, in which indirect information is not recorded at all. This demands random access to the export data files of all dependencies, so is not suitable for distributed build systems.)

-
If you have never contributed to the compiler before, a simple way to begin can be adding a log statement or

`panic("here")`

to get some initial insight into whatever you are investigating. -
The compiler itself provides logging, debugging and visualization capabilities, such as:

`$ go build -gcflags=-m=2 # print optimization info, including inlining, escape analysis $ go build -gcflags=-d=ssa/check_bce/debug # print bounds check info $ go build -gcflags=-W # print internal parse tree after type checking $ GOSSAFUNC=Foo go build # generate ssa.html file for func Foo $ go build -gcflags=-S # print assembly $ go tool compile -bench=out.txt x.go # print timing of compiler phases`

Some flags alter the compiler behavior, such as:

`$ go tool compile -h file.go # panic on first compile error encountered $ go build -gcflags=-d=checkptr=2 # enable additional unsafe pointer checking`

There are many additional flags. Some descriptions are available via:

`$ go tool compile -h # compiler flags, e.g., go build -gcflags='-m=1 -l' $ go tool compile -d help # debug flags, e.g., go build -gcflags=-d=checkptr=2 $ go tool compile -d ssa/help # ssa flags, e.g., go build -gcflags=-d=ssa/prove/debug=2`

There are some additional details about

`-gcflags`

and the differences between`go build`

vs.`go tool compile`

in a[section below](#-gcflags-and-go-build-vs-go-tool-compile). -
In general, when investigating a problem in the compiler you usually want to start with the simplest possible reproduction and understand exactly what is happening with it.


-
Be sure to read the

[Quickly testing your changes](https://go.dev/doc/contribute#quick_test)section of the Go Contribution Guide. -
Some tests live within the cmd/compile packages and can be run by

`go test ./...`

or similar, but many cmd/compile tests are in the top-level[test](https://github.com/golang/go/tree/master/test)directory:`$ go test cmd/internal/testdir # all tests in 'test' dir $ go test cmd/internal/testdir -run='Test/escape.*.go' # test specific files in 'test' dir`

For details, see the

[testdir README](https://github.com/golang/go/tree/master/test#readme). The`errorCheck`

method in[testdir_test.go](https://github.com/golang/go/blob/master/src/cmd/internal/testdir/testdir_test.go)is helpful for a description of the`ERROR`

comments used in many of those tests.In addition, the

`go/types`

package from the standard library and`cmd/compile/internal/types2`

have shared tests in`src/internal/types/testdata`

, and both type checkers should be checked if anything changes there. -
The new

[application-based coverage profiling](https://go.dev/testing/coverage/)can be used with the compiler, such as:`$ go install -cover -coverpkg=cmd/compile/... cmd/compile # build compiler with coverage instrumentation $ mkdir /tmp/coverdir # pick location for coverage data $ GOCOVERDIR=/tmp/coverdir go test [...] # use compiler, saving coverage data $ go tool covdata textfmt -i=/tmp/coverdir -o coverage.out # convert to traditional coverage format $ go tool cover -html coverage.out # view coverage via traditional tools`


-
Many of the compiler tests use the version of the

`go`

command found in your PATH and its corresponding`compile`

binary. -
If you are in a branch and your PATH includes

`<go-repo>/bin`

, doing`go install cmd/compile`

will build the compiler using the code from your branch and install it to the proper location so that subsequent`go`

commands like`go build`

or`go test ./...`

will exercise your freshly built compiler. -
[toolstash](https://pkg.go.dev/golang.org/x/tools/cmd/toolstash)provides a way to save, run, and restore a known good copy of the Go toolchain. For example, it can be a good practice to initially build your branch, save that version of the toolchain, then restore the known good version of the tools to compile your work-in-progress version of the compiler.Sample set up steps:

`$ go install golang.org/x/tools/cmd/toolstash@latest $ git clone https://go.googlesource.com/go $ cd go $ git checkout -b mybranch $ ./src/all.bash # build and confirm good starting point $ export PATH=$PWD/bin:$PATH $ toolstash save # save current tools`

After that, your edit/compile/test cycle can be similar to:

`<... make edits to cmd/compile source ...> $ toolstash restore && go install cmd/compile # restore known good tools to build compiler <... 'go build', 'go test', etc. ...> # use freshly built compiler`

-
toolstash also allows comparing the installed vs. stashed copy of the compiler, such as if you expect equivalent behavior after a refactor. For example, to check that your changed compiler produces identical object files to the stashed compiler while building the standard library:

`$ toolstash restore && go install cmd/compile # build latest compiler $ go build -toolexec "toolstash -cmp" -a -v std # compare latest vs. saved compiler`

-
If versions appear to get out of sync (for example, with errors like

`linked object header mismatch`

with version strings like`devel go1.21-db3f952b1f`

), you might need to do`toolstash restore && go install cmd/...`

to update all the tools under cmd.

-
[compilebench](https://pkg.go.dev/golang.org/x/tools/cmd/compilebench)benchmarks the speed of the compiler. -
[benchstat](https://pkg.go.dev/golang.org/x/perf/cmd/benchstat)is the standard tool for reporting performance changes resulting from compiler modifications, including whether any improvements are statistically significant:`$ go test -bench=SomeBenchmarks -count=20 > new.txt # use new compiler $ toolstash restore # restore old compiler $ go test -bench=SomeBenchmarks -count=20 > old.txt # use old compiler $ benchstat old.txt new.txt # compare old vs. new`

-
[bent](https://pkg.go.dev/golang.org/x/benchmarks/cmd/bent)facilitates running a large set of benchmarks from various community Go projects inside a Docker container. -
[perflock](https://github.com/aclements/perflock)helps obtain more consistent benchmark results, including by manipulating CPU frequency scaling settings on Linux. -
[view-annotated-file](https://github.com/loov/view-annotated-file)(from the community) overlays inlining, bounds check, and escape info back onto the source code. -
[godbolt.org](https://go.godbolt.org)is widely used to examine and share assembly output from many compilers, including the Go compiler. It can also[compare](https://go.godbolt.org/z/5Gs1G4bKG)assembly for different versions of a function or across Go compiler versions, which can be helpful for investigations and bug reports.

-
`-gcflags`

is a go command[build flag](https://pkg.go.dev/cmd/go#hdr-Compile_packages_and_dependencies).`go build -gcflags=<args>`

passes the supplied`<args>`

to the underlying`compile`

invocation(s) while still doing everything that the`go build`

command normally does (e.g., handling the build cache, modules, and so on). In contrast,`go tool compile <args>`

asks the`go`

command to invoke`compile <args>`

a single time without involving the standard`go build`

machinery. In some cases, it can be helpful to have fewer moving parts by doing`go tool compile <args>`

, such as if you have a small standalone source file that can be compiled without any assistance from`go build`

. In other cases, it is more convenient to pass`-gcflags`

to a build command like`go build`

,`go test`

, or`go install`

. -
`-gcflags`

by default applies to the packages named on the command line, but can use package patterns such as`-gcflags='all=-m=1 -l'`

, or multiple package patterns such as`-gcflags='all=-m=1' -gcflags='fmt=-m=2'`

. For details, see the[cmd/go documentation](https://pkg.go.dev/cmd/go#hdr-Compile_packages_and_dependencies).

To dig deeper into how the SSA package works, including its passes and rules,
head to [cmd/compile/internal/ssa/README.md](/golang/go/blob/master/src/cmd/compile/internal/ssa/README.md).

Finally, if something in this README or the SSA README is unclear
or if you have an idea for an improvement, feel free to leave a comment in
[issue 30074](https://go.dev/issue/30074).