命令行艺术
命令行
![curl -s 'https://raw.githubusercontent.com/jlevy/the-art-of-command-line/master/README.md' | egrep -o '`\w+`' | tr -d '`' | cowsay -W50](/data/attachment/album/201506/27/151622zjf2gg5cgpso2kcv.png)


流畅地使用命令行是一个常被忽略的技能，或被认为是神秘的奥义。但是，它会以明显而微妙的方式改善你作为工程师的灵活度和生产力。这是我在 Linux 上工作时发现的有用的命令行使用小窍门和笔记的精粹。有些小窍门是很基础的，而有些是相当地特别、复杂、或者晦涩难懂。这篇文章不长，但是如果你可以使用并记得这里的所有内容，那么你就懂得很多了。


其中大部分[最初](http://www.quora.com/What-are-some-lesser-known-but-useful-Unix-commands)[出现](http://www.quora.com/What-are-the-most-useful-Swiss-army-knife-one-liners-on-Unix)在[Quora](http://www.quora.com/What-are-some-time-saving-tips-that-every-Linux-user-should-know)上，但是考虑到兴趣所在，似乎更应该放到 Github 上，这里的人比我更能提出改进建议。如果你看到一个错误，或者更好的某种东西，请提交问题或 PR！（当然，提交前请看看必读小节和已有的 PR/Issue。）


必读
--


范围：


* 本文是针对初学者和专业人员的，选题目标是覆盖面广（全都很重要）、有针对性（大多数情况下都给出具体实例）而简洁（避免不必要内容以及你能在其它地方轻松找到的离题的内容）。每个小窍门在某种情形下都很必需的，或者能比替代品大大节省时间。
* 这是为 Linux 写的。绝大部分条目都可以同样应用到 MacOS（或者甚至 Cygwin）。
* 主要针对交互式 Bash，尽管大多数小窍门也可以应用到其它 shell，以及常规 Bash 脚本。
* 包括了“标准的”UNIX 命令以及那些需要安装的软件包（它们很重要，值得安装）。


注意：


* 为了能在一篇文章内展示尽量多的东西，一些具体的信息会被放到引用页里。你可以使用 Google 来获得进一步的内容。（如果需要的话，）你可以使用 `apt-get`/`yum`/`dnf`/`pacman`/`pip`/`brew` 来安装这些新的程序。
* 使用 [Explainshell](http://explainshell.com/) 来获取命令、参数、管道等内容的解释。


基础
--


* 学习基本 Bash 技能。实际上，键入`man bash`，然后至少浏览一遍所有内容；它很容易理解，没那么长。其它 shell 也不错，但是 Bash 很强大，而且到处都可以找到（如果在你自己的笔记本上*只*学习 zsh、fish 之类，会在很多情形下受到限制，比如使用现存的服务器时）。
* 至少学好一种基于文本的编辑器。理想的一个是 Vim（`vi`），因为在终端中编辑时随时都能找到它（即使大多数时候你在使用 Emacs、一个大型的 IDE、或一个现代的时髦编辑器）。
* 学习怎样使用 `man` 来阅读文档（好奇的话，用 `man man` 来列出分区号，比如 1 是常规命令，5 是文件描述，8 用于管理员）。用 `apropos` 找到帮助页。了解哪些命令不是可执行程序，而是 Bash 内置的，你可以用 `help` 和 `help -d` 得到帮助。
* 学习使用 `>` 和 `<` 来进行输出和输入重定向，以及使用 `|` 来管道重定向，学习关于 stdout 和 stderr 的东西。
* 学习 `*`（也许还有 `?` 和 `{`...`}` ）文件通配扩展和应用，以及双引号 `"` 和单引号 `'` 之间的区别。（更多内容请参看下面关于变量扩展部分）。
* 熟悉 Bash 作业管理：`&`， **ctrl-z**， **ctrl-c**， `jobs`， `fg`， `bg`， `kill` 等等。
* 掌握`ssh`，以及通过 `ssh-agent`，`ssh-add` 等进行无密码验证的基础技能。
* 基本的文件管理：`ls` 和 `ls -l`（特别是，知道`ls -l`各个列的意义），`less`， `head`， `tail` 和 `tail -f`（或者更好的`less +F`），`ln` 和 `ln -s`（知道硬链接和软链接的区别，以及硬链接相对于软链接的优势），`chown`，`chmod`，`du`（用于查看磁盘使用率的快速摘要：`du -sk *`）。文件系统管理：`df`， `mount`，`fdisk`，`mkfs`，`lsblk`。
* 基本的网络管理： `ip` 或 `ifconfig`，`dig`。
* 熟知正则表达式，以及各种使用`grep`/`egrep`的选项。`-i`，`-o`，`-A` 和 `-B` 选项值得掌握。
* 学会使用 `apt-get`，`yum` ，`dnf` 或 `pacman`（这取决于你的发行版）来查找并安装软件包。确保你可以用 `pip` 来安装基于 Python 的命令行工具（下面的一些东西可以很容易地通过 `pip` 安装）。


日常使用
----


* 在Bash中，使用 **tab** 补完参数，使用 **ctrl-r** 来搜索命令历史。
* 在Bash中，使用 **ctrl-w** 来删除最后的单词，使用 **ctrl-u** 来删除整行，返回行首。使用 **alt-b** 和 **alt-f** 来逐词移动，使用 **ctrl-k** 来清除到行尾的内容，以及使用 **ctrl-l** 清屏。参见 `man readline` 来查看 Bash 中所有默认的键盘绑定，有很多。例如，**alt-.** 可以循环显示先前的参数，而**alt-** 扩展通配。（LCTT 译注：关于 Bash 下的快捷键，可以参阅： </article-5660-1.html> ）
* 另外，如果你喜欢 vi 风格的键盘绑定，可以使用 `set -o vi`。
* 要查看最近用过的命令，请使用 `history` 。 有许多缩写形式，比如 `!$`（上次的参数）和`!!`（上次的命令），虽然使用 `ctrl-r` 和 `alt-.` 更容易些。（LCTT 译注：关于历史扩展功能，可以参阅： </article-5658-1.html> ）
* 返回先前的工作目录： `cd -`
* 如果你命令输入到一半，但是改变主意了，可以敲 **alt-#** 来添加一个 `#` 到开头，然后将该命令作为注释输入（或者使用快捷键 **ctrl-a**， **#**，**enter** 输入）。然后，你可以在后面通过命令历史来回到该命令。
* 使用 `xargs`（或 `parallel`），它很强大。注意，你可以控制每行（`-L`）执行多少个项目，以及并行执行（`-P`）。如果你不确定它是否会做正确的事情，可以首先使用 `xargs echo`。同时，使用 `-I{}` 也很方便。样例：



```
  find . -name '*.py' | xargs grep some_function
  cat hosts | xargs -I{} ssh root@{} hostname

```
* `pstree -p` 对于显示进程树很有帮助。
* 使用 `pgrep` 和 `pkill` 来按名称查找进程或给指定名称的进程发送信号（`-f` 很有帮助）。
* 掌握各种可以发送给进程的信号。例如，要挂起进程，可以使用 `kill -STOP [pid]`。完整的列表可以查阅 `man 7 signal`。
* 如果你想要一个后台进程一直保持运行，使用 `nohup` 或 `disown`。
* 通过 `netstat -lntp` 或 `ss -plat` 检查哪些进程在监听（用于 TCP，对 UDP 使用 `-u` 替代 `-t`）。
* `lsof`来查看打开的套接字和文件。
* 在 Bash 脚本中，使用 `set -x` 调试脚本输出。尽可能使用严格模式。使用 `set -e` 在遇到错误时退出。也可以使用 `set -o pipefail`，对错误进行严格处理（虽然该话题有点微妙）。对于更复杂的脚本，也可以使用 `trap`。
* 在 Bash 脚本中，子 shell（写在括号中的）是组合命令的便利的方式。一个常见的例子是临时移动到一个不同的工作目录，如：



```
  # 在当前目录做些事
  (cd /some/other/dir; other-command)
  # 继续回到原目录

```
* 注意，在 Bash 中有大量的各种各样的变量扩展。检查一个变量是否存在：`${name:?error message}`。例如，如果一个Bash脚本要求一个单一参数，只需写 `input_file=${1:?usage: $0 input_file}`。算术扩展：`i=$(( (i + 1) % 5 ))`。序列： `{1..10}`。修剪字符串：`${var%suffix}` 和 `${var#prefix}`。例如，if `var=foo.pdf` ，那么 `echo ${var%.pdf}.txt` 会输出 `foo.txt`。
* 命令的输出可以通过 `<(some command)` 作为一个文件来处理。例如，将本地的 `/etc/hosts` 和远程的比较：



```
  diff /etc/hosts <(ssh somehost cat /etc/hosts)

```
* 了解 Bash 中的“嵌入文档”，就像在 `cat <<EOF ...` 中。
* 在 Bash 中，通过：`some-command >logfile 2>&1` 同时重定向标准输出和标准错误。通常，要确保某个命令不再为标准输入打开文件句柄，而是将它捆绑到你所在的终端，添加 `</dev/null` 是个不错的做法。
* `man ascii` 可以得到一个不错的ASCII表，带有十六进制和十进制值两种格式。对于常规编码信息，`man unicode`，`man utf-8` 和 `man latin1` 将很有帮助。
* 使用 `screen` 或 `tmux` 来复用屏幕，这对于远程 ssh 会话尤为有用，使用它们来分离并重连到会话。另一个只用于保持会话的最小可选方案是 `dtach`。
* 在 ssh 中，知道如何使用 `-L` 或 `-D`（偶尔也用`-R`）来打开端口通道是很有用的，如从一台远程服务器访问网站时。
* 为你的 ssh 配置进行优化很有用；例如，这个 `~/.ssh/config` 包含了可以避免在特定网络环境中连接被断掉的情况的设置、使用压缩（这对于通过低带宽连接使用 scp 很有用），以及使用一个本地控制文件来开启到同一台服务器的多通道：



```
  TCPKeepAlive=yes
  ServerAliveInterval=15
  ServerAliveCountMax=6
  Compression=yes
  ControlMaster auto
  ControlPath /tmp/%r@%h:%p
  ControlPersist yes

```
* 其它一些与 ssh 相关的选项对会影响到安全，请小心开启，如各个子网或主机，或者在信任的网络中：`StrictHostKeyChecking=no`， `ForwardAgent=yes`
* 要获得八进制格式的文件的权限，这对于系统配置很有用而用 `ls` 又没法查看，而且也很容易搞得一团糟，可以使用像这样的东西：



```
  stat -c '%A %a %n' /etc/timezone

```
* 对于从另一个命令的输出结果中交互选择值，可以使用[`percol`](https://github.com/mooz/percol)。
* 对于基于另一个命令（如`git`）输出的文件交互，可以使用`fpp` ([路径选择器](https://github.com/facebook/PathPicker))。
* 要为当前目录（及子目录）中的所有文件构建一个简单的 Web 服务器，让网络中的任何人都可以获取，可以使用： `python -m SimpleHTTPServer 7777` （使用端口 7777 和 Python 2）。


处理文件和数据
-------


* 要在当前目录中按名称定位文件，`find . -iname '*something*'`（或者相类似的）。要按名称查找任何地方的文件，使用 `locate something`（但请记住，`updatedb` 可能还没有索引最近创建的文件）。
* 对于源代码或数据文件进行的常规搜索（要比 `grep -r` 更高级），使用 [`ag`](https://github.com/ggreer/the_silver_searcher)。
* 要将 HTML 转成文本：`lynx -dump -stdin`。
* 对于 Markdown、HTML，以及各种类型的文档转换，可以试试 [`pandoc`](http://pandoc.org/)。
* 如果你必须处理 XML，`xmlstarlet` 虽然有点老旧，但是很好用。
* 对于 JSON，使用`jq`。
* 对于 Excel 或 CSV 文件，[csvkit](https://github.com/onyxfish/csvkit) 提供了 `in2csv`，`csvcut`，`csvjoin`，`csvgrep` 等工具。
* 对于亚马逊 S3 ，[`s3cmd`](https://github.com/s3tools/s3cmd) 会很方便，而 [`s4cmd`](https://github.com/bloomreach/s4cmd) 则更快速。亚马逊的 [`aws`](https://github.com/aws/aws-cli) 则是其它 AWS 相关任务的必备。
* 掌握 `sort` 和 `uniq`，包括 uniq 的 `-u` 和 `-d` 选项——参见下面的单行程序。
* 掌握 `cut`，`paste` 和 `join`，它们用于处理文本文件。很多人会使用 `cut`，但常常忘了 `join`。
* 了解 `tee`，它会将 stdin 同时复制到一个文件和 stdout，如 `ls -al | tee file.txt`。
* 知道 locale 会以微妙的方式对命令行工具产生大量的影响，包括排序的顺序（整理）以及性能。大多数安装好的 Linux 会设置 `LANG` 或其它 locale 环境变量为本地设置，比如像 US English。但是，你要明白，如果改变了本地环境，那么排序也将改变。而且 i18n 过程会让排序或其它命令的运行慢*好多倍*。在某些情形中（如像下面那样的设置操作或唯一性操作），你可以安全地整个忽略缓慢的 i18n 过程，然后使用传统的基于字节的排序顺序 `export LC_ALL=C`。
* 了解基本的改动数据的 `awk` 和 `sed` 技能。例如，计算某个文本文件第三列所有数字的和：`awk '{ x += $3 } END { print x }'`。这可能比 Python 的同等操作要快3倍，而且要短3倍。
* 在一个或多个文件中，替换所有出现在特定地方的某个字符串：



```
  perl -pi.bak -e 's/old-string/new-string/g' my-files-*.txt

```
* 要立即根据某个模式对大量文件重命名，使用 `rename`。对于复杂的重命名，[`repren`](https://github.com/jlevy/repren) 可以帮助你达成。



```
  # 恢复备份文件 foo.bak -> foo:
  rename 's/\.bak$//' *.bak
  # 完整的文件名、目录名 foo -> bar:
  repren --full --preserve-case --from foo --to bar .

```
* 使用 `shuf` 来从某个文件中打乱或随机选择行。
* 了解 `sort` 的选项。知道这些键是怎么工作的（`-t`和`-k`）。特别是，注意你需要写`-k1,1`来只通过第一个字段排序；`-k1`意味着根据整行排序。
* 稳定排序（`sort -s`）会很有用。例如，要首先按字段2排序，然后再按字段1排序，你可以使用 `sort -k1,1 | sort -s -k2,2`
* 如果你曾经需要在 Bash 命令行中写一个水平制表符（如，用于 -t 参数的排序），按**ctrl-v** **[Tab]**，或者写`$'\t'`（后面的更好，因为你可以复制/粘贴）。
* 对源代码进行补丁的标准工具是 `diff` 和 `patch`。 用 `diffstat` 来统计 diff 情况。注意 `diff -r` 可以用于整个目录，所以可以用 `diff -r tree1 tree2 | diffstat` 来统计（两个目录的）差异。
* 对于二进制文件，使用 `hd` 进行简单十六进制转储，以及 `bvi` 用于二进制编辑。
* 还是用于二进制文件，`strings`（加上 `grep` 等）可以让你找出一点文本。
* 对于二进制文件的差异（delta 压缩），可以使用 `xdelta3`。
* 要转换文本编码，试试 `iconv` 吧，或者对于更高级的用途使用 `uconv`；它支持一些高级的 Unicode 的东西。例如，这个命令可以转换为小写并移除所有重音符号（通过扩展和丢弃）：



```
  uconv -f utf-8 -t utf-8 -x '::Any-Lower; ::Any-NFD; [:Nonspacing Mark:] >; ::Any-NFC; ' < input.txt > output.txt

```
* 要将文件分割成几个部分，来看看 `split`（按大小分割）和 `csplit`（按格式分割）吧。
* 使用 `zless`，`zmore`，`zcat` 和 `zgrep` 来操作压缩文件。


系统调试
----


* 对于 Web 调试，`curl` 和 `curl -I` 很方便灵活，或者也可以使用它们的同行 `wget`，或者更现代的 [`httpie`](https://github.com/jakubroztocil/httpie)。
* 要了解磁盘、CPU、网络的状态，使用 `iostat`，`netstat`，`top`（或更好的 `htop`）和（特别是）`dstat`。它们对于快速获知系统中发生的状况很好用。
* 对于更深层次的系统总览，可以使用 [`glances`](https://github.com/nicolargo/glances)。它会在一个终端窗口中为你呈现几个系统层次的统计数据，对于快速检查各个子系统很有帮助。
* 要了解内存状态，可以运行 `free` 和 `vmstat`，看懂它们的输出结果吧。特别是，要知道“cached”值是Linux内核为文件缓存所占有的内存，因此，要有效地统计“free”值。
* Java 系统调试是一件截然不同的事，但是对于 Oracle 系统以及其它一些 JVM 而言，不过是一个简单的小把戏，你可以运行 `kill -3 <pid>`，然后一个完整的堆栈追踪和内存堆的摘要（包括常规的垃圾收集细节，这很有用）将被转储到stderr/logs。
* 使用 `mtr` 作路由追踪更好，可以识别网络问题。
* 对于查看磁盘满载的原因，`ncdu` 会比常规命令如 `du -sh *` 更节省时间。
* 要查找占用带宽的套接字和进程，试试 `iftop` 或 `nethogs` 吧。
* （Apache附带的）`ab`工具对于临时应急检查网络服务器性能很有帮助。对于更复杂的负载测试，可以试试 `siege`。
* 对于更仔细的网络调试，可以用 `wireshark`，`tshark` 或 `ngrep`。
* 掌握 `strace` 和 `ltrace`。如果某个程序失败、挂起或崩溃，而你又不知道原因，或者如果你想要获得性能的大概信息，这些工具会很有帮助。注意，分析选项（`-c`）和使用 `-p` 关联运行进程。
* 掌握 `ldd` 来查看共享库等。
* 知道如何使用 `gdb` 来连接到一个运行着的进程并获取其堆栈追踪信息。
* 使用 `/proc`。当调试当前的问题时，它有时候出奇地有帮助。样例：`/proc/cpuinfo`，`/proc/xxx/cwd`，`/proc/xxx/exe`，`/proc/xxx/fd/`，`/proc/xxx/smaps`。
* 当调试过去某个东西为何出错时，`sar` 会非常有帮助。它显示了 CPU、内存、网络等的历史统计数据。
* 对于更深层的系统和性能分析，看看 `stap` ([SystemTap](https://sourceware.org/systemtap/wiki))，[`perf`](http://en.wikipedia.org/wiki/Perf_(Linux)) 和 [`sysdig`](https://github.com/draios/sysdig) 吧。
* 确认是正在使用的 Linux 发行版版本（支持大多数发行版）：`lsb_release -a`。
* 每当某个东西的行为异常时（可能是硬件或者驱动器问题），使用`dmesg`。


单行程序
----


这是将命令连成一行的一些样例：


* 有时候通过 `sort`/`uniq` 对文本文件做交集、并集和差集运算时，这个例子会相当有帮助。假定 `a` 和 `b` 是已经进行了唯一性处理的文本文件。这会很快，而且可以处理任意大小的文件，总计可达数千兆字节。（Sort不受内存限制，不过如果 `/tmp` 放在一个很小的根分区的话，你可能需要使用 `-T` 选项。）也可参见上面关于`LC_ALL`的注解和 `-u` 选项（参见下面例子更清晰）。  


```
sh cat a b | sort | uniq > c # c 是 a 和 b 的并集 
cat a b | sort | uniq -d > c # c 是 a 和 b 的交集 
cat a b b | sort | uniq -u > c # c 是 a 减去 b 的差集
```
* 使用 `grep . *` 来可视化查看一个目录中的所有文件的所有内容，例如，对于放满配置文件的目录： `/sys`， `/proc`， `/etc`。
* 对某个文本文件的第三列中所有数据进行求和（该例子可能比同等功能的Python要快3倍，而且代码也少于其3倍）：



```
  awk '{ x += $3 } END { print x }' myfile

```
* 如果想要查看某个文件树的大小/日期，该例子就像一个递归`ls -l`，但是比`ls -lR`要更容易读懂：



```
  find . -type f -ls

```
* 只要可以，请使用 `xargs` 或 `parallel`。注意，你可以控制每行（`-L`）执行多少个项目，以及并行执行（`-P`）。如果你不确定它是否会做正确的事情，可以首先使用 `xargs echo`。同时，使用 `-I{}` 也很方便。样例：



```
  find . -name '*.py' | xargs grep some_function
  cat hosts | xargs -I{} ssh root@{} hostname

```
* 比如说，你有一个文本文件，如 Web 服务器的日志，在某些行中出现了某个特定的值，如 URL 中出现的 `acct_id` 参数。如果你想要统计有多少个 `acct_id` 的请求：



```
cat access.log | egrep -o 'acct_id=[0-9]+' | cut -d= -f2 | sort | uniq -c | sort -rn

```
* 运行该函数来获得来自本文的随机提示（解析Markdown并从中提取某个项目）：



```
function taocl() {
    curl -s https://raw.githubusercontent.com/jlevy/the-art-of-command-line/master/README.md |
      pandoc -f markdown -t html |
      xmlstarlet fo --html --dropdtd |
      xmlstarlet sel -t -v "(html/body/ul/li[count(p)>0])[$RANDOM mod last()+1]" |
      xmlstarlet unesc | fmt -80
  }

```


晦涩难懂，但却有用
---------


* `expr`：实施算术或布林操作，或者求正则表达式的值
* `m4`：简单的宏处理器
* `yes`：大量打印一个字符串
* `cal`：漂亮的日历
* `env`：（以特定的环境变量设置）运行一个命令（脚本中很有用）
* `look`：查找以某个字符串开头的英文单词（或文件中的行）
* `cut` 和 `paste` 以及 `join`：数据处理
* `fmt`：格式化文本段落
* `pr`：格式化文本为页/列
* `fold`：文本折行
* `column`：格式化文本为列或表
* `expand` 和 `unexpand`：在制表符和空格间转换
* `nl`：添加行号
* `seq`：打印数字
* `bc`：计算器
* `factor`：分解质因子
* `gpg`：加密并为文件签名
* `toe`：terminfo 条目表
* `nc`：网络调试和数据传输
* `socat`：套接字中继和 tcp 端口转发（类似 `netcat`）
* `slurm`：网络流量可视化
* `dd`：在文件或设备间移动数据
* `file`：识别文件类型
* `tree`：以树形显示目录及子目录；类似 `ls`，但是是递归的。
* `stat`：文件信息
* `tac`：逆序打印文件
* `shuf`：从文件中随机选择行
* `comm`：逐行对比分类排序的文件
* `hd`和`bvi`：转储或编辑二进制文件
* `strings`：从二进制文件提取文本
* `tr`：字符转译或处理
* `iconv`或`uconv`：文本编码转换
* `split`和`csplit`：分割文件
* `units`：单位转换和计算；将每双周（fortnigh）一浪（浪，furlong，长度单位，约201米）转换为每瞬（blink）一缇（缇，twip，一种和屏幕无关的长度单位）（参见： /usr/share/units/definitions.units）（LCTT 译注：这都是神马单位啊！）
* `7z`：高比率文件压缩
* `ldd`：动态库信息
* `nm`：目标文件的符号
* `ab`：Web 服务器基准测试
* `strace`：系统调用调试
* `mtr`：用于网络调试的更好的路由追踪软件
* `cssh`：可视化并发 shell
* `rsync`：通过 SSH 同步文件和文件夹
* `wireshark` 和 `tshark`：抓包和网络调试
* `ngrep`：从网络层摘取信息
* `host` 和 `dig`：DNS查询
* `lsof`：处理文件描述符和套接字信息
* `dstat`：有用的系统统计数据
* [`glances`](https://github.com/nicolargo/glances)：高级，多个子系统概览
* `iostat`：CPU和磁盘使用率统计
* `htop`：top的改进版
* `last`：登录历史
* `w`：谁登录进来了
* `id`：用户/组身份信息
* `sar`：历史系统统计数据
* `iftop`或`nethogs`：按套接口或进程的网络使用率
* `ss`：套接口统计数据
* `dmesg`：启动和系统错误信息
* `hdparm`：SATA/ATA 磁盘操作/改善性能
* `lsb_release`：Linux 发行版信息
* `lsblk`：列出块设备，以树形展示你的磁盘和分区
* `lshw`：硬件信息
* `fortune`，`ddate` 和 `sl`：嗯，好吧，它取决于你是否认为蒸汽机车和 Zippy 引用“有用”


更多资源
----


* [超棒的shell](https://github.com/alebcay/awesome-shell): 一个shell工具和资源一览表。
* [严格模式](http://redsymbol.net/articles/unofficial-bash-strict-mode/) 用于写出更佳的shell脚本。


免责声明
----


除了非常小的任务外，其它都写出了代码供大家阅读。伴随力量而来的是责任。事实是，你*能*在Bash中做的，并不意味着是你所应该做的！；）




---


via: <https://github.com/jlevy/the-art-of-command-line>


作者：[jlevy](https://github.com/jlevy) 译者：[GOLinux](https://github.com/GOLinux) 校对：[wxy](https://github.com/wxy)


本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创翻译，[Linux中国](https://linux.cn/) 荣誉推出


OK
🌍
[Čeština](/jlevy/the-art-of-command-line/blob/master/README-cs.md) ∙ [Deutsch](/jlevy/the-art-of-command-line/blob/master/README-de.md) ∙ [Ελληνικά](/jlevy/the-art-of-command-line/blob/master/README-el.md) ∙ [English](/jlevy/the-art-of-command-line/blob/master/README.md) ∙ [Español](/jlevy/the-art-of-command-line/blob/master/README-es.md) ∙ [Français](/jlevy/the-art-of-command-line/blob/master/README-fr.md) ∙ [Indonesia](/jlevy/the-art-of-command-line/blob/master/README-id.md) ∙ [Italiano](/jlevy/the-art-of-command-line/blob/master/README-it.md) ∙ [日本語](/jlevy/the-art-of-command-line/blob/master/README-ja.md) ∙ [한국어](/jlevy/the-art-of-command-line/blob/master/README-ko.md) ∙ [polski](/jlevy/the-art-of-command-line/blob/master/README-pl.md) ∙ [Português](/jlevy/the-art-of-command-line/blob/master/README-pt.md) ∙ [Română](/jlevy/the-art-of-command-line/blob/master/README-ro.md) ∙ [Русский](/jlevy/the-art-of-command-line/blob/master/README-ru.md) ∙ [Slovenščina](/jlevy/the-art-of-command-line/blob/master/README-sl.md) ∙ [Українська](/jlevy/the-art-of-command-line/blob/master/README-uk.md) ∙ [简体中文](/jlevy/the-art-of-command-line/blob/master/README-zh.md) ∙ [繁體中文](/jlevy/the-art-of-command-line/blob/master/README-zh-Hant.md)

*Note: I'm planning to revise this and looking for a new co-author to help with expanding this into a more comprehensive guide. While it's very popular, it could be broader and a bit deeper. If you like to write and are close to being an expert on this material and willing to consider helping, please drop me a note at josh (0x40) holloway.com. – jlevy, Holloway. Thank you!*

[Meta](#meta)[Basics](#basics)[Everyday use](#everyday-use)[Processing files and data](#processing-files-and-data)[System debugging](#system-debugging)[One-liners](#one-liners)[Obscure but useful](#obscure-but-useful)[macOS only](#macos-only)[Windows only](#windows-only)[More resources](#more-resources)[Disclaimer](#disclaimer)

Fluency on the command line is a skill often neglected or considered arcane, but it improves your flexibility and productivity as an engineer in both obvious and subtle ways. This is a selection of notes and tips on using the command-line that we've found useful when working on Linux. Some tips are elementary, and some are fairly specific, sophisticated, or obscure. This page is not long, but if you can use and recall all the items here, you know a lot.

This work is the result of [many authors and translators](/jlevy/the-art-of-command-line/blob/master/AUTHORS.md).
Some of this
[originally](http://www.quora.com/What-are-some-lesser-known-but-useful-Unix-commands)
[appeared](http://www.quora.com/What-are-the-most-useful-Swiss-army-knife-one-liners-on-Unix)
on [Quora](http://www.quora.com/What-are-some-time-saving-tips-that-every-Linux-user-should-know),
but it has since moved to GitHub, where people more talented than the original author have made numerous improvements.
[ Please submit a question](https://airtable.com/shrzMhx00YiIVAWJg) if you have a question related to the command line.

[if you see an error or something that could be better!](/jlevy/the-art-of-command-line/blob/master/CONTRIBUTING.md)

**Please contribute**Scope:

- This guide is for both beginners and experienced users. The goals are
*breadth*(everything important),*specificity*(give concrete examples of the most common case), and*brevity*(avoid things that aren't essential or digressions you can easily look up elsewhere). Every tip is essential in some situation or significantly saves time over alternatives. - This is written for Linux, with the exception of the "
[macOS only](#macos-only)" and "[Windows only](#windows-only)" sections. Many of the other items apply or can be installed on other Unices or macOS (or even Cygwin). - The focus is on interactive Bash, though many tips apply to other shells and to general Bash scripting.
- It includes both "standard" Unix commands as well as ones that require special package installs -- so long as they are important enough to merit inclusion.

Notes:

- To keep this to one page, content is implicitly included by reference. You're smart enough to look up more detail elsewhere once you know the idea or command to Google. Use
`apt`

,`yum`

,`dnf`

,`pacman`

,`pip`

or`brew`

(as appropriate) to install new programs. - Use
[Explainshell](http://explainshell.com/)to get a helpful breakdown of what commands, options, pipes etc. do.

-
Learn basic Bash. Actually, type

`man bash`

and at least skim the whole thing; it's pretty easy to follow and not that long. Alternate shells can be nice, but Bash is powerful and always available (learning*only*zsh, fish, etc., while tempting on your own laptop, restricts you in many situations, such as using existing servers). -
Learn at least one text-based editor well. The

`nano`

editor is one of the simplest for basic editing (opening, editing, saving, searching). However, for the power user in a text terminal, there is no substitute for Vim (`vi`

), the hard-to-learn but venerable, fast, and full-featured editor. Many people also use the classic Emacs, particularly for larger editing tasks. (Of course, any modern software developer working on an extensive project is unlikely to use only a pure text-based editor and should also be familiar with modern graphical IDEs and tools.) -
Finding documentation:

- Know how to read official documentation with
`man`

(for the inquisitive,`man man`

lists the section numbers, e.g. 1 is "regular" commands, 5 is files/conventions, and 8 are for administration). Find man pages with`apropos`

. - Know that some commands are not executables, but Bash builtins, and that you can get help on them with
`help`

and`help -d`

. You can find out whether a command is an executable, shell builtin or an alias by using`type command`

. `curl cheat.sh/command`

will give a brief "cheat sheet" with common examples of how to use a shell command.

- Know how to read official documentation with
-
Learn about redirection of output and input using

`>`

and`<`

and pipes using`|`

. Know`>`

overwrites the output file and`>>`

appends. Learn about stdout and stderr. -
Learn about file glob expansion with

`*`

(and perhaps`?`

and`[`

...`]`

) and quoting and the difference between double`"`

and single`'`

quotes. (See more on variable expansion below.) -
Be familiar with Bash job management:

`&`

,**ctrl-z**,**ctrl-c**,`jobs`

,`fg`

,`bg`

,`kill`

, etc. -
Know

`ssh`

, and the basics of passwordless authentication, via`ssh-agent`

,`ssh-add`

, etc. -
Basic file management:

`ls`

and`ls -l`

(in particular, learn what every column in`ls -l`

means),`less`

,`head`

,`tail`

and`tail -f`

(or even better,`less +F`

),`ln`

and`ln -s`

(learn the differences and advantages of hard versus soft links),`chown`

,`chmod`

,`du`

(for a quick summary of disk usage:`du -hs *`

). For filesystem management,`df`

,`mount`

,`fdisk`

,`mkfs`

,`lsblk`

. Learn what an inode is (`ls -i`

or`df -i`

). -
Basic network management:

`ip`

or`ifconfig`

,`dig`

,`traceroute`

,`route`

. -
Learn and use a version control management system, such as

`git`

. -
Know regular expressions well, and the various flags to

`grep`

/`egrep`

. The`-i`

,`-o`

,`-v`

,`-A`

,`-B`

, and`-C`

options are worth knowing. -
Learn to use

`apt-get`

,`yum`

,`dnf`

or`pacman`

(depending on distro) to find and install packages. And make sure you have`pip`

to install Python-based command-line tools (a few below are easiest to install via`pip`

).

-
In Bash, use

**Tab**to complete arguments or list all available commands and**ctrl-r**to search through command history (after pressing, type to search, press**ctrl-r**repeatedly to cycle through more matches, press**Enter**to execute the found command, or hit the right arrow to put the result in the current line to allow editing). -
In Bash, use

**ctrl-w**to delete the last word, and**ctrl-u**to delete the content from current cursor back to the start of the line. Use**alt-b**and**alt-f**to move by word,**ctrl-a**to move cursor to beginning of line,**ctrl-e**to move cursor to end of line,**ctrl-k**to kill to the end of the line,**ctrl-l**to clear the screen. See`man readline`

for all the default keybindings in Bash. There are a lot. For example**alt-.**cycles through previous arguments, and**alt-*** expands a glob. -
Alternatively, if you love vi-style key-bindings, use

`set -o vi`

(and`set -o emacs`

to put it back). -
For editing long commands, after setting your editor (for example

`export EDITOR=vim`

),**ctrl-x****ctrl-e**will open the current command in an editor for multi-line editing. Or in vi style,**escape-v**. -
To see recent commands, use

`history`

. Follow with`!n`

(where`n`

is the command number) to execute again. There are also many abbreviations you can use, the most useful probably being`!$`

for last argument and`!!`

for last command (see "HISTORY EXPANSION" in the man page). However, these are often easily replaced with**ctrl-r**and**alt-.**. -
Go to your home directory with

`cd`

. Access files relative to your home directory with the`~`

prefix (e.g.`~/.bashrc`

). In`sh`

scripts refer to the home directory as`$HOME`

. -
To go back to the previous working directory:

`cd -`

. -
If you are halfway through typing a command but change your mind, hit

**alt-#**to add a`#`

at the beginning and enter it as a comment (or use**ctrl-a**,**#**,**enter**). You can then return to it later via command history. -
Use

`xargs`

(or`parallel`

). It's very powerful. Note you can control how many items execute per line (`-L`

) as well as parallelism (`-P`

). If you're not sure if it'll do the right thing, use`xargs echo`

first. Also,`-I{}`

is handy. Examples:

```
find . -name '*.py' | xargs grep some_function
cat hosts | xargs -I{} ssh root@{} hostname
```

-
`pstree -p`

is a helpful display of the process tree. -
Use

`pgrep`

and`pkill`

to find or signal processes by name (`-f`

is helpful). -
Know the various signals you can send processes. For example, to suspend a process, use

`kill -STOP [pid]`

. For the full list, see`man 7 signal`

-
Use

`nohup`

or`disown`

if you want a background process to keep running forever. -
Check what processes are listening via

`netstat -lntp`

or`ss -plat`

(for TCP; add`-u`

for UDP) or`lsof -iTCP -sTCP:LISTEN -P -n`

(which also works on macOS). -
See also

`lsof`

and`fuser`

for open sockets and files. -
See

`uptime`

or`w`

to know how long the system has been running. -
Use

`alias`

to create shortcuts for commonly used commands. For example,`alias ll='ls -latr'`

creates a new alias`ll`

. -
Save aliases, shell settings, and functions you commonly use in

`~/.bashrc`

, and[arrange for login shells to source it](http://superuser.com/a/183980/7106). This will make your setup available in all your shell sessions. -
Put the settings of environment variables as well as commands that should be executed when you login in

`~/.bash_profile`

. Separate configuration will be needed for shells you launch from graphical environment logins and`cron`

jobs. -
Synchronize your configuration files (e.g.

`.bashrc`

and`.bash_profile`

) among various computers with Git. -
Understand that care is needed when variables and filenames include whitespace. Surround your Bash variables with quotes, e.g.

`"$FOO"`

. Prefer the`-0`

or`-print0`

options to enable null characters to delimit filenames, e.g.`locate -0 pattern | xargs -0 ls -al`

or`find / -print0 -type d | xargs -0 ls -al`

. To iterate on filenames containing whitespace in a for loop, set your IFS to be a newline only using`IFS=$'\n'`

. -
In Bash scripts, use

`set -x`

(or the variant`set -v`

, which logs raw input, including unexpanded variables and comments) for debugging output. Use strict modes unless you have a good reason not to: Use`set -e`

to abort on errors (nonzero exit code). Use`set -u`

to detect unset variable usages. Consider`set -o pipefail`

too, to abort on errors within pipes (though read up on it more if you do, as this topic is a bit subtle). For more involved scripts, also use`trap`

on EXIT or ERR. A useful habit is to start a script like this, which will make it detect and abort on common errors and print a message:

```
set -euo pipefail
trap "echo 'error: Script failed: see failed command above'" ERR
```

- In Bash scripts, subshells (written with parentheses) are convenient ways to group commands. A common example is to temporarily move to a different working directory, e.g.

```
# do something in current dir
(cd /some/other/dir && other-command)
# continue in original dir
```

-
In Bash, note there are lots of kinds of variable expansion. Checking a variable exists:

`${name:?error message}`

. For example, if a Bash script requires a single argument, just write`input_file=${1:?usage: $0 input_file}`

. Using a default value if a variable is empty:`${name:-default}`

. If you want to have an additional (optional) parameter added to the previous example, you can use something like`output_file=${2:-logfile}`

. If`$2`

is omitted and thus empty,`output_file`

will be set to`logfile`

. Arithmetic expansion:`i=$(( (i + 1) % 5 ))`

. Sequences:`{1..10}`

. Trimming of strings:`${var%suffix}`

and`${var#prefix}`

. For example if`var=foo.pdf`

, then`echo ${var%.pdf}.txt`

prints`foo.txt`

. -
Brace expansion using

`{`

...`}`

can reduce having to re-type similar text and automate combinations of items. This is helpful in examples like`mv foo.{txt,pdf} some-dir`

(which moves both files),`cp somefile{,.bak}`

(which expands to`cp somefile somefile.bak`

) or`mkdir -p test-{a,b,c}/subtest-{1,2,3}`

(which expands all possible combinations and creates a directory tree). Brace expansion is performed before any other expansion. -
The order of expansions is: brace expansion; tilde expansion, parameter and variable expansion, arithmetic expansion, and command substitution (done in a left-to-right fashion); word splitting; and filename expansion. (For example, a range like

`{1..20}`

cannot be expressed with variables using`{$a..$b}`

. Use`seq`

or a`for`

loop instead, e.g.,`seq $a $b`

or`for((i=a; i<=b; i++)); do ... ; done`

.) -
The output of a command can be treated like a file via

`<(some command)`

(known as process substitution). For example, compare local`/etc/hosts`

with a remote one:

` diff /etc/hosts <(ssh somehost cat /etc/hosts)`

- When writing scripts you may want to put all of your code in curly braces. If the closing brace is missing, your script will be prevented from executing due to a syntax error. This makes sense when your script is going to be downloaded from the web, since it prevents partially downloaded scripts from executing:

```
{
# Your code here
}
```

- A "here document" allows
[redirection of multiple lines of input](https://www.tldp.org/LDP/abs/html/here-docs.html)as if from a file:

```
cat <<EOF
input
on multiple lines
EOF
```


-
In Bash, redirect both standard output and standard error via:

`some-command >logfile 2>&1`

or`some-command &>logfile`

. Often, to ensure a command does not leave an open file handle to standard input, tying it to the terminal you are in, it is also good practice to add`</dev/null`

. -
Use

`man ascii`

for a good ASCII table, with hex and decimal values. For general encoding info,`man unicode`

,`man utf-8`

, and`man latin1`

are helpful. -
Use

`screen`

orto multiplex the screen, especially useful on remote ssh sessions and to detach and re-attach to a session.`tmux`

`byobu`

can enhance screen or tmux by providing more information and easier management. A more minimal alternative for session persistence only is.`dtach`

-
In ssh, knowing how to port tunnel with

`-L`

or`-D`

(and occasionally`-R`

) is useful, e.g. to access web sites from a remote server. -
It can be useful to make a few optimizations to your ssh configuration; for example, this

`~/.ssh/config`

contains settings to avoid dropped connections in certain network environments, uses compression (which is helpful with scp over low-bandwidth connections), and multiplex channels to the same server with a local control file:

```
TCPKeepAlive=yes
ServerAliveInterval=15
ServerAliveCountMax=6
Compression=yes
ControlMaster auto
ControlPath /tmp/%r@%h:%p
ControlPersist yes
```


-
A few other options relevant to ssh are security sensitive and should be enabled with care, e.g. per subnet or host or in trusted networks:

`StrictHostKeyChecking=no`

,`ForwardAgent=yes`

-
Consider

an alternative to ssh that uses UDP, avoiding dropped connections and adding convenience on the road (requires server-side setup).`mosh`

-
To get the permissions on a file in octal form, which is useful for system configuration but not available in

`ls`

and easy to bungle, use something like

` stat -c '%A %a %n' /etc/timezone`

-
For interactive selection of values from the output of another command, use

or`percol`

.`fzf`

-
For interaction with files based on the output of another command (like

`git`

), use`fpp`

([PathPicker](https://github.com/facebook/PathPicker)). -
For a simple web server for all files in the current directory (and subdirs), available to anyone on your network, use:

`python -m SimpleHTTPServer 7777`

(for port 7777 and Python 2) and`python -m http.server 7777`

(for port 7777 and Python 3). -
For running a command as another user, use

`sudo`

. Defaults to running as root; use`-u`

to specify another user. Use`-i`

to login as that user (you will be asked for*your*password). -
For switching the shell to another user, use

`su username`

or`su - username`

. The latter with "-" gets an environment as if another user just logged in. Omitting the username defaults to root. You will be asked for the password*of the user you are switching to*. -
Know about the

[128K limit](https://wiki.debian.org/CommonErrorMessages/ArgumentListTooLong)on command lines. This "Argument list too long" error is common when wildcard matching large numbers of files. (When this happens alternatives like`find`

and`xargs`

may help.) -
For a basic calculator (and of course access to Python in general), use the

`python`

interpreter. For example,

```
>>> 2+3
5
```


-
To locate a file by name in the current directory,

`find . -iname '*something*'`

(or similar). To find a file anywhere by name, use`locate something`

(but bear in mind`updatedb`

may not have indexed recently created files). -
For general searching through source or data files, there are several options more advanced or faster than

`grep -r`

, including (in rough order from older to newer),`ack`

("the silver searcher"), and`ag`

(ripgrep).`rg`

-
To convert HTML to text:

`lynx -dump -stdin`

-
For Markdown, HTML, and all kinds of document conversion, try

. For example, to convert a Markdown document to Word format:`pandoc`

`pandoc README.md --from markdown --to docx -o temp.docx`

-
If you must handle XML,

`xmlstarlet`

is old but good. -
For JSON, use

. For interactive use, also see`jq`

and`jid`

.`jiq`

-
For YAML, use

.`shyaml`

-
For Excel or CSV files,

[csvkit](https://github.com/onyxfish/csvkit)provides`in2csv`

,`csvcut`

,`csvjoin`

,`csvgrep`

, etc. -
For Amazon S3,

is convenient and`s3cmd`

is faster. Amazon's`s4cmd`

and the improved`aws`

are essential for other AWS-related tasks.`saws`

-
Know about

`sort`

and`uniq`

, including uniq's`-u`

and`-d`

options -- see one-liners below. See also`comm`

. -
Know about

`cut`

,`paste`

, and`join`

to manipulate text files. Many people use`cut`

but forget about`join`

. -
Know about

`wc`

to count newlines (`-l`

), characters (`-m`

), words (`-w`

) and bytes (`-c`

). -
Know about

`tee`

to copy from stdin to a file and also to stdout, as in`ls -al | tee file.txt`

. -
For more complex calculations, including grouping, reversing fields, and statistical calculations, consider

.`datamash`

-
Know that locale affects a lot of command line tools in subtle ways, including sorting order (collation) and performance. Most Linux installations will set

`LANG`

or other locale variables to a local setting like US English. But be aware sorting will change if you change locale. And know i18n routines can make sort or other commands run*many times*slower. In some situations (such as the set operations or uniqueness operations below) you can safely ignore slow i18n routines entirely and use traditional byte-based sort order, using`export LC_ALL=C`

. -
You can set a specific command's environment by prefixing its invocation with the environment variable settings, as in

`TZ=Pacific/Fiji date`

. -
Know basic

`awk`

and`sed`

for simple data munging. See[One-liners](#one-liners)for examples. -
To replace all occurrences of a string in place, in one or more files:


` perl -pi.bak -e 's/old-string/new-string/g' my-files-*.txt`

- To rename multiple files and/or search and replace within files, try
. (In some cases the`repren`

`rename`

command also allows multiple renames, but be careful as its functionality is not the same on all Linux distributions.)

```
# Full rename of filenames, directories, and contents foo -> bar:
repren --full --preserve-case --from foo --to bar .
# Recover backup files whatever.bak -> whatever:
repren --renames --from '(.*)\.bak' --to '\1' *.bak
# Same as above, using rename, if available:
rename 's/\.bak$//' *.bak
```

- As the man page says,
`rsync`

really is a fast and extraordinarily versatile file copying tool. It's known for synchronizing between machines but is equally useful locally. When security restrictions allow, using`rsync`

instead of`scp`

allows recovery of a transfer without restarting from scratch. It also is among the[fastest ways](https://web.archive.org/web/20130929001850/http://linuxnote.net/jianingy/en/linux/a-fast-way-to-remove-huge-number-of-files.html)to delete large numbers of files:

`mkdir empty && rsync -r --delete empty/ some-dir && rmdir some-dir`

-
For monitoring progress when processing files, use

,`pv`

,`pycp`

,`pmonitor`

,`progress`

`rsync --progress`

, or, for block-level copying,`dd status=progress`

. -
Use

`shuf`

to shuffle or select random lines from a file. -
Know

`sort`

's options. For numbers, use`-n`

, or`-h`

for handling human-readable numbers (e.g. from`du -h`

). Know how keys work (`-t`

and`-k`

). In particular, watch out that you need to write`-k1,1`

to sort by only the first field;`-k1`

means sort according to the whole line. Stable sort (`sort -s`

) can be useful. For example, to sort first by field 2, then secondarily by field 1, you can use`sort -k1,1 | sort -s -k2,2`

. -
If you ever need to write a tab literal in a command line in Bash (e.g. for the -t argument to sort), press

**ctrl-v****[Tab]**or write`$'\t'`

(the latter is better as you can copy/paste it). -
The standard tools for patching source code are

`diff`

and`patch`

. See also`diffstat`

for summary statistics of a diff and`sdiff`

for a side-by-side diff. Note`diff -r`

works for entire directories. Use`diff -r tree1 tree2 | diffstat`

for a summary of changes. Use`vimdiff`

to compare and edit files. -
For binary files, use

`hd`

,`hexdump`

or`xxd`

for simple hex dumps and`bvi`

,`hexedit`

or`biew`

for binary editing. -
Also for binary files,

`strings`

(plus`grep`

, etc.) lets you find bits of text. -
For binary diffs (delta compression), use

`xdelta3`

. -
To convert text encodings, try

`iconv`

. Or`uconv`

for more advanced use; it supports some advanced Unicode things. For example:

```
# Displays hex codes or actual names of characters (useful for debugging):
uconv -f utf-8 -t utf-8 -x '::Any-Hex;' < input.txt
uconv -f utf-8 -t utf-8 -x '::Any-Name;' < input.txt
# Lowercase and removes all accents (by expanding and dropping them):
uconv -f utf-8 -t utf-8 -x '::Any-Lower; ::Any-NFD; [:Nonspacing Mark:] >; ::Any-NFC;' < input.txt > output.txt
```

-
To split files into pieces, see

`split`

(to split by size) and`csplit`

(to split by a pattern). -
Date and time: To get the current date and time in the helpful

[ISO 8601](https://en.wikipedia.org/wiki/ISO_8601)format, use`date -u +"%Y-%m-%dT%H:%M:%SZ"`

(other options[are](https://stackoverflow.com/questions/7216358/date-command-on-os-x-doesnt-have-iso-8601-i-option)[problematic](https://unix.stackexchange.com/questions/164826/date-command-iso-8601-option)). To manipulate date and time expressions, use`dateadd`

,`datediff`

,`strptime`

etc. from.`dateutils`

-
Use

`zless`

,`zmore`

,`zcat`

, and`zgrep`

to operate on compressed files. -
File attributes are settable via

`chattr`

and offer a lower-level alternative to file permissions. For example, to protect against accidental file deletion the immutable flag:`sudo chattr +i /critical/directory/or/file`

-
Use

`getfacl`

and`setfacl`

to save and restore file permissions. For example:

```
getfacl -R /some/path > permissions.txt
setfacl --restore=permissions.txt
```

- To create empty files quickly, use
`truncate`

(creates[sparse file](https://en.wikipedia.org/wiki/Sparse_file)),`fallocate`

(ext4, xfs, btrfs and ocfs2 filesystems),`xfs_mkfile`

(almost any filesystems, comes in xfsprogs package),`mkfile`

(for Unix-like systems like Solaris, Mac OS).

-
For web debugging,

`curl`

and`curl -I`

are handy, or their`wget`

equivalents, or the more modern.`httpie`

-
To know current cpu/disk status, the classic tools are

`top`

(or the better`htop`

),`iostat`

, and`iotop`

. Use`iostat -mxz 15`

for basic CPU and detailed per-partition disk stats and performance insight. -
For network connection details, use

`netstat`

and`ss`

. -
For a quick overview of what's happening on a system,

`dstat`

is especially useful. For broadest overview with details, use.`glances`

-
To know memory status, run and understand the output of

`free`

and`vmstat`

. In particular, be aware the "cached" value is memory held by the Linux kernel as file cache, so effectively counts toward the "free" value. -
Java system debugging is a different kettle of fish, but a simple trick on Oracle's and some other JVMs is that you can run

`kill -3 <pid>`

and a full stack trace and heap summary (including generational garbage collection details, which can be highly informative) will be dumped to stderr/logs. The JDK's`jps`

,`jstat`

,`jstack`

,`jmap`

are useful.[SJK tools](https://github.com/aragozin/jvm-tools)are more advanced. -
Use

as a better traceroute, to identify network issues.`mtr`

-
For looking at why a disk is full,

saves time over the usual commands like`ncdu`

`du -sh *`

. -
To find which socket or process is using bandwidth, try

or`iftop`

.`nethogs`

-
The

`ab`

tool (comes with Apache) is helpful for quick-and-dirty checking of web server performance. For more complex load testing, try`siege`

. -
For more serious network debugging,

,`wireshark`

, or`tshark`

.`ngrep`

-
Know about

`strace`

and`ltrace`

. These can be helpful if a program is failing, hanging, or crashing, and you don't know why, or if you want to get a general idea of performance. Note the profiling option (`-c`

), and the ability to attach to a running process (`-p`

). Use trace child option (`-f`

) to avoid missing important calls. -
Know about

`ldd`

to check shared libraries etc — but[never run it on untrusted files](http://www.catonmat.net/blog/ldd-arbitrary-code-execution/). -
Know how to connect to a running process with

`gdb`

and get its stack traces. -
Use

`/proc`

. It's amazingly helpful sometimes when debugging live problems. Examples:`/proc/cpuinfo`

,`/proc/meminfo`

,`/proc/cmdline`

,`/proc/xxx/cwd`

,`/proc/xxx/exe`

,`/proc/xxx/fd/`

,`/proc/xxx/smaps`

(where`xxx`

is the process id or pid). -
When debugging why something went wrong in the past,

can be very helpful. It shows historic statistics on CPU, memory, network, etc.`sar`

-
For deeper systems and performance analyses, look at

`stap`

([SystemTap](https://sourceware.org/systemtap/wiki)),, and`perf`

.`sysdig`

-
Check what OS you're on with

`uname`

or`uname -a`

(general Unix/kernel info) or`lsb_release -a`

(Linux distro info). -
Use

`dmesg`

whenever something's acting really funny (it could be hardware or driver issues). -
If you delete a file and it doesn't free up expected disk space as reported by

`du`

, check whether the file is in use by a process:`lsof | grep deleted | grep "filename-of-my-big-file"`


A few examples of piecing together commands:

- It is remarkably helpful sometimes that you can do set intersection, union, and difference of text files via
`sort`

/`uniq`

. Suppose`a`

and`b`

are text files that are already uniqued. This is fast, and works on files of arbitrary size, up to many gigabytes. (Sort is not limited by memory, though you may need to use the`-T`

option if`/tmp`

is on a small root partition.) See also the note about`LC_ALL`

above and`sort`

's`-u`

option (left out for clarity below).

```
sort a b | uniq > c # c is a union b
sort a b | uniq -d > c # c is a intersect b
sort a b b | uniq -u > c # c is set difference a - b
```

- Pretty-print two JSON files, normalizing their syntax, then coloring and paginating the result:

```
diff <(jq --sort-keys . < file1.json) <(jq --sort-keys . < file2.json) | colordiff | less -R
```


-
Use

`grep . *`

to quickly examine the contents of all files in a directory (so each line is paired with the filename), or`head -100 *`

(so each file has a heading). This can be useful for directories filled with config settings like those in`/sys`

,`/proc`

,`/etc`

. -
Summing all numbers in the third column of a text file (this is probably 3X faster and 3X less code than equivalent Python):


` awk '{ x += $3 } END { print x }' myfile`

- To see sizes/dates on a tree of files, this is like a recursive
`ls -l`

but is easier to read than`ls -lR`

:

` find . -type f -ls`

- Say you have a text file, like a web server log, and a certain value that appears on some lines, such as an
`acct_id`

parameter that is present in the URL. If you want a tally of how many requests for each`acct_id`

:

` egrep -o 'acct_id=[0-9]+' access.log | cut -d= -f2 | sort | uniq -c | sort -rn`

-
To continuously monitor changes, use

`watch`

, e.g. check changes to files in a directory with`watch -d -n 2 'ls -rtlh | tail'`

or to network settings while troubleshooting your wifi settings with`watch -d -n 2 ifconfig`

. -
Run this function to get a random tip from this document (parses Markdown and extracts an item):


```
function taocl() {
curl -s https://raw.githubusercontent.com/jlevy/the-art-of-command-line/master/README.md |
sed '/cowsay[.]png/d' |
pandoc -f markdown -t html |
xmlstarlet fo --html --dropdtd |
xmlstarlet sel -t -v "(html/body/ul/li[count(p)>0])[$RANDOM mod last()+1]" |
xmlstarlet unesc | fmt -80 | iconv -t US
}
```

-
`expr`

: perform arithmetic or boolean operations or evaluate regular expressions -
`m4`

: simple macro processor -
`yes`

: print a string a lot -
`cal`

: nice calendar -
`env`

: run a command (useful in scripts) -
`printenv`

: print out environment variables (useful in debugging and scripts) -
`look`

: find English words (or lines in a file) beginning with a string -
`cut`

,`paste`

and`join`

: data manipulation -
`fmt`

: format text paragraphs -
`pr`

: format text into pages/columns -
`fold`

: wrap lines of text -
`column`

: format text fields into aligned, fixed-width columns or tables -
`expand`

and`unexpand`

: convert between tabs and spaces -
`nl`

: add line numbers -
`seq`

: print numbers -
`bc`

: calculator -
`factor`

: factor integers -
: encrypt and sign files`gpg`

-
`toe`

: table of terminfo entries -
`nc`

: network debugging and data transfer -
`socat`

: socket relay and tcp port forwarder (similar to`netcat`

) -
: network traffic visualization`slurm`

-
`dd`

: moving data between files or devices -
`file`

: identify type of a file -
`tree`

: display directories and subdirectories as a nesting tree; like`ls`

but recursive -
`stat`

: file info -
`time`

: execute and time a command -
`timeout`

: execute a command for specified amount of time and stop the process when the specified amount of time completes. -
`lockfile`

: create semaphore file that can only be removed by`rm -f`

-
`logrotate`

: rotate, compress and mail logs. -
`watch`

: run a command repeatedly, showing results and/or highlighting changes -
: runs any command you specify whenever it sees file changed. See`when-changed`

`inotifywait`

and`entr`

as well. -
`tac`

: print files in reverse -
`comm`

: compare sorted files line by line -
`strings`

: extract text from binary files -
`tr`

: character translation or manipulation -
`iconv`

or`uconv`

: conversion for text encodings -
`split`

and`csplit`

: splitting files -
`sponge`

: read all input before writing it, useful for reading from then writing to the same file, e.g.,`grep -v something some-file | sponge some-file`

-
`units`

: unit conversions and calculations; converts furlongs per fortnight to twips per blink (see also`/usr/share/units/definitions.units`

) -
`apg`

: generates random passwords -
`xz`

: high-ratio file compression -
`ldd`

: dynamic library info -
`nm`

: symbols from object files -
`ab`

or: benchmarking web servers`wrk`

-
`strace`

: system call debugging -
: better traceroute for network debugging`mtr`

-
`cssh`

: visual concurrent shell -
`rsync`

: sync files and folders over SSH or in local file system -
: grep for the network layer`ngrep`

-
`host`

and`dig`

: DNS lookups -
`lsof`

: process file descriptor and socket info -
`dstat`

: useful system stats -
: high level, multi-subsystem overview`glances`

-
`iostat`

: Disk usage stats -
`mpstat`

: CPU usage stats -
`vmstat`

: Memory usage stats -
`htop`

: improved version of top -
`last`

: login history -
`w`

: who's logged on -
`id`

: user/group identity info -
: historic system stats`sar`

-
`ss`

: socket statistics -
`dmesg`

: boot and system error messages -
`sysctl`

: view and configure Linux kernel parameters at run time -
`hdparm`

: SATA/ATA disk manipulation/performance -
`lsblk`

: list block devices: a tree view of your disks and disk partitions -
`lshw`

,`lscpu`

,`lspci`

,`lsusb`

,`dmidecode`

: hardware information, including CPU, BIOS, RAID, graphics, devices, etc. -
`lsmod`

and`modinfo`

: List and show details of kernel modules. -
`fortune`

,`ddate`

, and`sl`

: um, well, it depends on whether you consider steam locomotives and Zippy quotations "useful"

These are items relevant *only* on macOS.

-
Package management with

`brew`

(Homebrew) and/or`port`

(MacPorts). These can be used to install on macOS many of the above commands. -
Copy output of any command to a desktop app with

`pbcopy`

and paste input from one with`pbpaste`

. -
To enable the Option key in macOS Terminal as an alt key (such as used in the commands above like

**alt-b**,**alt-f**, etc.), open Preferences -> Profiles -> Keyboard and select "Use Option as Meta key". -
To open a file with a desktop app, use

`open`

or`open -a /Applications/Whatever.app`

. -
Spotlight: Search files with

`mdfind`

and list metadata (such as photo EXIF info) with`mdls`

. -
Be aware macOS is based on BSD Unix, and many commands (for example

`ps`

,`ls`

,`tail`

,`awk`

,`sed`

) have many subtle variations from Linux, which is largely influenced by System V-style Unix and GNU tools. You can often tell the difference by noting a man page has the heading "BSD General Commands Manual." In some cases GNU versions can be installed, too (such as`gawk`

and`gsed`

for GNU awk and sed). If writing cross-platform Bash scripts, avoid such commands (for example, consider Python or`perl`

) or test carefully. -
To get macOS release information, use

`sw_vers`

.

These items are relevant *only* on Windows.

-
Access the power of the Unix shell under Microsoft Windows by installing

[Cygwin](https://cygwin.com/). Most of the things described in this document will work out of the box. -
On Windows 10, you can use

[Windows Subsystem for Linux (WSL)](https://msdn.microsoft.com/commandline/wsl/about), which provides a familiar Bash environment with Unix command line utilities. -
If you mainly want to use GNU developer tools (such as GCC) on Windows, consider

[MinGW](http://www.mingw.org/)and its[MSYS](http://www.mingw.org/wiki/msys)package, which provides utilities such as bash, gawk, make and grep. MSYS doesn't have all the features compared to Cygwin. MinGW is particularly useful for creating native Windows ports of Unix tools. -
Another option to get Unix look and feel under Windows is

[Cash](https://github.com/dthree/cash). Note that only very few Unix commands and command-line options are available in this environment.

-
You can perform and script most Windows system administration tasks from the command line by learning and using

`wmic`

. -
Native command-line Windows networking tools you may find useful include

`ping`

,`ipconfig`

,`tracert`

, and`netstat`

. -
You can perform

[many useful Windows tasks](http://www.thewindowsclub.com/rundll32-shortcut-commands-windows)by invoking the`Rundll32`

command.

-
Install additional Unix programs with the Cygwin's package manager.

-
Use

`mintty`

as your command-line window. -
Access the Windows clipboard through

`/dev/clipboard`

. -
Run

`cygstart`

to open an arbitrary file through its registered application. -
Access the Windows registry with

`regtool`

. -
Note that a

`C:\`

Windows drive path becomes`/cygdrive/c`

under Cygwin, and that Cygwin's`/`

appears under`C:\cygwin`

on Windows. Convert between Cygwin and Windows-style file paths with`cygpath`

. This is most useful in scripts that invoke Windows programs.

[awesome-shell](https://github.com/alebcay/awesome-shell): A curated list of shell tools and resources.[awesome-osx-command-line](https://github.com/herrbischoff/awesome-osx-command-line): A more in-depth guide for the macOS command line.[Strict mode](http://redsymbol.net/articles/unofficial-bash-strict-mode/)for writing better shell scripts.[shellcheck](https://github.com/koalaman/shellcheck): A shell script static analysis tool. Essentially, lint for bash/sh/zsh.[Filenames and Pathnames in Shell](http://www.dwheeler.com/essays/filenames-in-shell.html): The sadly complex minutiae on how to handle filenames correctly in shell scripts.[Data Science at the Command Line](http://datascienceatthecommandline.com/#tools): More commands and tools helpful for doing data science, from the book of the same name

With the exception of very small tasks, code is written so others can read it. With power comes responsibility. The fact you *can* do something in Bash doesn't necessarily mean you should! ;)

This work is licensed under a [Creative Commons Attribution-ShareAlike 4.0 International License](http://creativecommons.org/licenses/by-sa/4.0/).