Bash Commands


My commonly used commands

This is a page for my personal reference. TLDR pages has a more comprehensive command collection.

Background settings

Take the local repo of this blog as an example.

Show current path.

$ pwd
/home/vin100/quickstart

Distinguish files from folders .

$ ls -l
total 28
drwxrwxr-x 2 vin100 vin100 4096 Jun 28 21:00 archetypes
-rw-rw-r-- 1 vin100 vin100 2734 Aug 28 17:30 config.toml
drwxrwxr-x 4 vin100 vin100 4096 Jun 29 00:47 content
drwxrwxr-x 3 vin100 vin100 4096 Aug 25 15:29 layouts
drwxrwxr-x 9 vin100 vin100 4096 Jun 29 00:47 public
drwxrwxr-x 4 vin100 vin100 4096 Aug 25 13:36 static
drwxrwxr-x 3 vin100 vin100 4096 Aug 27 18:06 themes

apg

Avanced password generator.

My preferred way: partial management.

  1. Partial password saved: apg -a 1 -n 1 -m 6 -x 8 -M SNCL > output.txt
  2. manages partial passwords.
  3. (De)centralized local storage. (bare repo on USB devices)

Flags explanations: (in alphabetical order)

  • -a [0|1]: algorigthm
    • 0: can be “pronounced”
    • 1: “random” string
  • -m: minimum password length
  • -M: mode
    • c: should include capital letter
    • C: must include capital letter
    • l: should include lowercase letter
    • L: must include lowercase letter
    • n: should include number
    • N: must include number
    • s: should include special character
    • S: must include special character
  • -n: number of output passwords
  • -x: maximum password length

apt-get

CLI Package manager. Requires sudo priviledge.

sudo apt-get … Function
update update local repo
upgrade upgrade package version
install install a package
remove remove a package
purge remove a package and erase its folder
autoremove automatiqucally remove unnessary dependencies
clean remove unused local cache files

aptitude

I redirect interested readers to the post on my old blog to avoid duplicating efforts.

awk

I use it to extract column(s). I don’t know why double quotes " doesn’t work.

$ ls -dl * | awk '{print $9, $6}'
archetypes 4096
config.toml 2861
content 4096
layouts 4096
public 4096
static 4096
themes 4096

bash

GNU’s Bourne-again shell. Use this with -c [CMD] to execute a CMD.

Key SIGNAL effect
^C SIGINT interrupt current process in the foreground
^D NIL send EOF by default, terminate normally a bash utility
^Z SIGTSTP suspend current process (see also: fg and bg)

The caret notation ^★ means <C-★>. For example, ^D means <C-d>.

Some build-in commands:

  • pwd: print currect directory
  • cd [PATH]: change directory. If PATH is omitted, switch to ~.
  • history [n]: print last n command(s) with line numbers.
  • alias foo=[CMD]: set up foo as a shorthand of CMD, usually appear in ~/.bashrc
  • bg [jobspec] / fg [jobspec]: run suspended job in the background / foreground.
  • kill [jobspec]: send SIGTERM to jobspec to kill current job.

    jobspec meaning
    %+ current job, same as % and %%
    %- previous job
    %[n] job number n
    %foo a job named foo (the first word in cmd)
    %?foo a job matched foo

    In fact, bg and fg can be omitted. Simply calling the jobspec (say, %2) will run the job in fg. Appending & to the jobspec (say, %1 &) will run the job in bg.

    Some thoughts on : To suspend a running job (in either bg or fg, of whatever sign: +,-, , through ^Z or kill -TSTP), it’s automatically brought to fg. As a result,

    • newly suspended job → %+
    • %+%-

    Some thoughts on %foo and %?foo: if foo matches multiple jobs, bash will throw an error. However, Zsh won’t: the most recent matching job will be run.

  • read [VAR]: read STDIN and set it to shell VAR. This avoids exposing VAR value to bash history.

  • which [CMD]: find out the absolute path of CMD

  • command [-Vv] [CMD] [ARG(S)]

    • -v: print command absolute path or alias
    • -V: explain command type of CMD in a complete sentence
    • no flags: execute commands with colors stripped off.

bc

GNU’s basic calculator

Flags explanations: (in alphabetical order)

  • -l: load math library

    syntax function
    a $\arctan$
    s $\sin$
    c $\cos$
    l $\log$
  • -q: quiet, don’t display GNU’s standard welcome message

blkid

Display hard disk UUID.

cat

Catenate (combine) files and display them (in STDOUT).

  • -n: display line number

This can be used with the “null device” to show string without leaving a trace in the bash history.

$ cat > /dev/null
foo bar ...
$

convert

General usage: convert [input-option] [input-file] [output-option] [output-file].

Supported formats: JPG, PNG, GIF, SVG, etc

GUI softwares (e.g. GIMP enable preview of processed images, which is necessary in image sharpening. Therefore, I only list a few options below.

format conversion

$ convert foo.ppm -quality [num] bar.jpg

[num] takes value from 1 to 100. The higher the value, the better the quality and the larger the file.

image manipulation

This is good for batch processing.

Options:

  • -crop WxH+X+Y: crop image of W px $\times$ H px from (X,Y)
  • -rotate [DEG]: rotate input-file by DEG clockwisely.
  • -resize [DIM1] [DIM2]: resize image (if DIM2 is missing, largest dimension will be taken)

curl

Like wget, grab stuff from the Internet. Support all common protocols. (HTTP(S), (S)FTP, etc)

Basic usage:

$ curl [FLAG(S)] [URL]

It writes to STDOUT. An -o flag can be passed to specify output file.

$ curl https://api.staticman.net/v2/encrypt/testing -o foo.txt

File downloading

To keep the file name, use -O.

$ curl -O http://mae2.eng.uci.edu/~fjabbari//me270b/chap5.pdf

To download multiple files, use -O in series.

$ curl -O [URL1] -O [URL2] ...

To continue an interrupted download, use -C like the following line.

$ curl -C - -O http://mae2.eng.uci.edu/~fjabbari//me270b/chap5.pdf

This saves time and bandwidth in case of network interruption, and this can be applied to large files like ISO files for GNU/Linux distros.

URL redirects

If a page is “moved (permanently)” or the URL is the shortlink of another page, use -L to enable URL redirects.

$ curl https://lstu.fr/3s       # no output
$ curl -L https://lstu.fr/3s    # many lines of output

GET request

$ curl -i -X GET https://api.staticman.net/v2/encrypt/1q2w3e4r

See also: wget

References:

  1. HowtoForge
  2. curl tutorial
  3. curl POST examples

date

Display or adjust system date. Default to current time (zone).

Flags explanations: (in alphabetical order)

  • -d [STR]: convert STR to +%c

    $ date -d '@1536336779'
    Friday, September 07, 2018 PM06:12:59 CEST
  • -f [FILE]: read from FILE line by line and convert to +%c

  • -I[d|h|m|s|n]: ISO 8601 (default to d)

    [dhmsn] output
    n 2018-09-07T18:12:59,822423484+02:00
    s 2018-09-07T18:12:59+02:00
    m 2018-09-07T18:12+02:00
    h 2018-09-07T18+02:00
    d 2018-09-07
  • -R: for sending emails 📧

    $ date -R -d "2018-09-07 18:12:59"
    Fri, 07 Sep 2018 18:12:59 +0200
  • --rfc-3339=[dsn]: similar to -I with little differences (T, ,.)

    [dsn] output
    n 2018-09-07 18:12:59.822423484+02:00
    s 2018-09-07 18:12:59+02:00
    d 2018-09-07

Formats:

$ echo $LC_TIME
en_HK.UTF-8
$ date +%A
Friday
+% e.g. Remarks
z +0200
:z +02:00
::z +02:00:00
:::z +02 shortest numeric time zone
Z CEST
c Friday, September 07, 2018 PM06:12:59 CEST locale’s date and time
Y 2018
C 20
y 18
q 3 quarter
m 09
B September
b Sep same as %h
U 35 Week no. (week starts from Sunday, 00–53)
V 36 ISO week no. (week starts from Monday, 01–53)
W 35 Week no. (week starts from Monday, 00–53)
j 250 jour in French (001–366)
F 2018-09-07 Full date: %Y-%m-%d
x Friday, September 07, 2018 locale’s date
w 6
A Friday
a Fri
d 07
e 7 %_d
p PM blank if unknown
P pm idem
r PM06:12:59 CEST locale’s 12-hour time
T 18:12:59
X 06:12:29 CEST locale’s time
R 18:12
H 18
k 18 %_H
I 06
l 6 %_I
M 12
s 1536336779 seconds elapsed since 01/01/1970 00:00:00 UTC
S 59
n Enter ↵
t Tab ↹

 

Optional flags in between % and [char] meaning
- no padding
_ pad with
0 pad with 0
^ try uppercase

 

Acronym meaning
LC locale
CEST central European standard time

df

Disk free. Return amount of used and available disk space.

If a file is specified in the argument, df will return the row which represents the file system containing the file.

$ df
Filesystem     1K-blocks     Used Available Use% Mounted on
udev             3982080        0   3982080   0% /dev
tmpfs             802728     1304    801424   1% /run
/dev/sda7       29396988 10706500  17174152  39% /
tmpfs            4013620    21500   3992120   1% /dev/shm
tmpfs               5120        4      5116   1% /run/lock
tmpfs            4013620        0   4013620   0% /sys/fs/cgroup
/dev/sda6         463826   151423    283936  35% /boot
tmpfs             802724       12    802712   1% /run/user/1000
  • -B[SIZE]: set unit to SIZE. GB is the SI counterpart of G.

    $ df -BGB
    Filesystem     1GB-blocks  Used Available Use% Mounted on
    udev                  5GB   0GB       5GB   0% /dev
    tmpfs                 1GB   1GB       1GB   1% /run
    /dev/sda7            31GB  11GB      18GB  39% /
    tmpfs                 5GB   1GB       5GB   1% /dev/shm
    tmpfs                 1GB   1GB       1GB   1% /run/lock
    tmpfs                 5GB   0GB       5GB   0% /sys/fs/cgroup
    /dev/sda6             1GB   1GB       1GB  35% /boot
    tmpfs                 1GB   1GB       1GB   1% /run/user/1000

    This flag doesn’t give accurate results due to rounding errors. Use this with care.

  • -h: human readable sizes

    $ df -h
    Filesystem      Size  Used Avail Use% Mounted on
    udev            3.8G     0  3.8G   0% /dev
    tmpfs           784M  1.3M  783M   1% /run
    /dev/sda7        29G   11G   17G  39% /
    tmpfs           3.9G   22M  3.9G   1% /dev/shm
    tmpfs           5.0M  4.0K  5.0M   1% /run/lock
    tmpfs           3.9G     0  3.9G   0% /sys/fs/cgroup
    /dev/sda6       453M  148M  278M  35% /boot
    tmpfs           784M   12K  784M   1% /run/user/1000
  • -H: SI counterpart of -h

    $ df -H
    Filesystem      Size  Used Avail Use% Mounted on
    udev            4.1G     0  4.1G   0% /dev
    tmpfs           822M  1.4M  821M   1% /run
    /dev/sda7        31G   11G   18G  39% /
    tmpfs           4.2G   23M  4.1G   1% /dev/shm
    tmpfs           5.3M  4.1k  5.3M   1% /run/lock
    tmpfs           4.2G     0  4.2G   0% /sys/fs/cgroup
    /dev/sda6       475M  156M  291M  35% /boot
    tmpfs           822M   13k  822M   1% /run/user/1000
  • -t: file system type

    $ df -t ext4
    Filesystem     1K-blocks     Used Available Use% Mounted on
    /dev/sda7       29396988 10706516  17174136  39% /
    /dev/sda6         463826   151423    283936  35% /boot
  • --total: produce a sum of disk spaces at the bottom row and complete that row.

  • -x: opposite of -t

See also: du

diff

🆚 Display the difference between two text files.

Basic usage: diff [FILE1] [FILE2]

More useful form: show diff hunk. diff -u [FILE1] [FILE2]

dpkg

Dealing with packages. I only know the listing functionalities.

Flags explanations: (in alphabetical order)

  • -l [PKG(S)]: display package name, version, architecture and description
  • -L [PKG]: show installed files’ absolute path

    $ dpkg -L g++
    /.
    /usr
    /usr/bin
    /usr/share
    /usr/share/doc
    /usr/share/man
    /usr/share/man/man1
    /usr/bin/g++
    /usr/bin/x86_64-linux-gnu-g++
    /usr/share/doc/g++
    /usr/share/man/man1/g++.1.gz
    /usr/share/man/man1/x86_64-linux-gnu-g++.1.gz

du

Display disk usage in KB. Only folders are shown by default.

General usage: du [FLAG(S)] [DIR(S)] ...

If the argument [DIR(S)] is omitted, the size of every (sub)folder (including hidden ones) will be displayed.

$ du
...

Display the size of each subfolder in the folder layouts.

$ du layouts
12 layouts/partials
20 layouts/

Flags explanations: (in alphabetical order)

  • -a: also include all files
  • -c: include grand total at the bottom

    $ du -c layouts static
    12  layouts/partials
    20  layouts
    8   static/css
    196 static/img
    212 static
    232 total
  • -d [n]: max depth

    Process contents of [DIR] up to at most n level(s) deep.

    I found the concept of “level” hard to understand when I ran this in . because the output was cluttered with folders holding binary objects.

    Let me illustrate this idea with the following example.

    $ du -d 2 content
    8       content/post/2018-08-29-csb-theorem
    16      content/post/2018-07-17-rodeo
    248     content/post/2018-07-07-upgraded-to-linux-mint-19
    500     content/post/2018-08-18-ubuntu-18-04-installation-on-fujitsu-ah557
    76      content/post/2018-07-26-web-image-optimisation-with-gimp
    920     content/post/2018-07-04-fujitsu-lh532-fan-cleaning
    388     content/post/2018-08-23-brighten-image-with-gimp
    1412    content/post/2018-07-23-fujitsu-lh532-keyboard-cleaning
    3624    content/post
    12      content/page/bash-commands
    20      content/page┆┆
    3652    content┆┆   ┆┆
            ├──────┤├───┤├────────────────────────────────────────────────────┤
              Lv 0  Lv 1                         Lv 2
  • -h: human readable

    $ du -h layouts
    12K layouts/partials
    20K layouts/
  • --exclude=[FILE]

  • -s: summary, display only [DIR(S)]’s size. (equivalent to -d 0) This can be used to measure the size of a folder .

    $ du -s static content
    212     static
    3656    content
  • --time: also display time in the middle of each row

    $ du --time static
    8       2018-08-28 16:58        static/css
    196     2018-07-26 15:47        static/img
    212     2018-08-28 16:58        static

See also: df

echo

Display all arguments (in STDOUT).

$ echo foo bar
foo bar
  • -e: enable regular expressions

    • \a: shell beep (disabled in Ubuntu by default)

          $ echo -e "\a"
      
          $
    • \n: newline ↵

    • \t: tab ↹

  • -n: don’t output ↵ at the end

    $ echo -en "\a"
    $

find

Find files under PATH, -print and/or -execute command(s). PATH defaults to current path ..

General usage:

  • print / -print0: Display files: find [PATH] [FLAG(S)]

    • You may add -print at the end. This won’t change the display, but useful in conjonction with -exec.
    • You may use -print0 so that each output is delimited by null character \0 instead of newline ↵.

    In Ubuntu’s default location of Sublime Text 3 user config files, compare the output of -print

        find ~/.config/sublime-text-3/ -maxdepth 1 -print | od -c
        0000000   /   h   o   m   e   /   v   i   n   1   0   0   /   .   c   o
        0000020   n   f   i   g   /   s   u   b   l   i   m   e   -   t   e   x
        0000040   t   -   3   /  \n   /   h   o   m   e   /   v   i   n   1   0
        0000060   0   /   .   c   o   n   f   i   g   /   s   u   b   l   i   m
        0000100   e   -   t   e   x   t   -   3   /   I   n   s   t   a   l   l
        0000120   e   d       P   a   c   k   a   g   e   s  \n   /   h   o   m
        0000140   e   /   v   i   n   1   0   0   /   .   c   o   n   f   i   g
        0000160   /   s   u   b   l   i   m   e   -   t   e   x   t   -   3   /
        0000200   L   o   c   a   l  \n   /   h   o   m   e   /   v   i   n   1
        0000220   0   0   /   .   c   o   n   f   i   g   /   s   u   b   l   i
        0000240   m   e   -   t   e   x   t   -   3   /   L   i   b  \n   /   h
        0000260   o   m   e   /   v   i   n   1   0   0   /   .   c   o   n   f
        0000300   i   g   /   s   u   b   l   i   m   e   -   t   e   x   t   -
        0000320   3   /   C   a   c   h   e  \n   /   h   o   m   e   /   v   i
        0000340   n   1   0   0   /   .   c   o   n   f   i   g   /   s   u   b
        0000360   l   i   m   e   -   t   e   x   t   -   3   /   P   a   c   k
        0000400   a   g   e   s  \n
        0000405

    with -print0 using od -c.

        find ~/.config/sublime-text-3/ -maxdepth 1 -print0 | od -c
        0000000   /   h   o   m   e   /   v   i   n   1   0   0   /   .   c   o
        0000020   n   f   i   g   /   s   u   b   l   i   m   e   -   t   e   x
        0000040   t   -   3   /  \0   /   h   o   m   e   /   v   i   n   1   0
        0000060   0   /   .   c   o   n   f   i   g   /   s   u   b   l   i   m
        0000100   e   -   t   e   x   t   -   3   /   I   n   s   t   a   l   l
        0000120   e   d       P   a   c   k   a   g   e   s  \0   /   h   o   m
        0000140   e   /   v   i   n   1   0   0   /   .   c   o   n   f   i   g
        0000160   /   s   u   b   l   i   m   e   -   t   e   x   t   -   3   /
        0000200   L   o   c   a   l  \0   /   h   o   m   e   /   v   i   n   1
        0000220   0   0   /   .   c   o   n   f   i   g   /   s   u   b   l   i
        0000240   m   e   -   t   e   x   t   -   3   /   L   i   b  \0   /   h
        0000260   o   m   e   /   v   i   n   1   0   0   /   .   c   o   n   f
        0000300   i   g   /   s   u   b   l   i   m   e   -   t   e   x   t   -
        0000320   3   /   C   a   c   h   e  \0   /   h   o   m   e   /   v   i
        0000340   n   1   0   0   /   .   c   o   n   f   i   g   /   s   u   b
        0000360   l   i   m   e   -   t   e   x   t   -   3   /   P   a   c   k
        0000400   a   g   e   s  \0
        0000405
  • -exec: Execute commands for each matching file: find [PATH] [FLAG(S)] -exec [CMD]

    $ find archetypes -exec file {} \;
    archetypes: directory
    archetypes/default.md: ASCII text

    {} \; is necessary for representing an instance of matching file.

    -exec expects a bash command instead of an if-else statement or a for loop. Therefore, there’s no way to place them under -exec unless they are wrapped with sh -c. However, I’ve never tried this since I’ve no idea how to put {} \; inside sh -c.

More options:

  • type [d|f|l]: file type

      type
    d directory
    f file
    l symbolic link
  • -mindepth [n], -maxdepth [n]: works like du’s -d flag.

  • -and: AND operator, allow conjunction of -print(0) and -exec.

  • -path [PATH]: matches PATH.

  • -o: OR operator, useful when used with -prune

  • -prune: return TRUE when a directory is matched. This is useful to exclude path when used with -path.

    $ ls static
    css  google191a8de293cb8fe1.html  img
    $ find static -path static/css -prune -o
    find: expected an expression after '-o'
    $ find static -path static/css -prune -o -print
    static
    static/img
    static/img/globe.gif
    static/img/file-compress.ico
    static/img/globe16.ico
    static/google191a8de293cb8fe1.html
  • -[a|c|m][time|min] [+|-][n]: operate on recently X‘ed file last n Y ago. n should be an integer.

      X
    a accessed
    c status changed
    m modified

     

      Y
    time days
    min minutes

git

The most popular VCS (version control system). Here’s a minimal collection of commands needed for starters.

getting started

start from cmd
scratch git init
existing project git clone <source_url> <target_path>

add files to be tracked

Basic usage:

goal cmd
add some files git add <PAT>
add all files git add .

Technically, git add stages the file(s) matching the <PAT>tern to the index for committing.

Adding empty folder is forbidden.

unstage files

Avoid committing changes in certain file(s) to the version history. To be used for files in progress.

$ git reset HEAD -- <unready_file>

(re)move files

To rename files, use git mv <PAT>, which mv the file(s) matching the PATtern and then “inform” of this operation. (stage the mv operation)

To remove (unnecessary) files from , consider git rm, which rm the file(s) matching the PATtern and then “inform” of this operation. (stage the rm operation)

Keep the file? example cmd meaning
RSA/GPG private keys git rm --cached <PAT> remove matching file(s) from only
system generated files git rm <PAT> remove matching file(s) from and file system

ignore files

Simply put the files that you don’t wish to be tracked into ./.gitignore, one file per line. will read this GITIGNORE file take it into account.

show status

If you aren’t sure whether your files are changed, use this.

$ git status
On branch master
Your branch is up to date with 'origin/master'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

    modified:   content/page/bash-commands/index.md

no changes added to commit (use "git add" and/or "git commit -a")

You may run git diff / git diff --cached to view the unstaged / staged differences.

commit changes

Write the changes on the staged files to the history. Vim will pop up by default.

$ git commit

If you don’t want to work with Vim, you may input your commit message using STDIN.

$ git commit -m "commit message"

view changes

  • show commits: git log
  • show code change: git show
  • view project at a certain commit: git checkout <SHA1-hash>

At this stage, one will be able to manage his/her own repo locally.

work with different parallel versions

  • branching: for testing new features on a new branch without cracking the core stuff

    $ git branch <branch-name>
  • check branch(es): git branch only shows local branch(es)

    • -a: show remote branch(es) also
    • -v: show the SHA-1 hash at the tip of branch
  • switch betwween branches: git checkout <branch-name>

  • delete branch: git branch -d <branch-name>

  • compare branches: git diff <branch1>..<branch2>

  • merge changes from other local branch: git merge <branch-name>

work with others

If you want to save your code to , , , or send upload your repo to an SSH remote for sharing.

  1. Add/set remote:
    • Existing repo: git remote set-url <remote-name> <addr>
    • New repo started from scratch: git remote add <remote-name> <addr>
    • <remote-name>: take this to be origin for the first time if you’re newbie
    • <addr>: an HTTPS/SSH remote. For example,
      • SSH: git@gitlab.com:VincentTam/vincenttam.gitlab.io
      • HTTPS: https://gitlab.com/VincentTam/vincenttam.gitlab.io
  2. Submit your changes:
    • first time: git push -u <remote-name> <branch-name>. Set upstream branch to track in remote followed by sending out your commits.
    • afterwards: git push
  3. Get updates from others:
    • git fetch <remote-name> <branch-name>: download a remote branch without merging it against the current branch.
    • git checkout <remote-name>/<branch-name>: view a remote branch
    • git merge <remote-name>/<branch-name>: merge it against the current branch.
    • git pull <remote-name> <branch-name>: perform git fetch and git merge.

git-annex

A librarian like VCS on binary files through symbolic links.

Similar to git, I’ll only put the necessary commands to keep a repo running.

getting started

$ git init
$ git annex init "machine1"

add files

$ git annex add <file>

commit staged changes

$ git commit -m "<commit-msg>"

After git annex add, one can operate on file as in git.

add local remote

For example, backup data on a usb drive.

  1. Get git-annex started on usb.
  2. Add usb as remote on machine1: git remote add usb <path>
  3. and vice versa: git remote add machine1 <path>
  4. Sync remotes: git annex sync [--remote=<remote-name>]
  5. Send some files to usb: git annex sync --content

Working with the “library”:

  • Update catalogue: git annex sync
  • Catalogue check: git annex whereis <file>
  • Demand a copy: git annex copy <file> --from usb
  • Drop unnecessary copies: git annex drop <file>

If one of the remote is down, git annex sync throws an error. To avoid this, set up --remote=....

Before dropping a copy, ask git annex whereis <file>’s other copies.

If, for some reasons, you don’t want git-annex to touch your project, add an empty file .noannex at root level. This is useful if your project is being managed by other software. (say, rsync)

Further reading:

  1. Lee Hinman’s tutorial
  2. Git annex — un cas d’utilisation
  3. Git annex’s official walkthrough

grep

Get regular expression and print.

General usage:

  • read from file(s): grep [FLAG(S)] [PATTERN] [FILE]
  • pipe from STDIN: [cmd] | grep [FLAG(S)] [PATTERN]
  • read from STDIN:

    1. Type grep [PATTERN].

      $ grep foo
    2. Input the content to be searched. After you’ve finished, terminate the line with newline ↵.

      $ grep foo
      foobar
    3. Your search content is duplicated, with the matching part highlighted.

Flags explanations: (in alphabetical order)

  • -A [n]: print n lines after the matching line
  • -B [n]: print n lines before the matching line

    $ grep -B 3 Author config.toml
    #  src = "img/hexagon.jpg"
    #  desc = "Hexagon"
    
    [Author]
  • -C [n]: use this when the arguments for -A and -B are the same

  • -c: output the number of matching lines instead of the matching parts

    $ grep -c menu config.toml
    8
  • -E: use regular expression in the search string

  • -i: ignore case

  • -I: ignore binary files

    $ cd content/post/2018-07-23-fujitsu-lh532-keyboard-cleaning
    $ grep if *
    Binary file 20180708_234331_HDR.jpg matches
    Binary file 20180709_001359_HDR.jpg matches
    ...
    index.md:Finally the N key was fixed: *no* noticeable difference from other norm
    al size

    Use this flag to ignore the JPG files (and match the text files only).

    $ grep -I if *
    ...
    index.md:Finally the N key was fixed: *no* noticeable difference from other norm
    al size
  • -l: show matched file path

  • -L: show unmatched file path

  • -n: also show matching line number

    $ grep -n menu config.toml
    70:[[menu.main]]
    ...
    108:[[menu.main]]
  • -r: recursive

    $ grep -r bl[au] content static
    content/post/2018-06-28-xubuntu-dualboot-on-fujitsu-lh532.md:fa-laptop" aria-hid
    den></i>, Windows 10 often falls into blue screen.
    ...
    content/post/2018-07-04-fujitsu-lh532-fan-cleaning/index.md:blanket can put a ti
    ny electronic component into intensive care.  To
    content/post/2018-08-23-brighten-image-with-gimp/index.md:B | <i class="fa fa-sq
    uare blue"></i>--<i class="fa fa-square yellow" aria-hidden></i>
    static/css/custom.css:.blue {color: #0000FF;}
  • -R: like -r, but dereference links

  • -v: inverse match

  • -w: match whole word, similar to \< and \> in Vim.

  • -q: quiet

Print first n lines of a file or STDOUT. (n = 10 by default) Works like but opposite of tail.

hexdump

directory Display binary files as blocks of hexadecimal numbers.

  • -c: character

See alse: od

ifconfig

Display and/or modify connection info.

  • connections
    • e...: ethernet, wired connection
    • lo: localhost
    • w...: Wi-Fi
  • inet: internal IP address

To get external IP address, one needs to send a request to an external server . See wget for the command to do so.

info

More informative than man.

General usage: info [cmd]

kill

Kills specified job(s).

Usgae: kill [OPTION(S)] [PID]. PID can be consulted from ps aux.

  • -TSTP: “polite” stop (SIGTSTP), suspend a job instead of killing it. (Thanks to Steve Burdine .)
  • -STOP: “hard” stop (SIGSTOP), use -TSTP instead if possible
  • -CONT: continue job (SIGCONT)

less

View without editing the file in a seperate popup session with some Vim like key bindings.

General usage: less [FILE]

$ less config.toml

Compare with:

  • more: leave nothing on the screen after finish reading [FILE], ideal for text/code files / with a lot of lines.
  • vim: some keys are borrowed from Vim, loads faster due to its limited functionalities. This suits previewing important files, especially system files.
Key Function
b Scroll one page backward
f Scroll one page forward
d Scroll half page down
u Scroll half page up
g Jump to first line
G Jump to last line
j Move the cursor one line down
k Move the cursor one line up
/ Forward search
? Backword search
n Next match
N Previous match
h Show help page
q Quit

Some of the above keys can be quantified by prepending a number as in Vim.

ls

List files in a directory .

Default bash aliases:

Flags added
l -CF
la -A
ll -alF

Flags explanations: (in alphabetical order)

  • -a: all
  • -A: almost all (except current directory . and parent directory ..)
  • --block-size=[SIZE]: SIZE represents a unit (B, K, M, G).
  • -d I don’t understand what “list directories themselves, not their contents” means, as it can display the contents of a directory. Running this on any folder containing files and subfolders illustrates my doubts.

    # see what '-d' does on current path '.'
    $ ls -ld
    drwxrwxr-x 9 vin100 vin100 4096 Aug 27 22:04 .
    
    # see what '-d' does on every non-hidden folders and files `*`
    $ ls -ld *
    drwxrwxr-x 2 vin100 vin100 4096 Jun 28 21:00 archetypes
    -rw-rw-r-- 1 vin100 vin100 2734 Aug 28 17:30 config.toml
    drwxrwxr-x 4 vin100 vin100 4096 Jun 29 00:47 content
    drwxrwxr-x 3 vin100 vin100 4096 Aug 25 15:29 layouts
    drwxrwxr-x 9 vin100 vin100 4096 Jun 29 00:47 public
    drwxrwxr-x 4 vin100 vin100 4096 Aug 25 13:36 static
    drwxrwxr-x 3 vin100 vin100 4096 Aug 27 18:06 themes
    
    # take away '-d', compare the two outputs
    $ ls -l *
    -rw-rw-r-- 1 vin100 vin100 2734 Aug 28 17:30 config.toml
    
    archetypes:
    total 4
    -rw-rw-r-- 1 vin100 vin100 84 Jun 28 21:00 default.md
    
    content:
    total 12
    -rw-rw-r-- 1 vin100 vin100  279 Aug 28 02:23 _index.md
    drwxrwxr-x 3 vin100 vin100 4096 Aug 28 17:18 page
    drwxrwxr-x 9 vin100 vin100 4096 Aug 23 13:36 post
    ...

    I use -d with * to keep ls from expanding the subfolders. .

man

Display manual page in a separate session. Less detailed than info.

General usage: man [num] [cmd].

mkdir

Make directory.

more

Works like cat, but for files with more lines. Opposite of less.

General usage: more [FILE]

od

Display binary files as blocks of octal numbers.

  • -c: character

See also: hexdump

openssl

OpenSSL tools

I only know two usages.

  1. random base64 encoded string generation.

    $ openssl rand -out output.txt -base64 12 $ cat output.txt 0DNyuVqNxecZI+Fo

  2. RSA key generation

    Generate a 2048-bit RSA key pair and output key.pem

    $ openssl genrsa -out key.pem 2048

    To add extra security to the private key, it may be encrypted with -des3.

    To export a corresponding public key of a private key, use -pubout.

    openssl rsa -in key.pem -outform PEM -pubout -out pubkey.pem

pdfgrep

Grep string from PDF file(s). .

General usage:

  • Single file: pdfgrep "foo" bar.pdf
  • Multiple files: pdfgrep -R "foo" ./some/path

pgrep

ps + grep

Basic usage: pgrep [PROC]. Only one pattern can be provided.

  • -f: match full pattern instead of process name
  • -l: show PROC at the 2nd column

It took me a while to understand what full pattern meant. Process 2339 bears the name Web Content, but it’s triggered by firefox. Viewing its command (by ps aux), I’ve understoood why this process appeared in the output of pgrep -fl firefox.

$ pgrep -l firefox
2292 firefox
$ pgrep -lf firefox
2292 firefox
2339 Web Content
17363 Web Content
$ ps aux | grep 2339
vin100    2339 24.4  5.2 2230872 419804 ?      Sl   01:43  28:34 /usr/lib/firefox/firefox -contentproc -childID 1 -isForBrowser -prefsLen 10155 -schedulerPrefs 0001,2 -parentBuildID 20180905220717 -greomni /usr/lib/firefox/omni.ja -appomni /usr/lib/firefox/browser/omni.ja -appdir /usr/lib/firefox/browser 2292 true tab
vin100   26892  0.0  0.0  22008  1104 pts/1    S+   03:40   0:00 grep --color=auto 2339

ps

Prints process status.

I only know ps aux, which displays every process. ps ux only shows jobs owned by root.

  • -C [cmd]: show process cmd only

qpdf

Split/merge/rotate/encrypt/decrypt PDF files.

I don’t have to explore the encryption functionalities for the moment.

Generic usage:

  • --linearize: optimize file for web.
  • --rotate=[+|-]angle:page-range: possible parameters for angle are 90, 180 and 270.
  • --split-pages=n [INPDF] [OUTPDF]: split file into groups of n pages. n defaults to 1.

    $ qpdf --split-pages foo.pdf bar.pdf
    $ ls bar.pdf
    bar-01.pdf  bar-03.pdf  bar-05.pdf  bar-07.pdf  bar-09.pdf  bar-11.pdf
    bar-02.pdf  bar-04.pdf  bar-06.pdf  bar-08.pdf  bar-10.pdf  bar-12.pdf

    An optional string %d in OUTPDF is replaced with a zero-padded page.

    $ qpdf --split-pages twopage.pdf o%dut.pdf
    o1ut.pdf  o2ut.pdf
  • --empty: discard INPDF metadata

  • --pages [INPDF] [RANGE] [[INPDF2] [RANGE]...] --: extract pages

    • RANGE: apart from the basic 1-3,5, you may reverse selection order.
    • INPDF*: multiple files don’t need to be seperated by --. (The command in Mankier is wrong.)

      $ qpdf --empty --pages input1.pdf 1,6-8 input2.pdf -- output.pdf

References:

  1. Official manual
  2. Mankier

rpm

An RPM cheat sheet on Cyberciti.

rsync

Remote synchronization. Supports many protocols like FTP, SSH, etc.

General usage: rsync [FLAGS] [SRC] [DEST].

  • SRC: source
    • files
    • folders
      • with trailing /: send contents inside the folder
      • without trailing /: send whole folder
  • DEST: destination, a file path

Flags explanations: (in alphabetical order)

  • -a: archive, preserve file attributes, permissions and [acm]times
  • -u: update, only send file(s) in SRC newer than DEST
  • -v: verbose, print every operations to STDOUT
  • -z: zip, reduce transferred data.

screendump

Record a TTY session into a text file (with one single long line). Require sudo privileges.

$ sudo screendump 1 > ~/mylog.txt
[sudo] password for vin100:
$ cat ~/mylog.txt

Ubuntu 18.04.1 LTS vin100-LIFEBOOK-LH532 tty1

vin100-LIFEBOOK-LH532 login:

...

sed

Stream editor

$ sed [FLAG(S)] '[range][cmd]'

Flags explanations: (in alphabetical order)

  • -e: extended mode
  • -i: in place editing
  • -n: no normal output

[range] can refer to a line number ($ meaning the last line), or a scope /[PAT]/. The later can be used to remove empty lines.

$ git status
On branch master
Your branch is up to date with 'origin/master'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

        modified:   content/page/bash-commands/index.md
        modified:   content/page/sublime/index.md

no changes added to commit (use "git add" and/or "git commit -a")
$ git status | sed '/^$/d'
On branch master
Your branch is up to date with 'origin/master'.
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)
        modified:   content/page/bash-commands/index.md
        modified:   content/page/sublime/index.md
no changes added to commit (use "git add" and/or "git commit -a")

A [range] can be inverted by !, so sed '$!d' works like tail -1.

shrinkpdf

I’ve added Alfred Klomp’s simple Ghostscript wrapper to ~/bin, which has been added to $PATH. The > is optional. The third argument indicates the quality, and it defaults to 90, which is barely enough for pure text documents .

$ ./shrinkpdf.sh foo.pdf > bar.pdf
$ ./shrinkpdf.sh foo.pdf bar.pdf 150

quality/size tradeoff:

  • 150 suffices for text documents in emails 📧.
  • 200 suffices for documents with text and photos.
  • 300 for usual storage 💾
  • 450 / 600 for superior quality. (e.g. , , etc)

shuf

Shuffle input. (no repetition by default)

Flags explanations: (in alphabetical order)

  • -i: input range

    $ shuf -i 1-10 -n 3
    4
    9
    7
  • -n: output number

  • -r: allow repetition

sleep

Start an idle process for n seconds, which n is the first argument.

Remarks: I can’t say that the shell is suspended despite its apparent effect as appending & to the command allows it to run in background.

sort

Sort input file (in ascending alphabetical order by default).

  • -k[n]: sort according to column [n]
  • -n: use numeric order instead
  • -r: reverse order
  • -u: preserve only unique lines after sort

ssh

Secure shell: access remote desktop on the network through an encrypted “tunnel”.

  • simplest syntax: ssh [USER]@[DOMAIN]

    $ ssh vin100@192.168.1.2

    The shell then prompts you to input the password for [USER] up. Upon success, you’ll logged in as [USER].

  • one-spot connection: ssh [USER]@[DOMAIN] [CMD]

    $ ssh vin100@192.168.1.2 ls ~/quickstart
    archetypes  config.toml  content  layouts  public  static  themes

stty

Show info of TTY.

  • Return number of rows and columns

    $ stty size
    43 132
  • Set buffer sizes

    $ stty cols 80
    $ stty rows 32

See also: tty

tail

Print last n lines of a file or STDOUT. (n = 10 by default) Opposite of head

  • -c [n]: output last n bytes. Useful for EOL detection at EOF.
  • -n [m]: output last m lines
  • -z: use null character \0 instead of newline ↵ as line delimiter. It can be used to process the output of find ... -print0.

tee

Redirect command output into STDOUT and a file. To be used for inspecting and capturing command output simultaneously.

$ ls | tee eles
archetypes
config.toml
content
layouts
public
static
themes
$ cat eles
archetypes
config.toml
content
layouts
public
static
themes

> captures the command output without showing it (except errors).

time

Record the time taken for running a command.

General usage: time [command]

$ time sleep 5

real    0m5.002s
user    0m0.002s
sys     0m0.000s

tty

Output absolute file path of the current terminal. (no argument needed)

  • GUI: /dev/pts/1
  • TTYn: /dev/tty[n]

See also: stty

tr

Translate or remove a certain characters. Like other GNU coreutils (e.g. cat, grep, etc), it accepts STDIN and/or input file(s), and write to STDOUT.

General usage:

  • Replace character

    $ tr ' ' '_'
    foo bar
    foo_bar
  • Delete character

    $ tr -d ' '
    foo bar
    foobar

uniq

Output “locallyunique lines. i.e. Remove neighbouring duplicate lines of input file(s) or STDIN.

$ echo '1\n1\n1\n2' | uniq
1
2
$ echo '1\n2\n1\n2' | uniq
1
2
1
2

See also: sort -u

vim

Improved text-editor from vi, which is preloaded on every GNU/Linux and FreeBSD distro. (even on Mac OS)

  • -R: read-only mode
Normal mode key Function
<C-b> Scroll one page backward
<C-f> Scroll one page forward
<C-d> Scroll half page down
<C-u> Scroll half page up
g Jump to first line
G Jump to last line
h Move the cursor one character left
j Move the cursor one character down
k Move the cursor one character up
l Move the cursor one character right
/ Forward search
? Backword search
n Next match
N Previous match
i Insert character under the cursor
q Quit

P.S. It was my favorite editor.

wc

Word count

  1. Use files: output character, word and line counts, followed by file name

    $ wc .gitmodules
      4  11 133 .gitmodules
  2. Use STDOUT: also show these three counts, but without file name

    $ cat .gitmodules | wc
          4      11     133
  • c: character count
  • w: word count
  • l: line count

wget

From web, get stuff (with its resources).

  • -c: works like -C in curl
  • -O: works like -o in curl
  • -q: quiet, don’t output to STDOUT

To get the external IP address, try

$ wget -qO- http//ipecho.net/plain ; echo

See also: curl

You may refer to the TLDR page for more useful commands like

  1. Download page with its resources
  2. Download full website
  3. Download recursively a remote folder
  4. Download via authenticated FTP

xargs

Rearrange and/or execute arguments.

Output of ls without -l flag is ascending column-wise.

$ ls -A .git
branches        description  hooks  logs     ORIG_HEAD
COMMIT_EDITMSG  FETCH_HEAD   index  modules  packed-refs
config          HEAD         info   objects  refs

xargs -n [num] treats input as arguments delimited by space ␣, tab ↹ and/or newline ↵. It outputs [num] arguments delimited by space ␣ on each line.

$ ls -A .git | xargs -n 3
branches COMMIT_EDITMSG config
description FETCH_HEAD HEAD
hooks index info
logs modules objects
ORIG_HEAD packed-refs refs

Observe the difference of the output below with the first block in this section.

$ ls -A .git | xargs -n 3 | xargs -n 5
branches COMMIT_EDITMSG config description FETCH_HEAD
HEAD hooks index info logs
modules objects ORIG_HEAD packed-refs refs

xwd

Take screeenshot of graphical desktop from TTY. (require sudo priviledges)

This can be useful for capturing the login screen.

The following only works for LightDM.

I’ve refined Neroshan’s command on Ask Ubuntu into a shell script.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
#!/bin/sh
# screenshot.sh
# USAGE: ./screenshot.sh [file-name]

chvt 7 # On Xubuntu 18.04
#chvt 1# On Ubuntu 18.04
DISPLAY=:0 XAUTHORITY=/var/run/lightdm/root/:0 xwd -root -out ~/screenshot.xwd
convert ~/screenshot.xwd $1
rm ~/screenshot.xwd
chvt `tty | sed 's:/dev/tty::'`

This script requires one single argument: output file name (with extension name), which is passed to $1. The idea is simple.

  1. Switch to GUI from CLI (TTY1–TTY6 on Xubuntu 18.04; TTY2–TTY7 on Ubuntu 18.04)
  2. Add necessary shell variables. (Adapt it to GDM or other display manager)
  3. Create a temporary XWD file.
  4. Convert this file to a file with your specified file name.
  5. Remove the temporary XWD file.
  6. Switch back to CLI.
Xubuntu 18.04 error after login

Screenshot by xwd

Taken with the above script from TTY on Xubuntu 18.04