Missing Newline at EOL

Background

Despite my experience in Vim, the multi-cursor functionality in Sublime Text 3 has seduced my to change my editor.

Unlike Vim, a nonempty new line at the end of Sublime Text 3 file buffer causes the file to end without a newline character. In fact, it’s a POSIX standard to include a newline character at the EOL (end of file). (c.f. No Newline at End of File)

Problem

Unaware of the above POSIX standard and Sublime Text 3 convention, I have edited many lines of code in the repository for this blog and the one for my custom Beautiful Hugo. These edited files were almost everywhere in these repositories, and they polluted their remote counterparts on GitLab .

Solution

1. List files from Git index or the file tree.
2. Omit binary files (especially image files ).
3. Look into the file and append a newline if necessary.

Core part

It’s possible to write shell script which appends a newline provided that the last character tail -c 1 $1 matches an empty string ‘’, but I prefer a oneline solution.

find the way out?

Since the folders public, themes and .git have to be pruned, the pattern -path [DIR] -prune -o has to be repeated three times. This pattern is found in the man page of find, so there’s no other simpler way of finding files with some [DIR(S)] excluded.

In addition, -exec waits for a command instead of an if-else statement. Therefore, there’s no way to bring the above part into -execution.

A low-level attempt

Git has a low-level command git ls-files to list the files. However, this didn’t work well with my grep command for omitting image files due to the presence of the submodule for the theme.

$ for f in `git ls-files`; do if grep -Iq '' $f; then
for then> echo "$f is an ASCII file"; fi; done
.gitignore is an ASCII file
.gitlab-ci.yml is an ASCII file
...
static/css/print.min.css is an ASCII file
static/google191a8de293cb8fe1.html is an ASCII file
grep: themes/beautifulhugo: Is a directory

Even though the installation instructions of Beautiful Hugo suggest cloning independently, doing so will lead to failure during GitLab CI/CD. Therefore, one has better follow the official Hugo’s approach: grab the theme as a Git submodule.

Solution for my custom contents and themes

$ pwd
/home/vin100/quickstart

$ for f in `git grep --cached -Il ''`; do if [ "$(tail -c 1 $f)" != '' ]; then
for then> echo >> $f; fi; done
git status
On branch master
Your branch is ahead of 'origin/master' by 1 commit.
(use "git push" to publish your local commits)

Changes not staged for commit:
(use "git add <file>..." to update what will be committed)
(use "git checkout -- <file>..." to discard changes in working directory)

      modified:   content/page/bash-commands/index.md
      modified:   content/post/2018-07-07-upgraded-to-linux-mint-19/index.md
      modified:   content/post/2018-07-14-hugo-image-path-refactoring.md
      modified:   content/post/2018-07-23-fujitsu-lh532-keyboard-cleaning/index.md
      modified:   content/post/2018-07-26-web-image-optimisation-with-gimp/index.md
      modified:   content/post/2018-08-15-tlp-prevents-login-on-ubuntu-18-04.md
      modified:   content/post/2018-08-23-brighten-image-with-gimp/index.md
      modified:   layouts/partials/footer_custom.html
      modified:   layouts/partials/head_custom.html
      modified:   static/css/custom.css
      modified:   static/css/print.min.css
      modified:   static/google191a8de293cb8fe1.html

I used to thinking that git grep operates on the index by default. In fact, an additional --cached flag needs to be passed to obtain this behavior.

Solution for my enhanced Hugo theme

I had taken special care with the assets in static: exclude this folder also. However, I couldn’t think of a better solution without digging deeper into “bash inside bash”. Therefore, I resorted to a basic solution.

1. First loop through all text files and perform necessary modifications.

    $ for f in `git grep -lq ''`; do
    for> if [ "$(tail -c 1 $f)" != '' ]; then
    for then> echo >> $f
    for then> fi
    for> done
   

2. Then undo the previous step for some files, say the minified CSS and JS files in ./static.

    $ git checkout -- static
   

   Since git reset --hard doesn’t accept double-hyphen -- on the right-hand side, one has to run the above commands a few times against different subfolders to obtain the desired result.

Prevent making the same mistake again

To err is human, but repeating the same error isn’t wise. However, it’s inefficient to manually inspect the end of file for each edited files.

$ for f in "$(git diff --name-only)"; do echo $f && tail -c 1 $f | od -c; done

To prevent making this mistake in Sublime Text 3, I added one simple line to the user preferences.

"ensure_newline_at_eof_on_save": true