Adventures in Vim: Lee and Jim figure out how to change the colors of comments

On a beautiful Monday morning, Ars Technica’s senior technology editor, Lee Hutchinson, came to me with a problem: the colors in his text editor, in his humble opinion, had started to suck.

In about 20 years of using Vim, Lee has gotten used to commenting lines in his code and configuration files being rendered in dark blue. But after upgrading a machine to Ubuntu 20.04, Vim started rendering comments in cyan – and as the “Identifier” syntax category also rendered in cyan, he was unhappy enough to decide to change the defaults.

At first glance, Vim seems to adhere to roughly the same configuration pattern that many, if not most Unix-like systems and applications do – there is a set of system-wide settings in /etc, which can be individually replaced by a user with changes made to an optional configuration file in that user’s home directory. In the case of Vim, this is ~/.vimrc– just like the Bash settings can be overridden in ~/.bashrc.

But when Lee tried to do his One Simple Change in Vim’s syntax highlighting – turning comments from the new cyan back to dark blue, which he preferred – things got interesting.

Hutchinson’s way of setting up comment highlighting

After searching Google a bit, the command Lee found to change the comment’s color looked very simple: highlight comment ctermcfg=19, where 19 is the color code that Vim uses for dark blue. The problem is, making the change in ~/.vimrc it didn’t really work.

To be more specific, did work – briefly – but almost immediately after opening the file, the comments have changed from dark blue to cyan again. On a fast local machine, the change happened too quickly to be noticed; but Lee was getting into a remote machine, and that gave him enough delay to see his color preference applied initially, but quickly reversed.

After researching Google extensively, Lee discovered an ugly workaround. There is a very old joke that says Vim is not really a text editor – it is an operating system in its own right, which simply disguises itself as a text editor. Like most good jokes, this one is a bit exaggerated, but it has a truth to it – Vim configuration files don’t simply assign values ​​to configuration variables; they can actually run code on their own.

In Lee’s case, he decided that since there was a delay of about 100 ms between his dark blue comments being applied and Vim changing them back, he could simply wait for the program to wait 100 ms to apply the change first of all:

function DelayedSetVariables(timer)
    highlight comment ctermfg=19
endfunction

let timer=timer_start(16,'DelayedSetVariables')

For sure, the ugly hack worked: now, instead of seeing comments in dark blue initially that later went back to the hated cyan, Hutchinson saw comments in cyan that later went back to his favorite dark blue.

This worked well enough for your purposes … but what’s the point of being a senior technology editor if you can’t pass the problem on to a technology reporter who reports to you?

The wrong way … in fact, several wrong ways

When Lee brought his problem half solved for me, it certainly looked like a bug – I may not be a Vim user, but with over 20 years of experience in a Unix-like operating system under my own belt, I also expected a user configuration file profile to cleanly replace an entire system configuration. THE unbalanced speech the coherent, focused problem report Lee offered me included a warning: there were, in his words, “about 20 different places where Vim configuration changes are applied”, so tracking the problem was unusually difficult.

I am not a user of Vim – I am one of those pagans who never saw a particular reason to learn more about Vim than :q! I needed to get out of it, but my immediate suspicion was that a bug was causing Vim’s configuration files to be applied out of order. So I researched Google how to check which settings were applied to a running Vim instance: it turns out that there is a special command :scriptnames that will provide just that.

  1: /usr/share/vim/vimrc
  2: /usr/share/vim/vim81/debian.vim
  3: /usr/share/vim/vim81/syntax/syntax.vim
  4: /usr/share/vim/vim81/syntax/synload.vim
  5: /usr/share/vim/vim81/syntax/syncolor.vim
  6: /usr/share/vim/vim81/filetype.vim
  7: ~/.vimrc
  8: /usr/share/vim/vim81/plugin/getscriptPlugin.vim
  9: /usr/share/vim/vim81/plugin/gzip.vim
 10: /usr/share/vim/vim81/plugin/logiPat.vim
 11: /usr/share/vim/vim81/plugin/manpager.vim
 12: /usr/share/vim/vim81/plugin/matchparen.vim
 13: /usr/share/vim/vim81/plugin/netrwPlugin.vim
 14: /usr/share/vim/vim81/plugin/rrhelper.vim
 15: /usr/share/vim/vim81/plugin/spellfile.vim
 16: /usr/share/vim/vim81/plugin/tarPlugin.vim
 17: /usr/share/vim/vim81/plugin/tohtml.vim
 18: /usr/share/vim/vim81/plugin/vimballPlugin.vim
 19: /usr/share/vim/vim81/plugin/zipPlugin.vim
 20: /usr/share/vim/vim81/scripts.vim
 21: /usr/share/vim/vim81/syntax/perl.vim
 22: /usr/share/vim/vim81/syntax/pod.vim
Press ENTER or type command to continue

Lee was not kidding about the wide range of configuration files to examine: my system loaded 22 separate configuration files, 15 of which took effect after the .vimrc in my home directory! Thus began the beginning of a long, tortuous and ultimately fruitless path in the evening primrose: I wanted to find instances of the color of the comment being changed somewhere after mine ~/.vimrc, and I found that it just wasn’t happening.

The only place I could find where the color of the comment was set to Cyan I was /usr/share/vim/vim81/syncolor.vim, some spaces forward of my people .vimrc. In theory, the change in ~/.vimrc should have replaced the one in syncolor.vim– but in practice, without the horrible hack of Lee’s timer, the only way I found to change the color of the comment was inside syncolor.vim in itself.

" Many terminals can only use six different colors (plus black and white).
" Therefore the number of colors used is kept low. It doesn't look nice with
" too many colors anyway.
" Careful with "cterm=bold", it changes the color to bright for some terminals.
" There are two sets of defaults: for a dark and a light background.
if &background == "dark"
  SynColor Comment      term=bold cterm=NONE ctermfg=Cyan ctermbg=NONE gui=NONE guifg=#80a0ff guibg=NONE

Changing ctermfg=Cyan inside syncolor.vim for ctermfg=19—Or, even better, ctermfg=DarkBlue, which produced an easier-to-read shade of blue – worked as expected and produced the output Lee wanted without the terrible timer hack. But he applied the change across the system, not just to Lee’s own user account – and more importantly, he didn’t explain how or why the original change in ~/.vimrc refused to work as expected.

I could still smell an out of order insect, so I dug more.

" Vim syntax support file
" Maintainer:   Bram Moolenaar 
" Last Change:  2001 Sep 12

" This file sets up the default methods for highlighting.
" It is loaded from "synload.vim" and from Vim for ":syntax reset".
" Also used from init_highlight().

According to the comments at the top of syncolor.vim, changes to that file were applied in three cases – when synload.vim is analyzed during Vim startup, when the user issues the command :syntax reset, and within the Vim function init_highlight(). I didn’t know that neither Lee nor I were calling :syntax reset, so I started to find the invocation of syncolor.vim from the inside synload.vim.

" Set the default highlighting colors.  Use a color scheme if specified.
if exists("colors_name")
  exe "colors " . colors_name
else
  runtime! syntax/syncolor.vim
endif

If I put the simple highlight comment ctermfg=19 back to mine ~/.vimrc, and commented the runtime! syntax/syncolor.vim at the synload.vim, I believed that everything should work correctly: this would still qualify as an ugly hack, of course, but it would restrict the source of the problem’s behavior and allow me to write a more accurate bug report to file with the Vim project.

Unfortunately, it didn’t work that way: even with runtime! syntax/syncolor.vim commented, the cyan comments that the file specified replaced the simple configuration in my ~/.vimrc. That meant that the settings there were being called by Vim’s init_highlight() occupation after analyzed ~/.vimrc.

On the one hand, it certainly still smelled like a bug to me: I couldn’t replace a simple configuration setting for my user-level rc file. On the other hand, did I mention more than 20 years of open source experience? I needed to make sure that something obvious was missing that would cause a bug report to be rejected with a #WONTFIX because I had lost some deliberate idiosyncrasy of Vim.

Finding the right way

Since the Vim configuration files had self-documenting comments, it was time to read them in more detail. I had already learned that the content of syncolor.vim were applied by init_highlight() and synload.vim– but I needed to dig more.

I couldn’t go any further with the documentation comments at the top of the synload.vim or syncolor.vim, but the next clue came from the code in syncolor.vim in itself:

if syntax_cmd == "enable"
    " ":syntax enable" keeps any existing colors
    command -nargs=* SynColor hi def 
    command -nargs=* SynLink hi def link 
  elseif syntax_cmd == "reset"
    " ":syntax reset" resets all colors to the default
    command -nargs=* SynColor hi 
    command -nargs=* SynLink hi! link 
  else
    " User defined syncolor file has already set the colors.
    finish
  endif

Clearly, there was some appropriate way of defining user-defined colors, since this if the block specifically avoided configuring them if a “user-defined syncolor file” already had them. So the next step was to search Google for “vim user-defined syncolor file”. The main result of the research was the source of syncolor.vim on Github, but the second result brought me to the Vim documentation on SourceForge.

Running a ctrl-F syncolor the browser search in this 5,128 line document ended up getting me the information I needed, about 90 percent of the way down the page:

If you want to use different colors for syntax highlighting, you can add a Vim
script file to set these colors.  Put this file in a directory in
'runtimepath' which comes after $VIMRUNTIME, so that your settings overrule
the default colors.  This way these colors will be used after the ":syntax
reset" command.

For Unix you can use the file ~/.vim/after/syntax/syncolor.vim.

Finally, I found the right answer to the seemingly simple question “How do I change the color of the comment in Vim?”: after creation ~/.vim, ~/.vim/after, and ~.vim/after/syntax, you can finally create the file ~/.vim/after/syntax/syncolor.vim—And changes made to the syntax highlight colors over there applied the way Lee and I expected.

Petting the furry dog

Hopefully, you learned something along the way by reading this horrible furry dog ​​story about setting up a Linux application. Maybe you also just wanted to change some colors in a text editor – in that case, I took you an absurdly long way just to come up with a relatively short answer.

But more importantly, I hope that the full exercise can serve as a broader problem-solving exercise. Happy Linux-ing!