Project Wide Search in Vim

I finally managed to switch to Vim, Neovim actually, full time about four months ago. I have tried to move to Vim a number of times over the years, but I always felt too busy to deal with the initial productivity loss. There was always some part of my workflow that I 1) couldn't live without, and 2) couldn't quite figure out how to do in Vim.

This time I had a bit of downtime that I could spend on just getting to a comfortable level using Vim. For the first month or so, would find myself reverting back to Atom for certain tasks. Over time I figured out how to deal with more and more editing situations. Now I am completely converted.

I still have a huge amount of learning to do before I would even call myself a competent Vim user. There are many, many ways that I can still improve my editing speed, but that is one of the nice things about Vim, it is an editor to grow with.

There are some rough edges that I am still working on. One of those edges is project wide search. As I said, I was previously using Atom, and Textmate before that. Both had simple project wide search, just enter Cmd-Shift-f, enter the search term (or if text was selected it would be pre-populated in the find dialog), and press Return. Boom, there is a list of all occurrences of that search term in the entire project. Click on an item in the list and a new tab opens the selected file and the search term is highlighted.

I'd like something similar while working in Vim. A nice thing, and also sometimes a bad thing about Vim, is its flexibility. Vim has two ways to search multiple files, not including the current file search commands (/ and ?). By the end of this article I will show you how to have essentially the same project wide search functionality as Atom and you will find out how to create your own Vim key mappings.

System Grep

Vim has a :grep Ex command that uses the grep program on your system. It works quickly, but, since Vim is shelling out to the system, it is asynchronous. Vim will be unresponsive until the grep command is complete. For small to medium projects this probably isn't a problem. For larger projects this could lock up Vim for an unacceptable amount of time.

Another issue with this approach is the length of the command line. Grep does not search recursively by default. You must supply a command line argument (-R) to make it search through all of the directories in your project.

That isn't so bad, you are probably thinking to yourself, just adding one argument wouldn't be hard. The thing is, the -R argument looks at every sub-directory in your project. Including .git, node_modules and tmp to name a few. Search hits in these directories typically clog up the results with a bunch of garbage you don't care about.

There has to be a way to exclude directories right? Yes, grep has an --exclude-dir argument, and unfortunately there is no short version of the argument. You can add multiple directories by grouping them, comma separated, within brackets. The final :grep command would look something like the following.

:grep -Ri --exclude-dir={.git,node_modules,tmp} foo .

That is, find the pattern foo in all files -R starting at the current directory ., ignoring case -i, but exclude directories named .git, node_modules and tmp.

I don't know about you, but that seems like a lot of typing. Vim has a way that you can simplify the command quite a bit, but at the cost of flexibility. You can change the way Vim calls the system grep with the grepprg setting. If you wish to display the value of the setting type :set grepprg? inside Vim. Note the question mark at the end, that will display the setting rather than alter it. By default grepprg is:

grep -n $* /dev/null

The -n tells grep to add the line number inside the file that the pattern was found in. This allows Vim to move to that line when it opens the file. The $* is replaced with any arguments typed into the :grep Ex command. Honestly, I don't know what the /dev/null part of the command is for. I understand that anything sent to the /dev/null device is just thrown away. I just don't know why it is useful here. I spent a little time trying to search for an answer, but didn't turn up anything. (If anyone can shed light on this please feel free to contact me.)

You can change the default grepprg so that it includes the other arguments above; something like:

:set grepprg=grep -Rin --exclude-dir={.git,node_modules,tmp} $* /dev/null

That would work, and make project wide searches easier. However, it would force all :grep commands to be recursive and exclude those directories. That might not be what you want all of the time.

Key Mapping In Vim

Instead of overriding the built in grepprg setting we can create a key mapping with our long grep command arguments. This will allow us to continue to use the standard grepprg arguments for other search cases, and our special arguments just when we are searching our code.

I briefly toyed with the idea of trying to create a key mapping for the same shortcut that Atom and Textmate use, Cmd-Shift-F, but it seems there isn't a reliable way to refer to the Cmd key in Vim. It wouldn't be portable to other machines anyway, so I'll try to stick to the Vim Way ™.

I didn't have the Leader key set in my configuration files, so that is the first step. If you aren't aware of the Vim Leader key, it is a way to create additional Vim shortcuts without overriding exiting shortcuts. The Leader defaults to the \\ key, but most people remap it to the , key because it is easier to reach.

Add this to your .vimrc or init.vim file if you don't already have it.

let mapleader=","

Now we can create our mapping. The format of a Vim key map is as follows.

cmd [attr] key(s) command(s)

Map Commands

The cmd is one of the built in mapping commands. The basic commands are :map and :map!. The :map version creates a key mapping available in Normal, Visual and Operator Pending modes. Using :map! makes the key map available only in Insert and Command modes.

There are also variations for each specific mode, so creating a key map with :nmap will only be available in Normal mode. Creating a key map with :vmap will only be available in Visual mode.

By default key mappings are recursive. When you trigger a key mapping the assigned command will be run, that command can contain other key mappings. You can have a chain of key mappings all triggered together from a single key map. This can be handy in some situations, but it isn't what we are looking for here.

There are mapping command variations that do not try to recursively run other key mappings. The primary one is :noremap. Like :map it creates a key mapping for Normal, Visual and Operator Pending modes. There are also Mode specific versions, such as :nnoremap which is the Normal mode only version.

For more information about the various map commands look at :help map-commands.

Key Map Attributes

Next is the optional [attr] portion. If an attribute is specified it must be one of <buffer>, <nowait>, <silent>, <script>, <expr> and <unique>. These change the behavior of the key mapping in various ways. For example, the <buffer> attribute makes the mapping available only in the current buffer. This would allow you to use the same mapping for a different command in a different buffer.

The <silent> attribute tells Vim that the command associated with the key mapping should not be echoed in the command line. We will use this as part of our mapping, just to keep the Vim UI uncluttered.

Keys

Next comes the actual keystroke(s) you want to use to trigger the command. For regular character keys you can just use the actual key. They are case sensitive, so using F for the keystroke would actually require the shift key as well.

There are many system keys available as well. These are surrounded by < and > characters. To use the F1 key in a key mapping you would use .

I want to use the leader key as part of my key mapping. It can be written just like a system key, so f, which is what this key mapping will use, means to press the leader key, whatever it is set too, then the f key.

For a complete list of available system keys use :help key-codes.

Command

The command portion of a key mapping is the full command that you want to trigger when the key mapping is triggered. We've already specified the grep command pretty thoroughly above. It is just a matter of plugging it into the key mapping now.

Well, sort of. A couple modifications will come in handy. Instead of the :grep command we'll use :grep!. Adding the exclamation point to the command will keep if from jumping to the first file where the pattern is found. I'd rather see the list and then decide how to deal with it.

Getting access to the word that the cursor is currently sitting on is done with the operator. It just finds the current word and inserts it into the command at the position it is placed.

The final command is:

:grep! -Rin --exclude-dir={.git,node_modules,tmp,log} <cword> .

It is pretty close to what we had above. I added log to the list of excluded directories because it seemed like a likely candidate to add. Feel free to add any other directories you might need to exclude in your situation.

The Final Key Mapping

Here is the final key mapping. Add it to your .vimrc or init.vim file.

nnoremap <silent> <Leader>f :grep! -Rin --exclude-dir={.git,node_modules,tmp,log} <cword> .<Cr>:cw<Cr>

To use it move the cursor to a function name, constant, variable or any other identifier in a project you are working on. Make sure you are in Normal Mode and then press the ,f keys to trigger the grep command.

The :grep command populates Vim's Quickfix list, which is a buffer that is available to all open Vim windows. It can be opened with either the :cw or :copen commands. This is why the <Cr>:cw<Cr> bit was added to the :grep command above. The <Cr> system key is the return key. It completes the :grep command. Once the :grep command is done the rest of the command is run. At which point the :cw command is executed and the Quickfix list is opened.

Navigating the Quickfix List

There isn't enough room to fully cover the Quickfix list here, but this little bit should be enough to get you started. By default the Quickfix list will open in a horizontal split window at the bottom of the current Vim window.

You can move between Vim windows with the <Ctrl-w><w> shortcut. That really means to press the <Ctrl> and w keys together, then release both and then press the w key again. It works just as well if you continue to hold down the <Ctrl> key however. That is how I use the command. Press the <Ctrl> key and then tap the w key twice. That will move you through each open window one at a time. If you only have the Quickfix list and a file open it will just toggle between the two windows.

You don't necessarily need to move to the Quickfix list window though. You can leave the source code window active and use the :cnext, :cprev, :cfirst and :clast commands to activate items in the Quickfix list. When you use one of those movement commands it will highlight the current item in the Quickfix list and the selected file will be opened in a new buffer and positioned on the line where the pattern was matched.

If you are done with the Quickfix list you can close it with the :cclose command. For additional information about the Quickfix list look at :help quickfix in Vim.

That was a bit of a haul, but you should start to see how configurable Vim is. Every aspect of the editing environment is malleable, which makes it possible to mold the editor into the exact configuration you need.