I use Vim as my primary text editor.
Vim is not perfect. It is great in some aspects, but it sucks in others.
There are lots of IDEs out there, and some time ago I even tried to switch to an IDE, but I failed, even though most IDEs have Vim-emulation mode. Nothing else can give that true “talking-to-editor” feeling to me. But, well, Vim is not perfect.
One of the things I really miss is that there's no such thing as a “project” in Vim. Vim is a text-editor, not an IDE, and unfortunately in 2015 we still don't have a standard interface between a text editor and an IDE. That's the real shame in my opinion, and so, we have to struggle with rather hackish solutions all the time.
One of the things I undoubtedly need for my daily development is the code navigation: you know, the ability to quickly jump to some symbol's definition. When I moved to Vim about 5 years ago, I quickly found out there is no convenient way to do that: all of the existing solutions had some limitations.
The Vim's traditional way to perform the code navigation is by using tags. Tags are generated by the external utility Exuberant Ctags, which, while being rather simple parser, supports lots of languages.
Surely we can be pure geeks, and use ctags directly, like this:
$ ctags -R -f /path/to/output/tagfile /path/to/project
And then, in Vim, use our tags by typing:
:set tags=/path/to/output/tagfile
After this, Vim is aware of where project's symbols are located. We can now place our cursor on some symbol, press Ctrl+], and Vim will bring us to the symbol's definition. Or, we can just type :tag my_symbol_name.
It works. The problem is that generating tags manually is just a little bit inconvenient. And given that we actually have to re-generate tags every time we make any changes to source files, it becomes completely unacceptable.
So, that's what various plugins are for: they try to make the process of dealing with tags somewhat more convenient. But sadly, and surprisingly enough, I failed to find the solution that would satisfy me.
What I want is to be able to just specify where my project files are located (to generate tags for), and then forget about tags at all. Tags should be generated and re-generated automatically when appropriate, and the whole process should be completely transparent to the user (i.e. to me). Existing plugins, however, required me to perform too many actions to be really convenient.
In the absence of existing solutions, and taking into account my strong wish to keep using Vim, I had to roll my own.
I named my brand new plugin “Indexer”. It is hosted at bitbucket: Indexer.
Its primary job is to get a list of your project's files, generate tags for them, and when you save some file from your project, transparently update tags. Sounds like rather easy thing to do, but it turned out to be harder than it sounds:
Okay. But, before Indexer can proceed with its job, as I said before, it needs to “get a list of your project's files”. So, how does it do that?
We have two options:
At the moment of writing Indexer for the first time (i.e. in 2010), I was using the Project plugin. I don't use it these days, but still, Indexer can accompany the Project plugin pretty well. If you're like me today, and you don't use Project plugin, you can pretty much skip this section.
If, however, you use Project plugin, and your projects file is stored in the default location ~/.vimprojects, then you'll find that it's very easy to set Indexer up: no extra configuration is needed!
Indexer automatically parses your ~/.vimprojects, your tags are automatically generated and updated as necessary. Out of the box!
If your projects file is named differently, just point Indexer there by setting an option:
let g:indexer_projectsSettingsFilename = '/path/to/my/vimprojects'
But I believe the majority of readers don't use Project plugin, so, proceed to the next option:
If you don't use Project plugin, use Indexer's own file to define our projects. Its default location is ~/.indexer_files, and it may look like this:
[my_first_project] /path/to/first_project/src /path/to/first_project/some_other_src [my_second_project] /path/to/second_project/src
As you see, the format is very simple: we specify the name of the project in square brackets, and then specify one or more paths to where source files to generate tags for are located.
Whatever way we use to specify our projects, every time you open some file from any of your projects, Indexer will do whatever it takes to generate tags for the whole project, and set Vim's &tags appropriately. Multiple projects are handled correctly.
For clarity about how things work, let's consider little concrete example.
Assume we have two simple projects: first and second. They are located in /home/dimon/projects, and they have the following files:
/home/dimon/projects/first ├── main.c └── test.c /home/dimon/projects/second ├── main.c └── myfile.c
So, the first project has the following files:
int my_value = 5; int my_func(void) { return my_value; } int main(void) { return my_func(); }
int my_test(void) { return 10; }
Whereas the second project has:
extern int my_func(void); int main(void) { return my_func(); }
int my_value = 5; int my_func(void) { return my_value; }
And let's make Indexer aware of these pieces of art. Make sure our ~/.indexer_files contains the following:
[My first project] /home/dimon/projects/first [My second project] /home/dimon/projects/second
After you've edited the .indexer_files, it's better to restart Vim. Sorry about that! It's not very convenient, but it doesn't hurt much, since we edit it rarely.
Now, let's see Indexer in action! Open file from the first project: /home/dimon/projects/first/main.c. At this moment, Indexer will notice that opened file belongs to the project “My first project”, which is not yet opened. So, it will generate tags for the whole project (well, even though it consists of just a couple of files), save tags file as /home/dimon/.indexer_files_tags/My_first_project, and set &tags to this location.
Now, you can type in your Vim:
:tag my_value
And Vim will bring you where the my_value is defined in our first project.
I promised that multiple projects are supported as well, so, let's check that claim: open file from the second project: /home/dimon/projects/second/main.c. At this moment, Indexer will notice that opened file belongs to the project “My second project”, which is not yet opened. So, it will generate tags for all files under /home/dimon/projects/second, save tags file as /home/dimon/.indexer_files_tags/My_second_project, and set &tags to this location.
Now, you can type in your Vim:
:tag my_value
And Vim will bring you where the my_value is defined in our second project, that is, to myfile.c. As you see, two projects don't interfere with each other, even though they contain the same symbols. And more: after you typed :tag my_value, Vim has opened the file myfile.c, Indexer has noticed that opened file belongs to the project “My second project”, which is already opened, and active. So, no special actions were done: tags are not regenerated (since they're already up-to-date).
Okay, going further. Let's open file test.c from the first project. Then, Indexer notices that opened file belongs to the project “My first project”, which is already opened, but inactive. So, it will not generate any tags now, but instead just set &tags to already generated /home/dimon/.indexer_files_tags/My_first_project.
It just works!
Indexer updates tags for some particular project when you save any file from that project. However, as was mentioned before, on rather large projects it would take noticeable time to rebuild tags for the whole project (even in though tags are generated in background, it is inconvenient), so, Indexer does its best to avoid that.
On Linux and Mac, by default Indexer does not rebuild tags for the whole project. Instead, it removes tags for saved file by sed, and then runs ctags in append mode. This way, we don't have any stale tags (since they're removed by sed), all relevant tags are saved, and the whole thing works much faster.
However, on Windows, all versions of sed that I was able to find are buggy: one of them couldn't handle Windows line endings correctly, another one works “most of the time”, but sometimes corrupts tags file, etc.
After all, I gave up, and on Windows tags are rebuilt for the whole project every time you save every single file, by default. There is an option g:indexer_ctagsJustAppendTagsAtFileSave, if you want to change the default behaviour on any platform.
We need to make a note about background tags generation. First of all, you need your vim to be built with +servername. All popular pre-built binaries have this feature enabled, so you're unlikely to have troubles with it.
Then, it will work out-of-the-box for Gvim, but not for terminal Vim. If you want it to work in terminal, you should run your Vim like this:
$ vim --servername MY_SERVERNAME
What is the servername? I'll explain briefly: in order to achieve background tags generation, Indexer has to run ctags asynchronously: separate process with ctags is spawn, Vim goes to do its own business, and later, when ctags process is done, Vim gets notified about that, and Indexer proceeds further.
The crucial part is to talk to running Vim about something (in this case - about finished ctags process). For this to be done, the running instance of Vim should have some servername: that way, we can even work with multiple running Vim instances.
Gvim is started with default servername GVIM, so, it works by default. I have no idea why terminal vim doesn't have servername set.
Okay, now things work, but I actually find it deeply wrong that we have some centralized ~/.indexer_files (or ~/.vimprojects), instead of keeping that data in the project repository. More, if we move our project somewhere, Indexer will stop working for that project, since ~/.indexer_files needs to be updated as well! That's not what I like.
Well, the problem is actually much deeper and more general than this particular case with Indexer plugin: in Vim, we have no way to store per-project options. Again, Vim is “just” a text editor, and it doesn't know what the “project” is, at all. So, while struggling with it, I had to come up with one more plugin: Vimprj.
The idea is quite simple: in the root of our project directory, we create the .vimprj directory, in which we can store any number of *.vim files (usually, 1 file is enough) with project-specific settings. When we open some file in Vim, Vimprj walks up by tree, looking for the .vimprj directory. For each .vimprj found, Vimprj sources all .vimprj/*.vim files, and continues to go up, looking further for other .vimprj, until it reaches the root of filesystem.
Example tree may look like this:
my_project ├── .vimprj │ └── my.vim ├── main.c └── any_other_file.c
Of course, it is optimized: if we open some file from the location for which we have already applied settings, the .vimprj/*.vim files won't be sourced again. But if we switch project (open some file from different location, with its own .vimprj), then, of course, all settings will be re-applied.
The easiest use-case for Vimprj is probably an indentation options. Assume I'd like to use 4-space indent in my projects. No tabs, exactly four spaces. One day I need to work on another project written by someone else, and there's 2-space indent. Or maybe tabs.
Sounds like a perfect job for Vimprj!
So, in my own projects, I create a file .vimprj/my.vim:
let &tabstop = 4 let &shiftwidth = 4 set expandtab
And in the project with 2-space indentation, I put the following settings:
let &tabstop = 2 let &shiftwidth = 2 set expandtab
Now, every time I open any file under my_project, the &tabstop and &shiftwidth options will be set to 4. When I open any file from other_project, these options will be set to 2. Very convenient, no pain! More, we can put whatever other project-specific settings here: some mappings, other plugins' settings.. Whatever. And we can nest projects: for example, we may have some “environment” project with things that are common for inner projects. And each particular project can set more precise settings. We'll talk about that later, in the section about Indexer's subprojects.
Okay, that sounds good. But actually, with the current design, we have an issue. Can you spot it?
Assume I open file from my_project: tab is 4-space. Good, now, open some file from other_project: tab is 2-space. Still good! And now, open some file that is not contained in any project. Oops.
Of course, I'd like tab to be 4-space, since it is my preferred settings. But, as you might have guessed, with the design described above, it's still 2-space: nothing to source to get 4-space settings, so, the last applied settings are still in effect. That leads us to the fact that we want to have some default settings.
Vimprj provides hooks for other plugins. For instance, Indexer (since version 4.0) uses these hooks to achieve correct behaviour when user works on different project simultaneously.
At the moment, I have not documented yet all these hooks. I'm going to tell you about just one hook, which lets us specify our default options: SetDefaultOptions. Typical usage is to put code like this into .vimrc:
function! <SID>SetMainDefaults() " your default options set tabstop=4 set shiftwidth=4 set expandtab endfunction " apply defaults right now call <SID>SetMainDefaults() " initialize vimprj plugin call vimprj#init() " define a hook function! g:vimprj#dHooks['SetDefaultOptions']['main_options'](dParams) call <SID>SetMainDefaults() endfunction
Now, Vimprj will call our function SetMainDefaults() just before sourcing all .vimprj/*.vim files, as well as when you open file not from any
project. In any words, when we're going to leave current project.
For convenience, Vimprj also sets the variable $VIMPRJ_PROJECT_ROOT that points to the root of the currently active project.
Let's recall what we have started with: we wanted to get rid of centralized .indexer_files, which we can't even include in the project's repository.
The solution is to have separate .indexer_files for each project, and refer to it from our .vimprj/my.vim file. We can put .indexer_files pretty much anywhere inside our project, but I prefer to store it right into .vimprj dir. It feels natural.
So, let's try to refactor our first project from the examples above, so that it doesn't depend on central ~/.indexer_files. We end up with the following tree:
first ├── .vimprj │ ├── .indexer_files │ └── my.vim ├── main.c └── test.c
And our .vimprj/my.vim should contain the following settings for Indexer:
" path to .vimprj folder let s:sVimprjPath = expand('<sfile>:p:h') " point Indexer to our local .indexer_files let g:indexer_indexerListFilename = s:sVimprjPath.'/.indexer_files' " TODO: here may be any other project-specific settings, such as tabstop, etc
And in our .indexer_files, we don't have to use absolute filenames anymore: it's much more flexible to take advantage of the $VIMPRJ_PROJECT_ROOT variable, which is carefully set by Vimprj for us. So, new .indexer_files looks like this:
[My first project] $VIMPRJ_PROJECT_ROOT
Now, we can (but not have to) remove your old record of the first project from the central ~/.indexer_files. And let's check it: open some file from our first project!
$ gvim /home/dimon/projects/first/main.c
This time, Indexer will use our local file .vimprj/.indexer_files. You can verify that by issuing the :IndexerInfo command:
* Indexer version: 4.15 * Ctags version: Exuberant Ctags 5.9~svn20110310, Copyright (C) 1996-2009 Darren Hiebert * Filelist: indexer file: /home/dimon/projects/first/.vimprj/.indexer_files * Index-mode: DIRS. (option g:indexer_ctagsDontSpecifyFilesIfPossible is ON) * At file save: remove tags for saved file by SED, and just append tags * Background tags generation: YES * Projects indexed: My first project * Root paths: /home/dimon/projects/first * Paths for ctags: /home/dimon/projects/first * Files for ctags: * Paths (with all subfolders): .,/usr/include,,,/home/dimon/projects/first, * Tags file: ./tags,./TAGS,tags,TAGS,/home/dimon/projects/first/.vimprj/.indexer_files_tags/My_first_project
Among others, it shows the filelist file being used: indexer file: /home/dimon/projects/first/.vimprj/.indexer_files.
And with this setup, tags file is saved into first/.vimprj/.indexer_files_tags/My_first_project, that is, under the project directory. So, I always put tags directory to my .hgignore list, like this:
vimprj[/\\].+_tags
Now, it's much better, isn't it? All necessary project information is kept into repository, and we can move our project in just any place in our filesystem, it will just work.
If you use Project plugin, then you probably want to do the same with .vimprojects: store it in your project's tree, instead of using central ~/.vimprojects.
On the Indexer part, it's as easy as for .indexer_files: you just put your .vimprojects to the directory .vimprj, and in your .vimprj/my.vim point Indexer to it:
" path to .vimprj folder let s:sVimprjPath = expand('<sfile>:p:h') " point Indexer to our local .vimprojects let g:indexer_projectsSettingsFilename = s:sVimprjPath.'/.vimprojects' " TODO: here may be any other project-specific settings, such as tabstop, etc
It's enough for Indexer, but we also want the Project plugin to use our local .vimprojects, don't we?
Unfortunately, the author of the Project plugin, Aric Blumer, decided not to provide an option to set the path to .vimprojects; instead, the only way to use different file is to use :Project /path/to/vimprojects command, which actually opens a Project window with specified file opened.
So, in our .vimprj/my.vim, we can add the following:
" open our local vimprojects file in Project plugin exec "Project ".s:sVimprjPath.'/.vimprojects'
But this way, the :Project command will be executed each time we switch the project, causing Project window to open, which is not very convenient. I'd still prefer to just set the variable with path to .vimprojects. For example, I never actually type :Project command; instead, I use mapping for toggle project window, like this:
nmap <silent> <F9> <Plug>ToggleProject
So I just hit F9, and project window is opened or closed. And I want it to open or close the project that is stored in the variable.
I asked Aric Blumer to provide this simple functionality, but he refused by answering that I'm the first person who asks about this. Sounds pretty strange to me, but then, I had to hack on Project plugin a bit, as it's just a matter of a few lines. A diff command:
$ diff -u project.vim project_new.vim
--- project.vim 2006-10-13 17:47:08.000000000 +0400
+++ project_new.vim 2015-10-12 11:38:10.923919092 +0300
@@ -1269,7 +1269,11 @@
if !exists("*<SID>DoToggleProject()") "<<<
function! s:DoToggleProject()
if !exists('g:proj_running') || bufwinnr(g:proj_running) == -1
- Project
+ if !exists("g:proj_running") && exists('g:proj_project_filename')
+ exec('Project '.g:proj_project_filename)
+ else
+ Project
+ endif
else
let g:proj_mywindow = winnr()
Project
When this patch is applied, we can use the variable g:proj_project_filename. All in all, our .vimprj/my.vim looks like this:
" path to .vimprj folder let s:sVimprjPath = expand('<sfile>:p:h') " point Indexer to our local .vimprojects let g:indexer_projectsSettingsFilename = s:sVimprjPath.'/.vimprojects' " point Project to our local .vimprojects let g:proj_project_filename = s:sVimprjPath.'/.vimprojects' " TODO: here may be any other project-specific settings, such as tabstop, etc
This way, I open some file from my project, hit F9, and Project opens my local .vimprojects.
Well, at the moment, Indexer does not support sub-projects. But the good news is that we can work around this and get what we need with the flexibility of Vimprj! Let's look at how it is done.
Remember that in our .vimprj/*.vim files we can set any options. So, the main idea is to manually set up tags of needed libraries. Like this:
set tags+=/path/to/some/lib1/tags set tags+=/path/to/some/lib2/tags
For rather large projects, where I have lots of libraries, I usually have the “environment” repository, which includes several library sub-repositories, together with a main project (as just one more sub-repository). This technique described, for example, in the Mercurial's wiki.
So, let's assume we have the project myproj that uses a couple of libraries. As described above, we'll have the environment directory (let's name it myproj_env), which will have both libraries and the myproj itself. We end up with the following hierarchy:
myproj_env
├── lib1
│ ├── lib1.c
│ └── .vimprj
│ ├── .indexer_files
│ └── my.vim
├── lib2
│ ├── src
│ │ └── lib2.c
│ └── .vimprj
│ ├── .indexer_files
│ └── my.vim
├── myproj
│ ├── main.c
│ ├── test.c
│ └── .vimprj
│ ├── .indexer_files
│ └── my.vim
└── .vimprj
└── env.vim
Notice that the environment directory myproj_env has its own .vimprj. We use it to set up variables with paths to libraries:
" path to .vimprj dir let s:sVimprjPath = expand('<sfile>:p:h') " path to project dir let s:sProjectPath = simplify(s:sVimprjPath.'/..') " paths to all libraries let $VIMPRJ_ENV__PATH__LIB_1 = s:sProjectPath."/lib1" let $VIMPRJ_ENV__PATH__LIB_2 = s:sProjectPath."/lib2" " paths to all libraries tags (generated by Indexer) let $VIMPRJ_ENV__PATH__LIB_1__TAGS = $VIMPRJ_ENV__PATH__LIB_1."/.vimprj/.indexer_files_tags/lib1" let $VIMPRJ_ENV__PATH__LIB_2__TAGS = $VIMPRJ_ENV__PATH__LIB_2."/.vimprj/.indexer_files_tags/lib2"
As you see, we specify paths to tag files ($VIMPRJ_ENV__PATH__LIB_1__TAGS, $VIMPRJ_ENV__PATH__LIB_2__TAGS), so that we can use these variables in our myproj_env/myproj/.vimprj/my.vim:
" path to .vimprj folder let s:sVimprjPath = expand('<sfile>:p:h') " point Indexer to our local .indexer_files let g:indexer_indexerListFilename = s:sVimprjPath.'/.indexer_files' " use libraries tags exec "set tags+=".$VIMPRJ_ENV__PATH__LIB_1__TAGS exec "set tags+=".$VIMPRJ_ENV__PATH__LIB_2__TAGS " setup indentation options for project set tabstop=4 set shiftwidth=4 set expandtab
This way, when we open any file from myproj, tags for libraries will be used by Vim.
Each library has its own .vimprj with .indexer_files and my.vim, which are very simple (just like ones for the first project, in the section Store .indexer_files inside .vimprj). You may find the whole working example in the Indexer repository, doc/examples/vimprj_subprojects.
I must admit that such an implementation of sub-projects is a way too hackish. I see two clear drawbacks:
myproj, tags for libraries won't be generated automatically. We have to open each library in Vim, so that Indexer will generate tags for each of them;env.vim, we have to specify exact path to tags file, like $VIMPRJ_ENV__PATH__LIB_2."/.vimprj/.indexer_files_tags/lib2", which is an implementation detail of Indexer actually.Maybe one day Indexer will support sub-projects internally, and then, all of these inconveniences will be out. However, at the moment, it's much better than nothing: once we set things up and generated tags for all libraries, the whole thing works pretty nice.
Sometimes, it makes sense to fine-tune options that Indexer gives to ctags. For example, we may want limit ctags to generate tags only for files of specific type, and ignore everything else. I'm going to show an example of my actual .indexer_files for C project:
[some_project] option:ctags_params = "--langmap=c:.c.h --languages=c" $VIMPRJ_PROJECT_ROOT
As you see, we've just added option:ctags_params = “….” option after the project name.
The meaning of the options given is as follows:
–languages=c limits ctags to generate files for C files only.–langmap=c:.c.h specifies that .c and .h files should be treated as C files. This is needed because ctags treats .h files as C++ by default.
Full list of ctags options can be found here: http://ctags.sourceforge.net/ctags.html. It may be worth examining; at least, I find the aforementioned options languages and langmap very useful.
There is a convenient trick I use often: instead of specifying project name in square brackets manually, we can ask Indexer to use directory name instead. Consider:
[%dir_name(..)%] $VIMPRJ_PROJECT_ROOT
So, we can use %dir_name(/path/to/dir)%, where path is relative to path of the .indexer_files. For example, if we use such a trick for our first project above, Indexer will assume that the project is named first.
This trick is useful when you just copy your .indexer_files to other projects: with the dir_name() trick, you don't have to adjust .indexer_files for each project.
All of the above is quite nice, and it allows me to set up my projects accordingly to my needs. However, it sounds like an overkill if I occasionally download some third-party project, and want to just quickly peek at the source code, being able to navigate it: I have to create .vimprj directory inside with my.vim and .indexer_files… I'm far too lazy.
Instead, I've implemented simple feature: we can specify that some directory contains different projects. Then, Indexer will treat every directory inside as a separate project, without any additional change to .indexer_files!
For such third-party projects, I use the directory ~/projects/workspace. I just download some-cool-project, and save it as ~/projects/workspace/some-cool-project.
And in my generic ~/.indexer_files, I have the following lines:
[PROJECTS_PARENT] /home/dimon/projects/workspace
That's all! The key here is a special “project name”: PROJECTS_PARENT. Every single directory inside workspace is now treated by Indexer as a separate project. I download some new project to workspace, restart the Vim, and open any file from newly saved project. Indexer generates tags for it, and I can navigate the code immediately. That's simple, eh?
By the way, this is the only thing I use generic ~/.indexer_files for.
Again, sorry for obliging you to restart Vim. When it starts, Indexer fetches the list of all directories under PROJECTS_PARENT, and this list remains statically for the whole Vim runtime. I hope I'll find time to remove this limitation in the future.
Indexer sucks at really large projects (for example, Linux Kernel). When project is so large, ctags may pretty much run for several minutes while generating tags. And even though we use trick with sed and ctags -a when saving file, anyway it works a way too slow to be useful. If you get your processor loaded 100% for 30-60 seconds at every file save, it's not good at all.
More, by default, Indexer will re-generate tags when the project is opened for the first time in the particular Vim session (since Indexer has no reliable way to check whether tags are up-to-date: checking dates of every single file in vimscript will probably be even slower than just generate new tags).
So, when I use Indexer to navigate Linux Kernel, I turn this option:
let g:indexer_dontUpdateTagsIfFileExists = 1
Then, if tags file already exists, Indexer will not re-generate it.
But luckily, my projects aren't that big, so, Indexer works quite well for me.
You need three plugins:
I've written these plugins a long time ago: the first version of Indexer was released in 2010. They are not written in particularly elegant way, but they work for me very well: the time has proven that Indexer + Vimprj is a pretty decent solution for small- and medium-sized projects.
They work on Linux, Mac and Windows.
They are also hosted at vim.org:
If you like them, you may vote for them there.
I hope that this article will help you to get started quickly. For details on each plugin, refer to the help:
:help indexer :help vimprj
If you have any questions or other feedback, feel free to leave a comment below.