Vim: convenient code navigation for your projects

I use Vim as my primary text editor.

Vim is not perfect. It is great in some aspects, but it sucks in others.

There are lots of IDEs out there, and some time ago I even tried to switch to an IDE, but I failed, even though most IDEs have Vim-emulation mode. Nothing else can give that true “talking-to-editor” feeling to me. But, well, Vim is not perfect.

One of the things I really miss is that there's no such thing as a “project” in Vim. Vim is a text-editor, not an IDE, and unfortunately in 2015 we still don't have a standard interface between a text editor and an IDE. That's the real shame in my opinion, and so, we have to struggle with rather hackish solutions all the time.

Code navigation

One of the things I undoubtedly need for my daily development is the code navigation: you know, the ability to quickly jump to some symbol's definition. When I moved to Vim about 5 years ago, I quickly found out there is no convenient way to do that: all of the existing solutions had some limitations.

The Vim's traditional way to perform the code navigation is by using tags. Tags are generated by the external utility Exuberant Ctags, which, while being rather simple parser, supports lots of languages.

Surely we can be pure geeks, and use ctags directly, like this:

$ ctags -R -f /path/to/output/tagfile /path/to/project

And then, in Vim, use our tags by typing:

:set tags=/path/to/output/tagfile

After this, Vim is aware of where project's symbols are located. We can now place our cursor on some symbol, press Ctrl+], and Vim will bring us to the symbol's definition. Or, we can just type :tag my_symbol_name.

It works. The problem is that generating tags manually is just a little bit inconvenient. And given that we actually have to re-generate tags every time we make any changes to source files, it becomes completely unacceptable.

So, that's what various plugins are for: they try to make the process of dealing with tags somewhat more convenient. But sadly, and surprisingly enough, I failed to find the solution that would satisfy me.

What I want is to be able to just specify where my project files are located (to generate tags for), and then forget about tags at all. Tags should be generated and re-generated automatically when appropriate, and the whole process should be completely transparent to the user (i.e. to me). Existing plugins, however, required me to perform too many actions to be really convenient.

In the absence of existing solutions, and taking into account my strong wish to keep using Vim, I had to roll my own.

Meet Indexer

I named my brand new plugin “Indexer”. It is hosted at bitbucket: Indexer.

Its primary job is to get a list of your project's files, generate tags for them, and when you save some file from your project, transparently update tags. Sounds like rather easy thing to do, but it turned out to be harder than it sounds:

  • We don't want our Vim to hang while tags are being generated, so, ctags should run asynchronously;
  • Re-building tags for the whole project every time we perform a little change in every single file might work too slow, so, Indexer tries to be smarter;
  • We should be careful not to run out of the command maximum length limitation: at least, on Windows 7, we have about 8 kB limitation, which is very few. On Linux, we have about 2 MB, which is not impossible to run out of as well;
  • We should take platform differences in account;
  • etc.

Okay. But, before Indexer can proceed with its job, as I said before, it needs to “get a list of your project's files”. So, how does it do that?

We have two options:

Accompany the "Project" plugin

At the moment of writing Indexer for the first time (i.e. in 2010), I was using the Project plugin. I don't use it these days, but still, Indexer can accompany the Project plugin pretty well. If you're like me today, and you don't use Project plugin, you can pretty much skip this section.

If, however, you use Project plugin, and your projects file is stored in the default location ~/.vimprojects, then you'll find that it's very easy to set Indexer up: no extra configuration is needed!

Indexer automatically parses your ~/.vimprojects, your tags are automatically generated and updated as necessary. Out of the box!

If your projects file is named differently, just point Indexer there by setting an option:

let g:indexer_projectsSettingsFilename = '/path/to/my/vimprojects'

But I believe the majority of readers don't use Project plugin, so, proceed to the next option:

Use Indexer's own file .indexer_files

If you don't use Project plugin, use Indexer's own file to define our projects. Its default location is ~/.indexer_files, and it may look like this:

~/.indexer_files
[my_first_project]
 
/path/to/first_project/src
/path/to/first_project/some_other_src
 
 
[my_second_project]
 
/path/to/second_project/src

As you see, the format is very simple: we specify the name of the project in square brackets, and then specify one or more paths to where source files to generate tags for are located.

Whatever way we use to specify our projects, every time you open some file from any of your projects, Indexer will do whatever it takes to generate tags for the whole project, and set Vim's &tags appropriately. Multiple projects are handled correctly.

For clarity about how things work, let's consider little concrete example.

Simple example workflow

Assume we have two simple projects: first and second. They are located in /home/dimon/projects, and they have the following files:

/home/dimon/projects/first
├── main.c
└── test.c

/home/dimon/projects/second
├── main.c
└── myfile.c

So, the first project has the following files:

/home/dimon/projects/first/main.c
int my_value = 5;
 
int my_func(void)
{
    return my_value;
}
 
int main(void)
{
    return my_func();
}
/home/dimon/projects/first/test.c
int my_test(void)
{
    return 10;
}

Whereas the second project has:

/home/dimon/projects/second/main.c
extern int my_func(void);
 
int main(void)
{
    return my_func();
}
/home/dimon/projects/second/myfile.c
int my_value = 5;
 
int my_func(void)
{
    return my_value;
}

And let's make Indexer aware of these pieces of art. Make sure our ~/.indexer_files contains the following:

~/.indexer_files
[My first project]
/home/dimon/projects/first
 
[My second project]
/home/dimon/projects/second

After you've edited the .indexer_files, it's better to restart Vim. Sorry about that! It's not very convenient, but it doesn't hurt much, since we edit it rarely.

Now, let's see Indexer in action! Open file from the first project: /home/dimon/projects/first/main.c. At this moment, Indexer will notice that opened file belongs to the project “My first project”, which is not yet opened. So, it will generate tags for the whole project (well, even though it consists of just a couple of files), save tags file as /home/dimon/.indexer_files_tags/My_first_project, and set &tags to this location.

Now, you can type in your Vim:

:tag my_value

And Vim will bring you where the my_value is defined in our first project.

I promised that multiple projects are supported as well, so, let's check that claim: open file from the second project: /home/dimon/projects/second/main.c. At this moment, Indexer will notice that opened file belongs to the project “My second project”, which is not yet opened. So, it will generate tags for all files under /home/dimon/projects/second, save tags file as /home/dimon/.indexer_files_tags/My_second_project, and set &tags to this location.

Now, you can type in your Vim:

:tag my_value

And Vim will bring you where the my_value is defined in our second project, that is, to myfile.c. As you see, two projects don't interfere with each other, even though they contain the same symbols. And more: after you typed :tag my_value, Vim has opened the file myfile.c, Indexer has noticed that opened file belongs to the project “My second project”, which is already opened, and active. So, no special actions were done: tags are not regenerated (since they're already up-to-date).

Okay, going further. Let's open file test.c from the first project. Then, Indexer notices that opened file belongs to the project “My first project”, which is already opened, but inactive. So, it will not generate any tags now, but instead just set &tags to already generated /home/dimon/.indexer_files_tags/My_first_project.

It just works!

How tags are updated

Indexer updates tags for some particular project when you save any file from that project. However, as was mentioned before, on rather large projects it would take noticeable time to rebuild tags for the whole project (even in though tags are generated in background, it is inconvenient), so, Indexer does its best to avoid that.

On Linux and Mac, by default Indexer does not rebuild tags for the whole project. Instead, it removes tags for saved file by sed, and then runs ctags in append mode. This way, we don't have any stale tags (since they're removed by sed), all relevant tags are saved, and the whole thing works much faster.

However, on Windows, all versions of sed that I was able to find are buggy: one of them couldn't handle Windows line endings correctly, another one works “most of the time”, but sometimes corrupts tags file, etc.

After all, I gave up, and on Windows tags are rebuilt for the whole project every time you save every single file, by default. There is an option g:indexer_ctagsJustAppendTagsAtFileSave, if you want to change the default behaviour on any platform.

Background tags generation

We need to make a note about background tags generation. First of all, you need your vim to be built with +servername. All popular pre-built binaries have this feature enabled, so you're unlikely to have troubles with it.

Then, it will work out-of-the-box for Gvim, but not for terminal Vim. If you want it to work in terminal, you should run your Vim like this:

$ vim --servername MY_SERVERNAME

What is the servername? I'll explain briefly: in order to achieve background tags generation, Indexer has to run ctags asynchronously: separate process with ctags is spawn, Vim goes to do its own business, and later, when ctags process is done, Vim gets notified about that, and Indexer proceeds further.

The crucial part is to talk to running Vim about something (in this case - about finished ctags process). For this to be done, the running instance of Vim should have some servername: that way, we can even work with multiple running Vim instances.

Gvim is started with default servername GVIM, so, it works by default. I have no idea why terminal vim doesn't have servername set.

Okay, now things work, but I actually find it deeply wrong that we have some centralized ~/.indexer_files (or ~/.vimprojects), instead of keeping that data in the project repository. More, if we move our project somewhere, Indexer will stop working for that project, since ~/.indexer_files needs to be updated as well! That's not what I like.

Keep project-specific data in the project's directory

Well, the problem is actually much deeper and more general than this particular case with Indexer plugin: in Vim, we have no way to store per-project options. Again, Vim is “just” a text editor, and it doesn't know what the “project” is, at all. So, while struggling with it, I had to come up with one more plugin: Vimprj.

Meet Vimprj

The idea is quite simple: in the root of our project directory, we create the .vimprj directory, in which we can store any number of *.vim files (usually, 1 file is enough) with project-specific settings. When we open some file in Vim, Vimprj walks up by tree, looking for the .vimprj directory. For each .vimprj found, Vimprj sources all .vimprj/*.vim files, and continues to go up, looking further for other .vimprj, until it reaches the root of filesystem.

Example tree may look like this:

my_project
├── .vimprj
│   └── my.vim
├── main.c
└── any_other_file.c

Of course, it is optimized: if we open some file from the location for which we have already applied settings, the .vimprj/*.vim files won't be sourced again. But if we switch project (open some file from different location, with its own .vimprj), then, of course, all settings will be re-applied.

The easiest use-case for Vimprj is probably an indentation options. Assume I'd like to use 4-space indent in my projects. No tabs, exactly four spaces. One day I need to work on another project written by someone else, and there's 2-space indent. Or maybe tabs.

Sounds like a perfect job for Vimprj!

So, in my own projects, I create a file .vimprj/my.vim:

my_project/.vimprj/my.vim
    let &tabstop = 4
    let &shiftwidth = 4
    set expandtab

And in the project with 2-space indentation, I put the following settings:

other_project/.vimprj/my.vim
    let &tabstop = 2
    let &shiftwidth = 2
    set expandtab

Now, every time I open any file under my_project, the &tabstop and &shiftwidth options will be set to 4. When I open any file from other_project, these options will be set to 2. Very convenient, no pain! More, we can put whatever other project-specific settings here: some mappings, other plugins' settings.. Whatever. And we can nest projects: for example, we may have some “environment” project with things that are common for inner projects. And each particular project can set more precise settings. We'll talk about that later, in the section about Indexer's subprojects.

Okay, that sounds good. But actually, with the current design, we have an issue. Can you spot it?

Assume I open file from my_project: tab is 4-space. Good, now, open some file from other_project: tab is 2-space. Still good! And now, open some file that is not contained in any project. Oops.

Of course, I'd like tab to be 4-space, since it is my preferred settings. But, as you might have guessed, with the design described above, it's still 2-space: nothing to source to get 4-space settings, so, the last applied settings are still in effect. That leads us to the fact that we want to have some default settings.

Default settings

Vimprj provides hooks for other plugins. For instance, Indexer (since version 4.0) uses these hooks to achieve correct behaviour when user works on different project simultaneously.

At the moment, I have not documented yet all these hooks. I'm going to tell you about just one hook, which lets us specify our default options: SetDefaultOptions. Typical usage is to put code like this into .vimrc:

.vimrc
function! <SID>SetMainDefaults()
 
    " your default options
    set tabstop=4
    set shiftwidth=4
    set expandtab
 
endfunction
 
" apply defaults right now
call <SID>SetMainDefaults()
 
" initialize vimprj plugin
call vimprj#init()
 
" define a hook
function! g:vimprj#dHooks['SetDefaultOptions']['main_options'](dParams)
    call <SID>SetMainDefaults()
endfunction

Now, Vimprj will call our function SetMainDefaults() just before sourcing all .vimprj/*.vim files, as well as when you open file not from any project. In any words, when we're going to leave current project.

Project root

For convenience, Vimprj also sets the variable $VIMPRJ_PROJECT_ROOT that points to the root of the currently active project.

Store .indexer_files inside .vimprj

Let's recall what we have started with: we wanted to get rid of centralized .indexer_files, which we can't even include in the project's repository.

The solution is to have separate .indexer_files for each project, and refer to it from our .vimprj/my.vim file. We can put .indexer_files pretty much anywhere inside our project, but I prefer to store it right into .vimprj dir. It feels natural.

So, let's try to refactor our first project from the examples above, so that it doesn't depend on central ~/.indexer_files. We end up with the following tree:

first
├── .vimprj
│   ├── .indexer_files
│   └── my.vim
├── main.c
└── test.c

And our .vimprj/my.vim should contain the following settings for Indexer:

.vimprj/my.vim
" path to .vimprj folder
let s:sVimprjPath = expand('<sfile>:p:h')
 
" point Indexer to our local .indexer_files
let g:indexer_indexerListFilename = s:sVimprjPath.'/.indexer_files'
 
" TODO: here may be any other project-specific settings, such as tabstop, etc

And in our .indexer_files, we don't have to use absolute filenames anymore: it's much more flexible to take advantage of the $VIMPRJ_PROJECT_ROOT variable, which is carefully set by Vimprj for us. So, new .indexer_files looks like this:

.vimprj/.indexer_files
[My first project]
 
$VIMPRJ_PROJECT_ROOT

Now, we can (but not have to) remove your old record of the first project from the central ~/.indexer_files. And let's check it: open some file from our first project!

$ gvim /home/dimon/projects/first/main.c

This time, Indexer will use our local file .vimprj/.indexer_files. You can verify that by issuing the :IndexerInfo command:

* Indexer version: 4.15
* Ctags version: Exuberant Ctags 5.9~svn20110310, Copyright (C) 1996-2009 Darren Hiebert
* Filelist: indexer file: /home/dimon/projects/first/.vimprj/.indexer_files
* Index-mode: DIRS. (option g:indexer_ctagsDontSpecifyFilesIfPossible is ON)
* At file save: remove tags for saved file by SED, and just append tags
* Background tags generation: YES
* Projects indexed: My first project
* Root paths: /home/dimon/projects/first
* Paths for ctags: /home/dimon/projects/first
* Files for ctags: 
* Paths (with all subfolders): .,/usr/include,,,/home/dimon/projects/first,
* Tags file: ./tags,./TAGS,tags,TAGS,/home/dimon/projects/first/.vimprj/.indexer_files_tags/My_first_project

Among others, it shows the filelist file being used: indexer file: /home/dimon/projects/first/.vimprj/.indexer_files.

And with this setup, tags file is saved into first/.vimprj/.indexer_files_tags/My_first_project, that is, under the project directory. So, I always put tags directory to my .hgignore list, like this:

.hgignore
vimprj[/\\].+_tags

Now, it's much better, isn't it? All necessary project information is kept into repository, and we can move our project in just any place in our filesystem, it will just work.

Store .vimprojects inside .vimprj

If you use Project plugin, then you probably want to do the same with .vimprojects: store it in your project's tree, instead of using central ~/.vimprojects.

On the Indexer part, it's as easy as for .indexer_files: you just put your .vimprojects to the directory .vimprj, and in your .vimprj/my.vim point Indexer to it:

.vimprj/my.vim
" path to .vimprj folder
let s:sVimprjPath = expand('<sfile>:p:h')
 
" point Indexer to our local .vimprojects
let g:indexer_projectsSettingsFilename = s:sVimprjPath.'/.vimprojects'
 
" TODO: here may be any other project-specific settings, such as tabstop, etc

It's enough for Indexer, but we also want the Project plugin to use our local .vimprojects, don't we?

Unfortunately, the author of the Project plugin, Aric Blumer, decided not to provide an option to set the path to .vimprojects; instead, the only way to use different file is to use :Project /path/to/vimprojects command, which actually opens a Project window with specified file opened.

So, in our .vimprj/my.vim, we can add the following:

" open our local vimprojects file in Project plugin
exec "Project ".s:sVimprjPath.'/.vimprojects'

But this way, the :Project command will be executed each time we switch the project, causing Project window to open, which is not very convenient. I'd still prefer to just set the variable with path to .vimprojects. For example, I never actually type :Project command; instead, I use mapping for toggle project window, like this:

nmap <silent> <F9> <Plug>ToggleProject

So I just hit F9, and project window is opened or closed. And I want it to open or close the project that is stored in the variable.

I asked Aric Blumer to provide this simple functionality, but he refused by answering that I'm the first person who asks about this. Sounds pretty strange to me, but then, I had to hack on Project plugin a bit, as it's just a matter of a few lines. A diff command:

$ diff -u project.vim project_new.vim  
--- project.vim	2006-10-13 17:47:08.000000000 +0400
+++ project_new.vim	2015-10-12 11:38:10.923919092 +0300
@@ -1269,7 +1269,11 @@
 if !exists("*<SID>DoToggleProject()") "<<<
     function! s:DoToggleProject()
         if !exists('g:proj_running') || bufwinnr(g:proj_running) == -1
-            Project
+            if !exists("g:proj_running") && exists('g:proj_project_filename')
+                exec('Project '.g:proj_project_filename)
+            else
+                Project
+            endif
         else
             let g:proj_mywindow = winnr()
             Project

When this patch is applied, we can use the variable g:proj_project_filename. All in all, our .vimprj/my.vim looks like this:

.vimprj/my.vim
" path to .vimprj folder
let s:sVimprjPath = expand('<sfile>:p:h')
 
" point Indexer to our local .vimprojects
let g:indexer_projectsSettingsFilename = s:sVimprjPath.'/.vimprojects'
 
" point Project to our local .vimprojects
let g:proj_project_filename = s:sVimprjPath.'/.vimprojects'
 
" TODO: here may be any other project-specific settings, such as tabstop, etc

This way, I open some file from my project, hit F9, and Project opens my local .vimprojects.

Indexer tweaks and tricks

Sub-projects (libraries)

Well, at the moment, Indexer does not support sub-projects. But the good news is that we can work around this and get what we need with the flexibility of Vimprj! Let's look at how it is done.

Remember that in our .vimprj/*.vim files we can set any options. So, the main idea is to manually set up tags of needed libraries. Like this:

set tags+=/path/to/some/lib1/tags
set tags+=/path/to/some/lib2/tags

For rather large projects, where I have lots of libraries, I usually have the “environment” repository, which includes several library sub-repositories, together with a main project (as just one more sub-repository). This technique described, for example, in the Mercurial's wiki.

So, let's assume we have the project myproj that uses a couple of libraries. As described above, we'll have the environment directory (let's name it myproj_env), which will have both libraries and the myproj itself. We end up with the following hierarchy:

myproj_env
├── lib1
│   ├── lib1.c
│   └── .vimprj
│       ├── .indexer_files
│       └── my.vim
├── lib2
│   ├── src
│   │   └── lib2.c
│   └── .vimprj
│       ├── .indexer_files
│       └── my.vim
├── myproj
│   ├── main.c
│   ├── test.c
│   └── .vimprj
│       ├── .indexer_files
│       └── my.vim
└── .vimprj
    └── env.vim

Notice that the environment directory myproj_env has its own .vimprj. We use it to set up variables with paths to libraries:

myproj_env/.vimprj/env.vim
" path to .vimprj dir
let s:sVimprjPath = expand('<sfile>:p:h')
 
" path to project dir
let s:sProjectPath = simplify(s:sVimprjPath.'/..')
 
" paths to all libraries
let $VIMPRJ_ENV__PATH__LIB_1    = s:sProjectPath."/lib1"
let $VIMPRJ_ENV__PATH__LIB_2    = s:sProjectPath."/lib2"
 
" paths to all libraries tags (generated by Indexer)
let $VIMPRJ_ENV__PATH__LIB_1__TAGS  = $VIMPRJ_ENV__PATH__LIB_1."/.vimprj/.indexer_files_tags/lib1"
let $VIMPRJ_ENV__PATH__LIB_2__TAGS  = $VIMPRJ_ENV__PATH__LIB_2."/.vimprj/.indexer_files_tags/lib2"

As you see, we specify paths to tag files ($VIMPRJ_ENV__PATH__LIB_1__TAGS, $VIMPRJ_ENV__PATH__LIB_2__TAGS), so that we can use these variables in our myproj_env/myproj/.vimprj/my.vim:

myproj_env/myproj/.vimprj/my.vim
" path to .vimprj folder
let s:sVimprjPath = expand('<sfile>:p:h')
 
" point Indexer to our local .indexer_files
let g:indexer_indexerListFilename = s:sVimprjPath.'/.indexer_files'
 
" use libraries tags
exec "set tags+=".$VIMPRJ_ENV__PATH__LIB_1__TAGS
exec "set tags+=".$VIMPRJ_ENV__PATH__LIB_2__TAGS
 
" setup indentation options for project
set tabstop=4
set shiftwidth=4
set expandtab

This way, when we open any file from myproj, tags for libraries will be used by Vim.

Each library has its own .vimprj with .indexer_files and my.vim, which are very simple (just like ones for the first project, in the section Store .indexer_files inside .vimprj). You may find the whole working example in the Indexer repository, doc/examples/vimprj_subprojects.

I must admit that such an implementation of sub-projects is a way too hackish. I see two clear drawbacks:

  • When we've just cloned our repositories, and open some file from myproj, tags for libraries won't be generated automatically. We have to open each library in Vim, so that Indexer will generate tags for each of them;
  • In the env.vim, we have to specify exact path to tags file, like $VIMPRJ_ENV__PATH__LIB_2."/.vimprj/.indexer_files_tags/lib2", which is an implementation detail of Indexer actually.

Maybe one day Indexer will support sub-projects internally, and then, all of these inconveniences will be out. However, at the moment, it's much better than nothing: once we set things up and generated tags for all libraries, the whole thing works pretty nice.

Fine-tuning ctags options

Sometimes, it makes sense to fine-tune options that Indexer gives to ctags. For example, we may want limit ctags to generate tags only for files of specific type, and ignore everything else. I'm going to show an example of my actual .indexer_files for C project:

[some_project]
option:ctags_params = "--langmap=c:.c.h --languages=c"

$VIMPRJ_PROJECT_ROOT

As you see, we've just added option:ctags_params = “….” option after the project name.

The meaning of the options given is as follows:

  • –languages=c limits ctags to generate files for C files only.
  • –langmap=c:.c.h specifies that .c and .h files should be treated as C files. This is needed because ctags treats .h files as C++ by default.

Full list of ctags options can be found here: http://ctags.sourceforge.net/ctags.html. It may be worth examining; at least, I find the aforementioned options languages and langmap very useful.

Let's be lazy: let Indexer figure out project's name

There is a convenient trick I use often: instead of specifying project name in square brackets manually, we can ask Indexer to use directory name instead. Consider:

[%dir_name(..)%]

$VIMPRJ_PROJECT_ROOT

So, we can use %dir_name(/path/to/dir)%, where path is relative to path of the .indexer_files. For example, if we use such a trick for our first project above, Indexer will assume that the project is named first.

This trick is useful when you just copy your .indexer_files to other projects: with the dir_name() trick, you don't have to adjust .indexer_files for each project.

Let's be even more lazy: PROJECTS_PARENT

All of the above is quite nice, and it allows me to set up my projects accordingly to my needs. However, it sounds like an overkill if I occasionally download some third-party project, and want to just quickly peek at the source code, being able to navigate it: I have to create .vimprj directory inside with my.vim and .indexer_files… I'm far too lazy.

Instead, I've implemented simple feature: we can specify that some directory contains different projects. Then, Indexer will treat every directory inside as a separate project, without any additional change to .indexer_files!

For such third-party projects, I use the directory ~/projects/workspace. I just download some-cool-project, and save it as ~/projects/workspace/some-cool-project.

And in my generic ~/.indexer_files, I have the following lines:

[PROJECTS_PARENT]

/home/dimon/projects/workspace

That's all! The key here is a special “project name”: PROJECTS_PARENT. Every single directory inside workspace is now treated by Indexer as a separate project. I download some new project to workspace, restart the Vim, and open any file from newly saved project. Indexer generates tags for it, and I can navigate the code immediately. That's simple, eh?

By the way, this is the only thing I use generic ~/.indexer_files for.

Again, sorry for obliging you to restart Vim. When it starts, Indexer fetches the list of all directories under PROJECTS_PARENT, and this list remains statically for the whole Vim runtime. I hope I'll find time to remove this limitation in the future.

Indexer limitations

Large projects

Indexer sucks at really large projects (for example, Linux Kernel). When project is so large, ctags may pretty much run for several minutes while generating tags. And even though we use trick with sed and ctags -a when saving file, anyway it works a way too slow to be useful. If you get your processor loaded 100% for 30-60 seconds at every file save, it's not good at all.

More, by default, Indexer will re-generate tags when the project is opened for the first time in the particular Vim session (since Indexer has no reliable way to check whether tags are up-to-date: checking dates of every single file in vimscript will probably be even slower than just generate new tags).

So, when I use Indexer to navigate Linux Kernel, I turn this option:

let g:indexer_dontUpdateTagsIfFileExists = 1

Then, if tags file already exists, Indexer will not re-generate it.

But luckily, my projects aren't that big, so, Indexer works quite well for me.

Installation

You need three plugins:

  • DFUtil: simple library which contains some common code for my plugins. I don't think it will be useful for anyone but me, so, just install it as a dependency of Indexer and Vimprj, and it won't disturb you. I'm thinking of getting rid of this dependency, but at the moment, it's still there.

Conclusion

I've written these plugins a long time ago: the first version of Indexer was released in 2010. They are not written in particularly elegant way, but they work for me very well: the time has proven that Indexer + Vimprj is a pretty decent solution for small- and medium-sized projects.

They work on Linux, Mac and Windows.

They are also hosted at vim.org:

If you like them, you may vote for them there.

I hope that this article will help you to get started quickly. For details on each plugin, refer to the help:

:help indexer
:help vimprj

If you have any questions or other feedback, feel free to leave a comment below.

Discussion

Romick, 2015/10/24 04:38

Hello. Mistype?

`Assume I'd like to use 4-space indent in my projects. No tabs, exactly three spaces.'' 3?

Dmitry Frank, 2015/10/24 07:54

Hi Romick, good catch, thanks! I used to have 3 spaces before, and after I switched to 4 spaces, I failed to adjust the text properly.

Martin Baute, 2016/01/19 13:14

I just found this, and I am impressed.

However, since I had no use for pathogen before, this requires the installation of *four* seperate plugins. Plus, the documentation nicely leads me through the “thought process” why things are necessary this way, and all the options, but does not give a Summary / Quick Start if all I want is setting this up real quick.

Together with a failed first attempt at getting this to run, I am afraid I will pass for now.

A single vimball plus quick-start docs, that's what I would love to see.

Michal, 2016/03/03 12:44

Hi Dmitry, Your setup is amazing :) thanks for sharing! Now I am trying to add something like tagbar into my setup but tagbar is generating its own tag files and I am also not sure if its not interfere somehow with indexer and vimprj setup. Do you have some ideas what would be some reasonable solution to have list of variables, macros, etc on the side? Thanks ;)

Dmitry Frank, 2016/03/04 18:17

Hi Michael! Thanks for the comment, I'm glad you liked the setup. :)

As to tagbar, it does not interfere with Indexer, since it does not actually change &tags variable in Vim. Some time ago I tried to make it use my custom tags file(s), but IIRC I failed to make such setup, and I just use tagbar as is is: after all, it is already pretty decent.

Michal, 2016/03/06 22:29

Thanks for response. Yes I did the same finally. I would have one more question. What would be the best approach if I would like to search for all occurrences of the variable in the project? If I just search for it, e.g. variable “Data” I am also getting “DataAlg” etc. Do you have some neat tactics? :) Btw your whole blog such and inspirational :) THANKS!

Dmitry Frank, 2016/03/07 06:57

Yeah, good question: I also want to search the whole word quite often.

Well, first of all, in my vimrc, I have set variable g:vimprj_changeCurDirIfVimprjFound to 1, so that vimprj changes current working directory to the currently active vimprj root.

Then, I use plugin EasyGrep: https://github.com/dkprice/vim-easygrep ; it has a few keymappings, one of them is <leader>vV, which takes a word under cursor, and looks for this whole word starting from current working directory. Alternatively, the command :Grep Data does the same.

You might not like the fact that your cwd gets changed though; probably there is a way to set EasyGrep so that it uses some different path; but I'm not sure how to do that. I'm fine with the volatile cwd.

Michal, 2016/03/07 11:40

This setup looks really nice. I just encounter another issue :D if I have some additional plugin open on the side like minimap and do <Leader>-vV the results are opened on the slim right minibuffer window. It's quite nasty so I have to figure out how to force it to open in main window :)

Dmitry Frank, 2016/03/08 17:10

Sadly I have no idea off the top of my head. Let me now if you get is sorted, please! :)

Michal Gonda, 2016/03/11 14:51

Just for info, I tried easygrep but behave strangely so I gave up and just made my own vimprj friendly searching function with vimgrep and some globbing according to my needs. I usually working on C and python projects so this is so far all I need but it can be easily extended. Here it is if someone is interested:

"Search in project function:
function SearchMyProject()
    if (&ft == "c")
        let filetype = ".[hc]"
    elseif (&ft == "cpp")
        let filetype = ".[hc]"
    elseif (&ft == "python")
        let filetype = ".py"
    elseif (&ft == "sh")
        let filetype = ".sh"
    else
        let filetype = ""
    endif

    if ( $VIMPRJ_PROJECT_ROOT != "" )
        :execute "vimgrep /" . expand("<cword>") . "/j " . expand($VIMPRJ_PROJECT_ROOT) . "/**/*" . expand(filetype) | cw
    else
        if ( filetype == "" )
            :execute "vimgrep /" . expand("<cword>") . "/gj " . expand("%") | cw
        else
            :execute "vimgrep /" . expand("<cword>") . "/j *" . expand(filetype) | cw
        endif
    endif
endfunction

"Search across files:
nnoremap <C-f> :call SearchMyProject()<CR>
nnoremap <Leader>k :cn<CR>
nnoremap <Leader>j :cp<CR>
"because I don’t use <C-f> for page down ever, I use <C-u> and <C-d> which does about
"half a page at a time. So I can have the cursor on a word, hit <C-f> and it’ll search
"the project for me, and open the ‘quickfix’ list with the results, then I can use
"<Leader>j and <Leader>k to go to next / previous hit.
Sergey, 2016/10/17 09:03

I have found an issue.

I have next folder:

/home/user/workspace/projects/project1

where 'projects' is symbolic link on /media/user/projects (in my case it's another disk)

I add in .indexer_files :

[PROJECT1]

/home/user/workspace/projects/project1

When I open file from project1 tags file isn't generated!

But when I change .indexer_files :

[PROJECT1]

/media/user/projects/project1

all works fine!

May be it's not indexer issue but vim issue.

Dmitry Frank, 2016/10/17 09:15

Thanks for reporting that. Yeah, at the time of writing this plugin, I didn't care about symlinks, and recently I myself observed some weird behaviour w.r.t symlinks. Hopefully I'll get some time to fix it.

Nathan, 2016/10/24 13:06

Hi Dmitry,

Thanks for the inspiring combination of Vimprj and Indexer! It gives a solid base to build my own project workflow. I would like to propose a new feature for the Indexer: external / system header files indexing.

For example: I'm using the Qt5 libraries and would like to have these header files also indexed. But at the moment the tags will only be generated when opening a file “within” the project definition. It would be great to be able to force the indexing to generate the missing tags file.

Currently I'm generating the Qt5 tags manually. In my .vimprj/project.vim file I just add the path to the Qt5 tags file to the current tags. See example:

" External tags
let s:tags_qt = g:indexer_indexerListFilename . '_tags/tags-qt'
exec "set tags+=" . s:tags_qt

Thanks.

Dmitry Frank, 2016/10/24 18:40

Hi Nathan,

Yeah I agree it would be cool, I also suffer from exactly the same problem :) Hopefully I'll get some time to implement it, but no ETA at the moment.

Thanks for comment!

Nathan, 2016/10/27 14:56

Hi Dmitry,

After spending some time with the indexer plugin, I was able to add support for “external” projects. An external project is a directory which resides outside of the main project directory hierarchy. For example: the Qt5 (system) header files which I frequently use.

I'll explain how it works by showing my “indexer_files” file:

[%dir_name(..)%]

option:ctags_params = "--languages=c++,c"



$VIMPRJ_PROJECT_ROOT



[tags-qt]

option:ctags_params = "--languages=c++,c"

option:external = "static"



/usr/include/i386-linux-gnu/qt5


The main “project” (containing the actual code to develop) is defined as usual.

The second “project” called “tags-qt” is an external one. This is configured by the option:external setting:

  • static
  • dynamic

Tags of a static project will be created only when not created before. A dynamic project will follow the rules of a “normal” project. Eg. will be updated at startup unless the option g:dontUpdateTagsIfFileExists is set.

What do you think about this approach? If you like I can provide you a patch (via github?)

Dmitry Frank, 2016/10/27 17:33

Hi Nathan, thanks!

However, this particular design doesn't really fit into existing picture.

When I was thinking of the way to implement some kind of “library project”, I was planning that library project is going to have pretty regular section (without options like external). And now, project which uses library should indicate which library projects it wants to include, and how.

This way, we could have library project descriptor specified just once, in some file:

[tags-qt]
option:ctags_params = "--languages=c++,c"

/usr/include/i386-linux-gnu/qt5

And also we need some way to include other files into .indexer_files.

Dmitry Frank, 2016/10/27 21:13

So, in the main project (which uses a library), we're going to have something like this:

# somehow, include the file which defines tags-qt project
# (not sure about the exact syntax)
include /path/to/file/with/tags-qt/indexer_files

[%dir_name(..)%]
option:ctags_params = "--languages=c++,c"
option:library_project = "tags-qt,static"

$VIMPRJ_PROJECT_ROOT

What do you think?

Nathan, 2016/10/28 11:44

Hi Dmitry,

The approach of my current solution is a bit “rude”, but gets it done™. So I understand that it won't fit into the design of the indexer. An “include” is a much nicer solution 8-).

I'll spend some more time with the indexer to see if I can come up with an “include” approach.

Grtz,
Nathan.

Dmitry Frank, 2016/10/28 12:03

Cool, looking forward! :) Please let me know if there are some news on it.

I was planning to move repositories to GitHub, it seems now is the appropriate time. I hope I'll do that in the near future, so that it'll be easier for people to collaborate.

Thanks for your efforts!

Nathan, 2016/11/04 13:17

Hi Dmitry,

I have moved the include “block” inside of the project definition section like this:

Filename: “indexer_files”

[%dir_name(..)%]
include indexer_qt5

option:ctags_params = "--languages=c++,c"

$VIMPRJ_PROJECT_ROOT

This allows the project to contain the include(s).

The part after the “include” is a relative or absolute filename of the to be included indexer file. Relative includes are relative from the main indexer file. In this case relative from the “indexer_files” file (both files are in the same directory).

Filename: “indexer_qt5”

[tags_qt5]
option:ctags_params = "--exclude=*.cpp --exclude=3rdparty --file-scope=no --languages=c++,c"

/usr/include/i386-linux-gnu/qt5

For an included indexer file the tags will be generated / updated at startup only. Just like done for the main project.

Additional the paths to the generated (include) tags files are appended to the vim tags path:

tags=./tags,./TAGS,tags,TAGS,~/Projects/test/.vimprj/indexer_files_tags/test,~/Projects/test/.vimprj/indexer_files_tags/tags_qt5

What do you think of this approach?

In the meantime I'm going to setup a github project to share my changes.

Grtz,
Nathan.

Nathan, 2016/11/04 13:45

See https://github.com/nhuizing/indexer.tar.gz/tree/feature-include for the changes.

Nathan.

Dmitry Frank, 2016/11/04 15:54

Hi Nathan,

Thank you very much for your contribution,

I've created a real repo for indexer: https://github.com/dimonomid/indexer could you please re-create your fork from it instead of the one created by vimscripts? Sorry for the inconvenience :(

I'll review the changes tomorrow since today I'm busy with the company's work.

Natha, 2016/11/04 19:41

Done 8-)

https://github.com/nhuizing/indexer/tree/feature/include

Nathan.

Andrey, 2016/12/24 22:08

Hi Dmitry

Sorry in advance, my question is quite offtopic. However it relates to the indexing and project management in IDEs.

What approach do you use to get code auto-completion for your projects?

Of course vim allows to complete words from opened buffers. However sometimes it doesn't fit.

Dmitry Frank, 2016/12/25 09:06

Hi Andrey,

Obviously it depends on the language: e.g. for Java, I use eclim and run Eclipse in parallel; it provides a quick and flawless autocompletion, and it is awesome.

For C/C++, a very long time ago, I started with a tags-based approach: OmniCppComplete, but it sucks at more or less complicated things.

Then I switched to slightly modified version of clang_complete; it wasn't trivial to adjust for embedded projects (which can't be really compiled with clang), but it worked good enough.

But when I started working in Cesanta, I didn't bother to adjust clang_complete for the new codebase, so now I live without C/C++ autocompletion. Maybe, one day…

I've heard many good things about YouCompleteMe, but never tried it for real yet.

Enter your comment (please, English only). Wiki syntax is allowed:
   ____ __  __   ____ __  __ ______
  / __/ \ \/ /  /  _/ \ \/ //_  __/
 / _/    \  /  _/ /    \  /  / /   
/___/    /_/  /___/    /_/  /_/
 
articles/vim_project_code_navigation.txt · Last modified: 2015/11/07 19:28 by dfrank
Driven by DokuWiki Recent changes RSS feed Valid CSS Valid XHTML 1.0