Wednesday, May 11, 2011

Add-in Start-up Time in Visual Studio 2010

justcode-fastWhile I was in Dallas last week for a quick DNUG speaking tour, a good question came-up regarding the Telerik tools that extend Visual Studio: Should you expect them to add time to the Visual Studio loading time?

Clearly, different products integrate with Visual Studio in different ways (thus impacting the VS startup process differently), but we have invested focused energy making sure the Telerik tools do not significantly degrade the VS start-up (or runtime) process. As a result, the general answer to the question is that Telerik tools should not add more than a few seconds to VS start-up time.

But I decided a more specific answer was in order.

To more fully answer this question, I did a series of tests with my local copy of Visual Studio 2010 Ultimate to see how much different VS add-ins impacted the overall Visual Studio startup time.

Add-in Time Tests

The goal: understand how different add-ins installed on my machine impact the "baseline" Visual Studio start-up time. Since every computer configuration is going to impact the absolute values of a test like this, the most important thing to compare is the relative difference to the baseline. The baseline in this case is Visual Studio started in Safe Mode (devenv /safemode), which automatically disables any and every VS add-in.

MEASURING THE START-UP TIME
I tried a couple of different approaches for measurement. I wanted to use something precise that could take the measurements for me (like AppTimer), but no automatic solution could reliably determine when exactly Visual Studio was truly "loaded" and ready for action. So, a manual stopwatch was employed with precision dexterity to start the timer and run the devenv process simultaneously. (Multiple tests were conducted, too, to minimize error, of course.)

THE RAW TEST RESULTS
There is not much rhyme or reason to the add-ins that I measured (beyond the Q1 2011 Telerik tools) other than that the plug-ins were already running on my machine. So here are the results (times are rounded since millisecond precision here isn't that meaningful):

VS ADD-INS ENABLED Impact on Baseline Start-up Time
(in seconds)
Baseline (/safemode) -- (for the record, 1.8 sec)
No Plugins (w/o Start Page) +1 (with Start Page, +2)
Only JustMock -- (No measurable impact)
Only JustTrace --
Only JustCode +4
JustCode, JustMock, JustTrace +4
Only OpenAccess +1.5
JC,JM,JT,OA +5
NON-TELERIK ADD-INS  
Only BuildIncrementVersion -- (No measurable impact)
Only VisualGit +1
Only Chirpy --
Only .NET Reflector Pro +1
Only VMWare VMDebugger +10!

Out of curiosity (since I know many of you are), I decided to also test the latest versions of ReSharper and CodeRush/Refactor to see how they comparably impacted VS load time. Same test method. Results:

VS ADD-INS ENABLED Impact on Baseline Start-up Time
(in seconds)
Only ReSharper 5.1 +3.5*
Only CodeRush/Refactor 10.2.8 +3.5*

*These times need explanation. More on that below.

Analyzing the Start-up Test Results

There are few quick takeaways from these basic tests:

  • Many add-ins, including JustMock, JustTrace, Chripy, and BuildIncrementVersion have virtually no impact on the Visual Studio start-up time. As you might expect, though, these add-ins are doing very little to really integrate deeply into the VS IDE.
  • VMWare's VS add-in is a HUGE offender. It added over 10 seconds to raw startup time! I didn't realize this 'til my tests. You can bet this add-in will stay disabled from now on unless I need it!
  • JustCode only adds about 4 seconds to VS start time, which is roughly on par with similar VS productivity plug-ins.

I did observe that ReSharper and CodeRush seem to "lazy load," so while Visual Studio appears to open quickly, it remains unresponsive after opening while these plug-ins run through their loading routines. JustCode, meanwhile, fully loads during VS startup, so as soon as VS opens, it's ready to go. Ultimately, the add-ins all add roughly the same time to the VS start-up, just in different places. You can be sure Telerik will continue to invest heavily in optimizing the JustCode start-up routines so we can continue to set the new standard in Visual Studio productivity tools.

--

At the end of the day, I'm satisfied with these results. I don't launch instances of Visual Studio that often during a day (usually open and leave open), and an additional 4 or 5 seconds seems like a minimal start-up delay for all of the time JustCode, JustMock, and OpenAccess will save during development (especially since JustCode doesn't add crippling, time-sapping "lag" to Visual Studio like some add-ins). We'll keep innovating, though, and find new ways to make our impact event smaller.

In the meantime, if you're not seeing results like this, and your VS is taking a long time to load, look for other non-Telerik add-ins (VMWare?) or try "resetting" your VS by booting once in Safe Mode (this seems to work for me when VS load times get oddly slow over time). Hope this helps!

(And in case you missed it, be sure to check-out the top 10 ways JustCode makes Visual Studio better!)

4 comments:

Anonymous said...

Yes, but, I found that JustCode slows down my working after the load. I have a Solution with 33 projects and a fair amout of code. Often when I make a change, I have to wait, sometimes for more that 30 seconds for the UI to be available again. When I unistall JustCode the problem goes away. I do not have this issue with ReSharper. I prefer JustCode from a functional standpoint but can't use it from a performance standpoint.

Todd Anglin said...

@Anon- Sorry to hear you're having trouble like that with JustCode. It is *HIGHLY* unusual due to the way JustCode is built. Unlike ReSharper, which does all of its work inside the Visual Studio devenv process, JustCode does all of its work in a separate process and only runs a small, lightweight layer in VS for communicating the results of the analysis. In general, this means JustCode is architected to avoid "Visual Studio lag," and generally it's the best of the bunch at being transparent in VS.

We obviously would love to know more about your scenario so we can figure-out what's going on. Be sure to use our Support Ticket system or shoot me an email (anglin [at]) so we can learn more.

-Todd

Eric Schoenholzer said...

My experience is the opposite. JustCode performs so much better than ReSharper. I deinstalled R# because of Performance and price policy, and I'm using happily JC, which gets better with each update.

Brandon said...

I second Eric's comment - I've used ReSharper off and on for a couple of years now, and always end up turning it off when I want to use it for something (e.g., refactoring) and then turning it off as soon as I'm done because the run-time impact on Visual Studio 2008 is obscene (using Resharper 5, Visual Studio 2008 SP1, with a solution of 73 C# projects, 4 C++/CLI projects, and 2 native C++ projects). I wouldn't say that I "barely" notice that JustCode is running, but the impact is _very_ slight - Visual studio remains snappy. I'm very glad to have a good tool to use in our code base, which admittedly, is much larger and more complicated than most in JetBrain's target demographic.