LavaBlast Software Blog

Help your franchise business get to the next level.
AddThis Feed Button

Slash your ASP.NET compile/load time without any hard work

clock December 1, 2010 09:18 by author JKealey

Jason has a cool eye :) FranchiseBlast, our franchise management platform, is a ‘large’ solution inside Visual Studio which currently contains 35 projects. In total, we manage over 60 interrelated projects, and have always been concerned at improving the compilation/load performance on our development machines. This post is a quick summary of what we did to keep things as fast as possible.

Personally, when it starts taking over a minute to compile or load my application, I start throwing things. This was the case early this week and with a bit of work, I managed to cut down things by an order of magnitude (from around 140 seconds down to around 14 seconds) with only software changes.  This is what prompted me to write this post.

There are three things worth improving:

  1. Compilation time
  2. First load time (ASP.NET)
  3. Application speed / database performance

 

I only want to talk about the first two; the only tip I’ll give you for #3 is to get your hands on a profiler such as dotTrace as it is a real time saver. 

Get better hardware (Big impact)

You’ll get the best bang for your buck by upgrading your hard disk, especially if you’re using a single 5400 RPM or 7200 RPM drive (download benchmark software to evaluate your current disk). Our projects are stored on a solid state disk. We currently use Intel X25-M G2, but the RevoDrive x2 looks much faster (basically a RAID-0 array of SSDs) if you have an available PCI-Express x4 slot. If you’re cheap and find the SSDs to be too small, just get two large 7200 RPM drives and put them in RAID-0. Make sure you’ve got a robust backup solution.

We have 12GB RAM on our development machines and a recent Core i7 processor, but nothing fancy. This is more than sufficient.

Store your temporary IIS files on your fastest disk or a RAM disk

Depending on the amount of RAM you have, it may make sense to use a RAM disk. I use my RAM disk for my temporary internet files and for IIS’s temporary folder (compilation results). I haven’t measured specific performance details but since I have so much free RAM, might as well try use it in creative ways.

To speed up the first load time, you can tell IIS to store its temporary files on your RAM disk (or fastest disk) by changing the following setting in your web.config files:

<compilation ... tempDirectory="q:\temp\iistemp\"> ... </compilation>

 

You can either change your project files directly, or, if you’ve lazy and have numerous applications running on your development machine (like I do), update the system-wide web.config files. Note that you need to update this for each runtime version of the Framework and, if running a 64-bit machine, for both Framework and Framework64. On my machine, I needed to modify the following files:

   1:  C:\Windows\Microsoft.NET\Framework\v2.0.50727\CONFIG\Web.config
   2:  C:\Windows\Microsoft.NET\Framework\v4.0.30319\Config\Web.config
   3:  C:\Windows\Microsoft.NET\Framework64\v2.0.50727\CONFIG\Web.config
   4:  C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Config\Web.config

 

Trade-off: If you save your RAM disk when shutting down, you’ll notice how much longer it takes to reboot. I can live with that, rebooting only once every couple months.

Review a few magical settings (Most impact)

When an ASP.NET website is loaded for the first time, it pre-compiles all your pages and user controls. Once done, everything runs faster. This is great for production websites, but horrible for your development machine. Why?  When programming, you’re usually only modifying a page or two (or back-end code). You’ll iteratively make a change, compile, launch the website, test, and start over; often dozens of times. A two minute compile/load time (like we had) forces you to lose focus and get distracted. The following setting makes pre-compilation more selective, making the first load time massively faster in development scenarios. On my machine, it cut the first load time from around 74 seconds to 6 seconds.

<compilation ... batch="false"> ...</compilation>

 

While on the subject of random boolean flags that make your life better, I should mention the following:

<compilation ... optimizeCompilations="true"> ... </compilation>

This flag has a number of gotcha’s that are documented here. I’ve enabled it for now, although my quick tests didn’t show any significant performance improvements compared to the previous one. However, I was probably not testing the right thing.

Restructure your projects

Try multiple solutions

About a year ago, we took 50% of our least used projects and moved them into a secondary solution. We compile that solution only when changing one of these projects, once every couple months. After rebuilding the secondary solution, we copy the *.dll files to a separate folder. Our primary projects reference those pre-generated libraries. I should repeat this process again, but I like having all my projects in one place and splitting them was a pain. However, this is probably the only sustainable way to manage extremely large solutions.

Selectively build the necessary projects

Using Visual Studio’s Configuration Manager, I also created different active solution configurations in my primary solution with only a subset of the projects included in the build.  I can thus swap between configurations depending on my primary task (working on our point of sale instead of FranchiseBlast, for example). This can drastically reduce compilation time, but I don’t use it as often as I should because:

  1. I find the user interface slow to refresh (Visual Studio).
  2. My dependency tree is complex and going through the list of projects to remove those I don’t want built is a pain.
  3. I context-switch a lot and often hit cases where I modify a project and forget that is not included in the build.

 

A simpler option is to build only the current project (and its dependencies) instead of all projects. Just make sure you don’t accidentally break the build (recompile everything before a commit).

Parallel compilation (Big impact)

Following Scott Hanselman’s post, I setup my project to compile in parallel. I did add a few options to make it easier for me to see errors/warnings without MSBuild’s typical clutter. (I also had a post-build action using NAnt which I set to quiet). This reduced my full rebuild time from 96 seconds down to 16 (after a clean solution). In a typical recompilation, we’re talking roughly 8 seconds instead of around 66.

Title: Parallel Build
Command: C:\Windows\Microsoft.NET\Framework\v4.0.30319\MSBuild.exe
Arguments: /m $(SolutionFileName) /v:m /ds /nologo /clp:Summary;Verbosity=minimal
Initial Directory: $(SolutionDir)
Check: Use Output window.

Trade-off: Error messages aren’t nicely presented in the Visual Studio Error List tab and you are not notified when the build is complete. 

Other noteworthy attempts

Over time, I’ve tried a few things that didn’t work out. In general, they improved things slightly but I don’t use them on a daily basis due to the tradeoffs.

I tried putting the whole project (source and output folders) onto the RAM disk but its volatility scared me away, regardless of the performance enhancements (Off the top of my head, it was between 25% and 50%). I then tried putting my only project’s bin folders on my RAM disk (by creating symbolic links from my bin folders to my RAM disk). This also had a positive impact on performance, but not significant enough to warrant the kludge (around 25% reduction in compilation time).

I also found a few tips & tricks for larger projects on Stack Overflow. First, I tried putting ‘Copy Local’ to false for all project references. This gave me a 25% reduction in compilation time, but broke my deployment scripts which needed all the files in the bin folder.  Separately, I configured all my projects Output Paths to the same folder, avoiding content duplication on the disk. This also gave me a 25% reduction in compilation time. Oddly enough, moving this folder to my RAM disk did not impact performance.

Summary

I hope this post gave you some ideas on how to improve your compilation speeds and first load times. I didn’t intend to give exact benchmarks, as performance will vary greatly depending on your projects. However, the main lesson learned is that there are dozens of improvements you can make; it’s up to you to try them out and keep what works for you.

The top three time savers for us are:

  1. Setting batch=false in the ASP.NET configuration page. (Now ~10 times faster)
  2. Parallel compilation for our projects (Now ~8 times faster)
  3. Better hardware (Now between 2 and 5 times faster)

 

Honorable mention goes to splitting your solution files, even though it’s painful and time consuming process to setup.

Do you have any other tips & tricks you’d like to share with us?



Scripting an ASP.NET installation in Win2k3

clock March 27, 2008 08:26 by author JKealey

A month ago, I posted an article on a few console commands for managing ASP.NET applications and IIS. In the weeks that followed, I was contacted by my old friends at iWeb Technologies to help them automate their ASP.NET setup. I spent a couple hours creating the required scripts and I thought I'd post my real world example here!

By the way, iWeb is an exceptional web hoster (1TB of space, unmetered traffic, unlimited domains, $3 a month, and excellent technical support; what more could a web developer want?). I've been with them for nine years already!

Scripting the creation of a new website and configuring it for ASP.NET

Task: Given a domain name (lavablast.com) and a network/local path (\\fileserver\hosting\www.lavablast.com\web\ or c:\inetpub\wwwroot\www.lavablast.com\web\), setup IIS so that ASP.NET works on the root of the domain.

  1. Create an application pool
  2. Create a web site and associate it to the application pool
  3. Ensure both that the www subdomain works (www.lavablast.com and lavablast.com should load this website)
  4. Enable ASP.NET v2.0 on this website.
  5. Give ASP.NET read/write access to the folder.
  6. Add default.aspx to the default documents.

 

I ended up using the ADSUTIL.VBS script for most of these tasks. I used the iisweb command to create the web application, but it doesn't support network paths. I create it using a temp local path and end up using adsutil to change it to a network path. 

My googling skills are what made this task so short. Here are some of my references:

Finally, the IISBatchUtils collection of scripts provided the most help. Here's why. 

I have created batch files to make adding new websites to the server very quick and easy. The only tricky part about setting up new websites from batch file (or the command line) is that Microsoft's ADSUTL utility does not correctly add host headers like it says it does- rather than appending the new headers, it blindly sticks them in and possibly covers existing host headers that might already be set up.

I used both the original adsutil, Josh's modified adsutil, and some of his code to extract the site id from the IIS Metabase by using the site name.

The script

See the actual script for the variable definitions.

:Create
REM Step 1 - Create Application Pool
CSCRIPT //nologo %IWEB_ADSUTIL% CREATE w3svc/AppPools/%IWEB_DOMAIN% IIsApplicationPool
 
REM Step 2 - Create WebServer
iisweb /create %TEMP% %IWEB_DOMAIN% /d %IWEB_DOMAIN% /ap %IWEB_DOMAIN%
 
REM Step 3- Find new SiteID for further scripting. 
cscript //nologo iisbatchutils/translate.js "%IWEB_DOMAIN%" > siteid.txt
for /f %%I in (siteid.txt) do SET IWEB_SITEID=%%I
 
REM Step 4 - Add www. to site URL - uses their own custom adsutil because of bug in the normal one. 
cscript %IWEB_ADSUTIL2% append w3svc/%IWEB_SITEID%/serverbindings ":80:www.%IWEB_DOMAIN%"
 
REM Step 5 - set various website permissions. 
cscript //nologo %IWEB_ADSUTIL% set w3svc/%IWEB_SITEID%/accessread "true"
cscript //nologo %IWEB_ADSUTIL% set w3svc/%IWEB_SITEID%/accesswrite "true"
cscript //nologo %IWEB_ADSUTIL% set w3svc/%IWEB_SITEID%/root/AppFriendlyName %IWEB_DOMAIN%
cscript //nologo %IWEB_ADSUTIL% set w3svc/%IWEB_SITEID%/root/path %IWEB_Path%
cscript //nologo %IWEB_ADSUTIL% set w3svc/%IWEB_SITEID%/root/DefaultDoc Default.htm,Default.asp,index.htm,Default.aspx,index.asp,index.html
 
REM Step 6 - Cleanup
del siteid.txt
 
GOTO End

 

What about setting up IIS to use ASP.NET v2.0?

I did find a link showing me how to change the ASP.NET version from v1.1 to v2.0 using regiis, but later discovered that this stopped & restarted all websites... something catastrophic to do in a production environment. All the hosted websites (not only the new one) would lose their session state, for example. Fortunately, I found that you can change the root website and any new websites created afterwards will inherit the default ASP.NET version. Run the following once:

@echo off
echo WARNING: THIS WILL CHANGE THE DEFAULT ASP.NET VERSION FOR ALL NEW SITES TO V2.0
pause
 
REM will propagate to new sites. 
%windir%\Microsoft.NET\Framework\v2.0.50727\aspnet_regiis -sn W3SVC/
 
REM does not propagate
REM cscript %IWEB_ADSUTIL% set w3svc/accessread "true"
REM cscript %IWEB_ADSUTIL% set w3svc/accesswrite "true"

 

Scripting the deletion of a website

Deletion is much simpler, as you can see below.

 

:Delete
iisweb /delete %IWEB_DOMAIN%
CSCRIPT //nologo %IWEB_ADSUTIL% DELETE w3svc/AppPools/%IWEB_DOMAIN%
 
GOTO End

Conclusion

I had a fun time perfecting my scripting skills while creating a concrete example that solves someone's problem. I discovered that the devil is in the details, but that numerous people have worked on similar problems in the past. I ended up creating another script which runs this one numerous times, according to actions found in a local text file. Simply put, iWeb's PHP system appends operations to a file (create this site, delete that one, create this other site) and mine performs the operation and empties the task list. That way, the script can be run periodically and all the PHP coders need to do is append to a particular file.  

Download the code.

kick it on DotNetKicks.com



Common console commands for the typical ASP.NET developer

clock February 25, 2008 14:19 by author JKealey

iis As an ASP.NET web developer, there are a few tasks that I must perform often for which I am glad to be able to perform via the command line. GUIs are great, but there are some things that are simply faster to do via the command line. Although we do have Cygwin installed to enhance our tool belt with commands like grep, there are a few ASP.NET related commands that I wanted to share with you today. Some of these are more useful on Windows 2003 server (because you can run multiple worker processes), but I hope you will find them useful.

1) Restarting IIS

The iisreset command can be used to restart IIS easily from the command line. Self-explanatory.

Attempting stop...
Internet services successfully stopped
Attempting start...
Internet services successfully restarted

2) Listing all ASP.NET worker processes

You can use tasklist to get the running worker processes.

tasklist /FI "IMAGENAME eq w3wp.exe"

Image Name PID Session Name Session# Mem Usage
========================================================================
w3wp.exe 129504 Console 0 40,728 K

You can also use the following command if you have Cygwin installed (easier to remember)

 

tasklist | grep w3wp.exe

 

w3wp.exe 4456 Console 0 54,004 K
w3wp.exe 5144 Console 0 101,736 K
w3wp.exe 2912 Console 0 108,684 K
w3wp.exe 3212 Console 0 136,060 K
w3wp.exe 852 Console 0 133,616 K
w3wp.exe 352 Console 0 6,228 K
w3wp.exe 1556 Console 0 155,264 K
w3wp.exe 3480 Console 0 6,272 K

3) Associating a process ID with a particular application pool

Should you want to monitor memory usage for a particular worker process, the results shown above are not very useful. Use the iisapp command.

W3WP.exe PID: 4456 AppPoolId: .NET 1.1
W3WP.exe PID: 5144 AppPoolId: CustomerA
W3WP.exe PID: 2912 AppPoolId: CustomerB
W3WP.exe PID: 3212 AppPoolId: Blog
W3WP.exe PID: 852 AppPoolId: LavaBlast
W3WP.exe PID: 352 AppPoolId: CustomerC
W3WP.exe PID: 1556 AppPoolId: CustomerD
W3WP.exe PID: 3480 AppPoolId: DefaultAppPool

By using iisapp in conjunction with tasklist, you can know which task is your target for taskkill.

4) Creating a virtual directory

When new developers checkout your code for the first time (or when you upgrade your machine), you don’t want to spend hours configuring IIS. You could back up the metabase and restore it later on, but we simply use iisvdir. Assuming your root IIS has good default configuration settings for your project, you can create a virtual directory like so:

iisvdir /create “Default Web Site” franchiseblast c:\work\lavablast\franchiseblast\

 

5) Finding which folder contains the desired log files.

IIS saves its log files in %WINDOWS%\System32\LogFiles, but it creates a different subdirectory for each web application. Use iisweb /query to figure out which folder to go check out.

Connecting to server ...Done.
Site Name (Metabase Path) Status IP Port Host
==============================================================================

Default Web Site (W3SVC/1) STARTED ALL 80 N/A
port85 (W3SVC/858114812) STARTED ALL 85 N/A

6) Many more commands…

Take a peek at the following articles for more command-line tools that might be useful in your context:

http://www.microsoft.com/technet/prodtechnol/WindowsServer2003/Library/IIS/b8721f32-696b-4439-9140-7061933afa4b.mspx?mfr=true

http://www.tech-faq.com/using-iis-command-line-utilities-to-manage-iis.shtml

Conclusion

There are numerous command line tools distributed by Microsoft that help you manage your ASP.NET website. Obviously, the commands listed here are the tip of the iceberg! Although many developers know about these commands because they had to memorize them for some test, many are not even aware of their existence. Personally, I feel that if you write a single script that sets up IIS as you need it to develop, you’ll save time setting up new developers or when you re-install your operating system. Script it once and reap the rewards down the road.

kick it on DotNetKicks.com  



Month List

Disclaimer

The opinions expressed herein are my own personal opinions and do not represent my employer's view in anyway.

© Copyright 2014

Sign in