Nprof and ACT, Your Performance Assistants, Part 2By Klaus Salchner
Measuring Performance with Application Center Test (ACT)
Microsoft's Application Center Test (ACT) comes for free with Visual Studio .NET Enterprise Developer or Enterprise Architect editions, and is very easy to use. It is located under "Programs | Microsoft Visual Studio .NET 2003 | Visual Studio .NET Enterprise Features | Microsoft Application Center Test". When opened, create a new project with the menu "File | New Project". Give the project a new name and click Ok. Next, record the script you want to run. This records the pages you want to execute during your test. Click on the "New Test" button (clipboard with a star in the upper right corner) in the toolbar or select the menu "Actions | New Test."
This will bring up a wizard, which helps you to create your test script. Click Next, then select the option "record a new test" and click Next again. Now you select the script language, which for now just supports VBScript and then you click Next again. Finally you click on the "Start recording" button which brings up an empty browser. In the browser you navigate to your application and use it like your users would. Every page you requests will be recorded in the background so it can be replayed. When done you close the browser which brings you back to the wizard and shows you now in the list box all the page requests you made in the browser. When done you click on the "Stop recording" button and click Next. On the last screen you give the test a name, click Next and then click Finish. Now you have created a test, which you can play back.
Back in the main window of Application Center Test you see a tree on the left side. Expand the node called "Tests" with the plus sign and you find the test you just recorded. Clicking on it shows more details on the right side. It shows the recorded script at the bottom and allows you to enter some notes on the top. Right click on it and select "Start Test" from the popup menu. You see the "Test Status" window shown to the right.
In the left corner you see for how long the test is running and in the right corner you see the requests/sec as well as the total number of requests. The graph at the bottom shows the number of processed requests over time. This visualizes very well the requests/sec. When done click on the "Stop Test" button, wait till the test has completed and then click the Close button.
Back in the main window you can view the results by clicking on the "Results" node of the left side tree. It shows you a screen similar to the one to the left. The list "Test runs" lists you all the tests you ran. You can click on one test and it shows you the test summary at the bottom. Select from the "Report" drop down in the upper right corner the report called "Requests". It shows you at the bottom a summary which contains the "Time To First Byte" (TTFB) and "Time To Last Byte" (TTLB). Time to first byte is the time from, the web server receiving the request from the browser, processing the request and the first byte returned to the browser. So the time to first byte is considered the processing time for a page. The time to last byte includes the time it takes to return the complete page to the browser. Time to last byte minus time to first byte is the time it takes to transfer the processed page to the users' browser. The report shows you the average TTFB and TTLB which always need to stay below the five to 10 seconds.
Looking at the script, by clicking on the test in the tree, shows you that
fEnableDelays is set to
false, meaning there is no waiting time between each requested page. To make the test more real world set it to true
and for each requested page set the waiting time to a value between 5,000 msec and 15,000 msec. You find the call
Test.Sleep() at the beginning of each function (the script recorder creates a function for each requested
page). The value passed on the
Test.Sleep() is in msec, so you set it for example to
Diving into the Details
As you have just seen, Application Center Test can be used to show you how many requests/sec your application can handle and is a must-have tool for every developer. Use it early and frequently! Besides observing just the number of requests/sec your application can handle, you might be interested in a more detailed analysis, such as how often certain methods are called, and how much time is spent in these methods in relation to each other. That's where NProf comes into play. NProf is a tool that allows you to profile the "execution profile" for your application and analyze it.
NProf is invaluable when your application is suffering from some kind of performance problem, as it helps pinpoint where the problem is arising from. I would encourage you, though, to use NProf as often as possible, not just when things go awry. It can be used to help judge where your application's execution time is being utilized. Even if your ASP.NET page is fast, a simple change could make it even much faster, which frees up resources for other processing tasks.
NProf is a free, open-source tool that can be downloaded from http://nprof.sourceforge.net/Site/SiteHomeNews.html
(the current version is 0.8b at the time of writing). From the site, you'll be able to download a ZIP file
that contains installation instructions in the
Readme.txt file (which are very easy to follow).
Once you have installed NProf, launch
NProf.Application.exe. You will see a screen similar to the one
shown to the right.
Click on "File | New" and select the application you want to profile - in our case, ASP.NET. Click on the button "Create Project". Finally go to the menu "Project | Start project run" which will launch the application or restart IIS if you profile an ASP.NET application. The restart is needed because NProf relies on the CLR profiling API and it needs to re-launch the IIS/ASP.NET process with hooking itself into the application using the profiling API.
|Reduced Performance with NProf|
When running your application with NProf "hoooked into" it to ascertain the execution path and times, realize that your
application will be up to 20 times slower then normal. This is because NProf records its metrics through the
CLR profiling API, which inspects every single object/method call. That is naturally very resource intensive and
is also the reason why you have to look at execution time in percentage relative to each other. Absolute times
are skewed by the overhead of the profiling API.
This article will not describe the CLR profiling API itself. Detailed documentation is provided by Visual Studio
.NET 2003 itself (go to the following folder
Once you have hooked NProf into the ASP.NET process, you can now use the ASP.NET application as your normal user would. When done go back to NProf and click "Project | Stop project run". Stopping the profiling does not work through NProf if you use NProf with Windows 2003, which has IIS 6.0 (a known bug). You can work around it by going into the IIS Manager and stopping the Application Pool your application belongs to. So if your application runs under the Default Web Site you right click on "DefaultAppPool" (shown under the "Application Pools" node) and select "Recycle" from the popup menu. This tells IIS to shut down the active ASP.NET application process and bring up a new one. This is recognized by nprof and the profiling is stopped.
Now that we have seen how to use NProf to collect profiling information about an ASP.NET Web application, the next step is to examine how to analyze the data collected by NProf. We'll do this in Part 3 of this article.