Performance Testing Vs. Performance Engineering

Performance testing is an art of testing the application. This can be in two ways. 1. Manual and Automation.

Automation performance testing is conducted with tools such as load testing tools/monitoring tools/analysis tools including Loadrunner,Neoload,Jmeter,VSTS etc.

Performance testing is to measure the application responsiveness and other metrics also to check the capabilities such as scalability,availability etc.



On the otherhand, performance engineering is reengineering the application to acheive the application SLAs as defined.This can be in 2 ways. 1. ProActive 2. ReActive

ProActive performance engineering is during the application development and from architecting to unit testing.

ReActive performance engineering is during once performance bottlenecks are observed.

This is the process of profiling,analysing and tuning.

To perform this, many tools available in the market such as profilers,diagnostic tools etc.

Neoload Custom actions API Extensions

Neoload has a feature of custom actions with Jars and other api extensions that are useful with integrating with other tools.

Here is the link that provides more information on this -

Neoload Custom Actions extensions

ASP .NET application performance metrics - Perfmon in Web/App server

Here i am providing the list of metrics that can be captured through perfmon in Web/App server for a ASP.NET Application.These are really helpful in identifying memory/processor/Disk issues and code inefficiencies as well.

Processor --> % Processor time
Processor --> % User time
Processor --> % Previliged time
Processor --> % Idle time

Physical Disk --> % Disk Read time
Physical Disk -->% Disk write time
Physical Disk -->Avg. Disk queue length

Memory --> Available MBytes
Memory --> Committed Bytes

.NET CLR Memory --> %time in GC
.NET CLR Memory -->#Gen0 Collections
.NET CLR Memory -->#Gen1 Collections
.NET CLR Memory -->#Gen2 Collections

.NET CLR LocksandThreads --> Contention Rate/sec

.NET CLR Execeptions --> # of Exceps thrown/sec

ASP.NET --> worker process restarts

ASP.NET Applications --> Pipeline instance count
ASP.NET Applications --> Requests/sec
ASP.NET Applications --> Requests in application queue

System --> context swithces/sec

Hope this article is useful and if you still need more information on this please let me know. I can help you on that.

How Perfmon is useful in collecting metrics

Perfmon - A windows nativity monitoring tool helps in monitoring server health Online and offline. It can be plugged to various tools such as Loadrunner etc.

We can even import perfmon logs into loadrunner analysis file and so a combined report will be generated.

How to start :

1. Start -> Run -> Perfmon.exe

2. By default counters are displayed. To add counters click +

3. Select the Component(Eg.Processor) in the computer --> select metrics (Eg. % processor time)--> Select instnace(eg.Based on cores) and click Add.

4. Similarly add counters for objects to monitor for online monitoring.

For offline - 1. Add Data collector sets under user defined category --> Provide metrics and location to save. Provide timings also .

2. So that offline log is saved and we can view whenever needed.

Once we capture the metrics , as a next step we can move to analysis by looking at each metric.

Enjoy analysing.

How Performance requirements gathering plays a major Role

Performance requirements gathering is an important phase in performance testing life cycle.With out proper requirements, there is no meaning of testing.

Firstly we should understand Why,What,How and When we are testing. Here are the details that should be taken into consideration.

Scope - What is the scope of performance testing/business flows/aspects to consider and Why should we test

SLAs - What are the SLAs defined.Pass/Fail criteria

Application - What application and What is it about

Environment - Which environment is available for Testing and is it replica of production

Architecture - What are different components of architecture and how is the communication

Metrics - What are different types of metrics to be captured.

Toolset- What are different tools available for testing/monitoring/Analysis/Reporting.

Timelines - When is the window for the engagement

PoC- Point of contacts from different teams

Sharepath - place holder for documents/scripts/results

Performance testing in different types

Performance testing is defined in various types.

1. Load Testing: It is conducted to verify whether the application can meet expected SLAs in terms of Response times, Throughput and Resource utilization e.t.c under different loads of Vusers.

2. Stress Testing : This performance testing is done to identify the application maximum handling capacity of users, by determining the breakpoint.

3. Spike Testing : It is a performance test conducted to validate performance Characteristics when subjected to load volumes repeatedly increase beyond anticipated production operations for short periods of time.

4. Endurance/Soak Testing: It is conducted to verify application memory leaks and code inefficiencies. This test is performed for long durations like 5 hours, 10 hours, 1 day…

5. Volume Testing: These tests are more appropriate for messaging, Batch processing type situations. Here we concentrate more about throughput instead Response time.

How Internet performance testing is significant over traditional performance testing

Firstly, we should understand the difference between those two. In general , we simulate virtual users from LGs either from single place or multiple places based on their location. It does not give us practical results. The reason is , Users access the application hosted in internet from various geographic locations of the world.

So, We need to simulate the same to get exact experience. In order to acheive this, Internet performance testing is much helpful by simulating users from various geographic locations. Also, simualting with various bandwidth,browsers also important.

Here are the factors we should take into account as per my understanding and knowledge.

Geographic locations
Different Mix of Browsers
Different Network bandwidth

This can be acheived in Loadrunner through Shunra integration and in Neoload Cloud load generators.

Hope this article is useful.

Create Vugen script with SOAPUI where XML recording to be done

In some cases , we might need to capture Vugen scripts for web services load testing. Here is the way how to record scripts with XMLs available using both loadrunner and SOAPUI.

SOAP UI should be installed on your machine.

Navigate to Vugen -> File -> New ->Web (Http/Html) protocol.

Click Create and provide necessary details and Click record.

Fill the details as recording application micexec.exe (windows application).

Start Recording and SOAP UI will be opened.

Click File -> New SOAP UI Project and Provide necessary details in the page.

Click OK and we will be directed to the request page on Right pane.

Expand Execute XML -> Request.

Ensure the end point URL is correct.

Place the formatted xml correctly in the left request pane.

Click Execute button and so response will be displayed.

Repeat the same above steps for different Xmls and stop recording once done.

So,the Vugen script is ready.

Use the script for load testing and enjoy with the analysis.

Get value and Set value in Neoload can be used in Java script

In Neoload we have to use Javascript code wherever required if custom code is required.

Here is an example using Getvalue and Setvalue

var myVar1 = context.variableManager.getValue(" ");

context.variableManager.setValue("new variable that captured value passed to",myvar1);

How many virtual users does my Load generator support in Loadrunner

Many of us not sure how many virtual users does my LG support with out any memory exceptions/issues.

But before loadtesting we need to understand that . Remember, it differs from protocol to protocol.Web protocol consumes 2 MB/1Vuser where as Ajax truclient consumes 70-120MB/1 Vuser.


Here is a good document that helps in calculating the no.of users it LG supports.

http://h30499.www3.hp.com/t5/HP-LoadRunner-and-Performance/How-To-Understand-and-calculate-Virtual-User-quot-footprint-quot/ba-p/2407591