Friday, May 21, 2010

Difference In Monitoring System Resources From Performance Center & Controller

You might have monitored system resources from HP Performance Center or HP LoadRunner Controller application. But any time did you have faced any problem while collecting data (like %Processor Time, pages/sec etc for memory etc).
There is difference in adding measurements from Performance Center & from Controller. Have you ever seen that when you add System IP for collecting data from this IP while executing your scenario, but the data is not collected. I have faced this problem. Let me explain my scenario in this case:
In Performance Center:
1. You create a scenario, add scripts to execute in Design tabs and add the monitors in the Monitors tab (you provide the UserID/Password while adding the performance counters to be monitored on the machine).
2. You start the scenario, and the performance data is collected from this machine.

But do you really think it is so simple if you add performance counters for a machine on HP LoadRunner Controller.
Try adding the performance counter in LoadRunner Controller scenario and start the scenario. You will see that the window resources graph doesn't display the data. (it has happened with me) .
This is due to the fact that, while adding the machine under window resources, it doesn't ask for the system UserID/Password and the LoadRunner Controller machine UserID/Password overrides the UserID/Password of the system which is to be monitored.

So in this case, the password of the Controller machine and the Monitor machine should be same. Then only you can monitor the system resources added in the Controller scenario.

Saturday, April 24, 2010

Monitoring System Resources On Remote Machines Using PERFMON

We use to monitor the system resources using PERFMON and our machines without any issues, but I faced some different kind of issues while monitoring the remote machines using PERFMON tool from a centralized machine.

Before going into the detail of the PERFMON issues, please ensure few things on your machine where you are going to start the counter logs.
a) Check that Performance Logs and Alerts service is running
b) The service name is SysmonLog and the path to executable is "C:\window\system32\smlogsvc.exe". This service should be running in the services window.

I had 8 machines configured on the same domain and wanted to monitor these machine from a single point centralized machine. The required counters were created using "logman Create" method and to start and stop these perfmon counters I used to have a batch file. (This will monitor system resources on all the 8 machines from the centralized machine.

Now here comes the problem....!!!!

When I started the counter logs, it didn't start collecting data from these 8 machines. The reason for this is that, the user (for this centralized machine) doesn't have access to these machines so the perfmon counter logs were not getting started on these machines and hence the data is not collected.

The solution for this issue is that, after creating the counter logs on the centralized machine to monitor system resources of remote machines, perform the following actions:
1. Open Perfmon window
2. Go to properties of the counter logs (that has been created to monitor the remote machines)
3. On the System Overview properties (in XP, will be different in higher version of OS), type the User Name in the "Run As:" field and click on either "Set Password" or "Change" button (based on the OS version you are using)
4. On the Set Password window, provide the User Name, Password and Confirm Password and click on OK Button

Now, you are all set to monitor the system resources of the remote machine from a centralized machine.

Windows Excel Doesn't Support Big/Complex Formulas Spreadsheet To Open

You might have worked on MS Office Excel XP, 2003 or 2007. But did anyone tried opening an excel file of size more than 170 MB with lot of big and complex formulas in the excel cells?

If not, try once and then go below to read this article.

I faced this type of problem when I was working on one of my project and this project uses MS Excel to prepare the reports and these excel were of size almost more than 100 MB with lot of Excel complex formulas in it.

Excel doesn't work if there are lot of formulas in the spreadsheet. This is due to the fact that each excel cell formula occupies memory. And if you are trying to open these excel files with complex formulas the excel throws an error saying "Not enough resources".

Does someone knows why you get this error? If No, then here is the explanation......

This issue is due to the fact that this excel file has already reached the maximum memory limit per process on a 32-bit OS i.e 2GB per Process limit. This is the reason you will not be able to open an excel file that crosses 2GB memory on 32-bit OS.

This process can be eliminated if we use 64-bit OS with 64-bit Office Excel (if available), as 64-bit OS allows more than 2GB per process support.

In my case, I tried with Office 2010 on Windows 2008 or windows 7 and it worked (excel file has utilized more than 5 GB of memory)

So next time when you face this type of issue don't blame Office Excel but think broadly and try other alternatives.

Wednesday, March 3, 2010

Calculating Processor(_Total)\%Processor Time For System Using Process(*)\%Processor Time Performance Counter

Before going to detailed description on this topic, I would like to start it with a question...!!!!
"Can we calculate %Processor Time for the System which has multi-core processor if Processor(_Total)\%Processor Time counter is not included in the Perfmon?" [Also, provided we have all the necessary counters for the Process Object in the Perfmon, for e.g. Process(*)\%Processor Time is included]

Any Guess...!!!!!
Let me tell you how we can do this using the Process(*)\%Processor Time counters for the Process Object.
We can do this by using the following formula which will reflect exactly the same values for the Processor Counter [i.e. Processor(_Total)\%Processor Time]
Use the below formula:
Processor(_Total)\%Processor Time =[Process(_Total)\% Processor Time - Process(Idle)\% Processor Time]/Number Of Cores

Considering Process(_Total)\% Processor Time = Number_Of_Cores * 100

Bit confusing..????
Let me explain you with some example which will give you more understanding on this measurement.
Consider the below raw data which I have from the Perfmon utility every 1 second. I have considered here only 3 performance counters to make you understand:
1. Process(_Total)\% Processor Time
2. Process(Idle)\% Processor Time
3. Processor(_Total)\% Processor Time

Process(_Total)\ % Processor Time Process(Idle)\ % Processor Time
194.7602101 113.7708158
199.68128 174.72112
196.5612 170.04109
198.12127 174.72112

Processor(_Total)\% Processor Time
43.11458591
12.63944
14.97946
12.63944

Descriptions:
I have a Dual-Core machine, so let me consider the column \\MachineName\Process(_Total)\% Processor Time to evaluate it's values as:

\\MachineName\Process(_Total)\% Processor Time = 2 x 100 = 200 [So we can consider column values of this counter as 200 instead of 194.7602101, 199.68128 and so (this is nothing but the value 100% for each cores).

So, now let me calculate the Processor(_Total)\% Processor Time with the available Process related data.
e.g.
Processor(_Total)\% Processor Time = [200 - 113.7708158]/2
= 43.1145921 (which is almost equal to what it is shown in the third column of the Perfmon result i.e. equal to column values MachineName\Processor(_Total)\% Processor Time)

This explains the procedure to calculate the Processor(_Total)\% Processor Time if in case only Process counters are included in the Perfmon Utility.

Suggestions are always welcome........!!!!! :)

Monday, February 15, 2010

Copying Recent Updated Files From The Shared Drives To The Local Machine (From Same Network)

In case multiple log files are created on different physical machines connected on the same network, in my case the total physical machines were more than 30 and on each machine multiple log files were written by the application (say the average was 7-8 files). Even the size of the files does matter, as the size of the files written were ~700M and more.
Imagine copying these many files manually would always eat performance testers precious time. To eliminate this workload on the tester we can make use of the automated script to take care of copying the log files after each run.

What needs to be taken care:
1. If any services writes the log on these physical machine (same in my case), When the test is completed, execute the code which will collect the logs from different machines (my case: GAP service/PNL Service/DS/tradeserver logs).
2. The code should know how many logs and which logs it has to collect from which server.
3. The code sort the files based on the latest update (Date & Time) [latest files will be on top]
4. The code picks up the files which starts with text which is mentioned in the below If conditions.
5. The code picks up only the files which has the _error text in the name of the file
e.g. GAP1_BOND_FILT2_error_10142009_101408.log
HS1_BOND_FILT2_error_10142009_103028.log
6. The copied logs will be saved in the location that you provide during the execution of this code.

Code Goes Here:
package copyfiles_env2;
import java.io.*;
import java.io.File;
import javax.swing.*;
import java.util.Arrays;
import java.util.Comparator;

public class Main {
public static void main(String[] args) throws IOException
{
int forloop = 0;
boolean b1;
String string;
String[] servers={"\\\\server1b","\\\\server2b","\\\\server3b","\\\\server4b","\\\\server5b","\\\\server6b","\\\\server7b","\\\\server8b","\\\\server9b","\\\\server10b","\\\\server13b","\\\\server14b","\\\\server17b","\\\\server19b","\\\\server20b","\\\\server21b","\\\\server32b"};
BufferedReader TempReader;
BufferedWriter TempWriter;
File destFileName=null, DestFolderpath=null,tempfile=null;
String folderpathofscript = null, destfolderpath=null, temp1=null, s1=" ",s2=" "; // string to save the folder path selected by user

final JFileChooser fc = new JFileChooser();
fc.setFileSelectionMode(JFileChooser.DIRECTORIES_ONLY);
int returnVal1 = fc.showSaveDialog(null),fxpssearch2Count=0;
if (returnVal1 == JFileChooser.APPROVE_OPTION){
DestFolderpath = fc.getSelectedFile();
destfolderpath = String.valueOf(DestFolderpath);
JOptionPane.showMessageDialog(null, "Wait.....!!! Copying Files To The Destination Folder");

for (int j=0; j<=servers.length; j++){
folderpathofscript = servers[j]+"\\servers\\RTScripts";
if (servers[j]=="\\\\server1b"){
forloop = 2;
s1 = "GAP1_GAP_";
s2 = "GAP2_GAP_";
}
if (servers[j]=="\\\\server2b"){
forloop = 2;
s1 = "BVS_bondview_";
s2 = "BVS_collateralexceptionview_";
}
if (servers[j]=="\\\\server3b"){
forloop = 2;
s1 = "BVS_credderivdefaview_";
s2 = "BVS_dispexview_";
}
if (servers[j]=="\\\\server4b"){
forloop = 2;
s1 = "BVS_hedgeview_";
s2 = "BVS_incmsgview_";
}
if (servers[j]=="\\\\server5b"){
forloop = 2;
s1 = "BVS_requestsview_";
s2 = "BVS_riskview_";
}
if (servers[j]=="\\\\server6b")
{
forloop = 2;
s1 = "staticchangesserver_";
s2 = "tradeserver_";
}
if (servers[j]=="\\\\server7b"){
forloop = 2;
s1 = "GAP1_GAP_DERIV_";
s2 = "GAP2_GAP_DERIV_";
}
if (servers[j]=="\\\\server8b"){
forloop = 2;
s1 = "PRESTL_ACCT_TR_";
s2 = "PRESTL_BASKETOPT_";
}
if (servers[j]=="\\\\server9b"){
forloop = 2;
s1 = "PRESTL_EXOTIC_";
s2 = "PRESTL_FRA_";
}
if (servers[j]=="\\\\server10b"){
forloop = 2;
s1 = "PRESTL_MM_";
s2 = "PRESTL_MUST_TR_";
}
if (servers[j]=="\\\\server13b"){
forloop = 2;
s1 = "PNL1_PL_CDS_";
s2 = "PNL2_PL_CDS_";
}
if (servers[j]=="\\\\server14b"){
forloop = 2;
s1 = "PNL1_PL_BOND_";
s2 = "PNL2_PL_BOND_";
}
if (servers[j]=="\\\\server17b"){
forloop = 2;
s1 = "BDPS_";
s2 = "FXPS_";
}
if (servers[j]=="\\\\server19b"){
forloop = 2;
s1 = "HS1_BOND_";
s2 = "HS2_BOND_";
}
if (servers[j]=="\\\\server20b"){
forloop = 2;
s1 = "HS1_CDS_";
s2 = "HS2_CDS_";
}
if (servers[j]=="\\\\server21b"){
forloop = 2;
s1 = "HS1_DERIV_";
s2 = "HS2_DERIV_";
}
if (servers[j]=="\\\\server32b"){
forloop = 1;
s1 = "DS_";
}

File dir = new File(folderpathofscript);
File [] files = dir.listFiles();
Arrays.sort(files, new Comparator(){
public int compare(Object o1, Object o2) {
return compare( (File)o1, (File)o2);
}
private int compare( File f1, File f2){
long result = f2.lastModified() - f1.lastModified();
if( result > 0 ){
return 1;
} else if( result < 0 ){
return -1;
} else {
return 0;
}
}
});

for (int i=0;i<=forloop;i++){
if( files[i].getName().lastIndexOf(".log")!=-1 && files[i].getName().indexOf("error_")!=-1){
System.out.println("File Name: "+files[i].getName());
if(b1 = files[i].getName().indexOf(s1)!= -1 || files[i].getName().indexOf(s2)!= -1){
TempWriter = new BufferedWriter(new FileWriter(destfolderpath+"\\"+files[i].getName()));
TempReader = new BufferedReader(new FileReader(files[i].getPath()));
while((temp1=TempReader.readLine())!=null){
TempWriter.write(temp1+"\n");
}
TempWriter.close();
TempReader.close();
}
else if ((files[i].getName().lastIndexOf(".log")!=-1) && (files[i].getName().indexOf("STPAPI_")!=-1) || files[i].getName().lastIndexOf(".log")==-1 || files[i].getName().lastIndexOf("output_")==-1){
forloop = forloop + 1;
}
}
}
s1=" ";s2=" ";forloop=0;
}

JOptionPane.showMessageDialog(null, "Done.....!!! All The Required Files Are Copied Successfully....!!!");
}
}
}

Friday, January 8, 2010

Embedding Pacing In LoadRunner Script (Includes Pacing For Blocks Too)

We come accross many pacing logics in the LoadRunner script which is based on the multiple iterations. But if we need to have a pacing if there are Action blocks involved and if in case we need to re-calculate the pacing time after each iterations (which will be a good idea) as few iterations may not take the same time to execute in the LoadRunner Script, then, we need to implement a different Pacing Logic. Here it is.......
The below pacing logic calculates the pacing time after completion of each iterations and based on the test duration left.

This pacing logic have three different functions embedded in a .C function File.
1. InitCore()
2. TopOfIterations()
3. BottomOfIterations()

1. InitCore():
The InitCore() Function takes 3 parameters as mentioned in the below syntax:
InitCore(TestDuration(in Seconds), TotalBlockIterations, MaxNumberOfIterations);
Example: InitCore(180, 12, 2); - Script will execute for 24 iterations and for 180 seconds [i.e. 3 Min]
Refer the below screen for more understanding:

So, in this case the Total Number Of Iterations becomes 24 [TotalNoOfIterations = TotalBlockIterations + MaxNumberOfIterations].
This InitCore Function should be the first line in vuser_init section.

2. TopOfIterations():
This function calculates the Pacing Time based on the Test Duration and Total Number Of Iterations which will be used for the actual pacing time in the main script. This Function Call should be the first line in the Main Action Block.

3. BottomOfIterations():
This function does the main calculations of the Pacing Time when a Block Execution is completed based on the Test Durations left and the Total Number of Iterations left. This is same as re-calculating the Pacing Time instead of fixed Pacing Time each time the iterations is completed. Please note, here each BLOCK execution is considered as each ITERATIONS. This Function Call should be the last line in the Main Action Block.

These TopOfIterations() and BottomOfIterations() functions can be added in different Actions Blocks.

Bottleneck:
Since the Pacing Time is calculated based on the Test Durations left and Iterations left, there could be a possibility that few iterations might take long time to finish and if this is the case then at some point of time the Test Durations time becomes 0, in this case the rest of the Iterations are executed without calculating the Pacing Time and hence the Pacing Time becomes 0.

Actual Code Comes Here:

double TestDuration, pacing_time, Origpacing_time;
int Counter = 1,MaxCounter = 0, TotalIterations;
merc_timer_handle_t timer;
double duration;

InitCore(double TDuration, int TotalBlockIterations, int MaxNumberOfIterations)
{
TestDuration = TDuration;
TotalIterations =
TotalBlockIterations * MaxNumberOfIterations;
}

TopOfIterations()
{
if (Counter == 1) {
lr_output_message("Action1 - Total Test Durations - First Time: %lf",TestDuration);
pacing_time = (double)TestDuration / TotalIterations;
lr_output_message("Action1 - Total Iterations Left - First Time: %d",TotalIterations);
lr_output_message("Action1 - Pacing - First Time: %lf ",pacing_time);
}
}

BottomOfIterations()
{
if ((duration <> 0) {
TestDuration = TestDuration - pacing_time; //Is equal to Pacing time
MaxCounter = 1;
}

TotalIterations = TotalIterations - 1;
if (MaxCounter != 0) {
lr_think_time(pacing_time - duration);
pacing_time = TestDuration / TotalIterations;
MaxCounter = 0;
}
else if (TestDuration > 0) {
TestDuration = TestDuration - duration;
pacing_time = TestDuration / TotalIterations;
}

if (TestDuration <= 0) {
pacing_time = 0;
}
lr_output_message("Action1 - Total Iterations Left: %d",TotalIterations);
lr_output_message("Action1 - Total Test Durations Left: %lf",TestDuration);
lr_output_message("Action1 - New Pacing: %lf",pacing_time);
Counter = 0;
}
Enjoy Scripting.........