Thursday, February 7, 2019

Monitoring Java Application Running In Cloud Foundry (Platform-as-a-Service)

This article defines the approach to monitor Cloud Foundry Application via any Java Visualization Monitoring tool like JVisualVM or JConsole.

This approach will enable monitoring of Java Virtual Machine utilization (like GC, CPU, Memory etc). Also, it can monitor threads and perform CPU & Memory Profiling.

Step 1: 
Add below parameters in the Cloud Foundry Environment variables for the Cloud Foundry Application to enable Remote JMX monitoring.
JAVA_OPTS: -Djava.rmi.server.hostname=localhost -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=9480 -Dcom.sun.management.jmxremote.rmi.port=9480 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false
Restage the application to take the above parameters into affect.
    cf restage

Step 2:
Do local port forwarding for your Cloud Foundry application for JMX port running on Cloud Foundry Org/Space.
cf ssh -L 9480:localhost:9480

Step 3:
Use JVisualVM to monitor the Cloud Foundry Application.

CPU/Memory Profiling can also be enabled from JVisualVM          

Sunday, July 30, 2017

Learning From Performance Testing/Engineering:


ConcurrentModificationException occurs due to variables declared/used as Class Variables instead of variables are defined under Class Members.

Moving declared class variables in Class Members would help in resolving above ConcurrentModificationException in multi-threaded environment.


Wednesday, June 28, 2017

Performance Improvement To JSON Blobs Reads

The Dynamic Search feature is based on Amazon S3, RIAK KV as backend DB.
While performing Dynamic Search based with one or more than one field values, where large data objects/files are fetched from S3 buckets.

RIAK KV stores the metadata and link to S3 bucket for large data payload.
While searching for a data item with large json blob, ~100MB of data was sent to the client machine degrading the Search API performance.

The recommendation was to send meta data with link to the user and user can choose to download the item by clicking the link (each record of size 1MB, 2MB, 5MB or 10MB) when needed to see his observation data.

Wednesday, September 2, 2015

Deleting Duplicate Record In A Table

To delete duplicate records in a table use the below query:

delete /*+ PARALLEL(4) */ from letter_request A
WHERE a.rowid > 
   ANY (
     SELECT 
        B.rowid
     FROM 
        letter_request B
     WHERE 
      A.letter_name='WA20' and A.etl_batch_id=695
     AND
        A.loan_number = B.loan_number
     AND 
        A.sequence_number = B.sequence_number
        
     );

Wednesday, July 23, 2014

Capturing network packets using "tcpdump"

tcpdump is used for packet analyzer which allows the user to intercept and display TCP/IP and other packets being transmitted or received over a network to which the computer is attached.

Installation of tcpdump on Solaris/Linux:
   pkgutil is similar utility for Solaris compared to yum in Linux and apt-get in Ubuntu.
First let's install pkgutil using following command:
   /usr/sfw/bin/wget http://mirror.opencsw.org/opencsw/pkgutil.pkg
   pkgutil install at /opt/csw/bin/pkgutil on Solaris

Use below command to install tcpdump on Solaris:
pkgutil -i coreutils [This installs tcpdump at /usr/sbin/tcpdump]


Usage of tcpdump:
    tcpdump -w [output-file]  -i  [physical-interface] port [port]
    I noticed that capturing of the packets is only limited to 96 bytes. Checking manual of tcpdump I found that there is an option to disable the 96 bytes limitation by using the option "-s 0".  
Use the following command to capture 65535 bytes of packets.
    tcpdump -w [output-file] -i [physical-interface] port [port] -s 0

Example: tcpdump -w p001.pcap -i eth0 port 8080 -s 0
The above command captures any packet transfered on "physical-interface" on port 8080.

Analysis:
   The output file can be analyzed using "Wireshark" GUI application available for free.

Friday, April 18, 2014

Taking Thread Dumps At Regular Interval From CLI In Unix

Good to see posting some article on my blog after a long gap :)

Today I will explain on a very simple but important area needed for performance monitoring and analysis. This article is related to thread dump collection during performance execution in unix.

Topic: "How to collect thread dumps from CLI in Unix at regular interval"

Uuh... looks very simple post.. isn't it...!!! YES IT IS...

Use below command to capture thread dumps every 5 min:
nohup ./get_jstack  [Output-Log-Dir]  300  > /dev/null 2>&1 &

Now let's see what are the content of "get_jstack" file:

#!/bin/bash
LOGDIR=$1
WAIT=$2
mkdir $LOGDIR/serverjstack; mkdir $LOGDIR/muxjstack; mkdir $LOGDIR/wdjstack;
i=1
while [ $i -gt 0 ]
do
   PID1=`cat $LOGDIR/xmppd.pid`
   /usr/jdk/latest/bin/jstack $PID1 > $LOGDIR/serverjstack/jstack$i
   PID2=`cat $LOGDIR/mux.pid`
   /usr/jdk/latest/bin/jstack $PID2 > $LOGDIR/muxjstack/jstack$i
   PID3=`cat $LOGDIR/wd.pid`
   /usr/jdk/latest/bin/jstack $PID3 > $LOGDIR/wdjstack/jstack$i
    sleep $WAIT
    echo $i
    i=`expr $i + 1`
done

Make sure you provide the absolute path of jstack file. serverjstack, muxjstack and wdjstack are the folders for 3 processes which are running on the unix machine. 

here you go.. jstack will be created in the folder that you pass from command line. 
Enjoy Analyzing Thread Dumps For Performance Bottlenecks...

More to come soon.... :)

Thursday, July 12, 2012

Linux Multicore::Disable Core in Linux

We can disable the few cores from the multi-core linux machines by following the below commands:
$ sudo sh -c "echo 'n' > /sys/devices/system/cpu/cpu1/online"
E.g.
echo 0 > /sys/devices/system/cpu/cpu2/online; echo 0 > /sys/devices/system/cpu/cpu3/online;
The above line will disable the CPU 2 and 3 on a 4 CPU machine.
To enable it again:
 echo 1 > /sys/devices/system/cpu/cpu2/online; echo 1 > /sys/devices/system/cpu/cpu3/online;
The following command will display only enabled CPU:
cat /proc/cpuinfo