Running Shell Script having multiple programs dynamically in parallel

Question

I have a shell script which captures the Process ID, CPU and Memory of the JVM every nth second and writes the output to a file. Below is my code:

JVM="aaa001_bcdefx01"
systime=$(date +"%m-%d-%y-%T")
for i in {1..10}
do
PID=`ps -auxwww | grep "jdk" |  grep $JVM | grep -v grep | cut -c -30 | awk '{print  $2}'`
MEM=`ps -auxwww | grep "jdk" |  grep $JVM | grep -v grep | cut -c -30 | awk '{print  $4 }'`
CPU=`ps -auxwww | grep "jdk" |  grep $JVM | grep -v grep | cut -c -30 | awk '{print  $3 }'`
printf '%-5s %-20s %-20s %-20s %-20s \n' "$systime $JVM $PID $CPU $MEM "  >> $LOGFILE
sleep 5
done

This run perfectly fine when i have only one JVM in that server. How can i execute the same script in parallel and fetch the details if i have multiple JVM for a server.

I looked for some solutions and found & to be used in the script but couldn't understand how this can be implemented to my above script. Let's say I have 5 JVMs. How can i run the script and fetch the stats in parallel for all the below JVMs in parallel. Kindly guide. Any help would be appreciated.

JVM="aaa001_bcdefx01"
JVM="aaa002_bcdefx01"
JVM="aaa003_bcdefx01"
JVM="aaa004_bcdefx01"
JVM="aaa005_bcdefx01"

the above looks redundant. What's $systime and why for i in {1..10}? — RomanPerekhrest
– RomanPerekhrest, Commented Dec 8, 2017 at 15:20
systime=$(date +"%m-%d-%y-%T") is the time. updated in the script. i need to run the script for so many times with a sleep time(interval). — sdgd
– sdgd, Commented Dec 8, 2017 at 15:30
My golly - 18 processes to get PID, MEM and CPU! All you really need is two... read PID MEM CPU < <(ps ... | awk '...{print $2,$4,$3}') — Mark Setchell
– Mark Setchell, Commented Dec 8, 2017 at 15:48

Ole Tange · Accepted Answer · 2017-12-19 23:59:18Z

GNU Parallel is made for this kind of stuff

doit() {
  JVM="$1"
  systime=$(date +"%m-%d-%y-%T")
  for i in {1..10}
  do
    PID=`ps -auxwww | grep "jdk" |  grep $JVM | grep -v grep | cut -c -30 | awk '{print  $2}'`
    MEM=`ps -auxwww | grep "jdk" |  grep $JVM | grep -v grep | cut -c -30 | awk '{print  $4 }'`
    CPU=`ps -auxwww | grep "jdk" |  grep $JVM | grep -v grep | cut -c -30 | awk '{print  $3 }'`
    printf '%-5s %-20s %-20s %-20s %-20s \n' "$systime $JVM $PID $CPU $MEM "
    sleep 5
  done
}
export -f doit
parallel -j0 --linebuffer --tag doit ::: aaa00{1..5}_bcdefx01 >> $LOGFILE

The function is basically your code. The change is that it takes the JVM as argument and it prints to stdout (standard output). GNU Parallel calls the function with the arguments aaa00N_bcdefx01 where N = 1..5, and saves the output to $LOGFILE. It uses --linebuffer to pass the output as soon as there is a full line, and thus guarantees that you will not get half-a-line from one process mixed with a line from another process. --tag prepends the line with the JVM.

d95mtrx · Accepted Answer · 2017-12-08 15:51:44Z

0

How about using subshell?

Each JVM shell script should go inside '(' and ')'. Put '&' at the end so that it executes in the background.

An example is given here.

#!/bin/bash
echo > testfile.txt
echo "execute subshell 1"
(
#JVM 1 should go here
sleep 10
echo "subshell 1" >> testfile
)&

echo "execute subshell 2"
(
#JVM 2 should go here
sleep 10
echo "subshell 2" >> testfile
)&

echo "execute subshell 3"
(
#JVM 3 should go here
sleep 10
echo "subhsell 3" >> testfile
)&

Here each subshell writes data to testfile.txt after waiting for 10 seconds.

answered Dec 8, 2017 at 15:51

d95mtrx

176 bronze badges

3 Comments

sdgd Over a year ago

is this approach possible if i don't know the JVM count. How can i go about that way

d95mtrx Over a year ago

First find all the JVM info "aaa001_bcdefx01", "aaa002_bcdefx01".. and store it in a file. Use for loop to go through each JVM data and use subshell inside for loop

Ole Tange Over a year ago

You are tempting faith by appending to the same file in parallel. When you have small output it mostly does the right thing, but there is no guarantee for this. I will recommend that if you cannot explain when it is safe to do and when it is not safe to do, you should refrain from doing it.

Collectives™ on Stack Overflow

Running Shell Script having multiple programs dynamically in parallel

2 Answers 2

Comments

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related