How to run the bash script in parallel rather than sequentially?

Question

#!/bin/bash

some_array=($1)

echo "-- Setting-Up VM --"

for i in ${some_array[@]}; do
    echo "VM #: $i"

    case "$i" in

        "1")
        echo "Setting-Up VM $i"
        sshpass -p "root12"  ssh -tt -o StrictHostKeyChecking=no [email protected] <<EOF
        pwd
        nohup yes | /etc/rc.d/init.d/lifconfig
        su tarts
        nohup yes | vncserver
        sleep 10
        exit
        exit
EOF
        ;;

        "2")
        echo "Setting-Up VM $i"
        sshpass -p "root12"  ssh -tt -o StrictHostKeyChecking=no [email protected] <<EOF
        pwd
        nohup yes | /etc/rc.d/init.d/lifconfig
        su tarts
        nohup yes | vncserver
        sleep 10
        exit
        exit
EOF
        ;;

        "3")
        echo "Setting-Up VM $i"
        sshpass -p "root12"  ssh -tt -o StrictHostKeyChecking=no [email protected] <<EOF
        pwd
        nohup yes | /etc/rc.d/init.d/lifconfig
        su tarts
        nohup yes | vncserver
        sleep 10
        exit
        exit
EOF
        ;;
        *)
        echo "unknown VM!"
        ;;
    esac
done

Can someone please guide me I have the above script which is executed when we run the script for instance ./vmSetup.sh "1 2 3" but this is executed sequentially. I had created this script but now I want to run the cases in the script i.e. 1, 2 and 3 in parallel. Can someone also tell me how to run for instance 8 cases in parallel?

Note about heredocs: EOF must be the only characters on the line ending the heredoc: no leading spaces are allowed. You can use leading tabs if you do <<-EOF — glenn jackman
– glenn jackman, Commented Jun 4, 2021 at 15:48
@glennjackman That was my mistake when fixing the rest of the indentation. It was correct before my edit — Barmar
– Barmar, Commented Jun 4, 2021 at 17:28
Why do you have the case statement in the first place? All the cases do the same things. — Barmar
– Barmar, Commented Jun 4, 2021 at 17:31
How can I use just one case statement and then use any of the 25 cases I have given in the script? The difference between the cases is only the IP addresses. — Asad Javed
– Asad Javed, Commented Jun 4, 2021 at 17:55

Barmar · Accepted Answer · 2021-06-04 17:28:45Z

3

Put each case ... esac statement in the background by ending it with &. Then use wait after the loop to wait for all the background processes to finish.

#!/bin/bash

some_array=($1)

echo "-- Setting-Up VM --"

for i in ${some_array[@]}; do
    echo "VM #: $i"

    case "$i" in

        "1")
        echo "Setting-Up VM $i"
        sshpass -p "root12"  ssh -tt -o StrictHostKeyChecking=no [email protected] <<EOF
        pwd
        nohup yes | /etc/rc.d/init.d/lifconfig
        su tarts
        nohup yes | vncserver
        sleep 10
        exit
        exit
EOF
        ;;

        "2")
        echo "Setting-Up VM $i"
        sshpass -p "root12"  ssh -tt -o StrictHostKeyChecking=no [email protected] <<EOF
        pwd
        nohup yes | /etc/rc.d/init.d/lifconfig
        su tarts
        nohup yes | vncserver
        sleep 10
        exit
        exit
EOF
        ;;

        "3")
        echo "Setting-Up VM $i"
        sshpass -p "root12"  ssh -tt -o StrictHostKeyChecking=no [email protected] <<EOF
        pwd
        nohup yes | /etc/rc.d/init.d/lifconfig
        su tarts
        nohup yes | vncserver
        sleep 10
        exit
        exit
EOF
        ;;
        *)
        echo "unknown VM!"
        ;;
    esac &
done

wait

edited Jun 4, 2021 at 17:28

answered Jun 4, 2021 at 15:42

Barmar

789k57 gold badges554 silver badges669 bronze badges

Sign up to request clarification or add additional context in comments.

7 Comments

Asad Javed Over a year ago

Thanks Barmar but also is there a way to do it with GNU Parallel?

Barmar Over a year ago

Probably. But since I'm not familiar with GNU parallel I didn't use it in my answer.

Barmar Over a year ago

Why don't you just run the whole script using GNU Parallel?

Asad Javed Over a year ago

How can I do that with GNU Parallel? I don't know how to use the package even though I have installed it on my VM? Secondly, can you tell me how to run 8 cases in parallel at the same time?

Barmar Over a year ago

Like I said, I'm not familiar with GNU Parallel. Unix & Linux would be a better place to ask how to use that.

|

Paul Hodges · Accepted Answer · 2021-06-07 13:59:45Z

Why do you have a case statement at all when all these are identical?

#!/bin/bash
echo "-- Setting-Up VM(s) --"
for i in "$@"; do
case "$i" in
1) IP=1.2.3.4;;
2) IP=2.2.3.4;;
3) IP=3.2.3.4;;
*) echo "Invalid option '$i'" >&2; exit 1;;
esac
echo "Setting-Up VM $i"
sshpass -p root12 ssh -tt -o StrictHostKeyChecking=no root@$IP <<EOF &
  pwd
  nohup yes | /etc/rc.d/init.d/lifconfig
  su tarts
     nohup yes | vncserver
     sleep 10
     exit
  exit
EOF
done

To run a job in background, just use &

sshpass -p "root12"  ssh -tt -o StrictHostKeyChecking=no root@$IP <<EOF &

It's just confusing because of the here-doc, but the & metacharacter still parses correctly on my system. Try it.

Also, rather than quoting all your options in one string and then parsing them back out to an array, why not just simplify the whole thing and let them come in as separate arguments? Call it as

./vmSetup.sh 1 2 3 # NOT "1 2 3" with quotes

and the loop becomes just

for i in "$@" # properly quoted, though wouldn't matter for 1 2 3

This whole thing seems a lot simpler and easier to maintain.

Another option, which is what I would do: create a file that is a list of the DNS/IP addresses you need, then pass that file as the lone argument.

#!/bin/bash
while read -r addr || [[ -n "$addr" ]]
do echo "Setting-Up VM $addr"
sshpass -p "root12"  ssh -tt -o StrictHostKeyChecking=no root@$addr <<EOF &
  pwd
  nohup yes | /etc/rc.d/init.d/lifconfig
  su tarts
     nohup yes | vncserver
     sleep 10
     exit
  exit
EOF
done < "$1"

Better, add valid error checking first to make sure the file exists and is readable, etc, but as a simple case, this should work.

For even better, cleaner, safer code, read https://mywiki.wooledge.org/BashFAQ/001 and follow those suggestions. :)

Yes can you please tell me how to use just one case statement and shorten the script. The IP is the difference only.

Ole Tange · Accepted Answer · 2021-06-10 05:55:29Z

0

If ./vmSetup.sh 1 works as expected, these should work:

parallel -j8 ./vmSetup.sh {} ::: 1 2 3 ... 100
seq 100 | parallel -j8 ./vmSetup.sh

answered Jun 10, 2021 at 5:55

Ole Tange

34.1k9 gold badges93 silver badges111 bronze badges

Collectives™ on Stack Overflow

How to run the bash script in parallel rather than sequentially?

3 Answers 3

7 Comments

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

7 Comments

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related