0

Trying to run a simple shell script which consists of set of commands which I routinely execute to setup my Hadoop environmnent. First I'm connecting through 'ssh localhost' before I execute these commands manually, I want to automate this process, I'm the doing the following which is resulting in the errors listed below. Please note I'm able to execute these commands manually from command prompt. Please help me out.

What I did

$sudo chmod 777 hadoop-startup-script.sh
$ssh localhost
$./hadoop-startup-script.sh

This results in the following errors

(~($))sh hadoop-startup-script.sh
hadoop-startup-script.sh: 2: cd: can't cd to /home/hadoop/hadoop-0.20.204.0
hadoop-startup-script.sh: 3: hadoop-startup.sh: bin/hadoop: not found
hadoop-startup-script.sh: 4: hadoop-startup.sh: bin/start-all.sh: not found
hadoop-startup-script.sh: 5: hadoop-startup.sh: bin/hadoop: not found
hadoop-startup-script.sh: 6: hadoop-startup.sh: bin/hadoop: not found
hadoop-startup-script.sh: 7: hadoop-startup.sh: bin/hadoop: not found
hadoop-startup-script.sh: 8: hadoop-startup.sh: bin/hadoop: not found
hadoop-startup-script.sh: 9: hadoop-startup.sh: bin/hadoop: not found
hadoop-startup-script.sh: 10: hadoop-startup.sh: bin/hadoop: not found

Contents of hadoop-startup-script.sh

#! /bin/bash 
cd ~/hadoop-0.20.204.0
bin/hadoop namenode -format
bin/start-all.sh 
bin/hadoop fs -mkdir input
bin/hadoop fs -mkdir input/oreilly
bin/hadoop fs -put /home/hadoop/Dropbox/hadoop/hadoop-inputs/oreilly-programmingPig/*.* /user/hadoop/input/oreilly/
bin/hadoop fs -put /home/hadoop/Dropbox/hadoop/hadoop-inputs/oreilly-programmingPig/NYSE_dividends  /user/hadoop/input/oreilly/
bin/hadoop fs -put /home/hadoop/Dropbox/hadoop/hadoop-inputs/oreilly-programmingPig/baseball  /user/hadoop/input/oreilly/ 
bin/hadoop fs -put /home/hadoop/Dropbox/hadoop/hadoop-inputs/oreilly-programmingPig/NYSE_daily  /user/hadoop/input/oreilly/
Improve this question
3
  • So what’s your question? Commented Sep 22, 2013 at 8:32
  • 1
    The first error is the one that matters, all the others derive from it since you're never got to the folder where the other commands could work. Maybe check the permissions on that folder? Commented Sep 22, 2013 at 8:48
  • 1
    Does ~/hadoop-0.20.204.0 exist? Run ls -l ~/hadoop-0.20.204.0. Commented Sep 22, 2013 at 13:00

1 Answer 1

Reset to default
2

The errors are telling you that the locations ~/hadoop-0.20.204.0, bin/hadoop and bin/start-all.sh don't exist.

Did you mean ~/bin, /bin or ~/hadoop-0.20.204.0/bin? If you meant ~/hadoop-0.20.204.0/bin then the first failing cd explains the the following errors. Check the name and permissions on the folder carefully.

Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

OOps its typo in my cd command which was triggering the chain of erros. Sorry for bugging with such stupid question, correcting the hadoop folder name in the 'cd .. ' command did the trick.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.