Bash Concatenating Strings Improperly

Question

I have a text file with a list of Mercurial repositories in it, in the form:

IDE
Install
InstallShield

I'm writing a bash script to clone/pull/update all the repositories based on the text file. Right now I'm just echoing before I do any actual cloning. If I do:

while read line; do
    echo "hg clone" ${MASTER_HG}/${line};
done < Repos.txt

The output is as expected:

hg clone /media/fs02/IDE
hg clone /media/fs02/Install
hg clone /media/fs02/InstallShield

However, if I do:

while read line; do
    echo "hg clone" ${MASTER_HG}/${line} ${REPOROOT}/${line};
done < Repos.txt

The output is:

/var/hg/repos/IDE02/IDE 
/var/hg/repos/Installnstall
/var/hg/repos/InstallShieldShield

It seems to be replacing the beginning of the string with the end of the string. Is there some kind of character overflow or something going on? My apologies if this is a dumb question, but I'm a relative noob for bash.

Carriage returns? CRLF endings in Repos.txt?

Jonathan Leffler
– Jonathan Leffler

2013-07-31 20:48:50 +00:00
Commented Jul 31, 2013 at 20:48 — Jonathan Leffler
– Jonathan Leffler, Commented Jul 31, 2013 at 20:48
CRLF is the new square root of all evil.

devnull
– devnull

2013-07-31 20:50:09 +00:00
Commented Jul 31, 2013 at 20:50 — devnull
– devnull, Commented Jul 31, 2013 at 20:50
+1 for "a relative noob" with some actual code.

devnull
– devnull

2013-07-31 20:56:52 +00:00
Commented Jul 31, 2013 at 20:56 — devnull
– devnull, Commented Jul 31, 2013 at 20:56

chepner · Accepted Answer · 2013-07-31 20:59:54Z

6

Your file has DOS line endings; the \r at the end of $line causes the cursor to return to the beginning of the line, which only affects your output when $line is not the last thing being printed before the newline. You should remove them with something like dos2unix.

You can use something similar to Perl's chomp command to remove a trailing carriage return, if one is present:

# $'\r' is bash-only, but easy to type. For POSIX shell, you'll need to find
# someway of entering ASCII character 13; perhaps Control-V Control-M
line=${line%$'\r'}

Useful if, for whatever reason, you can't (or don't want to) fix the input before reading it.

edited Jul 31, 2013 at 20:59

answered Jul 31, 2013 at 20:48

chepner

538k77 gold badges594 silver badges746 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Chris Bush Over a year ago

Good grief. You're right. Thank you! I should have noticed the dos flag at the bottom of my editor. I'm embarrassed to say how long that stumped me.

SethMMorton Over a year ago

@ChrisBush I'm sure it's happened to all of us at least once.

CDspace · Accepted Answer · 2013-07-31 20:50:52Z

1

From the looks of it, ${REPOROOT} might already include ${line}, try echoing ${REPOROOT} by itself and see what you get.

answered Jul 31, 2013 at 20:50

CDspace

2,69919 gold badges32 silver badges39 bronze badges

3 Comments

Chris Bush Over a year ago

Thanks for the answer. It turned out to be DOS-esque carriage returns like chepner suggested.

CDspace Over a year ago

Ah, I see now. I wrote my answer before you edited the formatting, so I didn't notice that some of the characters were missing, rather than simply repeated the ${line} value... capital i = I != l = lowercase L

Chris Bush Over a year ago

That was my fault. Thanks to Ruben for highlighting the output.

Collectives™ on Stack Overflow

Bash Concatenating Strings Improperly

2 Answers 2

2 Comments

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related