3

I have a file called failedfiles.txt with the following content:

failed1
failed2
failed3

I need to use grep to return the content on each line in that file, and save the output in a list to be accessed. So I want something like this:

temp_list=$(grep "[a-z]" failedfiles.txt)

However, the problem with this is that when I type

echo ${temp_list[0]}

I get the following output:

failed1 failed2 failed3

But what I want is when I do:

echo ${temp_list[0]}

to print

failed1

and when I do:

echo ${temp_list[1]}

to print

failed2

Thanks.

1
  • Can you explain why you think you need to add the lines to an array? What is the underlying problem you are trying to solve? Commented Apr 1, 2014 at 7:32

3 Answers 3

13

@devnull's helpful answer explains why your code didn't work as expected: command substitution always returns a single string (possibly composed of multiple lines).

However, simply putting (...) around a command substitution to create an array of lines will only work as expected if the lines output by the command do not have embedded spaces - otherwise, each individual (whitespace-separated) word will become its own array element.


Capturing command output lines at once, in an array:

To capture the lines output by an arbitrary command in an array, use the following:

  • bash < 4 (e.g., on OSX as of OS X 10.9.2): use read -a
IFS=$'\n' read -rd '' -a linesArray <<<"$(grep "[a-z]" failedfiles.txt)"
  • bash >= 4: use readarray:
readarray -t linesArray <<<"$(grep "[a-z]" failedfiles.txt)"

Note:

  • <<< initiates a so-called here-string, which pipes the string to its right (which happens to be the result of a command substitution here) into the command on the left via stdin.
    • While command <<< string is functionally equivalent to echo string | command in principle, the crucial difference is that the latter creates subshells, which make variable assignments in command pointless - they are localized to each subshell.
  • An alternative to combining here-strings with command substitution is [input] process substitution - <(...) - which, simply put, allows using a command's output as if it were an input file; the equivalent of <<<"$(command)" is < <(command).
  • read: -a reads into an array, and IFS=$'\n' ensures that every line is considered a separate field and thus read into its own array element; -d '' ensures that ALL lines are read at once (before breaking them into fields); -r turns interpretation of escape sequence in the input off.
  • readarray (also callable as mapfile) directly breaks input lines into an array of lines; -t ensures that the terminating \n is NOT included in the array elements.

Looping over command output lines:

If there is no need to capture all lines in an array at once and looping over a command's output line by line is sufficient, use the following:

while IFS= read -r line; do
  # ...
done < <(grep "[a-z]" failedfiles.txt)
  • IFS= ensures that each line is read unmodified in terms of whitespace; remove it to have leading and trailing whitespace trimmed.
  • -r ensures that the lines are read 'raw' in that substrings in the input that look like escape sequences - e.g., \t - are NOT interpreted as such.
  • Note the use of [input] process substitution (explained above) to provide the command output as input to the read loop.
Sign up to request clarification or add additional context in comments.

Comments

3

You did not create an array. What you did was Command Substitution which would simply put the output of a command into a variable.

In order to create an array, say:

temp_list=( $(grep "[a-z]" failedfiles.txt) )

You might also want to refer to Guide on Arrays.

6 Comments

doesn't grep return a list of the lines where a match of the pattern was found?
So my argument ( well really im just trying to understand guru ) didn't I assign the return type to temp_list?
Yes, it does. Could you elaborate what you're attempting to clarify?
@kolonel I'm not sure what you mean by return type. However, what you did simply caused the output of the program to be assigned to a variable called temp_list. It did not create the array. If you quote the variable while printing, you might figure it: echo "$temp_list"
@kolonel Added a couple of links that might help.
|
2

The proper and portable way to loop over lines in a file is simply

while read -r line; do
    ... something with "$line"
done <failedfiles.txt

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.