Bash text database file read in loop

Question

Can anyone see the last syntax error I have made here? The issue is with incrementing the DB"X"FIELDS array. In a way, I'm simulating a mutlidimensional array by storing all records from each database into its own array so they can be called later. I can't figure out the syntax through to declare the DB array using an embedded variable that increments. Maybe I just need to Set the name first (including the variable, then declare the array.

function DATABASE.READ {
  cd databases
  DBARRAY=(*)
  cd ..
  local i
  local j
  for i in "${!DBARRAY[@]}"; do
    declare -a DB$iFIELDS                 ### Here is the problem. Declaring the array.
    while read LINE; do 
      for (( j=1; j<=25; j++ )); do
        VALUE="`echo $LINE | cut -d"|" -f"$j"`";
        DB$iFIELDS[$j]="$VALUE"           ### Also here, which I believe requires a different syntax
      done
    done < <(grep -v '#' databases/${DBARRAY[$i]})
  done
}

* UPDATE *

Ok, so I had a big realization that I was leaving out an entire dimension of the database. I have since revised the code to create the additional arrays to hold all database information. Yes I know this is starting to get a little out of hand. I appreciate all the help though. Thanks

function DATABASE.READ {
  cd databases
  DBARRAY=(*)
  cd ..
  local i
  local j
  local k
  i=1
  j=1
  for i in "${!DBARRAY[@]}"; do
    while read LINE; do
      declare -a "D${i}R${j}F"
      for (( k=1; k<=25; k++ )); do
        VALUE="`echo $LINE | cut -d"|" -f"$k"`";
        eval "D${i}R${j}F[$k]=\"$VALUE\""
      done
      let j=j+1
    done < <(grep -v '#' databases/${DBARRAY[$i]})
  done
  ########## EXAMPLE ECHO OF ONE SPECIFIC FIELD IN DATABASE1 RECORD1 FIELD1 ##########
  echo "${D1R1F[1]}"
}

* UPDATE *

THANKS to both of you. There is still a small logic error that I'm having trouble identifing, but I'm sure it's much simplier than what we were initially troubleshooting. The original issue appears to be resolved thanks to you guys. Here is the current code:

function DATABASE.READ {
  cd databases
  DBARRAY=(*)
  cd ..
  local i
  local j
  local k
  i=1
  for i in "${!DBARRAY[@]}"; do
    j=1
    while read LINE; do
      declare -a "D${i}R${j}F"
      for (( k=1; k<=25; k++ )); do
        VALUE="`echo $LINE | cut -d"|" -f"$k"`";
        printf -v "D${i}R${j}F[$k]" '%s' "$VALUE"
      done
      let j=j+1
    done < <(grep -v '#' databases/${DBARRAY[$i]})
    unset j
  done
  ########## ECHO TWO SPECIFIC FIELDS IN EACH DATABASE ##########
  echo "${D1R1F[1]}"
  echo "${D2R1F[1]}"
  echo "${D3R1F[1]}"
  echo "${D1R1F[3]}"
  echo "${D2R1F[3]}"
  echo "${D3R1F[3]}"
}

It looks like it's having trouble echoing both database 1 records. Trying to figure out what in the loop is goofing it up, or if maybe its a bigger problem.

* SOLUTION *

So it looks like the issue was with the database incrementing. When I don't specifically set i, and reference the databases starting at 0. It fixes it. Actually... it makes sense now. When the DB files are read into DBARRAY, they start with element 0 so yeah.....

So which of you do I give credit too? :)

$iFIELDS looks for a variable named iFIELDS, not a variable named i followed by the string FIELDS. Try DB${i}FIELDS. Also, the array access should look like ${DB${i}FIELDS[$j]}=..., I would think... Although you may need to wrap that entire line in eval, because bash doesn't generally do computed variable names the way you are trying to do. So eval "${DB${i}FIELDS[$j]}=\"$VALUE\"", maybe... — twalberg
– twalberg, Commented Apr 30, 2014 at 19:26
var/anm.fcn: line 25: ${DB${i}FIELDS[$j]}="$VALUE": bad substitution — Atomiklan
– Atomiklan, Commented Apr 30, 2014 at 19:40
Your problem now seems to be on line: declare -a "D${i}R${j}F" because the j=j+1 drags the j+ part, you have to change j=j+1 with let j=j+1, I have updated my answer, can you check it please? — dinox0r
– dinox0r, Commented Apr 30, 2014 at 20:34
Yeah I just caught that and updated above. I already added let. — Atomiklan
– Atomiklan, Commented Apr 30, 2014 at 20:36
Just a stupid remark: your j is never reset, so very likely there are no D1R1F[1] that are set. Apart from that, I can guarantee that your variables are set, and that printf is the way to do it, not eval. — gniourf_gniourf
– gniourf_gniourf, Commented Apr 30, 2014 at 21:08

dinox0r · Accepted Answer · 2014-04-30 21:08:21Z

0

Try the following updated version:

function DATABASE.READ {
   cd databases
   DBARRAY=(*)
   cd ..
   local i
   local j
   local k
   i=1
   j=1
   for i in "${!DBARRAY[@]}"; do
      declare -a "DB${i}FIELDS"
      while read LINE; do
         declare -a "D${i}R${j}F"
         for (( k=1; k<=25; k++ )); do
            VALUE="`echo $LINE | cut -d"|" -f"$k"`";
   echo "VALUE = '$VALUE'"
            eval "D${i}R${j}F[$k]=\"$VALUE\""
   eval "echo ${D${i}R${j}F[$k]}"
         done
         let j=j+1
      done < <(grep -v '#' databases/${DBARRAY[$i]})
   done
   ########## EXAMPLE ECHO OF ONE SPEFIC FIELD IN ##########
   echo "${D1R1F[1]}"
}

edited Apr 30, 2014 at 21:08

answered Apr 30, 2014 at 19:29

dinox0r

16.1k4 gold badges39 silver badges43 bronze badges

Sign up to request clarification or add additional context in comments.

10 Comments

dinox0r Over a year ago

mmm, weird, it should work, for example, the following test code works using eval as in the code: DB1=(a b c);echo "${DB1[@]}";i=1; eval "DB${i}[0]=\"b\""; echo "${DB1[@]}", it prints a b c and then prints b b c

Atomiklan Over a year ago

Currently there are three databases. i = 0 - 2. The first database is filled with test information in each record. If I try to do the following I get nothing. Maybe I'm just doing something really really stupid with the echo. echo "${DB0FIELDS[3]}"

Atomiklan Over a year ago

OH SHOOT... I forgot one very important piece... This does not account for each individual record. I will need a third layer to store every field from every record, from every database. Ugh need multidimensional arrays. Maybe it is time to move to a real language.

Atomiklan Over a year ago

Normally I process the database info directly in the loop. I kept recalling the database over and over again though in code. So I have been trying to write a function to do it ONCE and pull everything (since it will use everything) that way its only calling the DB once in code.

Atomiklan Over a year ago

I tried your update code above. Still no luck. VALUE is not being stored.

|

Collectives™ on Stack Overflow

Bash text database file read in loop

1 Answer 1

10 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

10 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related