How to use an array inside another array in scala in order to use each element from one array to append to corresponding element from other array?

Question

first array: var keyColumns = "A,B".split(",")

second array: var colValues = DataFrameTest.select("Y","Z").collect.map(row => row.toString) colValues: Array[String]= Array([1,2],[3,4],[5,6])

I want something as a result like: Array([A=1,B=2],[A=3,B=4],[A=5,B=6])

so that later I can iterate over this Array and can create my where clause like where (A=1 AND B=2) OR (A=3 AND B=4) OR (A=5 AND B=6)

Dima · Accepted Answer · 2021-04-09 11:18:25Z

1

First, don't convert structured data to string. Do .map(_.toSeq) after collect, not toString.

Then, something like this should work:

   colValues
     .map { _ zip keyColumns } 
     .map { _.map { case (v,k) => s"$k=$v" } }
     .map { _.mkString("(", " AND ", ")") }
     .mkString(" OR ")

You may find it helpful to run this step-by-step in REPL and see what each line does.

edited Apr 9, 2021 at 11:18

answered Apr 9, 2021 at 0:19

Dima

40.6k6 gold badges54 silver badges80 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

user8918714 Over a year ago

Thanks for the reply and the code snippet. It helps but .join does not work, 'value join is not a member of Seq[String]' I tried using map as below in place of join .map { _.map { e => e.split(",").mkString(" AND ")} It does not throw any error but it is not doing anything. When tried explicitly on the similar string, it works but not inside the chained transformations.

Dima Over a year ago

ugh, sorry ... mksString, not join ... I alway confuse these between different languages :) I'll update the answer.

Pritish · Accepted Answer · 2021-04-08 23:25:04Z

0

you can use regex expression, like:

scala> val keyColumns = "A,B".split(",")
keyColumns: Array[String] = Array(A, B)

scala> val colValues = "[1,2] [3,4] [5,6]".split(" ")
colValues: Array[String] = Array([1,2], [3,4], [5,6])

scala> val pattern = """^\[(.{1}),(.{1})\]$""".r //here, (.{1}) determines a regex group of exactly 1 any char
pattern: scala.util.matching.Regex = ^\[(.{1}),(.{1})\]$

scala> colValues.map { e => pattern.findFirstMatchIn(e).map { m => s"(${keyColumns(0)}=${m.group(1)} AND ${keyColumns(1)}=${m.group(2)})" }.getOrElse(e) }.mkString(" OR ")
res0: String = (A=1 AND B=2) OR (A=3 AND B=4) OR (A=5 AND B=6)

edited Apr 8, 2021 at 23:25

answered Apr 8, 2021 at 23:10

Pritish

5914 silver badges9 bronze badges

1 Comment

Dima Over a year ago

"(.{1}) determines a regex group of exactly 1 any char" -- and so does just . :D Or (.) if you want group.

Collectives™ on Stack Overflow

How to use an array inside another array in scala in order to use each element from one array to append to corresponding element from other array?

2 Answers 2

2 Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related