Help the world stop coronavirus! Stay home!

Prev Next

BigData / Apache Spark

Explain vectorAssembler in MLlib.

VectorAssembler is a transformer that combines a given list of columns into a single vector column.

VectorAssembler accepts the following input column types: all numeric types, boolean type, and vector type. In each row, the values of the input columns will be concatenated into a vector in the specified order.

scala> val vaDF = spark.read.option("multiLine",true).json("vectorAssemblerTest.data")
vaDF: org.apache.spark.sql.DataFrame = [id: bigint, mobile: double ... 3 more fields]
scala> vaDF.show
+---+------+---------+----+-----------------+
| id|mobile|otherData|time|     userFeatures|
+---+------+---------+----+-----------------+
|  1|   1.0|      yes|  18|[0.0, 11.0, 12.0]|
+---+------+---------+----+-----------------

scala> import org.apache.spark.ml.feature.VectorAssembler
import org.apache.spark.ml.feature.VectorAssembler

scala> val assembler = new VectorAssembler()
assembler: org.apache.spark.ml.feature.VectorAssembler = vecAssembler_dbd3d0a8c760

scala> val assembler = new VectorAssembler().setInputCols(Array("id","mobile","time")).setOutp
utCol("outputVectorColumn")
assembler: org.apache.spark.ml.feature.VectorAssembler = vecAssembler_65938f964d7f

scala> val output = assembler.transform(vaDF)
output: org.apache.spark.sql.DataFrame = [id: bigint, mobile: double ... 4 more fields]

scala> output.show
+---+------+---------+----+-----------------+------------------+
| id|mobile|otherData|time|     userFeatures|outputVectorColumn|
+---+------+---------+----+-----------------+------------------+
|  1|   1.0|      yes|  18|[0.0, 11.0, 12.0]|    [1.0,1.0,18.0]|
+---+------+---------+----+-----------------+------------------+


scala>

❤Cash Back At Stores you Love !!!❤

Earn your $10 reward when you make your first purchase through Ebates by signing up with clicking below button.

Ebates Coupons and Cash Back

More Related questions...

Show more question and Answers...

TensorFlow

Comments & Discussions