Help the world stop coronavirus! Stay home!
Prev
Next
BigData / Apache Spark
Explain vectorAssembler in MLlib.
VectorAssembler is a transformer that combines a given list of columns into a single vector column.
VectorAssembler accepts the following input column types: all numeric types, boolean type, and vector type. In each row, the values of the input columns will be concatenated into a vector in the specified order.
scala> val vaDF = spark.read.option("multiLine",true).json("vectorAssemblerTest.data")
vaDF: org.apache.spark.sql.DataFrame = [id: bigint, mobile: double ... 3 more fields]
scala> vaDF.show
+---+------+---------+----+-----------------+
| id|mobile|otherData|time| userFeatures|
+---+------+---------+----+-----------------+
| 1| 1.0| yes| 18|[0.0, 11.0, 12.0]|
+---+------+---------+----+-----------------
scala> import org.apache.spark.ml.feature.VectorAssembler
import org.apache.spark.ml.feature.VectorAssembler
scala> val assembler = new VectorAssembler()
assembler: org.apache.spark.ml.feature.VectorAssembler = vecAssembler_dbd3d0a8c760
scala> val assembler = new VectorAssembler().setInputCols(Array("id","mobile","time")).setOutp
utCol("outputVectorColumn")
assembler: org.apache.spark.ml.feature.VectorAssembler = vecAssembler_65938f964d7f
scala> val output = assembler.transform(vaDF)
output: org.apache.spark.sql.DataFrame = [id: bigint, mobile: double ... 4 more fields]
scala> output.show
+---+------+---------+----+-----------------+------------------+
| id|mobile|otherData|time| userFeatures|outputVectorColumn|
+---+------+---------+----+-----------------+------------------+
| 1| 1.0| yes| 18|[0.0, 11.0, 12.0]| [1.0,1.0,18.0]|
+---+------+---------+----+-----------------+------------------+
scala>
❤Cash Back At Stores you Love !!!❤
Earn your $10 reward when you make your first purchase through Ebates by signing up with clicking below button.
More Related questions...
