Prev Next

BigData / Apache Spark

Explain vectorAssembler in MLlib.

VectorAssembler is a transformer that combines a given list of columns into a single vector column.

VectorAssembler accepts the following input column types: all numeric types, boolean type, and vector type. In each row, the values of the input columns will be concatenated into a vector in the specified order.

scala> val vaDF = spark.read.option("multiLine",true).json("vectorAssemblerTest.data")
vaDF: org.apache.spark.sql.DataFrame = [id: bigint, mobile: double ... 3 more fields]
scala> vaDF.show
+---+------+---------+----+-----------------+
| id|mobile|otherData|time|     userFeatures|
+---+------+---------+----+-----------------+
|  1|   1.0|      yes|  18|[0.0, 11.0, 12.0]|
+---+------+---------+----+-----------------

scala> import org.apache.spark.ml.feature.VectorAssembler
import org.apache.spark.ml.feature.VectorAssembler

scala> val assembler = new VectorAssembler()
assembler: org.apache.spark.ml.feature.VectorAssembler = vecAssembler_dbd3d0a8c760

scala> val assembler = new VectorAssembler().setInputCols(Array("id","mobile","time")).setOutp
utCol("outputVectorColumn")
assembler: org.apache.spark.ml.feature.VectorAssembler = vecAssembler_65938f964d7f

scala> val output = assembler.transform(vaDF)
output: org.apache.spark.sql.DataFrame = [id: bigint, mobile: double ... 4 more fields]

scala> output.show
+---+------+---------+----+-----------------+------------------+
| id|mobile|otherData|time|     userFeatures|outputVectorColumn|
+---+------+---------+----+-----------------+------------------+
|  1|   1.0|      yes|  18|[0.0, 11.0, 12.0]|    [1.0,1.0,18.0]|
+---+------+---------+----+-----------------+------------------+


scala>

It's right time to invest in Cryptocurrencies Dogecoin! Earn free bitcoins up to $250 now by signing up.

Earn bitcoins upto $250 (free), invest in other Cryptocurrencies when you signup with blockfi. Use the referral link: Signup now and earn!

Using BlockFi, don't just buy crypto - start earning on it. Open an interest account with up to 8.6% APY, trade currencies, or borrow money without selling your assets.


Join CoinBase! We'll both receive $10 in free Bitcoin when they buy or sell their first $100 on Coinbase! Available in India also. Use the referral Join coinbase!


Invest now!!! Get Free equity stock (US, UK only)!

Use Robinhood app to invest in stocks. It is safe and secure. Use the Referral link to claim your free stock when you sign up!.

The Robinhood app makes it easy to trade stocks, crypto and more.


Webull! Receive free stock by signing up using the link: Webull signup.

More Related questions...

Show more question and Answers...

TensorFlow

Comments & Discussions