在Pyspark中使用UDF函数时，密集向量应为哪种类型？[重复]

在Pyspark中使用UDF函数时，密集向量应为哪种类型？[重复]

您可以将vector和VectorUDT与UDF结合使用，

from pyspark.ml.linalg import Vectors, VectorUDTfrom pyspark.sql import functions as Fud_f = F.udf(lambda r : Vectors.dense(r),VectorUDT())df = df.withColumn('b',ud_f('a'))df.show()+-------------------------+---------------------+|a  |b         |+-------------------------+---------------------+|[0.1, 0.2, 0.3, 0.4, 0.5]|[0.1,0.2,0.3,0.4,0.5]|+-------------------------+---------------------+df.printSchema()root  |-- a: array (nullable = true)  |    |-- element: double (containsNull = true)  |-- b: vector (nullable = true)

关于VectorUDT，http：
//spark.apache.org/docs/2.2.0/api/python/_modules/pyspark/ml/linalg.html