Word2Vecword2vec能將文本中出現的詞向量化，可以在捕捉語境信息的同時壓縮數據規模。Word2Vec實際上是兩種不同的方法：Continuous Bag of Words (CBOW) 和 Skip-gram。CBOW的目標是根據上下文來預測當前詞語的概率。Skip-gram剛好相反：根據當前詞語來預測上下文的概率。這兩種方法都利用人工神經網絡作為它們的分類算法。起初，每個單詞都是一個隨機 N 維向量。經過訓練之后，該算法利用 CBOW 或者 Skip-gram 的方法獲得了每個單詞的最優向量。?

實例代碼如下：

package sk.mlib;import java.util.Arrays; import java.util.List;import org.apache.spark.ml.feature.Word2Vec; import org.apache.spark.ml.feature.Word2VecModel; import org.apache.spark.sql.Dataset; import org.apache.spark.sql.Row; import org.apache.spark.sql.RowFactory; import org.apache.spark.sql.SparkSession; import org.apache.spark.sql.types.*;public class Word2VecDemo {public static void main(String[] args) {SparkSession spark = SparkSession.builder().appName("Word2VecDemo").getOrCreate();// Input data: Each row is a bag of words from a sentence or document.List<Row> data = Arrays.asList(RowFactory.create(Arrays.asList("Hi I heard about Spark".split(" "))),RowFactory.create(Arrays.asList("I wish Java could use case classes".split(" "))),RowFactory.create(Arrays.asList("Logistic regression models are neat".split(" "))));StructType schema = new StructType(new StructField[]{new StructField("text", new ArrayType(DataTypes.StringType, true), false, Metadata.empty())});Dataset<Row> documentDF = spark.createDataFrame(data, schema);// Learn a mapping from words to Vectors.Word2Vec word2Vec = new Word2Vec().setInputCol("text").setOutputCol("result").setVectorSize(3).setMinCount(0);Word2VecModel model = word2Vec.fit(documentDF);Dataset<Row> result = model.transform(documentDF);for (Row r : result.select("text","result").takeAsList(3)) {System.out.println(r);}spark.stop();} } /*執行結果：[WrappedArray(Hi, I, heard, about, Spark),[-0.028139343485236168,0.04554025698453188,-0.013317196490243079]] [WrappedArray(I, wish, Java, could, use, case, classes),[0.06872416580361979,-0.02604914902310286,0.02165239889706884]] [WrappedArray(Logistic, regression, models, are, neat),[0.023467857390642166,0.027799883112311366,0.0331136979162693]]*/

總結

以上是生活随笔為你收集整理的离线轻量级大数据平台Spark之MLib机器学习库Word2Vec实例的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。