...

Cloud Computing - RDD SPARK

Back to Course

Lesson Description


Lession - #1477 Distinct Function


Spark Distinct Function

In Spark, the Distinct capacity returns the particular components from the gave dataset.
Example of Distinct function
In this model, we disregard the copy components and recovers just the unmistakable components.
    To open the flash in Scala mode, follow the underneath order.

$spark-shell


    make a RDD utilizing parallelized assortment.

scala> val data = sc.parallelize(List(10,20,20,40>
>

Presently, we can peruse the created outcome by utilizing the accompanying order.
scala> data.collect


    Apply particular(>
    capacity to disregard copy components.

scala> val distinctfunc = data.distinct(>

Presently, we can peruse the produced outcome by utilizing the accompanying order.
scala> distinctfunc.collect