...

Cloud Computing - spark

Back to Course

Lesson Description


Lession - #775 Intersection Function


Spark Intersection Function

in Spark, Intersection work returns a new dataset that contains the crossing point of components present in the different datasets. Thus, it returns just a solitary line. This capacity acts very much like the INTERSECT inquiry in SQL.
Example of Intersection function In this example, we intersect the elements of two datasets.
    To open the Spark in Scala mode, follow the below command.

$ spark-shell



    Make a RDD utilizing the parallelized assortment.

scala> val data1 = sc.parallelize(List(1,2,3>
>


    Presently, we can peruse the produced outcome by utilizing the accompanying order.


scala> data1.collect



create another RDD utilizing parallelized assortment.

scala> val data2 = sc.parallelize(List(3,4,5>
>


Presently, we can peruse the created outcome by utilizing the accompanying order.

scala> data2.collect




    Apply crossing point(>
    capacity to return the convergence of the components.


scala> val intersectfunc = data1.intersection(data2>


Presently, we can peruse the created outcome by utilizing the accompanying order.

scala> intersectfunc.collect