...

Cloud Computing - spark

Back to Course

Lesson Description


Lession - #773 Union Function

Spark Union Function

In Spark, Union capacity returns a new dataset that contains the mix of components present in the different datasets.

Example of Union function
In this model, we consolidate the components of two datasets.

  • To open the flash in Scala mode, follow the underneath order.

$ spark-shell



  • Make a RDD utilizing parallelized assortment.

scala> val data1 = sc.parallelize(List(1,2>
>

  • Presently, we can peruse the created outcome by utilizing the accompanying order.

scala> data1.collect



    Create another RDD using parallelized collection.

      scala> val data2 = sc.parallelize(List(3,4,5>
      >

      • Presently, we can peruse the created outcome by utilizing the accompanying order.

      scala> data2.collect



      • Apply association(>
        capacity to return the association of the components.

      scala> val unionfunc = data1.union(data2>

        Presently, we can peruse the created outcome by utilizing the accompanying order.

      scala> unionfunc.collect






Distinct Function