Solve scala. collection. mutable. Map writing problem

Variable Map writing in scala

Use

val no2ID = scala.collection.mutable.Map[Int,String]()
no2ID += (1 -> "something")

In the following code, Map cannot add elements

val no2ID = scala.collection.mutable.Map[Int,String]()
sc.textFile(conf).foreach(line=>{
val splits = line.split( " , " )
val no = splits(0).trim.toInt
val ID = splits(1)
no2ID += (no -> ID)
})

Where the conf file is a number and a corresponding person name

After converting RDD to Array using collect, you can add elements to Map

val no2ID = scala.collection.mutable.Map[Int,String]()
sc.textFile(conf).collect().foreach(line=>{
val splits = line.split( " , " )
val no = splits(0).trim.toInt
val ID = splits(1)
no2ID += (no -> ID)
})

Use Example of Map in scala

Map structure is a very common structure, which has corresponding api in various programming languages. Because the underlying language of Spark is Scala, it is necessary to know how to use Map in Scala.

(1) Immutable Map

Features:

api is not very rich

If it is var modified, the reference is variable, and it supports reading and writing

In case of val modification, the reference is immutable, the value can only be written once, and then read only

var a:Map[String,Int]=Map("k1"->1,"k2"->2)// Initialize constructor 
    a += ("k3"->3)// Add Element 
    a += ("k4"->4)// Add Element 
    a += ("k1"->100)// Adding elements that already exist overrides 
    a -= ("k2","k1")// Delete Element 
//    a("k1") = "foo"// Not supported 
    println(a.contains("k6"))// Whether to contain an element 
    println(a.size)// Print size 
    println(a.get("k1").getOrElse("default")) // According to key Read the element and replace it with the default value if it does not exist 
    a.foreach{case (e,i) => println(e,i)} // Traversal printing 1
    for( (k,v)<-a ) println(k,v) // Traversal printing 2
    println(a.isEmpty)// Determine whether it is null 
    a.keys.foreach(println)// Print only key
    a.values.foreach(println)// Print only value
    a=Map()// Data Empty Use Again new
    println(a.size)
    a.toSeq.sortBy(_._1)// Ascending sort  key
    a.toSeq.sortBy(_._2)// Ascending sort  value
    a.toSeq.sortWith(_._1>_._1) // Descending sort  key
    a.toSeq.sortWith(_._2>_._2) // Descending sort  value
    
    // The following customization is sorted by English letters or numbers 
    implicit  val KeyOrdering=new Ordering[String] {
      override def compare(x: String, y: String): Int = {
        x.compareTo(y)
      }
    }
    println(a.toSeq.sorted)

(2) Variable Map Example

Features:

The richness of api is basically similar to that of Map in Java

If it is var modified, the reference is variable, and it supports reading and writing

If it is val modification, the reference is immutable and supports reading and writing

def map3(): Unit ={
  // Immutable Map+var Examples of keyword modification 
  var a:scala.collection.mutable.Map[String,Int]=scala.collection.mutable.Map("k1"->1,"k2"->2)// Initialize constructor 
  a += ("k3"->3)// Add Element 
  a += ("k4"->4)// Add Element 
  a += ("k1"->100)// Adding elements that already exist overrides 
  a += ("k1"->100,"k9"->9)// Add multiple elements 
  a -= ("k2","k1")// Delete Element 
  a ++= List("CA" -> 23, "CO" -> 25)// Append collection 
  a --= List("AL", "AZ")// Delete a collection 
 
  a.retain((k,v)=> k=="k1")// Keep only equal to k1 Element, other deletions 
  a.put("put1",200)//put
  a.remove("k2")//remove
  a.clear()// Empty 
  a("k3")=100// Support 
 
  println(a.contains("k6"))// Whether to contain an element 
  println(a.size)// Print size 
  println(a.get("k1").getOrElse("default")) // According to key Read the element and replace it with the default value if it does not exist 
  a.foreach{case (e,i) => println(e,i)} // Traversal printing 1
  for( (k,v)<-a ) println(k,v) // Traversal printing 2
  println(a.isEmpty)// Determine whether it is null 
  a.keys.foreach(println)// Print only key
  a.values.foreach(println)// Print only value
  a=scala.collection.mutable.Map()// Reference energy change 
  println(a.size)
  a.toSeq.sortBy(_._1)// Sort  key
  a.toSeq.sortBy(_._2)// Sort  value
  a.toSeq.sortWith(_._1>_._1) // Descending sort  key
  a.toSeq.sortWith(_._2>_._2) // Descending sort  value
  
      // The following customization is sorted by English letters or numbers 
  implicit  val KeyOrdering=new Ordering[String] {
    override def compare(x: String, y: String): Int = {
      x.compareTo(y)
    }
  }
  println(a.toSeq.sorted)
}