Solve scala. collection. mutable. Map writing problem
- 2021-09-12 01:14:00
- OfStack
Variable Map writing in scala
Use
val no2ID = scala.collection.mutable.Map[Int,String]()
no2ID += (1 -> "something")
In the following code, Map cannot add elements
val no2ID = scala.collection.mutable.Map[Int,String]()
sc.textFile(conf).foreach(line=>{
val splits = line.split( " , " )
val no = splits(0).trim.toInt
val ID = splits(1)
no2ID += (no -> ID)
})
Where the conf file is a number and a corresponding person name
After converting RDD to Array using collect, you can add elements to Map
val no2ID = scala.collection.mutable.Map[Int,String]()
sc.textFile(conf).collect().foreach(line=>{
val splits = line.split( " , " )
val no = splits(0).trim.toInt
val ID = splits(1)
no2ID += (no -> ID)
})
Use Example of Map in scala
Map structure is a very common structure, which has corresponding api in various programming languages. Because the underlying language of Spark is Scala, it is necessary to know how to use Map in Scala.
(1) Immutable Map
Features:
api is not very rich
If it is var modified, the reference is variable, and it supports reading and writing
In case of val modification, the reference is immutable, the value can only be written once, and then read only
var a:Map[String,Int]=Map("k1"->1,"k2"->2)// Initialize constructor
a += ("k3"->3)// Add Element
a += ("k4"->4)// Add Element
a += ("k1"->100)// Adding elements that already exist overrides
a -= ("k2","k1")// Delete Element
// a("k1") = "foo"// Not supported
println(a.contains("k6"))// Whether to contain an element
println(a.size)// Print size
println(a.get("k1").getOrElse("default")) // According to key Read the element and replace it with the default value if it does not exist
a.foreach{case (e,i) => println(e,i)} // Traversal printing 1
for( (k,v)<-a ) println(k,v) // Traversal printing 2
println(a.isEmpty)// Determine whether it is null
a.keys.foreach(println)// Print only key
a.values.foreach(println)// Print only value
a=Map()// Data Empty Use Again new
println(a.size)
a.toSeq.sortBy(_._1)// Ascending sort key
a.toSeq.sortBy(_._2)// Ascending sort value
a.toSeq.sortWith(_._1>_._1) // Descending sort key
a.toSeq.sortWith(_._2>_._2) // Descending sort value
// The following customization is sorted by English letters or numbers
implicit val KeyOrdering=new Ordering[String] {
override def compare(x: String, y: String): Int = {
x.compareTo(y)
}
}
println(a.toSeq.sorted)
(2) Variable Map Example
Features:
The richness of api is basically similar to that of Map in Java
If it is var modified, the reference is variable, and it supports reading and writing
If it is val modification, the reference is immutable and supports reading and writing
def map3(): Unit ={
// Immutable Map+var Examples of keyword modification
var a:scala.collection.mutable.Map[String,Int]=scala.collection.mutable.Map("k1"->1,"k2"->2)// Initialize constructor
a += ("k3"->3)// Add Element
a += ("k4"->4)// Add Element
a += ("k1"->100)// Adding elements that already exist overrides
a += ("k1"->100,"k9"->9)// Add multiple elements
a -= ("k2","k1")// Delete Element
a ++= List("CA" -> 23, "CO" -> 25)// Append collection
a --= List("AL", "AZ")// Delete a collection
a.retain((k,v)=> k=="k1")// Keep only equal to k1 Element, other deletions
a.put("put1",200)//put
a.remove("k2")//remove
a.clear()// Empty
a("k3")=100// Support
println(a.contains("k6"))// Whether to contain an element
println(a.size)// Print size
println(a.get("k1").getOrElse("default")) // According to key Read the element and replace it with the default value if it does not exist
a.foreach{case (e,i) => println(e,i)} // Traversal printing 1
for( (k,v)<-a ) println(k,v) // Traversal printing 2
println(a.isEmpty)// Determine whether it is null
a.keys.foreach(println)// Print only key
a.values.foreach(println)// Print only value
a=scala.collection.mutable.Map()// Reference energy change
println(a.size)
a.toSeq.sortBy(_._1)// Sort key
a.toSeq.sortBy(_._2)// Sort value
a.toSeq.sortWith(_._1>_._1) // Descending sort key
a.toSeq.sortWith(_._2>_._2) // Descending sort value
// The following customization is sorted by English letters or numbers
implicit val KeyOrdering=new Ordering[String] {
override def compare(x: String, y: String): Int = {
x.compareTo(y)
}
}
println(a.toSeq.sorted)
}