Geek asks and answers

Saving file contents in immutable map in scala

I am trying to implement a simple wordcount in scala using an immutable map (this is intentional), and the way I am trying to do this is as follows:

  • Create a blank immutable map
  • Create a scanner that reads the file.

  • So far, the .hasNext () scanner is true:

    • Check if the Card contains a word, if it does not contain a word, initializes the counter to zero
    • Create a new record with key = word and value = count + 1
    • Update map

  • At the end of the iteration, the map is filled with all the values.

My code is as follows:

val wordMap = Map.empty[String,Int]
val input = new java.util.scanner(new java.io.File("textfile.txt"))
while(input.hasNext()){
  val token = input.next()
  val currentCount = wordMap.getOrElse(token,0) + 1
  val wordMap = wordMap + (token,currentCount)
}

The idea is that wordMap will have all the words Counts at the end of the iteration ... Whenever I try to run this fragment, I get the following exception.

The recursive value of wordMap requires a type.

- , , ?

+3
2
val wordMap = wordMap + (token,currentCount)

. , wordMap var,

wordMap = wordMap + (token,currentCount)

?:

io.Source.fromFile("textfile.txt")            // read from the file
  .getLines.flatMap{ line =>                  // for each line
     line.split("\\s+")                       // split the line into tokens
       .groupBy(identity).mapValues(_.size)   // count each token in the line
  }                                           // this produces an iterator of token counts
  .toStream                                   // make a Stream so we can groupBy
  .groupBy(_._1).mapValues(_.map(_._2).sum)   // combine all the per-line counts
  .toList

, , . .

, ( ), Scala , Hadoop ( Hadoop Scala, Scrunch Scoobi).

EDIT: :

, flatMap. :

val line = "a b c b"
val tokens = line.split("\\s+") // Array(a, b, c, a, b)

identity is a function that just returns its argument, so if we groupBy (identity) `, :

val grouped = tokens.groupBy(identity) // Map(c -> Array(c), a -> Array(a), b -> Array(b, b))

, , :

val counts = grouped.mapValues(_.size) // Map(c -> 1, a -> 1, b -> 2)

, .

, flatMap? , , .

, :

a b c b
b c d d d
e f c

:

val countsByLine = 
  io.Source.fromFile("textfile.txt")            // read from the file
    .getLines.flatMap{ line =>                  // for each line
       line.split("\\s+")                       // split the line into tokens
         .groupBy(identity).mapValues(_.size)   // count each token in the line
    }                                           // this produces an iterator of token counts
println(countsByLine.toList) // List((c,1), (a,1), (b,2), (c,1), (d,3), (b,1), (c,1), (e,1), (f,1))

, . countsByLine Iterator, groupBy. Stream, . , , . groupBy .

val groupedCounts = countsByLine.toStream.groupBy(_._1)
println(groupedCounts.mapValues(_.toList)) // Map(e -> List((e,1)), f -> List((f,1)), a -> List((a,1)), b -> List((b,2), (b,1)), c -> List((c,1), (c,1), (c,1)), d -> List((d,3)))

, , , () :

val totalCounts = groupedCounts.mapValues(_.map(_._2).sum)
println(totalCounts.toList)
List((e,1), (f,1), (a,1), (b,3), (c,3), (d,3))

.

+7

: wordMap (val - ). , Map , var, ( ).

:

var wordMap = Map.empty[String,Int] withDefaultValue 0
val input = new java.util.Scanner(new java.io.File("textfile.txt"))
while(input.hasNext()){
  val token = input.next()
  wordMap += token -> (wordMap(token) + 1)
}
+3

More articles:


All Articles