Geek asks and answers

MongoDB: mapReduce side effects

During the development of our map-reducing operations, our MR code generates useful diagnostic data structures regardless of the data reducing the map. Is there an easy way to get this data into code called mapReduce or save it in Mongo? Just writing to the log file turns out to be very suboptimal, because (a) there is already a lot of data and (b) our diagnostic information is very structured and, in fact, we would like to run requests against This.

So far, my research says that MR data structures are passed by value (via serialization), so any data structures in memory are lost, including those tied to the "global" area. Namespaces are isolated from the main namespace on the JS server side, so dbevalI cannot reach them (or at least I don’t know where to look). And last but not least, although all database objects and functions are present, 10gen generates (confuses) error messages to prevent their use, for example, coll.insertit is not a function near , but typeof coll.insert === 'function'- true.

To be clear, I'm interested in doing this for development in a single node, because the logging / debugging support in MongoDB is pretty limited. This type of side effect is not very good in a work environment.

+5
source share
1 answer

As expected, it is impossible (as in MongoDB 2.2) to access another database from the Map / Reduce functions. In addition to the potential impact on performance, there is also the possibility of creating deadlocks and other unwanted side effects.

Unfortunately, this leaves print()mongo in the log as the only out-of-range output option.

Depending on your diagnostic output, one way you could try would be the following:

  • , ( )

  • tojson(), , print()

  • script tail mongod.log , ,

, M/R:

var diag = {
    'run' : diagrun,
    'phase': 'map',
    'key'  : z
}   
print("MAPDIAG:" + tojson(diag));

:

$ tail -f mongo.log | grep "^MAPDIAG"
MAPDIAG:{ "run" : "20120824", "phase" : "map", "key" : "dog" }
MAPDIAG:{ "run" : "20120824", "phase" : "map", "key" : "cat" }
MAPDIAG:{ "run" : "20120824", "phase" : "map", "key" : "cat" }
MAPDIAG:{ "run" : "20120824", "phase" : "map", "key" : "mouse" }
MAPDIAG:{ "run" : "20120824", "phase" : "map", "key" : "cat" }
MAPDIAG:{ "run" : "20120824", "phase" : "map", "key" : "dog" }
MAPDIAG:{ "run" : "20120824", "phase" : "reduce", "key" : "cat", "total" : 3 }
MAPDIAG:{ "run" : "20120824", "phase" : "reduce", "key" : "dog", "total" : 2 }
+2

More articles:


All Articles