Geek asks and answers

Should Mappers and Reducers in Hadoop be static classes?

I try to do something simple in Hadoop and find that when writing cartographers and reducers, they are defined as static everywhere. My task will be decomposed into several parts mapand one final reduce. What if I would like to reuse one of my cards in another job? If I have a mapper class defined as internal static, can I use it in another job? Also, non-trivial problems may require much more complex cartographers, so storing them in one giant file becomes terrible when saving.

Is there a way to have cards and reducers like ordinary classes (perhaps even in a separate bank) than the work itself?

+5
source share
2 answers

Is your question, should the class be static, maybe static, or maybe internal, or should be internal?

Hadoop itself should be able to create your reflections Mapperor Reducer, given the link to the class / name configured in yours Job. This will not work if it is a non-static inner class, since an instance can only be created in the context of some of your other classes, which apparently Hadoop knows nothing about. (If the inner class does not extend its encompassing class, I suppose.)

So, to answer the first question: it should not be unsteady, since this almost certainly makes it unsuitable. To answer the second and third: and this may be a static (inner) class.

a Mapper Reducer . , "Runner". , , . , . : , , .

: , Mapper Reducer JAR. Hadoop, JAR , , . Job. -, Mapper Reducer JAR.

+8

, . , , , . , .

, .

+1

More articles:


All Articles