How to write java log file using logger api when using hadoop

i Wrote code abbreviation code that I want to debug.

to do this, I cannot use standard output because the Hadoop platform does not display it unless an error occurs.

instead, I tried to use a log to create a log file.

i split it into two files using a handler, unfortunately, a “serious” log file leaves empty, and the general log file only logs events that occur in the main stream, and not in the map reduction functions.

The question is:

is there a problem with hadoop and log files or is it a problem with my logger configuration? if so, how to fix it.

log configuration code: I use one logger for the entire application (this is the root time logger)

public static Logger configureLogging() 
    {
        try
        {
            logger=Logger.getLogger("");
            //FileSystem hdfs=FileSystem.get(URI.create(Misc.S3FS),getConfiguration());
            logger.setLevel(Level.ALL);

            //StreamHandler handler=new StreamHandler(hdfs.create(new Path(Misc.LOGS_PATH+"mylog.log")),new SimpleFormatter());
            FileHandler handler=new FileHandler(Misc.LOGS_PATH+"mylog.xml",true);   
            FileHandler severeHandler=new FileHandler(Misc.LOGS_PATH+"mylogSevere.xml",true);
            severeHandler.setLevel(Level.INFO);
            logger.addHandler(handler);
            logger.addHandler(severeHandler);

        }
        catch (Exception e) 
        {
            e.printStackTrace();
        }
        return logger;

    }
+5
source share
1 answer

Hadoop comes with pre-configured log4j. All you have to do is import two classes:

import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;

And now you can define the registrar inside your cards, reducers and wherever you want:

private static final Log LOG = LogFactory.getLog(MyClass.class);

And write down what you need:

LOG.info("My message");

Messages will be displayed during the execution of the task. You can configure log4j configuration with

conf/log4j.properties
+7
source

All Articles