Geek asks and answers

Apache Pig, Suppress "Error checking output" "Output directory ... already exists"

After receiving help from orangeoctopus with this question , I now need to suppress the message "Error checking output" "The output directory ... already exists." I know that the directory exists, I want it like that. I am sure it will be a matter of redefining something in my UDF repository, but I find it hard to understand that. Brand new for Java, so bear with me. Thanks in advance.

+3
source share
2 answers

As far as I know, you cannot reuse the direct output directory. Hadoop prevents this. if I understand correctly, you are dealing with daily logs, so I suggest you set the parent output directory, called output, and set the output directory in the script for output / daily_date.

+3
source

Delete your output directory before the repository operation:

rmf $outpath;
0
source

More articles:


All Articles