Geek asks and answers

Pig, Oozie, and HBase - java.io.IOException: No FileSystem for schema: hbase

My Pig script works fine on its own until I put it in an Oozie workflow, where I get the following error:

ERROR 2043: Unexpected error during execution.

org.apache.pig.backend.executionengine.ExecException: ERROR 2043: Unexpected error during execution.
...
Caused by: java.io.IOException: No FileSystem for scheme: hbase

I successfully registered HBase and Zookeeper banks, but got the same error.

I also tried setting the Zookeeper quorum by adding a change to these lines in the Pig script:

SET hbase.zookeeper.quorum 'vm-myhost-001,vm-myhost-002,vm-myhost-003'

In some searches on the Internet, I was asked to add this to the top of my workflow.xml:

SET mapreduce.fileoutputcommitter.marksuccessfuljobs false

This solved the problem. I was even able to unregister the cans of HBase and Zookeeper and the Zookeeper quorum.

Now, after a double check, I noticed that my tasks are actually doing their job: they store the results in HBase, as expected. But, Oozi claims that there was a failure when he did not.

, mapreduce.fileoutputcommitter.marksuccessfuljobs false .

?

+3
1

, .

, -, , "".

0

More articles:


All Articles