Geek asks and answers

Hadoop block size and file size?

This may seem like a silly question, but in Hadoop, suppose blockize is X (usually 64 or 128 MB) and the local file size is Y (where Y is less than X). Now when I copy the Y file to HDFS, do they consume one block or will hasoop create smaller blocks?

+5
source share
1 answer

One block is used by Hadoop. This does not mean that memory capacity will be consumed in an equivalent way.

The output when watching HDFS from the Internet is as follows:

filename1   file    48.11 KB    3   128 MB  2012-04-24 18:36    
filename2   file    533.24 KB   3   128 MB  2012-04-24 18:36    
filename3   file    303.65 KB   3   128 MB  2012-04-24 18:37

You see that the size of each file is smaller than the block size, which is 128 MB. These files are in KB. HDFS capacity is consumed based on the actual file size, but a block is consumed for each file.

HDFS. , , . , Unix filsystem blockize, 512 . HDFS, 64-128 .

, map/reduce, , , , . , . , , . , , , .

, namenode, () . , .

:

+18

More articles:


All Articles