Which method is more efficient for concatenating large files in Java using FileChannels

I want to know which method is better than the two that I came up with to concatenate text files in Java. If anyone has an understanding, they can share what happens at the kernel level, which explains the difference between these writing methods in FileChannel, I would really appreciate it.

From what I understand from the documentation and other conversations, allocateDirect allocates space directly on the disk and basically avoids the use of RAM. I'm worried that a ByteBuffer created using allocateDirect might have the potential to overflow or not stand out if the File infile is large, say 1 GB. At this stage of software development, I guarantee that the file will be no more than 2 GB; but there is potential in the future that it may be 10 or 20 GB in size.

I noticed that the From transfer loop never goes through the loop more than once ... so it seems I managed to write all the inflation at once; but I have not tested it with files larger than 60 MB. I went in cycles because the documentation states that there is no guarantee of how much will be written right away. With transferFrom, which can only accept int32 in my system as the count parameter, I cannot specify more than 2 GB at a time ... And again, kernel expertise will help me understand.

Thanks in advance for your help!

Using ByteBuffer :

boolean concatFiles(StringBuffer sb, File infile, File outfile) {

    FileChannel inChan = null, outChan = null;

    try {

        ByteBuffer buff = ByteBuffer.allocateDirect((int)(infile.length() + sb.length()));
        //write the stringBuffer so it goes in the output file first:
        buff.put(sb.toString().getBytes());

        //create the FileChannels:
        inChan  = new RandomAccessFile(infile,  "r" ).getChannel();
        outChan = new RandomAccessFile(outfile, "rw").getChannel();

        //read the infile in to the buffer:
        inChan.read(buff);

        // prep the buffer:
        buff.flip();

        // write the buffer out to the file via the FileChannel:
        outChan.write(buff);
        inChan.close();
        outChan.close();
     } catch...etc

}

Using trasferTo (or transferFrom) :

boolean concatFiles(StringBuffer sb, File infile, File outfile) {

    FileChannel inChan = null, outChan = null;

    try {

        //write the stringBuffer so it goes in the output file first:    
        PrintWriter  fw = new PrintWriter(outfile);
        fw.write(sb.toString());
        fw.flush();
        fw.close();

        // create the channels appropriate for appending:
        outChan = new FileOutputStream(outfile, true).getChannel();
        inChan  = new RandomAccessFile(infile, "r").getChannel();

        long startSize = outfile.length();
        long inFileSize = infile.length();
        long bytesWritten = 0;

        //set the position where we should start appending the data:
        outChan.position(startSize);
        Byte startByte = outChan.position();

        while(bytesWritten < length){ 
            bytesWritten += outChan.transferFrom(inChan, startByte, (int) inFileSize);
            startByte = bytesWritten + 1;
        }

        inChan.close();
        outChan.close();
    } catch ... etc
+3
source share
1 answer

transferTo() , , , . , - .

, .

+3

All Articles