MappedByteBuffer 读取超过2G文件

 MappedByteBuffer处理大文件,一次只能读2G内容到内存中,为了读取大文件,需要循环读取处理。

protected void head() {
        //512m
        long length = 1L << 29;
        //4g
        long _4G = 1L << 32;
        long cur = 0L;
        try {
            MappedByteBuffer mappedByteBuffer;
            Random random = new Random();
            while (cur < _4G) {
                mappedByteBuffer = new RandomAccessFile("D:\\test\\bigfile.txt", "rw").getChannel()
                        .map(FileChannel.MapMode.READ_WRITE, cur, length);
                IntBuffer intBuffer = mappedByteBuffer.asIntBuffer();
                while (intBuffer.position() < intBuffer.capacity()) {
                    intBuffer.put(random.nextInt());
                }
                cur += length;
            }
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

需要注意的地方:移位运算(<<)

//这里的lL一定要带上L,不然是已int来计算的,结果会越界
long _4G = 1L << 32;
    @Override
    protected void head() {
        long _4g = 1<<32;
        //结果:1
        System.out.println(_4g);
        long _4G = 1L<<32;
        //结果:4294967296
        System.out.println(_4G);
    }

方法:java.nio.channels.FileChannel#map,这里的siez最大 = Integer.MAX_VALUE(2G)

    /*
     * @param  mode
     *         One of the constants {@link MapMode#READ_ONLY READ_ONLY}, {@link
     *         MapMode#READ_WRITE READ_WRITE}, or {@link MapMode#PRIVATE
     *         PRIVATE} defined in the {@link MapMode} class, according to
     *         whether the file is to be mapped read-only, read/write, or
     *         privately (copy-on-write), respectively
     *
     * @param  position
     *         The position within the file at which the mapped region
     *         is to start; must be non-negative
     *
     * @param  size
     *         The size of the region to be mapped; must be non-negative and
     *         no greater than {@link java.lang.Integer#MAX_VALUE}
     *
     * @return  The mapped byte buffer
     *
     * @throws NonReadableChannelException
     *         If the <tt>mode</tt> is {@link MapMode#READ_ONLY READ_ONLY} but
     *         this channel was not opened for reading
     *
     * @throws NonWritableChannelException
     *         If the <tt>mode</tt> is {@link MapMode#READ_WRITE READ_WRITE} or
     *         {@link MapMode#PRIVATE PRIVATE} but this channel was not opened
     *         for both reading and writing
     *
     * @throws IllegalArgumentException
     *         If the preconditions on the parameters do not hold
     *
     * @throws IOException
     *         If some other I/O error occurs
     *
     * @see java.nio.channels.FileChannel.MapMode
     * @see java.nio.MappedByteBuffer
     */
    public abstract MappedByteBuffer map(MapMode mode,
                                         long position, long size)
        throws IOException;

猜你喜欢

转载自blog.csdn.net/bpz31456/article/details/82255985