Detailed front-end binary: Blob, File, FileReader, ArrayBuffer, TypeArray, DataView

Currently, the front-end has many APIs available for binary data, which enriches the front-end's ability to process file data. With these capabilities, it is possible to perform various processing on image and other file data.
This article will focus on introducing some API knowledge related to front-end binary data processing, such as Blob, File, FileReader, ArrayBuffer, TypeArray, DataView, etc.

byte

Before introducing various APIs, we need to understand the knowledge related to bytes.

We know that the computer is a binary world, and a byte is a basic unit of binary data in computer technology. A byte has 8 binary bits, that is, 8 bits.
A bit is also called a bit. A bit of binary data is either 0 or 1. There are only two states, so 1 bit has 2 states.
1 byte has 8 bits, that is, 8 binary bits, which can represent 2**8 = 256a state, with values ​​from 00000000 to 11111111.

As the basic unit, bytes are used in many places, such as character encoding knowledge, see the character encoding knowledge that the front end needs to understand .
When binary data is stored, it is in bytes, and a knowledge about byte order is also involved here.

byte order

Endianness describes how computers store bytes.
Because we know that memory storage has index addresses, and each byte corresponds to an index address. One byte stores 8-bit binary, that is, between 0 and 255, but when it is necessary to store a value greater than 255, multiple bytes are required, and multiple bytes involve sorting issues.
So the byte order is: when multiple bytes are required to represent a value, what sorting method is used for these multiple bytes to be stored in memory.
There are mainly two sorting methods: big-endian and little-endian.

big endian storage and little endian storage

Big-endian storage is also called big-endian or high-endian. The method is that the low-order bytes are arranged at the high-address end of the memory, and the high-order bytes are arranged at the low-address end of the memory. Image files png and jpg are all in this way.
Little-endian storage is also called little endian and low endian. The method is that the low-order bytes are arranged at the low-address end of the memory, and the high-order bytes are arranged at the high-address end of the memory. The image file gif is little endian.

example

When we use different byte sequences to store numbers 0x12345678(here is the hexadecimal representation, the corresponding decimal system: 305419896. For the relevant knowledge of the base system, please refer to the base and base conversion in Javascript above :

The storage address stored in the memory of the big end:

the storage address stored in the memory of the little end:

here the high-low bits of the digital bytes are from left to right, the highest bit is 12, and the lowest bit is 78; while when stored in memory, from left to right is the low address - the high address.
So in big-endian, the high-order byte is 12at the leftmost low address bit of the memory, and the low-order byte 78is at the rightmost high-address bit of the memory; while little-endian is just the opposite.

From the perspective of visual habits, big-endian storage seems to be more pleasing to the eye, but no matter which method, the calculation result is the same, but this sorting method needs to be processed during calculation, which will be covered below.

Blob

Blob, Binary large Object, is essentially a binary object, which represents an immutable, original data file-like object.
Its immutability means it is read-only and cannot be changed.

The constructor syntax of the Blob object: new Blob(array, options).

Parameter array: It is a data array, which can be the data of various objects, including ArrayBuffer, Blob, String and so on.
Parameter options: optional object, specify two properties:

typeIndicates the MIME type of the Blob object data;
endingsspecifies how to write a string containing the line terminator \n.

We can directly create a new Blob object using the constructor:

const blob = new Blob(['123456789'], {
    
    type : 'text/plain'});

The newly created object instance has the following structure:

From the above example, we can see the methods and properties of the Blob object:

• instance attribute
  • size: The byte size of the data in the Blob object
  • type: String, indicating the MIME type of the Blob object data
• example method
  • arrayBuffer(): Returns a promise object of an ArrayBuffer in binary format containing all the contents of the Blob
  • stream(): returns the ReadableStream object that can read the Blob
  • text(): Returns a promise object containing a string (UTF-8 encoded) containing all the contents of the blob
  • slice([start [, end [, contentType]]])：
    • This method has three optional parameters that can be used to split blob data
    • It returns the data of the original blob in the range according to the specified start and end positions, and obtains a new blob object
    • The third parameter contentTypecan specify its own MIME type for the new Blob object

blobYou can operate on the above example:

blob.slice(0, 3).text().then(res => {
    
    
  console.log(res)
})
// 结果：123

The above code uses the slice() method to obtain the first three digits of the original blob data, and after generating a new blob instance, prints out the text content through the text() method.

Let's take a look at the application of Blob in interface requests. The object in the Fetch API Responsehas a blob method to get a Blob object.

const imgRequst = new Request('11.jpg')
fetch(imgRequst).then((response) => {
    
    
  return response.blob()
}).then((mBlob) => {
    
    
  console.log(mBlob)
})

Through the above code, a jpg image file is requested, and the response object blob()is converted to a Blob object through the method:

File

The File object inherits the Blob object, which is a special type of Blob that extends the support for system files.
File provides file information and can be accessed in javascript. It is generally <input>returned when using the tag to select a file, because <input>the tag allows multiple files to be selected, and the file list is returned here files.

In addition <input>to tags, there are two ways to return File objects:

• DataTransferObjects generated by free drag-and-drop operations .
• FileSystemFileHandleThe method of the object in the file system access API getFile().

The constructor of File: new File(bits, name[, options]).
There are three parameters:

• bits: It is a data array, which can be the data of various objects, similar to the Blob object
• name: file name
• options: optional attribute object, contains two options
  • type: MIME type string
  • lastModified: timestamp, indicating the last modification time of the file

The following code <input>reads the file through the tag:

<input id="input-file" type="file" accept="image/*" />

document.getElementById('input-file').onchange = (e) => {
    
    
  const file = e.target.files[0]
  console.log(file)
  // ...
}

This is a simple image upload, and the obtained file instance is printed out on the console:

Through the above picture (under the chrome browser), you can see that File inherits the properties and methods of Blob:

• Attributes In addition to size and type, File also has several attributes of its own
  • lastModified: read-only, timestamp, file last modification time
  • name: read-only, the file name
    lastModifiedDate: read-only, the Date object of the last modification time of the file, which is obsolete
  • webkitRelativePath: non-standard property, returns path or URL
• File does not have its own instance method, all inherit from Blob

Reading Blobs and Files

File inherits from Blob and is a read-only object. It has no other operation capabilities except using slice fragmentation, so if you need to use other APIs to process them.
The APIs mainly used to manipulate Blobs are: FileReader, URL.createObjectURL(), createImageBitmap() and XMLHttpRequest.send(). These methods are described below.

Both Blob and File WebAPIare provided by the browser environment, and the four objects mentioned above are also WebAPI.

FileReader

FileReader is used to asynchronously read the content of a file type (or raw data buffer), specifying a Blob or File object as the file data to be read.

FileReader cannot read files by pathname in the file system.

Constructor: new FileReader().

If you develop more file processing functions, you should be familiar with the FileReader object. Let's look at an example first:

document.getElementById('input-file').onchange = (e) => {
    
    
  const file = e.target.files[0]
  const reader = new FileReader()
  reader.onload = async (event) => {
    
    
    const img = new Image()
    img.src = event.target.result
  }
  reader.readAsDataURL(file)
}

The above code is very commonly used. Use FileReader to read the Base64 data of an image file, and then use the image object to load it. For Base64 knowledge, please refer to the previous [ In-depth understanding of Base64 encoded strings .
This code also involves properties, events, and methods of the FileReader object.

FileReader property events and methods

• properties (all read-only)
  • error: An error occurred while reading the file
  • readyState: Indicates the current read state

    constant name	value	status description
    EMPTY	0	not loaded
    LOADING	1	loading
    DONE	2	Completed all reads

  • result: file content, valid only when the read status is complete
• method
  • abort(): Abort the read operation. On return, the readyState property is DONE
  • readAsArrayBuffer(): Read the content in the Blob as an ArrayBuffer type
  • readAsBinaryString(): reads the contents of the Blob as a raw binary data type
  • readAsDataURL(): Read the content in the Blob as a Base64 string type
  • readAsText(): reads the content in the Blob as a text string
• event
  • onabort: triggered when the read operation is interrupted
  • onerror: Triggered when an error occurs in a read operation
  • onload: Triggered when the read operation completes
  • onloadstart: triggered when the read operation starts
  • onloadend: triggered when the read operation ends
  • onprogress: triggered when the blob is read

URL.createObjectURL()

URL is an interface object provided by the browser environment for processing url links. Through it, various url links can be parsed, constructed, normalized and encoded.
And a static method provided by URL createObjectURL()can be used to process Blob and File file objects.

Let's look at an example first:

document.getElementById('input-file').onchange = (e) => {
    
    
  const file = e.target.files[0]
  const url = URL.createObjectURL(file)
  const img = new Image()
  img.onload = () => {
    
    
    document.body.append(img)
  }
  img.src = url
}

Page display:

This code implements uploading pictures, URL.createObjectURLgenerates a local mapped url after reading, and then uses the Image object to load the picture.
By viewing the page elements, you can see the newly added image element, its src is a string similar to a link: blob:http://localhost:8088/29c8f4a5-9b47-436f-8983-03643c917f1c, through this string, the image can be loaded and displayed.
Again createObjectURL(), it returns a url containing a given Blob or File object, which can be loaded as a file resource. And the life cycle of this url is synchronized with its window, and the url is automatically released when the window is closed.

This url is an Object URL called a pseudo-protocol.

Object URL

Object URL, also known as Blob URL, is generally generated using Blob or File objects, and URL.createObjectURL()a unique URL is created through the method.
The format of the Object URL is: blob:origin/唯一标识(uuid).

The URL string generated above conforms to this format: blob:http://localhost:8088/29c8f4a5-9b47-436f-8983-03643c917f1c.

• Corresponding to origin http://localhost:8088/, if the local html file is opened directly, the origin is null.
• uuid corresponds 29c8f4a5-9b47-436f-8983-03643c917f1c.

The browser internally maintains a mapping from to to generate the Object URL URL. BlobThe Blob is kept in memory, and the browser will only release it when the current window document is unloaded.

If you want to manually release, you need another static method of URL: URL.revokeObjectURL(), which is used to destroy the previously created URL instance, and call it at the right time to destroy the Object URL.

URL.revokeObjectURL(url)

XMLHttpRequest.send()

XMLHttpRequest.send(body): Used to send the data body in the XHR HTTP request.
The body parameter here can be a variety of data types, including Blob objects.

const xhr = new XMLHttpRequest()
xhr.send(new Blob())

createImageBitmap()

createImageBitmap(): Mainly process image resources, accept different image resource objects as parameters, and generate an ImageBitmap object.
These parameters can be Blob and File objects.

ImageBitmap represents a bitmap image that can be drawn on the canvas.

createImageBitmap(file).then(imageBitmap => {
    
    
  const canvas = document.createElement('canvas')
  canvas.width = imageBitmap.width
  canvas.height = imageBitmap.height
  const ctx = canvas.getContext('2d')
  ctx.drawImage(imageBitmap, 0, 0)
  document.body.append(canvas)
})

As the above code, you can read the image file and use canvas to draw.

ArrayBuffer

The ArrayBuffer object represents a general-purpose, fixed-length raw binary buffer. It is a byte array, but its contents cannot be directly manipulated, but need to be processed by other means (such as TypeArray or DataView, etc.).

Constructor: new ArrayBuffer(length), returns an ArrayBuffer object of a specified size.
Parameter length: The byte size of the ArrayBuffer to be created. If it is greater than Number.MAX_SAFE_INTEGER (>= 2**53) or negative, a RangeError is thrown.

FileReaderNext, we first read the ArrayBuffer content of a file using the method introduced earlier :

document.getElementById('input-file').onchange = (e) => {
    
    
  const file = e.target.files[0]
  const reader = new FileReader()
  reader.onload = async (event) => {
    
    
    console.log(event.target.result)
  }
  reader.readAsArrayBuffer(file)
}

Console log printout:

From the above figure, you can see the instance properties and methods of ArrayBuffer:

• byteLength: Indicates the byte size, which cannot be changed
• slice(begin[, end]): Returns a new ArrayBuffer according to the specified position range, and can split the ArrayBuffer.

ArrayBuffer also has static properties and methods:

• ArrayBuffer.length: The length attribute of the constructor, the value is 1
• ArrayBuffer.isView(arg): Returns true if the argument is a view instance of ArrayBuffer.

Since we cannot directly manipulate ArrayBuffer, we need to use other objects to handle it, two of which will be introduced below.

TypeArray

TypeArray, a typed array, describes a class array of binary data buffers. TypeArray itself is not a usable object, but an auxiliary data type. As the prototype of all types of arrays, there are many types of arrays that are actually available, such as Int8Array, Uint8Array, etc.
Commonly used type arrays are shown in the following table:

object	The number of bytes occupied by the element	Ranges	describe
Int8Array	1	-128 - 127	8-bit signed integer array
Uint8Array	1	0 - 255	Array of 8-bit unsigned integers
Uint8ClampedArray	1	0 - 255	fixed array of 8-bit unsigned integers
Int16Array	2	-32768 - 32767	16-bit signed integer array
Uint16Array	2	0 - 65535	Array of 16-bit unsigned integers
int32array	4	-2147483648 - 2147483647	32-bit signed integer array
Uint32Array	4	0 - 4294967295	Array of 32-bit unsigned integers
Float32Array	4	1.2×10**-38 to 3.4×10**38	32-bit float array
Float64Array	8	5.0×10**-324 to 1.8×10**308	Array of 64-bit floats
BigInt64Array	8	-2**63 to 2**63-1	64-bit signed array
BigUint64Array	8	0 to 2**64-1	Array of 64-bit unsigned integers

Typed arrays are also similar to ordinary data, and they also have a series of methods and attributes, but they do not support push, pop, shift, unshift, spliceand other methods that can change the original array.
Since the data type is defined in the typed array, each element must be the same type of data, and the elements cannot be of different types like ordinary data; when the element data type is fixed and unified, the processing efficiency is better.

All types of arrays have the same syntax in terms of constructors, properties, methods, etc. The following is Uint8Arrayan example .

grammar

Uint8Array constructor:

new Uint8Array()
new Uint8Array(length)
new Uint8Array(typedArray)
new Uint8Array(object)
new Uint8Array(buffer [, byteOffset [, length]])

The maximum value of the length parameter

8-bit array-like is 2145386496
16-bit array-like is 1072693248
32-bit array-like is 536346624
32-bit array-like is 268173312

Static properties and methods

• BYTES_PER_ELEMENT: Returns the number of bytes occupied by the array element, the value in Uint8Array is 1, and the value in Uint32Array is 4, see the above table
• length: fixed length, the value in Uint8Array is 1, the value in Uint32Array is 3, basically useless
• name: The type array returns its own construction name, the Uint8Array type returns Uint8Array, the Uint32Array type returns Uint32Array, etc.
• from(source[, mapFn[, thisArg]]): returns a new array from the source type array
• of(element0[, element1[, …[, elementN]]]): creates an array of a new type with a variable number of arguments

Instance Properties and Methods

After introducing the static properties and methods, let's use an example to view the instance properties and methods of Uint8Array. The code is as follows.

const reader = new FileReader()
reader.onload = async (event) => {
    
    
  const aBuffer = event.target.result
  const uint8Array = new Uint8Array(aBuffer)
  console.log(uint8Array)
}
reader.readAsArrayBuffer(file)

The above code directly reads the ArrayBuffer data of the file, and then Uint8Arrayobtains the Uint8Array instance through the constructor function. View it on the console:

By loading a png image, get its Uint8Array array data, and you can see that most of the properties and methods of the type array are similar to ordinary arrays, except for the method of adding, deleting, and modifying the array mentioned above. Therefore, using subscripts, loops, etc. to read typed arrays is no different from ordinary functions.

The type array also has its own special properties (both read-only) and methods, as follows:

• buffer: returns an ArrayBuffer of type array reference
• byteLength: byte length
• byteOffset: the number of bytes offset relative to the source ArrayBuffer
• length: array length
• set(array[, offset]): read the element value from the given array and store it in the type array
• subarray(begin, end): Given a start and end index, returns a new typed array

Relationships Between Typed Arrays

To understand the relationship between common types of arrays, let’s first look at the following picture: the

picture shows the ArrayBuffer data of a png image. You can see that the byte length attribute of ArrayBuffer defaults to the length of an 8-bit integer array, which is consistent with the length of Int8Array and Uint8Array.
The length of Int8Array is 29848, which is exactly twice the length of Int16Array of 14924, and four times the length of Int32Array of 7462. It can be seen that here is the combined calculation of bytes:

• Int8Array(Uint8Array) to Int16Array(Uint16Array), need to combine two bytes in order to calculate the value.
• Int8Array(Uint8Array) to Int32Array(Uint32Array), need to combine four bytes in order to calculate the value.
• Int16Array(Uint16Array) to Int32Array(Uint32Array), need to combine two bytes in order to calculate the value.

Read GIF file example

The type array reads the content of the ArrayBuffer through an array, which can facilitate us to process the binary data of the file.
But when using typed arrays, when encountering multi-byte data, you need to consider the byte order.

Below, we take reading a GIF image stored in little endian as an example.

In the Uint8Array array data of the GIF picture, two bytes are used to store the width and height data, the 7-8 bits store the width of the picture, and the 9-10 bits store the height of the picture.

The width and height of the GIF images we loaded are both 600, and the byte order needs to be processed. The code is as follows:

const uint8Array = new Uint8Array(aBuffer)
let bufferIndex = 6
// 获取GIF宽度的两个字节的值
const width1 = uint8Array[bufferIndex]
// width1 结果：88
const width2 = uint8Array[bufferIndex + 1]
// width2 结果：2

// 得到各自的16进制数据
const width1hex = width1.toString(16)
const width2hex = width2.toString(16)
// 转换成实际的宽度大小，注意这里把两个字节的顺序做了调整，符合小端序
const width = parseInt(width2hex + width1hex, 16)
// width 结果：600

After using little-endian processing, the width result is equal to 600, which conforms to the actual width of the image.
It will be a little troublesome to manually handle the byte order by yourself. If you don't want to manually deal with the byte order problem, you can use another object: DataView.

DataView

DataView is an interface object that reads values ​​of various types from ArrayBufferit without regard to endianness. It is easy to use and has a series of get-and set-instance methods to manipulate data.

DataView's constructor: new DataView(buffer [, byteOffset [, byteLength]]).
parameter:

• buffer：源ArrayBuffer
• byteOffset: the byte offset in the buffer
• byteLength: byte length

DataView does not need to consider the byte order, and when reading the width of GIF, the code can be simplified:

const fileDataView = new DataView(arrBuffer)
let bufferIndex = 6
const width = fileDataView.getUint16(bufferIndex, true)
// 结果：600
bufferIndex += 2
const height = fileDataView.getUint16(bufferIndex, true)
// 结果：600

With the above code, it is very convenient to get the width and height data (600) of the GIF image, because it uses DataViewand its getUint16method, and there is no need to manually process the byte order.
getUint16The method has two parameters: the first parameter represents the byte index; the second parameter represents the byte order, the default big-endian, true means little-endian, GIF is little-endian, so the above code is true. There are more than a dozen similar instance methods
exceptgetUint16DataView

DataView's get and set series methods

The get series method obtains the corresponding value through the byte offset index, among which multi-byte data requires two parameters:

• byteOffset: byte offset when reading
• littleEndian: byte order, default big endian, set to true is little endian

name	parameter	describe
getInt8	(byteOffset)	signed 8-bit integer (1 byte)
getUint8	(byteOffset)	unsigned 8-bit integer (1 byte)
getInt16	(byteOffset [, littleEndian])	16-bit number (short integer, 2 bytes)
getUint16	(byteOffset [, littleEndian])	16-bit number (unsigned short integer, 2 bytes)
getInt32	(byteOffset [, littleEndian])	32-bit number (long integer, 4 bytes)
getUint32	(byteOffset [, littleEndian])	32-bit number (unsigned long integer, 4 bytes)
getFloat32	(byteOffset [, littleEndian])	32-bit floating point number (single precision floating point number, 4 bytes)
getFloat64	(byteOffset [, littleEndian])	64-bit number (double precision floating point, 8 bytes)
getBigInt64	(byteOffset [, littleEndian])	signed 64-bit integer (type long long) value
getBigUint64	(byteOffset [, littleEndian])	Unsigned 64-bit integer (type unsigned long long) value

The set series method corresponds to the get method, and processes the value of the corresponding byte offset index position. The parameters are as follows:

• byteOffset: byte offset when reading
• value: set the value of the corresponding type
• littleEndian: byte order, default big endian, set to true is little endian

name	parameter	describe
setInt8	(byteOffset, value)	8-bit number (one byte)
setUint8	(byteOffset, value)	8-bit number (unsigned byte)
setInt16	(byteOffset, value [, littleEndian])	16-bit number (short integer)
setUint16	(byteOffset, value [, littleEndian])	16-bit number (unsigned short integer)
setInt32	(byteOffset, value [, littleEndian])	32-bit number (long integer)
setUint32	(byteOffset, value [, littleEndian])	32-bit number (unsigned long integer)
setFloat32	(byteOffset, value [, littleEndian])	32-bit number (floating point type)
setFloat64	(byteOffset, value [, littleEndian])	64-bit number (double-precision floating-point type)
setBigInt64	(byteOffset, value [, littleEndian])	signed 64-bit integer (type long long) value
setBigUint64	(byteOffset, value [, littleEndian])	Unsigned 64-bit integer (type unsigned long long) value

Blobs and ArrayBuffers

For the two objects of Blob and ArrayBuffer, we can make a summary:

1. Blob is a Web API, provided by the browser environment. To read it, you can use Web APIs such as FileReader and URL.createObjectURL; ArrayBuffer is a built-in object in the JS language, and you need to use JS-APIs such as TypeArray and DataView to process it.
2. Blob represents immutable file-like data; ArrayBuffer represents raw data buffer.
3. Blob is used to read file-like data and does not correspond to memory; ArrayBuffer is used to read memory data.
4. Both Blob and ArrayBuffer require other objects to manipulate data.
5. Blob and ArrayBuffer can be converted between each other in different ways.
6. To operate byte binary data, you must rely on ArrayBuffer and its auxiliary operation objects.

Conversion between Blob and ArrayBuffer:

• Use the Blob constructor to read the ArrayBuffer and generate a new Blob.
• The corresponding ArrayBuffer can be obtained through the arrayBuffer() method of the Blob instance.
• Use the readAsArrayBuffer() method of the FileReader object to read the Blob as an ArrayBuffer.

The following code:

const aBuffer = new ArrayBuffer(4)
// 使用Blob构造函数
const blob = new Blob([aBuffer])
// Blob的arrayBuffer()方法（promise）
blob.arrayBuffer()
// FileReader
const reader = new FileReader()
reader.readAsArrayBuffer(blob)