Table of contents
1. Download from the official website
2. Decompress in the virtual machine
3). Transfer the decompressed spark to /usr/local
2. Configure the spark environment
3. Verify whether spark is installed
1. Install spark
1. Download from the official website
Download address:Index of /dist/spark
Download according to your own hadoop version (the version of spark is not important, what matters is which version of hadoop is based on it)
2. Decompress in the virtual machine
1) Open your own virtual machine, then drag or copy and paste the downloaded spark into the home directory
2), right-click on the desktop to open the terminal, enter the command: tar -zxvf spark-3.2.4-bin-hadoop2.7.tgz(I downloaded the 3.2.4 version of spark based on Hadoop 2.7)
3). Transfer the decompressed spark to /usr/local
Command: sudo mv spark-3.2.4-bin-hadoop2.7 /usr/local
The password you need to enter is the password you use to open your own virtual machine.
2. Configure the spark environment
1. Configure the environment
Click on the .bashrc file in the home directory and enter:
export SPARK_HOME=/usr/local/spark-3.2.4-bin-hadoop2.7
export PATH=$PATH:$SPARK_HOME/bin
2. Update the environment
After configuring the environment, you must update the environment before you can use it.
Right-click in the home directory to open the terminal, and then enter the command: source .bashrc to update the environment
3. Verify whether spark is installed
Find the location of spark-3.2.4-bin-hadoop2.7, click in it, right-click to open the terminal, and enter the command: spark-shell
As long as the big spark icon appears, the installation is complete.
The installation is complete!