配置系统
关闭防火墙
1
2
3
4
| /**
* 查看防火墙状态：sudo ufw status
*/
sudo ufw disable
|
配置主机名(spark-master)
修改hosts
1
2
3
4
5
6
| /**
* 如果发现hosts文件中有127.0.1.1或者除了127.0.0.1之外的其他内容，将他们删除
*/
sudo vim /etc/hosts
IP地址 spark-master
|
配置Java环境
请参考：Ubuntu下安装JDK环境
配置SSH免密码登录
请参考：Ubuntu配置SSH免密码登录
安装Tachyon
下载Tachyon
官方网站：Tachyon官网
配置Tachyon环境变量
1
2
3
4
5
6
7
8
9
10
| sudo vim /etc/profile
/**
* 添加Tachyon路径
*/
export TACHYON_HOME=/home/jabo/software/tachyon-0.7.1
export PATH=$TACHYON_HOME/bin:$PATH
//使配置生效
source /etc/profile
|
目录权限
1
| sudo chmod -R 775 tachyon-0.7.1/
|
复制Tachyon配置文件
1
2
3
4
5
|
cd /software/tachyon-0.7.1
cp ./conf/tachyon-env.sh.template ./conf/tachyon-env.sh
|
配置Tachyon
1
2
3
4
5
6
7
8
9
|
vim ./conf/tachyon-env.sh
export JAVA_HOME=/usr/lib/jvm/java
export JAVA="$JAVA_HOME/bin/java"
export TACHYON_MASTER_ADDRESS=spark-master
export TACHYON_UNDERFS_ADDRESS=$TACHYON_HOME/underfs
export TACHYON_WORKER_MEMORY_SIZE=512MB
|
配置slaves和workers
1
2
3
4
5
| //打开slaves配置文件，注释掉localhost，添加spark-master
vim ./conf/slaves
//打开workers配置文件，注释掉localhost，添加spark-master
vim ./conf/workers
|
初始化文件系统
1
2
3
4
| Connecting to spark-master as jabo...
Formatting Tachyon Worker @ spark-master
Connection to spark-master closed.
Formatting Tachyon Master @ spark-master
|
启动Tachyon的local模式
1
| ./bin/tachyon-start.sh local
|
1
2
3
4
5
6
7
8
| Killed 0 processes on spark-master
Killed 0 processes on spark-master
Connecting to spark-master as jabo...
Killed 0 processes on spark-master
Connection to spark-master closed.
Formatting RamFS: /mnt/ramdisk (512mb)
Starting master @ spark-master
Starting worker @ spark-master
|
查看Tachyon进程
1
2
3
4
5
| jps
7648 TachyonWorker
7835 Jps
7614 TachyonMaster
|
运行测试
查看Web UI
用浏览器打开：hostname:19999 (spark-master:19999)
