Add Hadoop User and Groups
Execute the following commands
1
2
3
4
5
mkdir -p /myhome/hadoop
groupadd -g 1000 hadoop
useradd -u 2000 -g hadoop -d /myhome/hadoop hadoop
chown -R hadoop:hadoop /myhome/hadoop
passwd hadoop
Check the group and user are correctly created
1
2
3
4
tail /etc/group
tail /etc/passwd
groups hadoop
id hadoop
Edit the home directroy of user
1
usermod -d /myhome/hadoop hadoop
Prepare the app and upload directories for hadoop
1
2
3
4
mkdir -p /app/hadoop
chown -R hadoop:hadoop /app/hadoop
mkdir /myhome/hadoop/upload
chown -R hadoop:hadoop /myhome/hadoop/upload
Grant sudo permission to hadoop
Edit /etc/sudoers to add the following lines
1
hadoop ALL =( ALL) NOPASSWD: ALL
Install and config JDK
Add the following environment under /etc/profile
1
2
3
export JAVA_HOME = /pcc/app/Linux_jdk1.7.0_x86_64
export PATH = $JAVA_HOME /bin:$PATH
export CLASSPATH = .:$JAVA_HOME /lib/dt.jar:$JAVA_HOME /lib/tools.jar
Verify after re-logon
Install and config Scala
1
2
3
wget http://downloads.lightbend.com/scala/2.11.8/scala-2.11.8.tgz /myhome/hadoop/upload
wget http://downloads.lightbend.com/scala/2.10.6/scala-2.10.6.tgz /myhome/hadoop/upload
tar -zxvf scala-2.10.6.tgz -C /app
Edit /etc/profile with SCALA configuration
1
2
export SCALA_HOME = /app/scala-2.10.6
export PATH = $PATH :${ SCALA_HOME } /bin
Verify after re-logon
1
2
ssh-keygen -t rsa
ssh-copy-id -i ~/.ssh/id_rsa.pub user@server_name
Show [user@hostname ~] when su user
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
cp /root/.bashrc /myhome/hadoop/
[ hadoop@qilibjtst3 ~]$ cat .bashrc
# .bashrc
# User specific aliases and functions
alias rm = 'rm -i'
alias cp = 'cp -i'
alias mv = 'mv -i'
# Source global definitions
if [ -f /etc/bashrc ] ; then
. /etc/bashrc
fi
1
2
3
4
5
6
7
8
9
10
11
12
13
14
cp /root/.bash_profile /myhome/hadoop/
[ hadoop@qilibjtst3 ~]$ cat .bash_profile
# .bash_profile
# Get the aliases and functions
if [ -f ~/.bashrc ] ; then
. ~/.bashrc
fi
# User specific environment and startup programs
PATH = $PATH :$HOME /bin
export PATH
Change the file permission after copy done
1
chown hadoop:hadoop /myhome/hadoop/.*