Ê×Ò³ > ÍøÂç > ÔÆ¼ÆËã > ÕýÎÄ
hadoopα·Ö²¼Ê½»·¾³´î½¨Óë²âÊÔ
2016-09-19       ¸öÆÀÂÛ      
ÊÕ²Ø    ÎÒҪͶ¸å

±êÇ©£¨¿Õ¸ñ·Ö¸ô£©£º hadoop α·Ö²¼Ê½ ¼¯Èº´î½¨

Ŀ¼

µÚÒ»²½ Linuxϵͳ»·¾³ÉèÖÃ
1. ϵͳ»·¾³½éÉÜ
2. ÅäÖù̶¨IPµØÖ·
3. ÐÞ¸ÄÖ÷»úÃû
4. ½¨Á¢Ö÷»úÓ³Éä
5. ¹Ø±Õ·À»ðǽ¡¢½ûÓÃselinux
6. ´´½¨ÆÕͨÓû§¡¢ÉèÖÃÓû§ÃÜÂë
7. Ϊ´´½¨µÄÆÕͨÓû§¸³È¨ÏÞ
8. Ð¶ÔØCentOS×Ô´øµÄOpenjdk
µÚ¶þ²½ ´´½¨hadoopĿ¼½á¹¹²¢Ê¹ÓÃFileZillaÉÏ´«Îļþ
1. ÒÔÆÕͨÓû§´´½¨hadoopĿ¼½á¹¹
2. ÐÞ¸ÄĿ¼¹éÊôÕß
3. ʹÓÃFileZillaÉÏ´«ÎļþÖÁÏàÓ¦µÄĿ¼ÖÐ
µÚÈý²½ °²×°JDK¡¢ÅäÖû·¾³±äÁ¿¡¢½âѹHadoop2.5.0
1. °²×°JDK
2. ÅäÖû·¾³±äÁ¿
3. ½âѹHadoop2.5.0Îļþ£¬×î´ó»¯¿ÉÓÿռä
µÚËIJ½ ÅäÖÃHDFS¡¢Æô¶¯ºÍ²âÊÔ¶ÁдÎļþ
1. ʹÓÃNotepad++ÅäÖÃhadoop-env.shÖеÄJAVA_HOME
2. ʹÓÃNotepad++ÅäÖÃhdfsÏà¹ØµÄxmlÊôÐÔ
3. ¸ñʽ»¯HDFSϵͳ¡¢Æô¶¯HDFSϵͳ¼°²âÊÔ¶ÁдÎļþ
µÚÎå²½ ÅäÖÃYARN¡¢Æô¶¯ºÍMapReduceÔËÐÐÔÚYARNÉÏ
1. ʹÓÃNotepad++ÅäÖÃyarn-env.shºÍmapred-env.shÖеÄJAVA_HOME
2. ʹÓÃNotepad++ÅäÖÃyarnÏà¹ØµÄxmlÊôÐÔ£¬²¢Æô¶¯YARN
3. ʹÓÃNotepad++ÅäÖÃmapredÏà¹ØµÄxmlÊôÐÔ£¬²¢Æô¶¯JobHistoryServer·þÎñ
4. ÔËÐÐMapReduce WordCount³ÌÐò
HDFS¡¢YARNÒÔ¼°MapReduce¹¦ÄÜÃèÊö


¡¾µÚÒ»²½¡¿£ºLinuxϵͳ»·¾³ÉèÖÃ

1. ϵͳ»·¾³½éÉÜ

1£© ÐéÄâ»ú»·¾³£º VMWare Workstation 12
2£© ²Ù×÷ϵͳ£º CentOS 6.5
3£© Ô¶³Ì¹¤¾ß£º SecureCRT£¨Ô¶³ÌÁ¬½Ó¹¤¾ß£©
FileZilla£¨Ô¶³ÌÎļþ´«Ê乤¾ß£©

2. ÅäÖù̶¨IPµØÖ·

  IPv4 Settings :
        IPADDR  : 192.168.1.100
        NETMASK : 255.255.255.0
        GATEWAY : 192.168.1.1
        DNS1    : 192.168.1.1
        DNS2    : 8.8.8.8
  setting file  £º/etc/sysconfig/network-scripts/ifcfg-eth0

 

1.png

 <†·Ÿ"/kf/ware/vc/" target="_blank" class="keylink">vcD4NCjxwPjxzdHJvbmc+My4g0N64xNb3u/rD+zwvc3Ryb25nPjwvcD4NCjxwcmUgY2xhc3M9"brush:java;"> [root@localhost ~]# hostname localhost [root@localhost ~]# vi /etc/sysconfig/network [root@localhost ~]# hostname hadoop-master.qianxin.com

 

2.png

 

4. ½¨Á¢Ö÷»úÓ³Éä

ÐÞ¸ÄLinuxÖ÷»úÓ³Éä
[root@localhost ~]# vi /hosts

 

2.png

 

ÐÞ¸ÄWindowsÖ÷»úÓ³Éä
ʹÓÃNotepad++´ò¿ªC:\Windows\System32\drivers\etcĿ¼ÏµÄhostsÎļþÌí¼ÓÒÔÏÂÐÅÏ¢£º
    192.168.1.100   hadoop-master.qianxin.com
ÐÞ¸ÄÍê³Éºó£¬ÖØÆôLinuxϵͳ¡£

 

2.png

 

5. ¹Ø±Õ·À»ðǽ¡¢½ûÓÃselinux

[root@hadoop-master ~]# service iptables stop
[root@hadoop-master ~]# chkconfig iptables off
[root@hadoop-master ~]# vi /etc/selinux/config

 

2.png

 

6. ´´½¨ÆÕͨÓû§¡¢ÉèÖÃÓû§ÃÜÂë

[root@hadoop-master ~]# useradd qianxin
[root@hadoop-master ~]# passwd qianxin

7. Ϊ´´½¨µÄÆÕͨÓû§¸³È¨ÏÞ

[root@hadoop-master ~]# visudo

 

2.png

 

8. Ð¶ÔØCentOS×Ô´øµÄOpenjdk

[root@hadoop-master ~]# rpm -qa | grep java
[root@hadoop-master ~]# rpm -e --nodeps [ÒѰ²×°µÄjdkÓ¦ÓÃÃû]

¡¾µÚ¶þ²½¡¿£º´´½¨hadoopĿ¼½á¹¹²¢Ê¹ÓÃFileZillaÉÏ´«Îļþ

1. ÒÔÆÕͨÓû§´´½¨hadoopĿ¼½á¹¹

[qianxin@hadoop-master ~]$ cd /opt
[qianxin@hadoop-master opt]$ sudo mkdir datas modules softwares tools
[qianxin@hadoop-master opt]$ ls -l
drwxr-xr-x 2 root root 4096 Sep 12 10:34 datas
drwxr-xr-x 4 root root 4096 Sep 12 10:45 modules
drwxr-xr-x 2 root root 4096 Sep 12 10:38 softwares
drwxr-xr-x 2 root root 4096 Sep 12 10:37 tools

2. ÐÞ¸ÄĿ¼¹éÊôÕß

[qianxin@hadoop-master opt]$ sudo chown -R qianxin:qianxin /opt/*
[qianxin@hadoop-master opt]$ ls -l

 

2.png

 

3. ʹÓÃFileZillaÉÏ´«ÎļþÖÁÏàÓ¦µÄĿ¼ÖÐ

2.png

 


¡¾µÚÈý²½¡¿£º°²×°JDK¡¢ÅäÖû·¾³±äÁ¿¡¢½âѹHadoop2.5.0

1. °²×°JDK
ʹÓÃÆÕͨÓû§½«/opt/softwaresĿ¼ÏµÄjdk-7u67-linux-x64.tar.gz½âѹÖÁ/opt/modulesĿ¼ÖС£

[qianxin@hadoop-master softwares]$ tar -zxvf jdk-7u67-linux-x64.tar.gz -C /opt/modules

2. ÅäÖû·¾³±äÁ¿

[qianxin@hadoop-master softwares]$ sudo vim /etc/profile

ÔÚÎļþĩβÌí¼ÓJAVA_HOME¡¢JRE_HOMEºÍPATHµÄ»·¾³±äÁ¿Öµ£¬¾ßÌåÖµÈçÏÂËùʾ¡£

export JAVA_HOME=/opt/modules/jdk1.7.0_67
export JRE_HOME=$JAVA_HOME/jre
export PATH=$PATH:$JAVA_HOME/bin:JRE_HOME/bin

 

2.png

 

ÅäÖÃÍê³Éºó±£´æ¹Ø±Õ£¬Ö´ÐÐÒÔÏÂÃüÁîʹÅäÖÃÉúЧ£¬¼ì²âJDKÊÇ·ñ°²×°³É¹¦¡£

[qianxin@hadoop-master softwares]$ sudo source /etc/profile
[qianxin@hadoop-master softwares]$ java -version
java version "1.7.0_67"
Java(TM) SE Runtime Environment (build 1.7.0_67-b01)
Java HotSpot(TM) 64-Bit Server VM (build 24.65-b04, mixed mode)

 

2.png

 

3. ½âѹHadoop2.5.0Îļþ£¬×î´ó»¯¿ÉÓÿռä

[qianxin@hadoop-master softwares]$ tar -zxvf hadoop-2.5.0.tar.gz -C /opt/modules
[qianxin@hadoop-master softwares]$ df -lh
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda1        60G  5.9G   51G  11% /
tmpfs           1.9G  372K  1.9G   1% /dev/shm
/dev/sda3        12G  160M   12G   2% /home
[qianxin@hadoop-master softwares]$ rm -rf /opt/modules/hadoop-2.5.0/share/docs
[qianxin@hadoop-master softwares]$ df -lh
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda1        60G  4.4G   52G   8% /
tmpfs           1.9G  372K  1.9G   1% /dev/shm
/dev/sda3        12G  160M   12G   2% /home

 

2.png

 


¡¾µÚËIJ½¡¿£ºÅäÖÃHDFS¡¢Æô¶¯ºÍ²âÊÔ¶ÁдÎļþ

1. ʹÓÃNotepad++ÅäÖÃhadoop-env.shÖеÄJAVA_HOME

·¾¶£º/opt/modules/hadoop-2.5.0/etc/hadoop/hadoop-env.sh

 

2.png

 

2. ʹÓÃNotepad++ÅäÖÃhdfsÏà¹ØµÄxmlÊôÐÔ
1)core-site.xmlÎļþÅäÖÃ

<£¡-- 1. core-site.xmlÎļþÅäÖà -->
<£¡-- ·¾¶£º/opt/modules/hadoop-2.5.0/etc/hadoop/core-site.xml -->
<£¡-- ÅäÖÃÄÚÈÝÈçÏ -->
<£¡-- ˵Ã÷£ºÅäÖÃÖ÷½ÚµãNameNodeλÖü°½»»¥¶Ë¿Ú -->

    fs.defaultFS
    hdfs://hadoop-master.qianxin.com:8020

<£¡-- ˵Ã÷£ºÖ¸¶¨hadoopÔËÐÐʱ²úÉúÎļþµÄ´æ´¢Ä¿Â¼ -->

    hadoop.tmp.dir
    /opt/modules/hadoop-2.5.0/data/tmp

 

2.png

 

2)slavesÎļþÅäÖÃ

<£¡-- 2. slavesÎļþÅäÖà -->
<£¡-- ·¾¶£º/opt/modules/hadoop-2.5.0/etc/hadoop/slaves -->
<£¡-- ˵Ã÷£ºÅäÖôӽڵãDataNodeλÖà -->
<£¡-- ÅäÖÃÄÚÈÝÈçÏ -->
hadoop-master.qianxin.com

 

2.png

 

3)hdfs-site.xmlÎļþÅäÖÃ

<£¡-- 3. hdfs-site.xmlÎļþÅäÖà -->
<£¡-- ·¾¶£º/opt/modules/hadoop-2.5.0/etc/hadoop/hdfs-site.xml -->
<£¡-- ÅäÖÃÄÚÈÝÈçÏ -->
<£¡-- ˵Ã÷£ºÅäÖÃhdfsϵͳÖÐÎļþ¿ìµÄÊý¾Ý¸±±¾¸öÊý -->

    dfs.replication
    1

<£¡-- ˵Ã÷£ºÅäÖÃSecondaryNameNodeλÖà -->

    dfs.namenode.secondary.http-address
    hadoop-master.qianxin.com:50090

 

2.png

 

3. ¸ñʽ»¯HDFSϵͳ¡¢Æô¶¯HDFSϵͳ¼°²âÊÔ¶ÁдÎļþ
1)¸ñʽ»¯HDFSÎļþϵͳ

[qianxin@hadoop-master hadoop-2.5.0]$ bin/hdfs namenode -format

³öÏÖÒÔÏÂÐÅÏ¢¼´Îª¸ñʽ»¯HDFSÎļþϵͳ³É¹¦£º

2.png

 

2)Æô¶¯HDFSÎļþϵͳ¼°²âÊÔ

[qianxin@hadoop-master hadoop-2.5.0]$ sbin/hadoop-daemon.sh start namenode
[qianxin@hadoop-master hadoop-2.5.0]$ sbin/hadoop-daemon.sh start secondarynamenode
[qianxin@hadoop-master hadoop-2.5.0]$ sbin/hadoop-daemon.sh start datanode
[qianxin@hadoop-master hadoop-2.5.0]$ jps
4161 NameNode
4240 SecondaryNameNode
4382 Jps
4304 DataNode

 

2.png

 

HDFS¹ÜÀí½çÃæ£ºhttp://hadoop-master.qianxin.com:50070/

2.png

 

²âÊÔ1£ºÔÚhdfsÉÏ´´½¨Îļþ¼Ð

[qianxin@hadoop-master hadoop-2.5.0]$ bin/hdfs dfs -mkdir -p /user/qianxin
[qianxin@hadoop-master hadoop-2.5.0]$ bin/hdfs dfs -ls -p /

 

2.png

2.png

×¢Ò⣺ʹÓÃ×Ô¼º±àÒëµÄnativeÎļþ¼ÐÌæ»»Ô­ÓÐ${HADOOP_HOME}/libÏÂÔ­ÓеÄnativeÎļþ¼Ð¼´¿É½â¾öÉÏÊöÎÞ·¨Ê¹Óñ¾µØ¿âµÄ¾¯¸æ¡£

 

²âÊÔ2£ºÉÏ´«ÎļþÖÁhdfs

[qianxin@hadoop-master hadoop-2.5.0]$ bin/hdfs dfs -put ./etc/hadoop/core-site.xml /user/qianxin
[qianxin@hadoop-master hadoop-2.5.0]$ bin/hdfs dfs -ls -p /

 

2.png

 

ʹÓÃHDFS¹ÜÀí½çÃæ²é¿´ÉÏ´«µÄÎļþ£º

2.png

 

²âÊÔ3£º´ÓhdfsÉ϶ÁÈ¡Îļþ

[qianxin@hadoop-master hadoop-2.5.0]$ bin/hdfs dfs -cat /user/qianxin/core-site.xml

 

2.png

 

²âÊÔ4£º´ÓhdfsÉÏÏÂÔØÎļþ

[qianxin@hadoop-master hadoop-2.5.0]$ bin/hdfs dfs -get /user/qianxin/core-site.xml /home/qianxin/qx-core-site.xml

 

2.png

 


¡¾µÚÎå²½¡¿£ºÅäÖÃYARN¡¢Æô¶¯ºÍMapReduceÔËÐÐÔÚYARNÉÏ

1. ʹÓÃNotepad++ÅäÖÃyarn-env.shºÍmapred-env.shÖеÄJAVA_HOME

·¾¶£º
yarn-env.sh     :   /opt/modules/hadoop-2.5.0/etc/hadoop/yarn-env.sh
mapred-env.sh   :   /opt/modules/hadoop-2.5.0/etc/hadoop/mapred-env.sh

2. ʹÓÃNotepad++ÅäÖÃyarnÏà¹ØµÄxmlÊôÐÔ£¬²¢Æô¶¯YARN
1)yarn-site.xmlÎļþÅäÖÃ

<£¡-- yarn-site.xmlÎļþÅäÖà -->
<£¡-- ·¾¶£º/opt/modules/hadoop-2.5.0/etc/hadoop/yarn-site.xml -->
<£¡-- ÅäÖÃÄÚÈÝÈçÏ -->
<£¡-- ˵Ã÷£ºÅäÖÃreduce»ñÈ¡Êý¾ÝµÄ·½Ê½ -->

    yarn.nodemanager.aux-services
    mapreduce_shuffle

<£¡-- ˵Ã÷£ºÖ¸¶¨ResourceManager -->

    yarn.resourcemanager.hostname
    hadoop-master.qianxin.com

 

2.png

 

2)Æô¶¯YARN

[qianxin@hadoop-master hadoop-2.5.0]$ sbin/yarn-daemon.sh start resourcemanager
[qianxin@hadoop-master hadoop-2.5.0]$ sbin/yarn-daemon.sh start nodemanager
[qianxin@hadoop-master hadoop-2.5.0]$ jps
4161 NameNode
4240 SecondaryNameNode
5908 NodeManager
6003 Jps
4304 DataNode
5666 ResourceManager

 

2.png

 

YARN¹ÜÀí½çÃæ£ºhttp://c:8088/

2.png

 

3. ʹÓÃNotepad++ÅäÖÃmapredÏà¹ØµÄxmlÊôÐÔ£¬²¢Æô¶¯JobHistoryServer·þÎñ
1)mapred-site.xmlÎļþÅäÖÃ

<£¡-- mapred-site.xmlÎļþÅäÖà -->
<£¡-- ·¾¶£º/opt/modules/hadoop-2.5.0/etc/hadoop/mapred-site.xml -->
<£¡-- ÅäÖÃÄÚÈÝÈçÏ -->
<£¡-- ˵Ã÷£ºÖ¸¶¨MapReduceÔËÐÐÔÚYARNÉÏ -->

    mapreduce.framework.name
    yarn

<£¡-- ˵Ã÷£ºÅäÖÃJobHistoryServer -->

    mapreduce.jobhistory.address
    hadoop-master.qianxin.com:10020


    mapreduce.jobhistory.webapp.address
    hadoop-master.qianxin.com:19888

 

2.png

 

2)Æô¶¯JobHistoryServer

[qianxin@hadoop-master hadoop-2.5.0]$ sbin/mr-jobhistory-daemon.sh start historyserver
[qianxin@hadoop-master hadoop-2.5.0]$ jps
4161 NameNode
6628 JobHistoryServer
4240 SecondaryNameNode
6375 NodeManager
4304 DataNode
6127 ResourceManager
6666 Jps

 

2.png

 

4. ÔËÐÐMapReduce WordCount³ÌÐò
1)ÔÚ/opt/datasĿ¼Ï´´½¨Ò»¸öwordcount_inputĿ¼£¬²¢´´½¨Ò»¸ö×Ô¶¨ÒåÎļþ

[qianxin@hadoop-master hadoop-2.5.0]$ mkdir /opt/datas/wordcount_input
[qianxin@hadoop-master hadoop-2.5.0]$ touch /opt/datas/wordcount_input/wc_input.txt
[qianxin@hadoop-master hadoop-2.5.0]$ vim /opt/datas/wordcount_input/wc_input.txt 

 

2.png

 

2)ÔÚhdfsÉÏ´´½¨Ð·¾¶

[qianxin@hadoop-master hadoop-2.5.0]$ bin/hdfs dfs -mkdir -p /user/qianxin/mapreduce/wordcount/input

 

2.png

 

3)ÉÏ´«×Ô¶¨ÒåµÄwc_input.txtÎļþÖÁhdfsÉϵÄinputĿ¼

[qianxin@hadoop-master hadoop-2.5.0]$ bin/hdfs dfs -put /opt/datas/wordcount_input/wc_input.txt /user/qianxin/mapreduce/wordcount/input
[qianxin@hadoop-master hadoop-2.5.0]$ bin/hdfs dfs -ls /user/qianxin/mapreduce/wordcount/input/*

 

2.png

 

4)ÔËÐÐWordCount³ÌÐò

[qianxin@hadoop-master hadoop-2.5.0]$ bin/yarn jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.5.0.jar wordcount /user/qianxin/mapreduce/wordcount/input /user/qianxin/mapreduce/wordcount/output

 

2.png

 

½çÃæ¼à¿ØMapReduce WordCount³ÌÐòµÄÔËÐУº

2.png

 

MapReduce WordCount³ÌÐò½áÊøºóÏÔʾµÄÔËÐÐÏêÇ飺

2.png

 

5)³ÌÐòÔËÐнá¹û
³ÌÐòÔËÐнáÊøºó»áÔÚ/user/qianxin/mapreduce/wordcount/Ŀ¼ÏÂÉú³ÉÒ»¸öoutputĿ¼¡£

2.png

 

ÏÔʾ³ÌÐòÔËÐеĽá¹û£º

[qianxin@hadoop-master hadoop-2.5.0]$ bin/yarn jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.5.0.jar wordcount /user/qianxin/mapreduce/wordcount/input /user/qianxin/mapreduce/wordcount/output

 

2.png

 


HDFS¡¢YARNÒÔ¼°MapReduce¹¦ÄÜÃèÊö

HDFS

Ò×ÓÚÀ©Õ¹µÄ·Ö²¼Ê½Îļþϵͳ£¬Îª½øÒ»²½·ÖÎö¼ÆËãÌṩÊý¾Ý»ù´¡£»ÔËÐÐÔÚ´óÁ¿ÆÕͨÁ®¼Û»úÆ÷ÉÏ£¬ÌṩÈÝ´í»úÖÆ£¬Êý¾Ý×Ô¶¯ÈßÓ࣬ÎÞÐèÁíÍⱸ·Ý£»Ìṩ·Ö²¼Ê½´æ´¢»úÖÆ£¬Îª´óÁ¿Óû§Ìṩ¿ÉÏÈÐÐÔö³¤µÄ´æ´¢ÄÜÁ¦ÒÔ¼°ÐÔÄܲ»´íµÄÎļþ´æÈ¡·þÎñ¡£
HDFSÈý´óÌåϵ½á¹¹¼°¹¦ÄÜ£º NameNode£ºÊÇÖ÷½Úµã£¬´æ´¢ÎļþµÄÔªÊý¾Ý£¬ÈçÎļþÃû£¬ÎļþĿ¼½á¹¹£¬ÎļþÊôÐÔ£¨Éú³Éʱ¼ä£¬¸±±¾Êý£¬ÎļþȨÏÞ£©ÒÔ¼°Ã¿¸öÎļþµÄ¿éÁбíºÍ¿éËùÔÚµÄDataNodeµÈ£» DataNode£º´æ·Å¾ßÌåÊý¾Ý£¬Êý¾Ý¿é´æ´¢ÔÚ²»Í¬µÄDataNodeÖУ¬ÔÚ±¾µØÎļþϵͳÖдæ·ÅÎļþ¿éÊý¾Ý£¬ÒÔ¼°Îļþ¿ìÊý¾ÝµÄУÑéºÍ£» SecondaryNameNode£ºÓÃÀ´¼à¿ØHDFS״̬µÄ¸¨Öúºǫ́³ÌÐò£¬Ã¿¸ôÒ»¶Îʱ¼ä»ñÈ¡HDFSÔªÊý¾ÝµÄ¿ìÕÕ¡£

YARN

Ö÷Òª¸ºÔð×÷ÒµµÄµ÷¶ÈºÍ×ÊÔ´µÄ¹ÜÀí·ÖÅ䣻½â¾öMapReduce 1.0¹ÌÓÐÎÊÌ⣻½â¾öHadoop 1.X×ÊÔ´ÀûÓÃÂʲ»¾ùºâµÄÎÊÌâ¡£
YARNËÄ´óÖØÒª×é¼þ¼°¹¦ÄÜ£º ResourceManager£ºÕû¸ö¼¯ÈºÖ»ÓÐÒ»¸ö£¬¸ºÔð¼¯Èº×ÊÔ´µÄͳһ¹ÜÀíºÍµ÷¶È¡£Ïêϸ¹¦ÄÜ£º
1£© ´¦Àí¿Í»§¶ËÇëÇó£»
2£© Æô¶¯/¼à¿ØApplicationMaster£»
3£© ¼à¿ØNodeManager£»
4£© ×ÊÔ´·ÖÅäÓëµ÷¶È¡£ NodeManager£ºÕû¸ö¼¯ÈºÓжà¸ö£¬¸ºÔðµ¥½Úµã×ÊÔ´¹ÜÀíºÍʹÓá£Ïêϸ¹¦ÄÜ£º
1£© µ¥¸ö½ÚµãÉϵÄ×ÊÔ´¹ÜÀíºÍÈÎÎñ¹ÜÀí£»
2£© ´¦ÀíÀ´×ÔResourceManagerµÄÃüÁ
3£© ´¦Àíñ®×ÓApplicationMasterµÄÃüÁî¡£ ApplicationMaster£ºÃ¿¸öÓ¦ÓÃÖ»ÓÐÒ»¸ö£¬¸ºÔðÓ¦ÓóÌÐòµÄ¹ÜÀí¡£Ïêϸ¹¦ÄÜ£º
1£© Êý¾ÝÇз֣»
2£© ΪӦÓóÌÐòÉêÇë×ÊÔ´£¬²¢½øÒ»²½·ÖÅ䏸ÄÚ²¿ÈÎÎñ£»
3£© ÈÎÎñ¼à¿ØºÍÈÝ´í¡£ Container£º¶ÔÈÎÎñÔËÐл·¾³µÄ³éÏó¡£Ïêϸ¹¦ÄÜ£º
1£© ÃèÊöÈÎÎñÔËÐÐ×ÊÔ´£¨½Úµã¡¢ÄÚ´æ¡¢CPU£©£»
2£© ÃèÊöÈÎÎñÆô¶¯ÃüÁ
3£© ÃèÊöÈÎÎñÔËÐл·¾³¡£

MapReduce

MapReduceÊÇÒ»¸ö·Ö²¼Ê½²¢ÐмÆËã¿ò¼Ü£» MapReduce˼Ï룺“·Ö¶øÖÎÖ®”£» MapReduceÖ÷ÒªÓÉMapºÍReduceÁ½´ó²¿·Ö¹¹³É£¬MapÈÎÎñʵΪ“·Ö”£¬Ã¿Ò»¸ömapÈÎÎñ¶¼»á¿ªÒ»¸öJavaÐéÄâ»úReduceÈÎÎñʵΪ“ºÏ”£¬½«MapµÄ½á¹û½øÐмÆËãºÍ¹æÔ¼¡£
µã»÷¸´ÖÆÁ´½Ó ÓëºÃÓÑ·ÖÏí!»Ø±¾Õ¾Ê×Ò³
ÉÏһƪ£ºPGM£ºÍ¼Ä£ÐÍѧϰ¸ÅÊö
ÏÂһƪ£ºhadoop-2.5.0Ô´Âë±àÒë
Ïà¹ØÎÄÕÂ
ͼÎÄÍÆ¼ö
ÎÄÕÂ
ÍÆ¼ö
ÈÈÃÅÐÂÎÅ

¹ØÓÚÎÒÃÇ | ÁªÏµÎÒÃÇ | ¹ã¸æ·þÎñ | Ͷ×ʺÏ×÷ | °æÈ¨ÉêÃ÷ | ÔÚÏß°ïÖú | ÍøÕ¾µØÍ¼ | ×÷Æ··¢²¼ | Vip¼¼ÊõÅàѵ
°æÈ¨ËùÓÐ: ºìºÚÁªÃË--ÖÂÁ¦ÓÚ×öʵÓõÄIT¼¼ÊõÑ§Ï°ÍøÕ¾