preparation

  • install jdk
  • User password free login

Installation reference article :

http://blog.csdn.net/circyo/article/details/46724335

http://www.linuxidc.com/Linux/2015-11/124800.htm

Common commands :

1、 find hadoop The installation directory , Start command “sbin/start-all.sh"

2、hdfs file system ”bin/hadoop fs"

3、“bin/hadoop fs -help" see hdfs File system common commands

4、 Configuration file in hadoop The installation directory “etc" Next

5、bin/hdfs fsck /   function DFS File detection and debugging

6、bin/hdfs namenode -format   format namenode

7、bin/hdfs dfsadmin -safemode  enter/leave/get/wait   ( safe mode : Enter to open / closed / get / wait for )

Be careful :

1、hadoop JAVA_HOME Need configuration

2、 After restarting the machine namenode It won't start , It can be formatted hdfs solve

3、datanode It won't start , Can be deleted by dfs Catalog resolution ( The possible reason is that the cluster information is incorrect , Maybe it's like formatting namenode relevant , The operation steps stop hadoop, Delete dfs Catalog , format hdfs, start-up hadoop)

eclipse  hadoop2.7.3 install

Reference article :http://www.cnblogs.com/duking1991/p/6056923.html

Be careful :

1、 After decompressing hadoop The path address needs no spaces or special characters , otherwise eclipse Unable to load hadoop Package file

2、 Add... To the project log4j To configure , Can be found in eclipse Output log information in

3、 Can't operate hadoop file , Users are not allowed to access (Permission denied), One solution , Change directory permissions ,"bin/hadoop fs -chmod 777 /user/***"

4、 stay main Middle configuration hadoop Catalog ,System.setProperty("hadoop.home.dir","d:\\hadoop-2.7.3");

my hadoop To configure :

core-site.xml

<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://master:9000</value>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>/home/hadoop/tmp</value>
</property>
<property>
<name>io.file.buffer.size</name>
<value>131702</value>
</property>
</configuration>

hdfs-site.xml

<configuration>
<property>
<name>dfs.namenode.name.dir</name>
<value>file:/home/hadoop/dfs/name</value>
</property>
<property>
<name>dfs.datanode.data.dir</name>
<value>file:/home/hadoop/dfs/data</value>
</property>
<property>
<name>dfs.replication</name>
<value>2</value>
</property>
<property>
<name>dfs.namenode.secondary.http-address</name>
<value>master:9001</value>
</property>
<property>
<name>dfs.webhdfs.enabled</name>
<value>true</value>
</property>
</configuration>

mapred-site.xml

<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
<final>true</final>
</property> <property>
<name>mapreduce.jobtracker.http.address</name>
<value>master:50030</value>
</property>
<property>
<name>mapreduce.jobhistory.address</name>
<value>master:10020</value>
</property>
<property>
<name>mapreduce.jobhistory.webapp.address</name>
<value>master:19888</value>
</property>
<property>
<name>mapred.job.tracker</name>
<value>http://master:9001</value>
</property> </configuration>

yarn-site.xml

<configuration>
<!-- Site specific YARN configuration properties -->
<property>
<name>yarn.resourcemanager.hostname</name>
<value>master</value>
</property> <property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.resourcemanager.address</name>
<value>master:8032</value>
</property>
<property>
<name>yarn.resourcemanager.scheduler.address</name>
<value>master:8030</value>
</property>
<property>
<name>yarn.resourcemanager.resource-tracker.address</name>
<value>master:8031</value>
</property>
<property>
<name>yarn.resourcemanager.admin.address</name>
<value>master:8033</value>
</property>
<property>
<name>yarn.resourcemanager.webapp.address</name>
<value>master:8088</value>
</property>
</configuration>

Password free login settings :centos 7 Password free login

centos 7 hadoop Installation and use of more related articles

  1. centos Next hadoop Installation

        hadoop It's not difficult to install , But it takes a lot of preparation . One .JDK It needs to be installed first jdk.centos You can go directly through yum install java-1.6.0-openjdk To install . Installation of different releases ...

  2. centos in -hadoop Single machine installation and pseudo distributed running instance

    Create user and join Authorization 1, establish hadoop user sudo useradd -m hadoop -s /bin/bash 2, modify sudo Configuration file for , be located /etc/sudoers, need root Permission to read ...

  3. centos And hadoop Installation

    Evernote Export First step The deployment environment Reference resources http://dblab.xmu.edu.cn/blog/install-hadoop-in-centos/ 1. establish hadoop user $su ...

  4. CentOS 7 Hadoop Installation configuration

    Preface : I used two computers for cluster configuration , If it's a stand-alone machine, there may be some problems . First, set the host names of the two computers root Permission open /etc/host file To set up hostname,root Permission open /etc/hos ...

  5. Centos 7 Installation and configuration in the environment Hadoop 3.0 Beta1 Brief notes

    Preface Because I have written an article before Centos 7 Installation and configuration in the environment 2.8 The essay of , So this one should be concise , Just select some important links and record them . The installation environment is : Both hosts are Centos 7.* operating system , The two machines are configured as : host ...

  6. Hadoop(2)-CentOS Under the jdk and hadoop Installation and configuration

    preparation download jdk8 and hadoop2.7.2 Use sftp By the way of hadoop100 Upper /opt/software Directory Configuration environment If you select... When installing the virtual machine open java, Please uninstall first rpm -q ...

  7. hadoop Cluster building --CentOS Deploy Hadoop service

    In understanding Hadoop After the relevant knowledge of , The next step is Hadoop Construction of environment , build Hadoop Environment is the beginning of learning big data , And then we start to build the environment ! We use the environment for :VMware 12+CentOS6.4 hadoo ...

  8. hadoop Stand alone installation and basic function demonstration

    Used in this article Linux The release version is :CentOS Linux release 7.4.1708 (Core) hadoop Stand alone installation preparation Create user useradd -m hadoop passwd ...

  9. CentOS Next Hadoop And ZooKeeper Environment building

    1. Test environment operating system CentOS 6.5. in total 5 Taiwan machine , The first two serve as namenode, be called nn01.nn02: The last three as datanode, be called dn01.dn02.dn03. Memory per machine ...

Random recommendation

  1. jQuery/javascript Implementation of a simple web calculator

    <html> <head> <meta charset="utf-8"> <title>jQuery Realization </title> ...

  2. VC6.0 error LNK2001: unresolved external symbol _main terms of settlement

    Study VC++ We often encounter link errors when we do LNK2001, This mistake is very annoying , Because for programmers , The best error to correct is the compilation error , Generally speaking, when a connection error occurs , The compilation has passed . There are many reasons for connection errors , especially LNK2001 error ...

  3. Yes n A circle of individuals , Sequence number . Count from the first person ( from 1 To 3 Number off ), Where to report 3 Of the people out of the circle , The last one left is the one with the original number .

    First write my thoughts , There's no pointer . If you're using a VC, Remove the sixth line . #include<stdio.h> #include<stdlib.h> int main() { setvb ...

  4. struts2 Use struts2-bootstrap-plugin plug-in unit

    1. Download plug-ins http://code.google.com/p/struts2-bootstrap/ 2. add to maven rely on <dependency> <groupId>com ...

  5. log4j.properties File configuration -- Official documents

    Default Initialization Procedure The log4j library does not make any assumptions about its environme ...

  6. UML_ Component diagram

    brief introduction as everyone knows , Component diagram is used to describe the relationship between components in the system . First we have to know what the definition of a component is , And then what are the relationships between the components . Sort this out , We will be able to Come in handy .UML The language's definition of components has changed dramatically ...

  7. stay Linux RedHatEL6 Installation and configuration in the environment JDK1.7 + Tomcat7.0 + MySQL5.6

    RedHatEL6 JDK The installation path : /usr/java/jdk1.7 Tomcat The installation path :/usr/local/tomcat7/ MySQL The installation path : /usr/local/mysql It is divided into three parts ...

  8. DBLookupCombobox Achieve pull-down linkage

    This time DBLookupCombobox Realize the province - City - Regional pull-down linkage , To learn DBLookupCombobox Some uses of design sketch : First set up the data table ( Data tables can be in many forms : Database table . Text .xml etc. , most ...

  9. Basic summary (04)-- display:none;&amp;&amp;visibility:hidden; difference

    display:none 1. Hide elements , No more space . 2. Dynamic operation will cause page reflow and redraw , Affect performance . 3. Child elements are also hidden and added display:block/visibility:visible nothing ...

  10. Ajax Receive the Boolean value and the specified string sent by the background

    backstage : aContext.getResponse().getWriter().println("" + result); front end : $.ajax({ url:encodeURI(en ...