Hadoop pseudo-distributed cluster construction
Software preparation
I'm using a virtual machine with CentOS-6.6, with the host name repo. Refer to the steps to install a Linux virtual machine in Windows, I installed JDK in that virtual machine, refer to the guide to installing JDK in Linux. In addition, the virtual machine is configured with a key-free login itself, and the settings for configuring key-free login between each virtual machine are referenced. The download address of the Hadoop installation package is: //m.sbmmt.com/link/8485694bae96aebc7c4fe6119599d0e0 .
-
Upload the Hadoop installation package to the server and decompress it
[root@repo ~]# tar zxvf hadoop-2.6.5.tar.gz -C /opt/apps/
-
Configure environment variables
# You can directly locate the last line of the file [root@repo hadoop-2.6.5]# vi /etc/profile export HADOOP_HOME=/opt/apps/hadoop-2.6.5 export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin [root@repo hadoop-2.6.5]# . /etc/profile
-
Modify the three configuration files of hadoop-env.sh, mapred-env.sh, and yarn-env.sh, and add JAVA_HOME
[root@repo hadoop]# pwd /opt/apps/hadoop-2.6.5/etc/hadoop [root@repo hadoop]# vi hadoop-env.sh export JAVA_HOME=/usr/local/jdk1.8.0_73 [root@repo hadoop]# vi mapred-env.sh export JAVA_HOME=/usr/local/jdk1.8.0_73 [root@repo hadoop]# vi yarn-env.sh export JAVA_HOME=/usr/local/jdk1.8.0_73
-
Modify the core-site.xml and hdfs-site.xml configuration files to configure pseudo-distributed related content
[root@repo hadoop]# vi core-site.xml <configuration> <property> <name>fs.defaultFS</name> <value>hdfs://repo:9000</value> </property> <property> <name>hadoop.tmp.dir</name> <value>/var/hadoop/pseudo</value> </property> </configuration> [root@repo hadoop]# vi hdfs-site.xml <configuration> <property> <name>dfs.replication</name> <value>1</value> </property> <property> <name>dfs.namenode.secondary.http-address</name> <value>repo:50090</value> </property> </configuration>
-
Modify the slaves configuration file and specify the node where the DataNode is located
[root@repo hadoop]# vi slaves repo
-
Format file system
[root@repo hadoop]# hadoop namenode --format # Success message17/09/16 21:17:11 INFO common.Storage: Storage directory /var/hadoop/pseudo/dfs/name has been successfully formatted.
-
Start HDFS and YARN
[root@repo hadoop]# start-dfs.sh Starting namesnodes on [repo] repo: starting namenode, logging to /opt/apps/hadoop-2.6.5/logs/hadoop-root-namenode-repo.out repo: starting datanode, logging to /opt/apps/hadoop-2.6.5/logs/hadoop-root-datanode-repo.out Starting secondary namesodes [repo] repo: starting secondarynamenode, logging to /opt/apps/hadoop-2.6.5/logs/hadoop-root-secondarynamenode-repo.out [root@repo hadoop]# start-yarn.sh starting yarn daemons starting resourcemanager, logging to /opt/hadoop-2.7.4/logs/yarn-root-resourcemanager-repo.out repo: starting nodemanager, logging to /opt/hadoop-2.7.4/logs/yarn-root-nodemanager-repo.out [root@repo hadoop]# jps 4368 Jps 3957 ResourceManager 3512 NameNode 3641 DataNode 4058 NodeManager 3805 SecondaryNameNode
Visit the WEB page

Build successfully!
The above is the detailed content of Hadoop pseudo-distributed cluster construction. For more information, please follow other related articles on the PHP Chinese website!
Hot AI Tools
Undress AI Tool
Undress images for free
Undresser.AI Undress
AI-powered app for creating realistic nude photos
AI Clothes Remover
Online AI tool for removing clothes from photos.
Clothoff.io
AI clothes remover
Video Face Swap
Swap faces in any video effortlessly with our completely free AI face swap tool!
Hot Article
Hot Tools
Notepad++7.3.1
Easy-to-use and free code editor
SublimeText3 Chinese version
Chinese version, very easy to use
Zend Studio 13.0.1
Powerful PHP integrated development environment
Dreamweaver CS6
Visual web development tools
SublimeText3 Mac version
God-level code editing software (SublimeText3)
Hot Topics
How to fix second monitor not detected in Windows?
Jul 12, 2025 am 02:27 AM
When Windows cannot detect a second monitor, first check whether the physical connection is normal, including power supply, cable plug-in and interface compatibility, and try to replace the cable or adapter; secondly, update or reinstall the graphics card driver through the Device Manager, and roll back the driver version if necessary; then manually click "Detection" in the display settings to identify the monitor to confirm whether it is correctly identified by the system; finally check whether the monitor input source is switched to the corresponding interface, and confirm whether the graphics card output port connected to the cable is correct. Following the above steps to check in turn, most dual-screen recognition problems can usually be solved.
How to Install PHP on Windows
Jul 15, 2025 am 02:46 AM
The key steps to install PHP on Windows include: 1. Download the appropriate PHP version and decompress it. It is recommended to use ThreadSafe version with Apache or NonThreadSafe version with Nginx; 2. Configure the php.ini file and rename php.ini-development or php.ini-production to php.ini; 3. Add the PHP path to the system environment variable Path for command line use; 4. Test whether PHP is installed successfully, execute php-v through the command line and run the built-in server to test the parsing capabilities; 5. If you use Apache, you need to configure P in httpd.conf
Meme Coin Mania: The Power of Dogecoin, Shiba Inu and Community Hype
Jul 10, 2025 pm 07:48 PM
The rise of meme coins reflects the key role of community power and social media influence in the cryptocurrency market. 1. Dogecoin was originally a satirical joke and was born in 2013; 2. Driven by tweets from celebrities such as Elon Musk, the attention soared; 3. The market value once reached tens of billions of dollars, becoming a mainstream digital asset. Shiba Inu Coin is positioned as a "dogcoin killer" and has rapidly risen through community-driven strategies, building a decentralized exchange ShibaSwap, and relies on low-priced units to attract a large number of users to participate. Its success also depends on circulation guarantees on mainstream platforms such as Binance, Coinbase, and OKX. The core driving forces of meme coins include: 1. Viral transmission mechanism, rapid spread of information; 2. Enhanced sense of community belonging
How to clear the print queue in Windows?
Jul 11, 2025 am 02:19 AM
When encountering the problem of printing task stuck, clearing the print queue and restarting the PrintSpooler service is an effective solution. First, open the "Device and Printer" interface to find the corresponding printer, right-click the task and select "Cancel" to clear a single task, or click "Cancel all documents" to clear the queue at one time; if the queue is inaccessible, press Win R to enter services.msc to open the service list, find "PrintSpooler" and stop it before starting the service. If necessary, you can manually delete the residual files under the C:\Windows\System32\spool\PRINTERS path to completely solve the problem.
What does the counter-referential meaning of the currency circle? Why do some people specifically operate in reverse? Market sentiment indicators
Jul 10, 2025 pm 09:27 PM
The "reverse reference" in the currency circle, as the name suggests, refers to those reference objects whose views or operations are often opposite to the actual market trend. When such people or groups are extremely optimistic, the market may face a decline; when they are extremely pessimistic, the market may instead rebound. This is not to say that these people deliberately provide wrong signals, but that their judgments may deviate from the mainstream trends in the market, or that their operating behavior happens to be a catalyst for market reversal in a specific situation.
Windows can't connect to this network
Jul 10, 2025 pm 12:40 PM
When encountering the problem of "Windowscan'tconnecttothisnetwork", you should first check whether the network name and password are correct, including case, spaces and special characters. If it is a Chinese WiFi name, try to change it to English, and then forget the network and reconnect; secondly, check whether the router limits the number of devices or enables MAC address filtering, and restart the router or log in to the background to adjust the settings; thirdly, check the system network settings and drivers, update the wireless network card driver, reset the network settings or confirm that the IP is automatically obtained; finally check whether the network authentication method is compatible, especially in enterprise or campus networks, you need to consult the administrator to configure the correct EAP protocol and installation certificate.
How to enable HTTP Strict Transport Security (HSTS) in Apache?
Jul 13, 2025 am 01:12 AM
Enable HSTS to force browsers to access websites through HTTPS, improving security. 1. To enable HTTPS in Apache, you must first configure HTTPS, and then add Strict-Transport-Security response header in the site configuration file or .htaccess; 2. To configure max-age (such as 31536000 seconds), includeSubDomains and preload parameters; 3. Make sure that the mod_headers module is enabled, otherwise run sudoa2enmodheaders and restart Apache; 4. You can optionally submit to the HSTSPreload list, but it must satisfy that both the main site and the subdomain support HTTPS.
How to check the airdrop of currency circle projects? How to avoid fake airdrop scams?
Jul 10, 2025 pm 09:12 PM
Finding airdrop opportunities for cryptocurrency projects is the way many participants want to acquire tokens for early-stage projects. These airdrops are usually a means for project parties to promote brand, community construction, or inspire early users. To find this information effectively, you need to rely on multiple reliable channels and methods.



Build successfully!
