Hadoop - Unable to load native-hadoop library for your platform

简介

运行hadoop或者spark(调用hdfs等)时,总出现这样的错误“Unable to load native-hadoop library for your platform”,其实是无法加载本地库

解决办法

1.环境变量是否设好(设好但是还不行尝试第二步)

export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib/native"

2.默认的libhadoop.so.1.0.0(lib/native)是否是32位,而机器是64位,此时需要编译hadoop,然后替换lib里面的文件

编译安装选项

mvn clean package -Pdist -Dtar -Dmaven.javadoc.skip=true -DskipTests -fail-at-end -Pnative

如果提示需要‘libprotoc 2.5.0

1 sudo apt-get install -y gcc g++ make maven cmake zlib zlib1g-dev libcurl4-openssl-dev
2           curl -# -O https://protobuf.googlecode.com/files/protobuf-2.5.0.tar.gz
3           gunzip protobuf-2.5.0.tar.gz
4           tar -xvf protobuf-2.5.0.tar
5           cd protobuf-2.5.0
6           ./configure --prefix=/usr
7           make
8           sudo make install9           sudo ldconfig

如果是在Ubuntu14.04里,提示libtool的错误,那么用/usr/bin/libtool来替换文件中的libtool

编译好的文件

设置HADOOP的日志级别

export HADOOP_ROOT_LOGGER=DEBUG,console

结果

参考:

http://stackoverflow.com/questions/30702686/building-apache-hadoop-2-6-0-throwing-maven-error

http://stackoverflow.com/questions/19943766/hadoop-unable-to-load-native-hadoop-library-for-your-platform-warning

http://stackoverflow.com/questions/19556253/trunk-doesnt-compile-because-libprotoc-is-old-when-working-with-hadoop-under-ec

时间: 07-20

Hadoop - Unable to load native-hadoop library for your platform的相关文章

关于cocos2dx 3.0升级崩溃报错(unable to load native library) 和(Fatal signal 11 (SIGSEGV) at 0x00000000)

最近一直在Windows平台开发cocos-2dx游戏,期间做了一次引擎升级,升级到了3.0正式版本.Windows平台上表现很正常,没有出现什么问题. 上周五准备发布一个安卓包,编译很轻松的就过了,没有花费多少时间,但是安装到手机后,发现运行就崩溃了.没办法只好用模拟机调试,再次吐槽Android的模拟器,真的太他妈慢了,不到万不得已我真的不想再去用它,google真的应该好好整一下了. 好不容易运行起来了,看到崩溃的时候logcat的报错是"unable to load native lib

Hadoop “Unable to load native-hadoop library for your platform” error on CentOS

在NameNode节点上一切正常,并没有提示这个信息,但是在DataNode上出现如下提示信息: 15/01/14 16:42:09 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 经查原来是DataNode子节点上/home/hadoop/hadoop2.2/lib目录下没有nativ

Hadoop “Unable to load native-hadoop library for y

http://devsolvd.com/questions/hadoop-unable-to-load-native-hadoop-library-for-your-platform-error-on-centos The answer depends... I just installed Hadoop 2.6 from tarball on 64-bit CentOS 6.6. The Hadoop install did indeed come with a prebuilt 64-bit

Unable to load native-hadoop library for your platform(已解决)

1.增加调试信息寻找问题 2.两种方式解决unable to load native-hadoop library for you platform 附:libc/glibc/glib简介 参考: 1.http://my.oschina.net/swuly302/blog/515853[66号公路: Hadoop Unable to load­ ­n­­ati­v­e­-­h­a­­d­o­o­­p library for your platform] 2.http://blog.sina.co

Hadoop 2.2.0 常见问题之:Ubuntu 64环境下“Unable to load native-hadoop library for your platform”问题”

问题 最近在学习Hadoop(2.2.0),打算写一个MapReduce的小程序在Ubuntu 64位的环境下测试一把,一切环境配置完毕后,执行的过程中,控制台输出下面的内容: WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 原因 在网上搜索了一番,得出如下结论: "The reason

WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable(spark加载hadoop本地库的时候出现不能加载的情况要怎么解决呢?)

原文地址: https://www.zhihu.com/question/23974067/answer/26267153 原文内容: 我64位机器,当时hadoop启动的时候出现这个问题是因为hadoop本身自带的本地库是32位的,我现在hadoop2.2.0已经替换了本地库为64位的了,并且编译spark的时候采用了相对应的版本:SPARK_HADOOP_VERSION=2.2.0 SPARK_YARN=true ./sbt/sbt assembly但是现在进入spark shell的时候依

HADOOP:WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable终于解决了

WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 参考了网上很多办法,不行. 这个警告总觉得不爽.. 最近部署SPARK时,顺带解决了. URL参考如下“ http://www.mak-blog.com/spark-on-yarn-setup-multinode.html 一,通常编译好的

Hadoop安装—— WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platf

今天在安装hadoop完成测试创建用户目录失败在网上找到了原因记录一下原文地址 http://blog.csdn.net/l1028386804/article/details/51538611 配置完hadoop启动运行的时候出现如下警告信息: WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applica

Hadoop中 Unable to load native-hadoop library for your platform... using builtin-java classes where applicable问题解决

环境 [[email protected] soft]# cat /etc/issue CentOS release 6.5 (Final) Kernel \r on an \m [[email protected] soft]# uname -a Linux vm8028 2.6.32-431.el6.x86_64 #1 SMP Fri Nov 22 03:15:09 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux [[email protected] soft