Hadoop - Unable to load native-hadoop library for your platform

简介

运行hadoop或者spark(调用hdfs等)时,总出现这样的错误“Unable to load native-hadoop library for your platform”,其实是无法加载本地库

解决办法

1.环境变量是否设好(设好但是还不行尝试第二步)

export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib/native"

2.默认的libhadoop.so.1.0.0(lib/native)是否是32位,而机器是64位,此时需要编译hadoop,然后替换lib里面的文件

编译安装选项

mvn clean package -Pdist -Dtar -Dmaven.javadoc.skip=true -DskipTests -fail-at-end -Pnative

如果提示需要‘libprotoc 2.5.0

1 sudo apt-get install -y gcc g++ make maven cmake zlib zlib1g-dev libcurl4-openssl-dev
2           curl -# -O https://protobuf.googlecode.com/files/protobuf-2.5.0.tar.gz
3           gunzip protobuf-2.5.0.tar.gz
4           tar -xvf protobuf-2.5.0.tar
5           cd protobuf-2.5.0
6           ./configure --prefix=/usr
7           make
8           sudo make install9           sudo ldconfig

如果是在Ubuntu14.04里,提示libtool的错误,那么用/usr/bin/libtool来替换文件中的libtool

编译好的文件

设置HADOOP的日志级别

export HADOOP_ROOT_LOGGER=DEBUG,console

结果

参考:

http://stackoverflow.com/questions/30702686/building-apache-hadoop-2-6-0-throwing-maven-error

http://stackoverflow.com/questions/19943766/hadoop-unable-to-load-native-hadoop-library-for-your-platform-warning

http://stackoverflow.com/questions/19556253/trunk-doesnt-compile-because-libprotoc-is-old-when-working-with-hadoop-under-ec

时间: 07-20

Hadoop - Unable to load native-hadoop library for your platform的相关文章

Hadoop “Unable to load native-hadoop library for your platform” error on CentOS

在NameNode节点上一切正常,并没有提示这个信息,但是在DataNode上出现如下提示信息: 15/01/14 16:42:09 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 经查原来是DataNode子节点上/home/hadoop/hadoop2.2/lib目录下没有nativ

Unable to load native-hadoop library for your platform(已解决)

1.增加调试信息寻找问题 2.两种方式解决unable to load native-hadoop library for you platform 附:libc/glibc/glib简介 参考: 1.http://my.oschina.net/swuly302/blog/515853[66号公路: Hadoop Unable to load­ ­n­­ati­v­e­-­h­a­­d­o­o­­p library for your platform] 2.http://blog.sina.co

Hadoop 2.2.0 常见问题之:Ubuntu 64环境下“Unable to load native-hadoop library for your platform”问题”

问题 最近在学习Hadoop(2.2.0),打算写一个MapReduce的小程序在Ubuntu 64位的环境下测试一把,一切环境配置完毕后,执行的过程中,控制台输出下面的内容: WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 原因 在网上搜索了一番,得出如下结论: "The reason

hadoop2.6.0实践:A02 问题处理 util.NativeCodeLoader: Unable to load native-hadoop library for your platform

############################################################# hadoop "util.NativeCodeLoader: Unable to load native-hadoop library for your platform" hadoop安装完以后,经常会提示以下警告: WARN util.NativeCodeLoader: Unable to load native-hadoop library for your

解决Unable to load native-hadoop library for your platform

使用hadoop fs相应命令时候总是出现 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 系统环境下 添加 export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native export HADOOP_OPTS="-Djava.li

Exception in thread "main" java.lang.UnsatisfiedLinkError: Unable to load library 'dll.msvcrt'

上网找了一下方法,说是jdk64的问题,也有说是路径不对问题. 代码片段 1 public interface CLibrary extends Library { 2 CLibrary INSTANCE = (CLibrary) Native.loadLibrary("dll.msvcrt", 3 CLibrary.class); 4 void printf(String format, Object... args); 5 } 在看到以下文章之后,觉得loadLibrary()加载

xvfb启动PyQt4程序报Unable to load library icui18n错误

xvfb启动PyQt4程序报如下错误: Unable to load library icui18n "Cannot load library icui18n: (libicui18n.so.48: cannot open shared object file: No such file or directory)" 解决方法: sudo apt-get install libicu48 参考:https://forums.virtualbox.org/viewtopic.php?f=

php 编译安装插件提示PHP Warning: PHP Startup: Unable to load dynamic library 报错

PHP Warning:  PHP Startup: Unable to load dynamic library '/usr/local/lib/php/extensions/no-debug-non-zts-20090626/"xcache.so"' - /usr/local/lib/php/extensions/no-debug-non-zts-20090626/"xcache.so": cannot open shared object file: No s

wamp在win7-64位下Unable to load dynamic library php_c

wamp在win7-64位下Unable to load dynamic library php_curl.dll网上很多资料都类似这个:PHP在Windows下启动时提示Unable to load dynamic library php_curl.dll问题解决作者:朦朧中的罪惡 发布于:2012-3-2 5:01 Friday 分类:疑难杂症这个问题容易手配PHP的时候出现, 在php.ini中正确开启了curl扩展后, 在PHPINFO中却看不到相关的信息, 而Apache的日志会显示以

PHP Warning: PHP Startup: Unable to load dynamic library '/usr/lib64/php/modules/module.so'

详细的错误截图如下所示: 第一个框中的错误,是因为版本混乱的原因,只需要删除/usr/lib/libxml2.so.2文件即可. 第二个框中的错误,是因为httpd重复加载了PHP的动态模块,只需要删除对应的ini文件即可. 比如:/usr/lib64/php/modules/odbc.so动态库对应的ini文件为:/etc/php.d/odbc.ini,删除这个文件即可. PHP Warning: PHP Startup: Unable to load dynamic library '/us