Category : hadoop-native-library

I’m using apache arrow project(https://arrow.apache.org/docs/python/filesystems.html#hadoop-file-system-hdfs) to do read parquet file located in HDFS. Apache arrow uses hadoop native library files to read is from HDFS. one of the file it uses is libhdfs.dylib OSError: Unable to load libhdfs: dlopen(/Users/sharad.mishra/Library/hadoop/hadoop-2.7.3/lib/native/libhdfs.dylib, 6): image not found I downloaded hadoop source files (https://github.com/apache/hadoop.git) and build it including native. Hadoop ..

Read more