使用spark访问elasticsearch的数据，前提是spark能访问hive，hive能访问es http://blog.csdn.net/ggz631047367/article/details/50445832 1.配置 cp elasticsearch-hadoop-2.1.1/dist/elasticsearch-hadoop-2.1.1.jar /usr/local/spark-1.5.2/lib/ 2.修改配置文件/usr/local/spark/conf/hive-site.xml,添加 <property>   <name>hive.aux.jars.path</name>   <value>file:///usr/local/spark-1.5.2/lib/elasticsearch-hadoop-2.1.1.jar</value>   <description>A comma separated list (with no spaces) of the jar files</description> </property> 3.配置/usr/local/hive-1.2.1/lib/MySQL-connector-java-5.1.31-bin.jar,在spark-env.sh添加 export SPARK_CLASSPATH=$SPARK_CLASSPATH:/usr/local/hive-1.2.1/lib/mysql-connector-java-5.1.31-bin.jar 4.启动 sbin/start-thriftserver.sh   --hiveconf hive.server2.thrift.port=10000  --hiveconf hive.server2.thrift.bind.host=hadoop-master     --master spark://10.8.2.100:7077  --jars /usr/local/spark-1.5.2/lib/elasticsearch-hadoop-2.1.1.jar 或者 bin/spark-sql –master spark://10.8.2.100:7077 –jars /usr/local/spark-1.5.2/lib/elasticsearch-hadoop-2.1.1.jar 5.测试 beeline -u jdbc:hive2://10.8.2.100:10000 (稍等一下执行此命令，报错有可能是服务还没起来) select count(dtime) from achi_ex limit 10; 注意：–driver-class-path和–jars的区别，使用sbin/start-thriftserver.sh -h查看 如果1中拷贝到所有slave中spark/lib中，后面不用—jars指定jar包 这种方式比直接访问hive中的数据慢一倍左右

 

http://blog.csdn.net/ggz631047367/article/details/50445790

http://blog.csdn.net/stark_summer/article/details/49743687