0
respostas

Erro Spark-Submit

Olá

Estou tendo o seguinte erro ao executar o exemplo python:

21/08/13 10:57:57 WARN Shell: Did not find winutils.exe: {}
java.io.FileNotFoundException: Hadoop bin directory does not exist: C:\Users\henri\Desktop\Hadoop\bin -see https://wiki.apache.org/hadoop/WindowsProblems
        at org.apache.hadoop.util.Shell.getQualifiedBinInner(Shell.java:608)
        at org.apache.hadoop.util.Shell.getQualifiedBin(Shell.java:592)
        at org.apache.hadoop.util.Shell.<clinit>(Shell.java:689)
        at org.apache.hadoop.util.StringUtils.<clinit>(StringUtils.java:78)
        at org.apache.hadoop.conf.Configuration.getTimeDurationHelper(Configuration.java:1814)
        at org.apache.hadoop.conf.Configuration.getTimeDuration(Configuration.java:1791)
        at org.apache.hadoop.util.ShutdownHookManager.getShutdownTimeout(ShutdownHookManager.java:183)
        at org.apache.hadoop.util.ShutdownHookManager$HookEntry.<init>(ShutdownHookManager.java:207)
        at org.apache.hadoop.util.ShutdownHookManager.addShutdownHook(ShutdownHookManager.java:302)
        at org.apache.spark.util.SparkShutdownHookManager.install(ShutdownHookManager.scala:181)
        at org.apache.spark.util.ShutdownHookManager$.shutdownHooks$lzycompute(ShutdownHookManager.scala:50)
        at org.apache.spark.util.ShutdownHookManager$.shutdownHooks(ShutdownHookManager.scala:48)
        at org.apache.spark.util.ShutdownHookManager$.addShutdownHook(ShutdownHookManager.scala:153)
        at org.apache.spark.util.ShutdownHookManager$.<init>(ShutdownHookManager.scala:58)
        at org.apache.spark.util.ShutdownHookManager$.<clinit>(ShutdownHookManager.scala)
        at org.apache.spark.util.Utils$.createTempDir(Utils.scala:326)
        at org.apache.spark.deploy.SparkSubmit.prepareSubmitEnvironment(SparkSubmit.scala:343)
        at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:894)
        at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
        at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
        at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
        at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1039)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1048)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
21/08/13 10:57:57 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Python nÒo encontrado; execute sem argumentos para instalar na Microsoft Store ou desabilite este atalho a partir de Configuraþ§es > Gerenciar Aliases de ExecuþÒo do Aplicativo.
log4j:WARN No appenders could be found for logger (org.apache.spark.util.ShutdownHookManager).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.

Alguma sugestão? Seria algum tipo de erro de instalação?

Obrigado