使用flink stream接收kafka消息数据,处理后存储到cb上,在本地运行没有问题,发布到yarn集群上启动报如下错误:
------------------------------------------------------------ The program finished with the following exception: java.lang.RuntimeException: Error deploying the YARN cluster at org.apache.flink.yarn.cli.FlinkYarnSessionCli.createCluster(FlinkYarnSessionCli.java:556) at org.apache.flink.yarn.cli.FlinkYarnSessionCli.createCluster(FlinkYarnSessionCli.java:72) at org.apache.flink.client.CliFrontend.createClient(CliFrontend.java:962) at org.apache.flink.client.CliFrontend.run(CliFrontend.java:243) at org.apache.flink.client.CliFrontend.parseParameters(CliFrontend.java:1086) at org.apache.flink.client.CliFrontend$2.call(CliFrontend.java:1133) at org.apache.flink.client.CliFrontend$2.call(CliFrontend.java:1130) at org.apache.flink.runtime.security.HadoopSecurityContext$1.run(HadoopSecurityContext.java:43) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1651) at org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:40) at org.apache.flink.client.CliFrontend.main(CliFrontend.java:1129) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.iqiyi.europa.EuropaBootstrapper$.startClass(EuropaBootstrapper.scala:56) at com.iqiyi.europa.EuropaBootstrapper$.main(EuropaBootstrapper.scala:37) at com.iqiyi.europa.EuropaBootstrapper.main(EuropaBootstrapper.scala) Caused by: java.lang.RuntimeException: Couldn't deploy Yarn cluster at org.apache.flink.yarn.AbstractYarnClusterDescriptor.deploy(AbstractYarnClusterDescriptor.java:447) at org.apache.flink.yarn.cli.FlinkYarnSessionCli.createCluster(FlinkYarnSessionCli.java:554) ... 19 more Caused by: org.apache.flink.yarn.AbstractYarnClusterDescriptor$YarnDeploymentException: The YARN application unexpectedly switched to state FAILED during deployment. Diagnostics from YARN: Application application_1529082330695_659780 failed 1 times due to AM Container for appattempt_1529082330695_659780_000001 exited with exitCode: 255 due to: Exception from container-launch. Container id: container_1529082330695_659780_01_000005 Exit code: 255 Stack trace: ExitCodeException exitCode=255: at org.apache.hadoop.util.Shell.runCommand(Shell.java:538) at org.apache.hadoop.util.Shell.run(Shell.java:455) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702) at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.launchContainer(LinuxContainerExecutor.java:281) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:299) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Shell output: main : command provided 1 main : user is qytt main : requested yarn user is qytt Container exited with a non-zero exit code 255 .Failing this attempt.. Failing the application. If log aggregation is enabled on your cluster, use this command to further investigate the issue: yarn logs -applicationId application_1529082330695_659780 at org.apache.flink.yarn.AbstractYarnClusterDescriptor.startAppMaster(AbstractYarnClusterDescriptor.java:904) at org.apache.flink.yarn.AbstractYarnClusterDescriptor.deployInternal(AbstractYarnClusterDescriptor.java:586) at org.apache.flink.yarn.AbstractYarnClusterDescriptor.deploy(AbstractYarnClusterDescriptor.java:445) ... 20 more
从该错误堆栈中无法找到具体的错误信息,只能知道是yarn启动错误,仔细查看日志,发现其中:
yarn logs -applicationId application_1529082330695_659780
于是在hadoop客户端命令行上执行上述命令,可以查看具体的错误信息:
[qytt@mbd-toutiao-worker-online056-jylt ~]$ yarn logs -applicationId application_1529082330695_659780 Container: container_1529082330695_659780_01_000005 on node153-154-134-jylt.abc.hadoop_8041 ============================================================================================== LogType: jobmanager.err LogLength: 3674 Log Contents: SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/data5/yarn/local/usercache/qytt/appcache/application_1529082330695_659780/filecache/16/flink-PingbackJinyingLog-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/data11/yarn/local/usercache/qytt/appcache/application_1529082330695_659780/filecache/13/lib/slf4j-log4j12-1.7.7.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/usr/lib/zookeeper/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] Uncaught error from thread [flink-akka.remote.default-remote-dispatcher-6] shutting down JVM since 'akka.jvm-exit-on-fatal-error' is enabled for ActorSystem[flink] java.lang.NoSuchMethodError: org.jboss.netty.channel.socket.nio.NioClientSocketChannelFactory.(Ljava/util/concurrent/Executor;ILorg/jboss/netty/channel/socket/nio/WorkerPool;Lorg/jboss/netty/util/Timer;)V at akka.remote.transport.netty.NettyTransport. (NettyTransport.scala:295) at akka.remote.transport.netty.NettyTransport. (NettyTransport.scala:251) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:526) at akka.actor.ReflectiveDynamicAccess$$anonfun$createInstanceFor$2.apply(DynamicAccess.scala:78) at scala.util.Try$.apply(Try.scala:191) at akka.actor.ReflectiveDynamicAccess.createInstanceFor(DynamicAccess.scala:73) at akka.actor.ReflectiveDynamicAccess$$anonfun$createInstanceFor$3.apply(DynamicAccess.scala:84) at akka.actor.ReflectiveDynamicAccess$$anonfun$createInstanceFor$3.apply(DynamicAccess.scala:84) at scala.util.Success.flatMap(Try.scala:230) at akka.actor.ReflectiveDynamicAccess.createInstanceFor(DynamicAccess.scala:84) at akka.remote.EndpointManager$$anonfun$9.apply(Remoting.scala:749) at akka.remote.EndpointManager$$anonfun$9.apply(Remoting.scala:741) at scala.collection.TraversableLike$WithFilter$$anonfun$map$2.apply(TraversableLike.scala:728) at scala.collection.Iterator$class.foreach(Iterator.scala:750) at scala.collection.AbstractIterator.foreach(Iterator.scala:1202) at scala.collection.IterableLike$class.foreach(IterableLike.scala:72) at scala.collection.AbstractIterable.foreach(Iterable.scala:54) at scala.collection.TraversableLike$WithFilter.map(TraversableLike.scala:727) at akka.remote.EndpointManager.akka$remote$EndpointManager$$listens(Remoting.scala:741) at akka.remote.EndpointManager$$anonfun$receive$2.applyOrElse(Remoting.scala:500) at akka.actor.Actor$class.aroundReceive(Actor.scala:467) at akka.remote.EndpointManager.aroundReceive(Remoting.scala:404) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:516) at akka.actor.ActorCell.invoke(ActorCell.scala:487) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:238) at akka.dispatch.Mailbox.run(Mailbox.scala:220) at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:397) at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) LogType: jobmanager.log LogLength: 26594 Log Contents: 2018-06-21 19:36:16,604 WARN org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2018-06-21 19:36:16,774 INFO org.apache.flink.yarn.YarnApplicationMasterRunner - -------------------------------------------------------------------------------- 2018-06-21 19:36:16,774 INFO org.apache.flink.yarn.YarnApplicationMasterRunner - Starting YARN ApplicationMaster / ResourceManager / JobManager (Version: 1.3.2, Rev:660f167, Date:19.12.2017 @ 11:58:34 CST) 2018-06-21 19:36:16,774 INFO org.apache.flink.yarn.YarnApplicationMasterRunner - Current user: qytt 2018-06-21 19:36:16,774 INFO org.apache.flink.yarn.YarnApplicationMasterRunner - JVM: Java HotSpot(TM) 64-Bit Server VM - Oracle Corporation - 1.7/24.65-b04 2018-06-21 19:36:16,774 INFO org.apache.flink.yarn.YarnApplicationMasterRunner - Maximum heap size: 2731 MiBytes 2018-06-21 19:36:16,774 INFO org.apache.flink.yarn.YarnApplicationMasterRunner - JAVA_HOME: /usr/java/default 2018-06-21 19:36:16,775 INFO org.apache.flink.yarn.YarnApplicationMasterRunner - Hadoop version: 2.5.0-cdh5.2.0 2018-06-21 19:36:16,776 INFO org.apache.flink.yarn.YarnApplicationMasterRunner - JVM Options: 2018-06-21 19:36:16,776 INFO org.apache.flink.yarn.YarnApplicationMasterRunner - -Xmx3072m 2018-06-21 19:36:16,776 INFO org.apache.flink.yarn.YarnApplicationMasterRunner - -Dlog.file=/data1/yarn/logs/application_1529082330695_659780/container_1529082330695_659780_01_000005/jobmanager.log 2018-06-21 19:36:16,776 INFO org.apache.flink.yarn.YarnApplicationMasterRunner - -Dlogback.configurationFile=file:logback.xml 2018-06-21 19:36:16,776 INFO org.apache.flink.yarn.YarnApplicationMasterRunner - -Dlog4j.configuration=file:log4j.properties 2018-06-21 19:36:16,776 INFO org.apache.flink.yarn.YarnApplicationMasterRunner - Program Arguments: (none) 2018-06-21 19:36:16,776 INFO org.apache.flink.yarn.YarnApplicationMasterRunner - Classpath: flink-PingbackJinyingLog-jar-with-dependencies.jar:lib/flink-dist_2.11-1.3.2.jar:lib/flink-python_2.11-1.3.2.jar:lib/flink-shaded-hadoop2-uber-1.3.2.jar:lib/log4j-1.2.17.jar:lib/slf4j-log4j12-1.7.7.jar:log4j.properties:logback.xml:flink.jar:flink-conf.yaml::/etc/hadoop/conf:/usr/lib/hadoop/hadoop-common-2.5.0-cdh5.2.0.jar:/usr/lib/hadoop/parquet-column.jar:/usr/lib/hadoop/hadoop-common-2.5.0-cdh5.2.0-tests.jar:/usr/lib/hadoop/hadoop-nfs.jar:/usr/lib/hadoop/parquet-jackson.jar:/usr/lib/hadoop/parquet-format.jar:/usr/lib/hadoop/parquet-common.jar:/usr/lib/hadoop/hadoop-nfs-2.5.0-cdh5.2.0.jar:/usr/lib/hadoop/hadoop-auth-2.5.0-cdh5.2.0.jar:/usr/lib/hadoop/hadoop-auth.jar:/usr/lib/hadoop/parquet-thrift.jar:/usr/lib/hadoop/parquet-generator.jar:/usr/lib/hadoop/parquet-protobuf.jar:/usr/lib/hadoop/parquet-avro.jar:/usr/lib/hadoop/parquet-scrooge.jar:/usr/lib/hadoop/parquet-hadoop.jar:/usr/lib/hadoop/parquet-cascading.jar:/usr/lib/hadoop/hadoop-common.jar:/usr/lib/hadoop/parquet-encoding.jar:/usr/lib/hadoop/hadoop-annotations-2.5.0-cdh5.2.0.jar:/usr/lib/hadoop/parquet-pig.jar:/usr/lib/hadoop/parquet-tools.jar:/usr/lib/hadoop/parquet-hadoop-bundle.jar:/usr/lib/hadoop/hadoop-annotations.jar:/usr/lib/hadoop/parquet-pig-bundle.jar:/usr/lib/hadoop/lib/slf4j-api-1.7.5.jar:/usr/lib/hadoop/lib/api-asn1-api-1.0.0-M20.jar:/usr/lib/hadoop/lib/avro.jar:/usr/lib/hadoop/lib/jetty-util-6.1.26.jar:/usr/lib/hadoop/lib/httpclient-4.2.5.jar:/usr/lib/hadoop/lib/asm-3.2.jar:/usr/lib/hadoop/lib/mockito-all-1.8.5.jar:/usr/lib/hadoop/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop/lib/guava-11.0.2.jar:/usr/lib/hadoop/lib/hadoop-lzo.jar:/usr/lib/hadoop/lib/commons-collections-3.2.1.jar:/usr/lib/hadoop/lib/log4j-1.2.17.jar:/usr/lib/hadoop/lib/curator-recipes-2.6.0.jar:/usr/lib/hadoop/lib/gson-2.2.4.jar:/usr/lib/hadoop/lib/jets3t-0.9.0.jar:/usr/lib/hadoop/lib/apacheds-i18n-2.0.0-M15.jar:/usr/lib/hadoop/lib/commons-el-1.0.jar:/usr/lib/hadoop/lib/jersey-core-1.9.jar:/usr/lib/hadoop/lib/curator-client-2.6.0.jar:/usr/lib/hadoop/lib/protobuf-java-2.5.0.jar:/usr/lib/hadoop/lib/commons-httpclient-3.1.jar:/usr/lib/hadoop/lib/commons-compress-1.4.1.jar:/usr/lib/hadoop/lib/jersey-json-1.9.jar:/usr/lib/hadoop/lib/jasper-compiler-5.5.23.jar:/usr/lib/hadoop/lib/commons-digester-1.8.jar:/usr/lib/hadoop/lib/jsr305-1.3.9.jar:/usr/lib/hadoop/lib/slf4j-log4j12.jar:/usr/lib/hadoop/lib/java-xmlbuilder-0.4.jar:/usr/lib/hadoop/lib/netty-3.6.2.Final.jar:/usr/lib/hadoop/lib/commons-io-2.4.jar:/usr/lib/hadoop/lib/jackson-core-asl-1.8.8.jar:/usr/lib/hadoop/lib/jsch-0.1.42.jar:/usr/lib/hadoop/lib/curator-framework-2.6.0.jar:/usr/lib/hadoop/lib/jettison-1.1.jar:/usr/lib/hadoop/lib/jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop/lib/jersey-server-1.9.jar:/usr/lib/hadoop/lib/commons-beanutils-core-1.8.0.jar:/usr/lib/hadoop/lib/stax-api-1.0-2.jar:/usr/lib/hadoop/lib/xz-1.0.jar:/usr/lib/hadoop/lib/jsp-api-2.1.jar:/usr/lib/hadoop/lib/zookeeper.jar:/usr/lib/hadoop/lib/api-util-1.0.0-M20.jar:/usr/lib/hadoop/lib/snappy-java-1.0.4.1.jar:/usr/lib/hadoop/lib/hadoop-lzo-0.4.15-cdh5.2.0.jar:/usr/lib/hadoop/lib/jasper-runtime-5.5.23.jar:/usr/lib/hadoop/lib/servlet-api-2.5.jar:/usr/lib/hadoop/lib/commons-lang-2.6.jar:/usr/lib/hadoop/lib/paranamer-2.3.jar:/usr/lib/hadoop/lib/commons-configuration-1.6.jar:/usr/lib/hadoop/lib/jackson-jaxrs-1.8.8.jar:/usr/lib/hadoop/lib/commons-cli-1.2.jar:/usr/lib/hadoop/lib/jackson-xc-1.8.8.jar:/usr/lib/hadoop/lib/commons-net-3.1.jar:/usr/lib/hadoop/lib/jaxb-api-2.2.2.jar:/usr/lib/hadoop/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/usr/lib/hadoop/lib/activation-1.1.jar:/usr/lib/hadoop/lib/commons-codec-1.4.jar:/usr/lib/hadoop/lib/commons-logging-1.1.3.jar:/usr/lib/hadoop/lib/commons-beanutils-1.7.0.jar:/usr/lib/hadoop/lib/commons-math3-3.1.1.jar:/usr/lib/hadoop/lib/httpcore-4.2.5.jar:/usr/lib/hadoop/lib/hamcrest-core-1.3.jar:/usr/lib/hadoop/lib/junit-4.11.jar:/usr/lib/hadoop/lib/jetty-6.1.26.jar:/usr/lib/hadoop/lib/xmlenc-0.52.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-nfs-2.5.0-cdh5.2.0.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-2.5.0-cdh5.2.0-tests.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-nfs.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-2.5.0-cdh5.2.0.jar:/usr/lib/hadoop-hdfs/lib/jetty-util-6.1.26.jar:/usr/lib/hadoop-hdfs/lib/asm-3.2.jar:/usr/lib/hadoop-hdfs/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop-hdfs/lib/guava-11.0.2.jar:/usr/lib/hadoop-hdfs/lib/log4j-1.2.17.jar:/usr/lib/hadoop-hdfs/lib/commons-el-1.0.jar:/usr/lib/hadoop-hdfs/lib/jersey-core-1.9.jar:/usr/lib/hadoop-hdfs/lib/protobuf-java-2.5.0.jar:/usr/lib/hadoop-hdfs/lib/jsr305-1.3.9.jar:/usr/lib/hadoop-hdfs/lib/netty-3.6.2.Final.jar:/usr/lib/hadoop-hdfs/lib/commons-io-2.4.jar:/usr/lib/hadoop-hdfs/lib/jackson-core-asl-1.8.8.jar:/usr/lib/hadoop-hdfs/lib/jersey-server-1.9.jar:/usr/lib/hadoop-hdfs/lib/jsp-api-2.1.jar:/usr/lib/hadoop-hdfs/lib/commons-daemon-1.0.13.jar:/usr/lib/hadoop-hdfs/lib/jasper-runtime-5.5.23.jar:/usr/lib/hadoop-hdfs/lib/servlet-api-2.5.jar:/usr/lib/hadoop-hdfs/lib/commons-lang-2.6.jar:/usr/lib/hadoop-hdfs/lib/commons-cli-1.2.jar:/usr/lib/hadoop-hdfs/lib/commons-codec-1.4.jar:/usr/lib/hadoop-hdfs/lib/commons-logging-1.1.3.jar:/usr/lib/hadoop-hdfs/lib/jetty-6.1.26.jar:/usr/lib/hadoop-hdfs/lib/xmlenc-0.52.jar:/usr/lib/hadoop-mapreduce/hadoop-rumen-2.5.0-cdh5.2.0.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-core-2.5.0-cdh5.2.0.jar:/usr/lib/hadoop-mapreduce/api-asn1-api-1.0.0-M20.jar:/usr/lib/hadoop-mapreduce/avro.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-app-2.5.0-cdh5.2.0.jar:/usr/lib/hadoop-mapreduce/jetty-util-6.1.26.jar:/usr/lib/hadoop-mapreduce/httpclient-4.2.5.jar:/usr/lib/hadoop-mapreduce/asm-3.2.jar:/usr/lib/hadoop-mapreduce/hadoop-datajoin-2.5.0-cdh5.2.0.jar:/usr/lib/hadoop-mapreduce/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop-mapreduce/guava-11.0.2.jar:/usr/lib/hadoop-mapreduce/commons-collections-3.2.1.jar:/usr/lib/hadoop-mapreduce/hadoop-sls.jar:/usr/lib/hadoop-mapreduce/log4j-1.2.17.jar:/usr/lib/hadoop-mapreduce/curator-recipes-2.6.0.jar:/usr/lib/hadoop-mapreduce/gson-2.2.4.jar:/usr/lib/hadoop-mapreduce/jets3t-0.9.0.jar:/usr/lib/hadoop-mapreduce/apacheds-i18n-2.0.0-M15.jar:/usr/lib/hadoop-mapreduce/commons-el-1.0.jar:/usr/lib/hadoop-mapreduce/jersey-core-1.9.jar:/usr/lib/hadoop-mapreduce/curator-client-2.6.0.jar:/usr/lib/hadoop-mapreduce/protobuf-java-2.5.0.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar:/usr/lib/hadoop-mapreduce/commons-httpclient-3.1.jar:/usr/lib/hadoop-mapreduce/commons-compress-1.4.1.jar:/usr/lib/hadoop-mapreduce/jersey-json-1.9.jar:/usr/lib/hadoop-mapreduce/jasper-compiler-5.5.23.jar:/usr/lib/hadoop-mapreduce/commons-digester-1.8.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-hs-2.5.0-cdh5.2.0.jar:/usr/lib/hadoop-mapreduce/jsr305-1.3.9.jar:/usr/lib/hadoop-mapreduce/hadoop-gridmix-2.5.0-cdh5.2.0.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient.jar:/usr/lib/hadoop-mapreduce/java-xmlbuilder-0.4.jar:/usr/lib/hadoop-mapreduce/hadoop-auth-2.5.0-cdh5.2.0.jar:/usr/lib/hadoop-mapreduce/hadoop-rumen.jar:/usr/lib/hadoop-mapreduce/hadoop-distcp.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-app.jar:/usr/lib/hadoop-mapreduce/commons-io-2.4.jar:/usr/lib/hadoop-mapreduce/hadoop-auth.jar:/usr/lib/hadoop-mapreduce/jackson-core-asl-1.8.8.jar:/usr/lib/hadoop-mapreduce/jsch-0.1.42.jar:/usr/lib/hadoop-mapreduce/hadoop-distcp-2.5.0-cdh5.2.0.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-2.5.0-cdh5.2.0.jar:/usr/lib/hadoop-mapreduce/curator-framework-2.6.0.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-hs.jar:/usr/lib/hadoop-mapreduce/jettison-1.1.jar:/usr/lib/hadoop-mapreduce/jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop-mapreduce/jersey-server-1.9.jar:/usr/lib/hadoop-mapreduce/commons-beanutils-core-1.8.0.jar:/usr/lib/hadoop-mapreduce/hadoop-extras-2.5.0-cdh5.2.0.jar:/usr/lib/hadoop-mapreduce/hadoop-sls-2.5.0-cdh5.2.0.jar:/usr/lib/hadoop-mapreduce/stax-api-1.0-2.jar:/usr/lib/hadoop-mapreduce/xz-1.0.jar:/usr/lib/hadoop-mapreduce/jsp-api-2.1.jar:/usr/lib/hadoop-mapreduce/metrics-core-3.0.1.jar:/usr/lib/hadoop-mapreduce/zookeeper.jar:/usr/lib/hadoop-mapreduce/api-util-1.0.0-M20.jar:/usr/lib/hadoop-mapreduce/snappy-java-1.0.4.1.jar:/usr/lib/hadoop-mapreduce/jasper-runtime-5.5.23.jar:/usr/lib/hadoop-mapreduce/servlet-api-2.5.jar:/usr/lib/hadoop-mapreduce/commons-lang-2.6.jar:/usr/lib/hadoop-mapreduce/paranamer-2.3.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-2.5.0-cdh5.2.0-tests.jar:/usr/lib/hadoop-mapreduce/commons-configuration-1.6.jar:/usr/lib/hadoop-mapreduce/jackson-jaxrs-1.8.8.jar:/usr/lib/hadoop-mapreduce/commons-cli-1.2.jar:/usr/lib/hadoop-mapreduce/hadoop-archives-2.5.0-cdh5.2.0.jar:/usr/lib/hadoop-mapreduce/jackson-xc-1.8.8.jar:/usr/lib/hadoop-mapreduce/commons-net-3.1.jar:/usr/lib/hadoop-mapreduce/hadoop-datajoin.jar:/usr/lib/hadoop-mapreduce/jaxb-api-2.2.2.jar:/usr/lib/hadoop-mapreduce/apacheds-kerberos-codec-2.0.0-M15.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-nativetask.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-hs-plugins.jar:/usr/lib/hadoop-mapreduce/activation-1.1.jar:/usr/lib/hadoop-mapreduce/commons-codec-1.4.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-shuffle.jar:/usr/lib/hadoop-mapreduce/commons-logging-1.1.3.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-core.jar:/usr/lib/hadoop-mapreduce/commons-beanutils-1.7.0.jar:/usr/lib/hadoop-mapreduce/commons-math3-3.1.1.jar:/usr/lib/hadoop-mapreduce/httpcore-4.2.5.jar:/usr/lib/hadoop-mapreduce/hadoop-streaming-2.5.0-cdh5.2.0.jar:/usr/lib/hadoop-mapreduce/hadoop-streaming.jar:/usr/lib/hadoop-mapreduce/hadoop-extras.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples-2.5.0-cdh5.2.0.jar:/usr/lib/hadoop-mapreduce/hamcrest-core-1.3.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-shuffle-2.5.0-cdh5.2.0.jar:/usr/lib/hadoop-mapreduce/junit-4.11.jar:/usr/lib/hadoop-mapreduce/jetty-6.1.26.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-hs-plugins-2.5.0-cdh5.2.0.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-nativetask-2.5.0-cdh5.2.0.jar:/usr/lib/hadoop-mapreduce/hadoop-archives.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-common-2.5.0-cdh5.2.0.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-common.jar:/usr/lib/hadoop-mapreduce/hadoop-gridmix.jar:/usr/lib/hadoop-mapreduce/xmlenc-0.52.jar:/usr/lib/hadoop-mapreduce/lib/aopalliance-1.0.jar:/usr/lib/hadoop-mapreduce/lib/guice-3.0.jar:/usr/lib/hadoop-mapreduce/lib/avro.jar:/usr/lib/hadoop-mapreduce/lib/asm-3.2.jar:/usr/lib/hadoop-mapreduce/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop-mapreduce/lib/log4j-1.2.17.jar:/usr/lib/hadoop-mapreduce/lib/jersey-core-1.9.jar:/usr/lib/hadoop-mapreduce/lib/protobuf-java-2.5.0.jar:/usr/lib/hadoop-mapreduce/lib/guice-servlet-3.0.jar:/usr/lib/hadoop-mapreduce/lib/commons-compress-1.4.1.jar:/usr/lib/hadoop-mapreduce/lib/netty-3.6.2.Final.jar:/usr/lib/hadoop-mapreduce/lib/commons-io-2.4.jar:/usr/lib/hadoop-mapreduce/lib/jackson-core-asl-1.8.8.jar:/usr/lib/hadoop-mapreduce/lib/jersey-server-1.9.jar:/usr/lib/hadoop-mapreduce/lib/xz-1.0.jar:/usr/lib/hadoop-mapreduce/lib/snappy-java-1.0.4.1.jar:/usr/lib/hadoop-mapreduce/lib/paranamer-2.3.jar:/usr/lib/hadoop-mapreduce/lib/javax.inject-1.jar:/usr/lib/hadoop-mapreduce/lib/hamcrest-core-1.3.jar:/usr/lib/hadoop-mapreduce/lib/junit-4.11.jar:/usr/lib/hadoop-mapreduce/lib/leveldbjni-all-1.8.jar:/usr/lib/hadoop-mapreduce/lib/jersey-guice-1.9.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-nodemanager-2.5.0-cdh5.2.0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-common-2.5.0-cdh5.2.0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-applicationhistoryservice.jar:/usr/lib/hadoop-yarn/hadoop-yarn-applications-unmanaged-am-launcher.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-tests.jar:/usr/lib/hadoop-yarn/hadoop-yarn-common.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-resourcemanager.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-web-proxy.jar:/usr/lib/hadoop-yarn/hadoop-yarn-api-2.5.0-cdh5.2.0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-common.jar:/usr/lib/hadoop-yarn/hadoop-yarn-api.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-web-proxy-2.5.0-cdh5.2.0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-applications-distributedshell.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-tests-2.5.0-cdh5.2.0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-applications-distributedshell-2.5.0-cdh5.2.0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.5.0-cdh5.2.0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-client.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-resourcemanager-2.5.0-cdh5.2.0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-client-2.5.0-cdh5.2.0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-applicationhistoryservice-2.5.0-cdh5.2.0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-nodemanager.jar:/usr/lib/hadoop-yarn/hadoop-yarn-common-2.5.0-cdh5.2.0.jar:/usr/lib/hadoop-yarn/lib/aopalliance-1.0.jar:/usr/lib/hadoop-yarn/lib/guice-3.0.jar:/usr/lib/hadoop-yarn/lib/jetty-util-6.1.26.jar:/usr/lib/hadoop-yarn/lib/asm-3.2.jar:/usr/lib/hadoop-yarn/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop-yarn/lib/guava-11.0.2.jar:/usr/lib/hadoop-yarn/lib/commons-collections-3.2.1.jar:/usr/lib/hadoop-yarn/lib/log4j-1.2.17.jar:/usr/lib/hadoop-yarn/lib/jersey-core-1.9.jar:/usr/lib/hadoop-yarn/lib/protobuf-java-2.5.0.jar:/usr/lib/hadoop-yarn/lib/commons-httpclient-3.1.jar:/usr/lib/hadoop-yarn/lib/guice-servlet-3.0.jar:/usr/lib/hadoop-yarn/lib/commons-compress-1.4.1.jar:/usr/lib/hadoop-yarn/lib/jersey-json-1.9.jar:/usr/lib/hadoop-yarn/lib/jsr305-1.3.9.jar:/usr/lib/hadoop-yarn/lib/commons-io-2.4.jar:/usr/lib/hadoop-yarn/lib/jersey-client-1.9.jar:/usr/lib/hadoop-yarn/lib/jackson-core-asl-1.8.8.jar:/usr/lib/hadoop-yarn/lib/jettison-1.1.jar:/usr/lib/hadoop-yarn/lib/jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop-yarn/lib/jersey-server-1.9.jar:/usr/lib/hadoop-yarn/lib/stax-api-1.0-2.jar:/usr/lib/hadoop-yarn/lib/xz-1.0.jar:/usr/lib/hadoop-yarn/lib/zookeeper.jar:/usr/lib/hadoop-yarn/lib/servlet-api-2.5.jar:/usr/lib/hadoop-yarn/lib/commons-lang-2.6.jar:/usr/lib/hadoop-yarn/lib/jackson-jaxrs-1.8.8.jar:/usr/lib/hadoop-yarn/lib/commons-cli-1.2.jar:/usr/lib/hadoop-yarn/lib/jackson-xc-1.8.8.jar:/usr/lib/hadoop-yarn/lib/jline-0.9.94.jar:/usr/lib/hadoop-yarn/lib/jaxb-api-2.2.2.jar:/usr/lib/hadoop-yarn/lib/activation-1.1.jar:/usr/lib/hadoop-yarn/lib/commons-codec-1.4.jar:/usr/lib/hadoop-yarn/lib/commons-logging-1.1.3.jar:/usr/lib/hadoop-yarn/lib/javax.inject-1.jar:/usr/lib/hadoop-yarn/lib/jetty-6.1.26.jar:/usr/lib/hadoop-yarn/lib/leveldbjni-all-1.8.jar:/usr/lib/hadoop-yarn/lib/jersey-guice-1.9.jar:/usr/lib/hbase/hbase-server-0.98.6-cdh5.2.0.jar:/usr/lib/hbase/hbase-hadoop2-compat-0.98.6-cdh5.2.0-tests.jar:/usr/lib/hbase/hbase-server-0.98.6-cdh5.2.0-tests.jar:/usr/lib/hbase/hbase-hadoop-compat-0.98.6-cdh5.2.0.jar:/usr/lib/hbase/hbase-it-0.98.6-cdh5.2.0-tests.jar:/usr/lib/hbase/hbase-server.jar:/usr/lib/hbase/hbase-hadoop2-compat-tests.jar:/usr/lib/hbase/hbase-testing-util.jar:/usr/lib/hbase/hbase-common.jar:/usr/lib/hbase/hbase-it-0.98.6-cdh5.2.0.jar:/usr/lib/hbase/hbase-testing-util-0.98.6-cdh5.2.0.jar:/usr/lib/hbase/hbase-hadoop-compat-tests.jar:/usr/lib/hbase/hbase-shell-0.98.6-cdh5.2.0.jar:/usr/lib/hbase/hbase-thrift-0.98.6-cdh5.2.0.jar:/usr/lib/hbase/hbase-examples-0.98.6-cdh5.2.0.jar:/usr/lib/hbase/hbase-prefix-tree.jar:/usr/lib/hbase/hbase-hadoop-compat-0.98.6-cdh5.2.0-tests.jar:/usr/lib/hbase/hbase-examples.jar:/usr/lib/hbase/hbase-protocol.jar:/usr/lib/hbase/hbase-server-tests.jar:/usr/lib/hbase/hbase-common-tests.jar:/usr/lib/hbase/hbase-prefix-tree-0.98.6-cdh5.2.0.jar:/usr/lib/hbase/hbase-hadoop2-compat-0.98.6-cdh5.2.0.jar:/usr/lib/hbase/hbase-shell.jar:/usr/lib/hbase/hbase-common-0.98.6-cdh5.2.0.jar:/usr/lib/hbase/hbase-it.jar:/usr/lib/hbase/hbase-client.jar:/usr/lib/hbase/hbase-common-0.98.6-cdh5.2.0-tests.jar:/usr/lib/hbase/hbase-it-tests.jar:/usr/lib/hbase/hbase-protocol-0.98.6-cdh5.2.0.jar:/usr/lib/hbase/hbase-hadoop2-compat.jar:/usr/lib/hbase/hbase-thrift.jar:/usr/lib/hbase/hbase-client-0.98.6-cdh5.2.0.jar:/usr/lib/hbase/hbase-hadoop-compat.jar:/usr/lib/hbase/lib/hsqldb-1.8.0.10.jar:/usr/lib/hbase/lib/slf4j-api-1.7.5.jar:/usr/lib/hbase/lib/commons-codec-1.7.jar:/usr/lib/hbase/lib/hbase-server-0.98.6-cdh5.2.0.jar:/usr/lib/hbase/lib/jersey-server-1.8.jar:/usr/lib/hbase/lib/api-asn1-api-1.0.0-M20.jar:/usr/lib/hbase/lib/avro.jar:/usr/lib/hbase/lib/hbase-hadoop2-compat-0.98.6-cdh5.2.0-tests.jar:/usr/lib/hbase/lib/httpclient-4.2.5.jar:/usr/lib/hbase/lib/guava-12.0.1.jar:/usr/lib/hbase/lib/asm-3.2.jar:/usr/lib/hbase/lib/jsp-api-2.1-6.1.14.jar:/usr/lib/hbase/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hbase/lib/hbase-server-0.98.6-cdh5.2.0-tests.jar:/usr/lib/hbase/lib/netty-3.6.6.Final.jar:/usr/lib/hbase/lib/core-3.1.1.jar:/usr/lib/hbase/lib/commons-collections-3.2.1.jar:/usr/lib/hbase/lib/hbase-hadoop-compat-0.98.6-cdh5.2.0.jar:/usr/lib/hbase/lib/hbase-it-0.98.6-cdh5.2.0-tests.jar:/usr/lib/hbase/lib/log4j-1.2.17.jar:/usr/lib/hbase/lib/curator-recipes-2.6.0.jar:/usr/lib/hbase/lib/gson-2.2.4.jar:/usr/lib/hbase/lib/jets3t-0.9.0.jar:/usr/lib/hbase/lib/htrace-core.jar:/usr/lib/hbase/lib/apacheds-i18n-2.0.0-M15.jar:/usr/lib/hbase/lib/commons-el-1.0.jar:/usr/lib/hbase/lib/curator-client-2.6.0.jar:/usr/lib/hbase/lib/protobuf-java-2.5.0.jar:/usr/lib/hbase/lib/commons-httpclient-3.1.jar:/usr/lib/hbase/lib/jsp-2.1-6.1.14.jar:/usr/lib/hbase/lib/metrics-core-2.2.0.jar:/usr/lib/hbase/lib/htrace-core-2.04.jar:/usr/lib/hbase/lib/jettison-1.3.1.jar:/usr/lib/hbase/lib/commons-compress-1.4.1.jar:/usr/lib/hbase/lib/high-scale-lib-1.1.1.jar:/usr/lib/hbase/lib/jasper-compiler-5.5.23.jar:/usr/lib/hbase/lib/jersey-core-1.8.jar:/usr/lib/hbase/lib/hbase-it-0.98.6-cdh5.2.0.jar:/usr/lib/hbase/lib/commons-digester-1.8.jar:/usr/lib/hbase/lib/jsr305-1.3.9.jar:/usr/lib/hbase/lib/commons-daemon-1.0.3.jar:/usr/lib/hbase/lib/slf4j-log4j12.jar:/usr/lib/hbase/lib/commons-math-2.1.jar:/usr/lib/hbase/lib/jruby-complete-1.6.8.jar:/usr/lib/hbase/lib/java-xmlbuilder-0.4.jar:/usr/lib/hbase/lib/commons-io-2.4.jar:/usr/lib/hbase/lib/jackson-core-asl-1.8.8.jar:/usr/lib/hbase/lib/jsch-0.1.42.jar:/usr/lib/hbase/lib/hbase-testing-util-0.98.6-cdh5.2.0.jar:/usr/lib/hbase/lib/curator-framework-2.6.0.jar:/usr/lib/hbase/lib/jaxb-api-2.1.jar:/usr/lib/hbase/lib/jamon-runtime-2.3.1.jar:/usr/lib/hbase/lib/hbase-shell-0.98.6-cdh5.2.0.jar:/usr/lib/hbase/lib/jaxb-impl-2.2.3-1.jar:/usr/lib/hbase/lib/commons-beanutils-core-1.8.0.jar:/usr/lib/hbase/lib/hbase-thrift-0.98.6-cdh5.2.0.jar:/usr/lib/hbase/lib/xz-1.0.jar:/usr/lib/hbase/lib/jsp-api-2.1.jar:/usr/lib/hbase/lib/findbugs-annotations-1.3.9-1.jar:/usr/lib/hbase/lib/hbase-examples-0.98.6-cdh5.2.0.jar:/usr/lib/hbase/lib/jersey-json-1.8.jar:/usr/lib/hbase/lib/hbase-hadoop-compat-0.98.6-cdh5.2.0-tests.jar:/usr/lib/hbase/lib/zookeeper.jar:/usr/lib/hbase/lib/api-util-1.0.0-M20.jar:/usr/lib/hbase/lib/snappy-java-1.0.4.1.jar:/usr/lib/hbase/lib/jasper-runtime-5.5.23.jar:/usr/lib/hbase/lib/servlet-api-2.5.jar:/usr/lib/hbase/lib/commons-lang-2.6.jar:/usr/lib/hbase/lib/paranamer-2.3.jar:/usr/lib/hbase/lib/commons-configuration-1.6.jar:/usr/lib/hbase/lib/jackson-jaxrs-1.8.8.jar:/usr/lib/hbase/lib/commons-cli-1.2.jar:/usr/lib/hbase/lib/jackson-xc-1.8.8.jar:/usr/lib/hbase/lib/commons-net-3.1.jar:/usr/lib/hbase/lib/jetty-sslengine-6.1.26.cloudera.2.jar:/usr/lib/hbase/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/usr/lib/hbase/lib/libthrift-0.9.0.jar:/usr/lib/hbase/lib/activation-1.1.jar:/usr/lib/hbase/lib/hbase-prefix-tree-0.98.6-cdh5.2.0.jar:/usr/lib/hbase/lib/hbase-hadoop2-compat-0.98.6-cdh5.2.0.jar:/usr/lib/hbase/lib/hbase-common-0.98.6-cdh5.2.0.jar:/usr/lib/hbase/lib/jetty-6.1.26.cloudera.2.jar:/usr/lib/hbase/lib/commons-beanutils-1.7.0.jar:/usr/lib/hbase/lib/commons-math3-3.1.1.jar:/usr/lib/hbase/lib/httpcore-4.2.5.jar:/usr/lib/hbase/lib/commons-logging-1.1.1.jar:/usr/lib/hbase/lib/hamcrest-core-1.3.jar:/usr/lib/hbase/lib/junit-4.11.jar:/usr/lib/hbase/lib/servlet-api-2.5-6.1.14.jar:/usr/lib/hbase/lib/hbase-common-0.98.6-cdh5.2.0-tests.jar:/usr/lib/hbase/lib/hbase-protocol-0.98.6-cdh5.2.0.jar:/usr/lib/hbase/lib/jetty-util-6.1.26.cloudera.2.jar:/usr/lib/hbase/lib/hbase-client-0.98.6-cdh5.2.0.jar:/usr/lib/hbase/lib/xmlenc-0.52.jar 2018-06-21 19:36:16,777 INFO org.apache.flink.yarn.YarnApplicationMasterRunner - -------------------------------------------------------------------------------- 2018-06-21 19:36:16,778 INFO org.apache.flink.yarn.YarnApplicationMasterRunner - Registered UNIX signal handlers for [TERM, HUP, INT] 2018-06-21 19:36:16,784 INFO org.apache.flink.yarn.YarnApplicationMasterRunner - remoteKeytabPrincipal obtained qytt 2018-06-21 19:36:16,784 INFO org.apache.flink.yarn.YarnApplicationMasterRunner - YARN daemon is running as: qytt Yarn client user obtainer: qytt@HADOOP.abc.COM 2018-06-21 19:36:16,790 INFO org.apache.flink.yarn.YarnApplicationMasterRunner - Loading config from directory /data6/yarn/local/usercache/qytt/appcache/application_1529082330695_659780/container_1529082330695_659780_01_000005 2018-06-21 19:36:16,794 INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: jobmanager.rpc.address, localhost 2018-06-21 19:36:16,794 INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: jobmanager.rpc.port, 6123 2018-06-21 19:36:16,794 INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: jobmanager.heap.mb, 1024 2018-06-21 19:36:16,794 INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: taskmanager.heap.mb, 1024 2018-06-21 19:36:16,794 INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: taskmanager.numberOfTaskSlots, 1 2018-06-21 19:36:16,794 INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: taskmanager.memory.preallocate, false 2018-06-21 19:36:16,794 INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: parallelism.default, 1 2018-06-21 19:36:16,795 INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: jobmanager.web.port, 8081 2018-06-21 19:36:16,796 INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: security.kerberos.login.use-ticket-cache, false 2018-06-21 19:36:16,796 INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: security.kerberos.login.keytab, /europa/local/kerberos/108394/qytt.keytab 2018-06-21 19:36:16,796 INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: security.kerberos.login.principal, qytt 2018-06-21 19:36:17,082 INFO org.apache.hadoop.security.UserGroupInformation - Login successful for user qytt using keytab file /data6/yarn/local/usercache/qytt/appcache/application_1529082330695_659780/container_1529082330695_659780_01_000005/krb5.keytab 2018-06-21 19:36:17,087 INFO org.apache.flink.runtime.security.modules.HadoopModule - Hadoop user set to qytt@HADOOP.abc.COM (auth:KERBEROS) 2018-06-21 19:36:17,126 INFO org.apache.flink.yarn.YarnApplicationMasterRunner - YARN assigned hostname for application master: node153-154-134-jylt.abc.hadoop 2018-06-21 19:36:17,127 INFO org.apache.flink.yarn.YarnApplicationMasterRunner - keytabPath: /data6/yarn/local/usercache/qytt/appcache/application_1529082330695_659780/container_1529082330695_659780_01_000005/krb5.keytab 2018-06-21 19:36:17,137 INFO org.apache.flink.yarn.YarnApplicationMasterRunner - TaskManagers will be created with 2 task slots 2018-06-21 19:36:17,137 INFO org.apache.flink.yarn.YarnApplicationMasterRunner - TaskManagers will be started with container size 4096 MB, JVM heap size 3072 MB, JVM direct memory limit -1 MB 2018-06-21 19:36:17,141 INFO org.apache.flink.yarn.YarnApplicationMasterRunner - Trying to start actor system at node153-154-134-jylt.abc.hadoop:49255 2018-06-21 19:36:17,704 INFO akka.event.slf4j.Slf4jLogger - Slf4jLogger started 2018-06-21 19:36:17,750 INFO Remoting - Starting remoting LogType: jobmanager.out LogLength: 0 Log Contents:
这里,我们欣喜的发现,终于可以看到一些有意义的信息了。在google上搜索java.lang.NoSuchMethodError: org.jboss.netty.channel.socket.nio.NioClientSocketChannelFactory
可以了解到应该是jar包冲突。我们打开eclipse,在pom.xml中搜索netty,可以看到是由于cb中的netty和flink中的冲突,exclude掉即可。