阿里云 blink 相关错误码问题解决

1、使用阿里云的flink平台是，有时重启任务会有shell cmd的报错信息，错误代码如下：Submit job for the [21] times,last error occurred at [Sun Jan 10 18:36:31 CST 2021],lastErrorMessage is as follows:run instance failed, BLINK error: baye

wppwpp1

1924人浏览 · 2021-01-13 11:42:23

wppwpp1 · 2021-01-13 11:42:23 发布

1、使用阿里云的flink平台是，有时重启任务会有shell cmd的报错信息，错误代码如下：

Submit job for the [21] times,last error occurred at [Sun Jan 10 18:36:31 CST 2021],lastErrorMessage is as follows:

run instance failed, BLINK error: bayes submit job failed.job name:[test_kafka], brs errorCode:[30013], brs errorMessage:

code:[30013], brief info:[error occur while run app], context info:[details:
&gt;&gt;&gt;&gt;&gt;
shell cmd execute failed

2021-01-10 18:36:26,104 INFO  org.apache.flink.yarn.cli.FlinkYarnSessionCli                 - No path for the flink jar passed. Using the location of class org.apache.flink.yarn.YarnClusterDescriptor to locate the jar
2021-01-10 18:36:26,104 INFO  org.apache.flink.yarn.cli.FlinkYarnSessionCli                 - No path for the flink jar passed. Using the location of class org.apache.flink.yarn.YarnClusterDescriptor to locate the jar
2021-01-10 18:36:26,105 INFO  org.apache.flink.yarn.cli.FlinkYarnSessionCli                 - org.apache.commons.cli.CommandLine@7fd50002
2021-01-10 18:36:26,105 INFO  org.apache.flink.yarn.cli.FlinkYarnSessionCli                 - org.apache.commons.cli.CommandLine@7fd50002
org.apache.commons.cli.CommandLine@7fd50002

------------------------------------------------------------
 The program finished with the following exception:

org.apache.flink.client.program.ProgramInvocationException: The main method caused an error.
	at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:610)
	at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:469)
	at org.apache.flink.client.program.OptimizerPlanEnvironment.getOptimizedPlan(OptimizerPlanEnvironment.java:94)
	at org.apache.flink.client.program.PackagedProgramUtils.createJobGraph(PackagedProgramUtils.java:81)
	at org.apache.flink.client.program.PackagedProgramUtils.createJobGraph(PackagedProgramUtils.java:134)
	at org.apache.flink.client.cli.CliFrontend.runProgram(CliFrontend.java:253)
	at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:217)
	at org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1075)
	at org.apache.flink.client.cli.CliFrontend.lambda$main$9(CliFrontend.java:1151)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1682)
	at org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
	at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1151)
Caused by: com.alibaba.blink.launcher.autoconfig.UnexpectedConfigurationException: 
************
ERR_ID:
     CLI-00000001
CAUSE:
     AutoConf: an error occurred while applying the resource configuration file.
     Node or edge number in stream graph and json file not matched, please generate a new json.
ACTION:
     The reason may be that the content of the SQL has been modified or the blink version number has been changed.
     Please regenerate the resource configuration file.
DETAIL:

************
	at com.alibaba.blink.launcher.autoconfig.StreamGraphConfigurer.checkValid(StreamGraphConfigurer.java:58)
	at com.alibaba.blink.launcher.autoconfig.StreamGraphConfigurer.configure(StreamGraphConfigurer.java:45)
	at com.alibaba.blink.launcher.JobLauncher.execute(JobLauncher.java:663)
	at com.alibaba.blink.launcher.JobLauncher.runStream(JobLauncher.java:545)
	at com.alibaba.blink.launcher.JobLauncher.main(JobLauncher.java:159)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:593)
	... 13 more
&lt;&lt;&lt;&lt;&lt;], cause:shell cmd execute failed

2021-01-10 18:36:26,104 INFO  org.apache.flink.yarn.cli.FlinkYarnSessionCli                 - No path for the flink jar passed. Using the location of class org.apache.flink.yarn.YarnClusterDescriptor to locate the jar
2021-01-10 18:36:26,104 INFO  org.apache.flink.yarn.cli.FlinkYarnSessionCli                 - No path for the flink jar passed. Using the location of class org.apache.flink.yarn.YarnClusterDescriptor to locate the jar
2021-01-10 18:36:26,105 INFO  org.apache.flink.yarn.cli.FlinkYarnSessionCli                 - org.apache.commons.cli.CommandLine@7fd50002
2021-01-10 18:36:26,105 INFO  org.apache.flink.yarn.cli.FlinkYarnSessionCli                 - org.apache.commons.cli.CommandLine@7fd50002
org.apache.commons.cli.CommandLine@7fd50002

------------------------------------------------------------
 The program finished with the following exception:

org.apache.flink.client.program.ProgramInvocationException: The main method caused an error.
	at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:610)
	at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:469)
	at org.apache.flink.client.program.OptimizerPlanEnvironment.getOptimizedPlan(OptimizerPlanEnvironment.java:94)
	at org.apache.flink.client.program.PackagedProgramUtils.createJobGraph(PackagedProgramUtils.java:81)
	at org.apache.flink.client.program.PackagedProgramUtils.createJobGraph(PackagedProgramUtils.java:134)
	at org.apache.flink.client.cli.CliFrontend.runProgram(CliFrontend.java:253)
	at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:217)
	at org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1075)
	at org.apache.flink.client.cli.CliFrontend.lambda$main$9(CliFrontend.java:1151)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1682)
	at org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
	at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1151)
Caused by: com.alibaba.blink.launcher.autoconfig.UnexpectedConfigurationException: 
************
ERR_ID:
     CLI-00000001
CAUSE:
     AutoConf: an error occurred while applying the resource configuration file.
     Node or edge number in stream graph and json file not matched, please generate a new json.
ACTION:
     The reason may be that the content of the SQL has been modified or the blink version number has been changed.
     Please regenerate the resource configuration file.
DETAIL:

************
	at com.alibaba.blink.launcher.autoconfig.StreamGraphConfigurer.checkValid(StreamGraphConfigurer.java:58)
	at com.alibaba.blink.launcher.autoconfig.StreamGraphConfigurer.configure(StreamGraphConfigurer.java:45)
	at com.alibaba.blink.launcher.JobLauncher.execute(JobLauncher.java:663)
	at com.alibaba.blink.launcher.JobLauncher.runStream(JobLauncher.java:545)
	at com.alibaba.blink.launcher.JobLauncher.main(JobLauncher.java:159)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:593)
	... 13 more

2、是因为改了执行计划，没有重新获取执行计划导致的，需要重新执行 “ 放弃当前配置” ==》“重新获取配置信息”==》“应用当前配置” 来解决这个问题

3、在执行监控的过程中，也发现某个流任务的停止和启动，也会影响下游流任务的延迟。

Kafka开源项目指南

Kafka开源项目指南提供详尽教程，助开发者掌握其架构、配置和使用，实现高效数据流管理和实时处理。它高性能、可扩展，适合日志收集和实时数据处理，通过持久化保障数据安全，是企业大数据生态系统的核心。

更多推荐

Kafka入门（一）概述、部署与API的简单使用

Kafka概述、部署与API的简单使用

Kafka开源项目指南

基于canal和kafka同步，实现binlog同步ElasticSearch

文章目录前言elasticsearch 安装canal安装canal-adapter 安装及配置mysql 安装zk及kafaka安装查看效果注意事项前言中间件版本elasticsearch7.5.2canal1.1.4client-adapter1.1.5-alpha-1zookeeper3.4.13kafka2.6.0mysql5.7.31elasticsearch 安装{"settings"

Kafka开源项目指南

基于 Iceberg 的湖仓一体架构在 B 站的实践

背景在B站，每天都有PB级的数据注入到大数据平台，经过离线或实时的ETL建模后，提供给下游的分析、推荐及预测等场景使用。面对如此大规模的数据，如何高效低成本地满足下游数据的分析需求，一直是我们重点的工作方向。我们之前的数据处理流程基本上是这样的：采集端将客户端埋点、服务端埋点、日志、业务数据库等数据收集到HDFS、Kafka等存储系统中，然后通过Hive、Spark、Fl...