org.apache.hadoop.mapreduce.counters.LimitExceededException: Too many counters: 121 max=120(org.apache.hadoop.mapreduce.counters.LimitExceededException:计数器太多:121 max=120)
问题描述
我正在运行一个有少量计数器和多输出的 hadoop 作业(来自 oozie).
I'm running a hadoop job ( from oozie ) that has few counters, and multioutput.
我收到如下错误:org.apache.hadoop.mapreduce.counters.LimitExceededException:计数器太多:121 max=120
I get error like: org.apache.hadoop.mapreduce.counters.LimitExceededException: Too many counters: 121 max=120
然后我删除了所有具有计数器的代码,并将 mout.setCountersEnabled 设置为 false.并且还在 hadoop 配置中将最大计数器设置为 240.
Then I removed all the code that has counters, and also set mout.setCountersEnabled to false. And also set the max counters to 240 in hadoop config.
现在我仍然遇到同样的错误org.apache.hadoop.mapreduce.counters.LimitExceededException:计数器太多:241 max=240
Now I still get the same error org.apache.hadoop.mapreduce.counters.LimitExceededException: Too many counters: 241 max=240
我该如何解决这个问题?是否有任何隐藏计数器存在的可能性?我怎样才能弄清楚之前有哪些计数器超过 240 ?(在我可以打印任何东西之前,这个过程似乎已经停止了?)
How can I solve this problem? Is there any possibility that any hidden counters exists? How can I make clear what counters there before exceeds 240 ? (The process looks like stopped before I can print anything? )
谢谢,新松
推荐答案
我用下面的方法解决了这个问题:vi $HADOOP_HOME/conf/mapred-site.xml
I solved the problem use the following method: vi $HADOOP_HOME/conf/mapred-site.xml
<property>
<name>mapreduce.job.counters.limit</name>
<!--<value>120</value>-->
<value>20000</value>
<description>Limit on the number of counters allowed per job. The default value is 200.</description>
</property>
这篇关于org.apache.hadoop.mapreduce.counters.LimitExceededException:计数器太多:121 max=120的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持编程学习网!
本文标题为:org.apache.hadoop.mapreduce.counters.LimitExceededException:计数器太多:121 max=120
基础教程推荐
- “未找到匹配项"使用 matcher 的 group 方法时 2022-01-01
- Java Keytool 导入证书后出错,"keytool error: java.io.FileNotFoundException &拒绝访问" 2022-01-01
- 减少 JVM 暂停时间 >1 秒使用 UseConcMarkSweepGC 2022-01-01
- 设置 bean 时出现 Nullpointerexception 2022-01-01
- 如何使用 Java 创建 X509 证书? 2022-01-01
- FirebaseListAdapter 不推送聊天应用程序的单个项目 - Firebase-Ui 3.1 2022-01-01
- 在 Libgdx 中处理屏幕的正确方法 2022-01-01
- 降序排序:Java Map 2022-01-01
- 无法使用修饰符“public final"访问 java.util.Ha 2022-01-01
- Java:带有char数组的println给出乱码 2022-01-01