贊同來自:

哦，我把yarn.nodemanager.pmem-check-enabled設(shè)置為true后，再次運行任務(wù)就報錯了。

您說的使用量超過limit 兩倍時，才啟動kill。應(yīng)該指得是虛擬內(nèi)存的使用超出虛擬內(nèi)存的兩倍，因為當物理內(nèi)存不夠用時，任務(wù)就會使用虛擬內(nèi)內(nèi)存，而一旦虛擬內(nèi)存超過設(shè)置的兩倍時，isOverLimit?就是true了，這樣就會觸發(fā)kill?container的動作了。

下面是我測試的報錯信息：

16/03/15 18:51:58 INFO mapreduce.Job: Task Id : attempt_1458038021988_0002_m_000000_0, Status : FAILED

Container [pid=9781,containerID=container_1458038021988_0002_01_000002] is running beyond physical memory limits. Current usage: 1.1 GB of 1 GB physical memory used; 4.9 GB of 2 GB virtual memory used. Killing container.

Dump of the process-tree for container_1458038021988_0002_01_000002 :

|- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE

|- 9781 9779 9781 9781 (bash) 0 0 108646400 301 /bin/bash -c /opt/beh-6.5/core/jdk/bin/java -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN? -Xmx4g -Djava.io.tmpdir=/opt/beh-6.5/metadata/yarn/usercache/user1/appcache/application_1458038021988_0002/container_1458038021988_0002_01_000002/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/opt/beh-6.5/logs/yarn/userlogs/application_1458038021988_0002/container_1458038021988_0002_01_000002 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA org.apache.hadoop.mapred.YarnChild 192.168.6.153 59177 attempt_1458038021988_0002_m_000000_0 2 1>/opt/beh-6.5/logs/yarn/userlogs/application_1458038021988_0002/container_1458038021988_0002_01_000002/stdout 2>/opt/beh-6.5/logs/yarn/userlogs/application_1458038021988_0002/container_1458038021988_0002_01_000002/stderr

|- 9785 9781 9781 9781 (java) 570 45 5128753152 286023 /opt/beh-6.5/core/jdk/bin/java -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Xmx4g -Djava.io.tmpdir=/opt/beh-6.5/metadata/yarn/usercache/user1/appcache/application_1458038021988_0002/container_1458038021988_0002_01_000002/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/opt/beh-6.5/logs/yarn/userlogs/application_1458038021988_0002/container_1458038021988_0002_01_000002 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA org.apache.hadoop.mapred.YarnChild 192.168.6.153 59177 attempt_1458038021988_0002_m_000000_0 2

Container killed on request. Exit code is 143

Container exited with a non-zero exit code 143

16/03/15 18:51:58 INFO mapreduce.Job: Task Id : attempt_1458038021988_0002_m_000000_1, Status : FAILED

Container [pid=9914,containerID=container_1458038021988_0002_01_000003] is running beyond physical memory limits. Current usage: 1.6 GB of 1 GB physical memory used; 4.9 GB of 2 GB virtual memory used. Killing container.

Dump of the process-tree for container_1458038021988_0002_01_000003 :

|- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE

|- 9914 9912 9914 9914 (bash) 0 0 108646400 302 /bin/bash -c /opt/beh-6.5/core/jdk/bin/java -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN? -Xmx4g -Djava.io.tmpdir=/opt/beh-6.5/metadata/yarn/usercache/user1/appcache/application_1458038021988_0002/container_1458038021988_0002_01_000003/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/opt/beh-6.5/logs/yarn/userlogs/application_1458038021988_0002/container_1458038021988_0002_01_000003 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA org.apache.hadoop.mapred.YarnChild 192.168.6.153 59177 attempt_1458038021988_0002_m_000000_1 3 1>/opt/beh-6.5/logs/yarn/userlogs/application_1458038021988_0002/container_1458038021988_0002_01_000003/stdout 2>/opt/beh-6.5/logs/yarn/userlogs/application_1458038021988_0002/container_1458038021988_0002_01_000003/stderr

|- 9918 9914 9914 9914 (java) 884 65 5106462720 416443 /opt/beh-6.5/core/jdk/bin/java -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Xmx4g -Djava.io.tmpdir=/opt/beh-6.5/metadata/yarn/usercache/user1/appcache/application_1458038021988_0002/container_1458038021988_0002_01_000003/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/opt/beh-6.5/logs/yarn/userlogs/application_1458038021988_0002/container_1458038021988_0002_01_000003 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA org.apache.hadoop.mapred.YarnChild 192.168.6.153 59177 attempt_1458038021988_0002_m_000000_1 3

Container killed on request. Exit code is 143

Container exited with a non-zero exit code 143

16/03/15 18:51:58 INFO mapreduce.Job: Task Id : attempt_1458038021988_0002_m_000000_2, Status : FAILED

Container [pid=10150,containerID=container_1458038021988_0002_01_000004] is running beyond physical memory limits. Current usage: 1.6 GB of 1 GB physical memory used; 4.9 GB of 2 GB virtual memory used. Killing container.

Dump of the process-tree for container_1458038021988_0002_01_000004 :

|- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE

|- 10150 10148 10150 10150 (bash) 0 0 108646400 302 /bin/bash -c /opt/beh-6.5/core/jdk/bin/java -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN? -Xmx4g -Djava.io.tmpdir=/opt/beh-6.5/metadata/yarn/usercache/user1/appcache/application_1458038021988_0002/container_1458038021988_0002_01_000004/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/opt/beh-6.5/logs/yarn/userlogs/application_1458038021988_0002/container_1458038021988_0002_01_000004 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA org.apache.hadoop.mapred.YarnChild 192.168.6.153 59177 attempt_1458038021988_0002_m_000000_2 4 1>/opt/beh-6.5/logs/yarn/userlogs/application_1458038021988_0002/container_1458038021988_0002_01_000004/stdout 2>/opt/beh-6.5/logs/yarn/userlogs/application_1458038021988_0002/container_1458038021988_0002_01_000004/stderr

|- 10154 10150 10150 10150 (java) 802 64 5106597888 418707 /opt/beh-6.5/core/jdk/bin/java -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Xmx4g -Djava.io.tmpdir=/opt/beh-6.5/metadata/yarn/usercache/user1/appcache/application_1458038021988_0002/container_1458038021988_0002_01_000004/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/opt/beh-6.5/logs/yarn/userlogs/application_1458038021988_0002/container_1458038021988_0002_01_000004 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA org.apache.hadoop.mapred.YarnChild 192.168.6.153 59177 attempt_1458038021988_0002_m_000000_2 4

Container killed on request. Exit code is 143

Container exited with a non-zero exit code 143

16/03/15 18:51:58 INFO mapreduce.Job: Task Id : attempt_1458038021988_0002_m_000000_3, Status : FAILED

Container [pid=4373,containerID=container_1458038021988_0002_01_000005] is running beyond physical memory limits. Current usage: 1.6 GB of 1 GB physical memory used; 4.9 GB of 2 GB virtual memory used. Killing container.

Dump of the process-tree for container_1458038021988_0002_01_000005 :

|- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE

|- 4377 4373 4373 4373 (java) 752 73 5141807104 419857 /opt/beh-6.5/core/jdk/bin/java -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Xmx4g -Djava.io.tmpdir=/opt/beh-6.5/metadata/yarn/usercache/user1/appcache/application_1458038021988_0002/container_1458038021988_0002_01_000005/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/opt/beh-6.5/logs/yarn/userlogs/application_1458038021988_0002/container_1458038021988_0002_01_000005 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA org.apache.hadoop.mapred.YarnChild 192.168.6.153 59177 attempt_1458038021988_0002_m_000000_3 5

|- 4373 4371 4373 4373 (bash) 0 0 108650496 302 /bin/bash -c /opt/beh-6.5/core/jdk/bin/java -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN? -Xmx4g -Djava.io.tmpdir=/opt/beh-6.5/metadata/yarn/usercache/user1/appcache/application_1458038021988_0002/container_1458038021988_0002_01_000005/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/opt/beh-6.5/logs/yarn/userlogs/application_1458038021988_0002/container_1458038021988_0002_01_000005 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA org.apache.hadoop.mapred.YarnChild 192.168.6.153 59177 attempt_1458038021988_0002_m_000000_3 5 1>/opt/beh-6.5/logs/yarn/userlogs/application_1458038021988_0002/container_1458038021988_0002_01_000005/stdout 2>/opt/beh-6.5/logs/yarn/userlogs/application_1458038021988_0002/container_1458038021988_0002_01_000005/stderr

Container killed on request. Exit code is 143

Container exited with a non-zero exit code 143

16/03/15 18:51:58 INFO mapreduce.Job: Job job_1458038021988_0002 failed with state FAILED due to:

總結(jié)

以上是生活随笔為你收集整理的hadoop job 未跑满资源_mapreduce任务占满整个集群资源的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯，歡迎將生活随笔推薦給好友。