This repository was archived by the owner on May 13, 2019. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 0
GPU error
Daniel Topping edited this page Mar 27, 2019
·
2 revisions
2019-03-27 00:24:45.782891: W tensorflow/core/common_runtime/bfc_allocator.cc:267] Allocator (GPU_0_bfc) ran out of memory trying to allocate 48.83MiB. Current allocation summary follows.
2019-03-27 00:24:45.787647: I tensorflow/core/common_runtime/bfc_allocator.cc:597] Bin (256): Total Chunks: 114, Chunks in use: 114. 28.5KiB allocated for chunks. 28.5KiB in use in bin. 2.1KiB client-requested in use in bin.
2019-03-27 00:24:45.794282: I tensorflow/core/common_runtime/bfc_allocator.cc:597] Bin (512): Total Chunks: 127, Chunks in use: 126. 63.5KiB allocated for chunks. 63.0KiB in use in bin. 62.9KiB client-requested in use in bin.
2019-03-27 00:24:45.799102: I tensorflow/core/common_runtime/bfc_allocator.cc:597] Bin (1024): Total Chunks: 21, Chunks in use: 21. 29.3KiB allocated for chunks. 29.3KiB in use in bin. 28.5KiB client-requested in use in bin.
2019-03-27 00:24:45.805892: I tensorflow/core/common_runtime/bfc_allocator.cc:597] Bin (2048): Total Chunks: 32, Chunks in use: 32. 96.0KiB allocated for chunks. 96.0KiB in use in bin. 96.0KiB client-requested in use in bin.
2019-03-27 00:24:45.810661: I tensorflow/core/common_runtime/bfc_allocator.cc:597] Bin (4096): Total Chunks: 1, Chunks in use: 1. 4.0KiB allocated for chunks. 4.0KiB in use in bin. 3.9KiB client-requested in use in bin.
2019-03-27 00:24:45.817567: I tensorflow/core/common_runtime/bfc_allocator.cc:597] Bin (8192): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2019-03-27 00:24:45.823669: I tensorflow/core/common_runtime/bfc_allocator.cc:597] Bin (16384): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2019-03-27 00:24:45.828308: I tensorflow/core/common_runtime/bfc_allocator.cc:597] Bin (32768): Total Chunks: 4, Chunks in use: 4. 224.0KiB allocated for chunks. 224.0KiB in use in bin. 224.0KiB client-requested in use in bin.
2019-03-27 00:24:45.834521: I tensorflow/core/common_runtime/bfc_allocator.cc:597] Bin (65536): Total Chunks: 19, Chunks in use: 19. 1.19MiB allocated for chunks. 1.19MiB in use in bin. 1.19MiB client-requested in use in bin.
2019-03-27 00:24:45.840083: I tensorflow/core/common_runtime/bfc_allocator.cc:597] Bin (131072): Total Chunks: 4, Chunks in use: 4. 640.0KiB allocated for chunks. 640.0KiB in use in bin. 640.0KiB client-requested in use in bin.
2019-03-27 00:24:45.846591: I tensorflow/core/common_runtime/bfc_allocator.cc:597] Bin (262144): Total Chunks: 13, Chunks in use: 13. 4.06MiB allocated for chunks. 4.06MiB in use in bin. 4.06MiB client-requested in use in bin.
2019-03-27 00:24:45.851537: I tensorflow/core/common_runtime/bfc_allocator.cc:597] Bin (524288): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2019-03-27 00:24:45.858665: I tensorflow/core/common_runtime/bfc_allocator.cc:597] Bin (1048576): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2019-03-27 00:24:45.865023: I tensorflow/core/common_runtime/bfc_allocator.cc:597] Bin (2097152): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2019-03-27 00:24:45.868800: I tensorflow/core/common_runtime/bfc_allocator.cc:597] Bin (4194304): Total Chunks: 2, Chunks in use: 1. 9.16MiB allocated for chunks. 4.58MiB in use in bin. 4.58MiB client-requested in use in bin.
2019-03-27 00:24:45.875380: I tensorflow/core/common_runtime/bfc_allocator.cc:597] Bin (8388608): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2019-03-27 00:24:45.880140: I tensorflow/core/common_runtime/bfc_allocator.cc:597] Bin (16777216): Total Chunks: 4, Chunks in use: 4. 97.66MiB allocated for chunks. 97.66MiB in use in bin. 97.66MiB client-requested in use in bin.
2019-03-27 00:24:45.886225: I tensorflow/core/common_runtime/bfc_allocator.cc:597] Bin (33554432): Total Chunks: 7, Chunks in use: 7. 342.14MiB allocated for chunks. 342.14MiB in use in bin. 341.97MiB client-requested in use in bin.
2019-03-27 00:24:45.891333: I tensorflow/core/common_runtime/bfc_allocator.cc:597] Bin (67108864): Total Chunks: 1, Chunks in use: 1. 95.75MiB allocated for chunks. 95.75MiB in use in bin. 48.83MiB client-requested in use in bin.
2019-03-27 00:24:45.898236: I tensorflow/core/common_runtime/bfc_allocator.cc:597] Bin (134217728): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2019-03-27 00:24:45.904683: I tensorflow/core/common_runtime/bfc_allocator.cc:597] Bin (268435456): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2019-03-27 00:24:45.908771: I tensorflow/core/common_runtime/bfc_allocator.cc:613] Bin for 48.83MiB was 32.00MiB, Chunk State:
2019-03-27 00:24:45.911075: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000500D80000 of size 1280
2019-03-27 00:24:45.915379: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000500D80500 of size 256
2019-03-27 00:24:45.917574: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000500D80600 of size 256
2019-03-27 00:24:45.920207: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000500D80700 of size 25600000
2019-03-27 00:24:45.922688: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005025EA700 of size 512
2019-03-27 00:24:45.926973: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005025EA900 of size 3584
2019-03-27 00:24:45.929498: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005025EB700 of size 2560
2019-03-27 00:24:45.932341: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005025EC100 of size 1536
2019-03-27 00:24:45.937313: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005025EC700 of size 512
2019-03-27 00:24:45.939602: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005025EC900 of size 65536
2019-03-27 00:24:45.942361: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005025FC900 of size 327680
2019-03-27 00:24:45.946363: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 000000050264C900 of size 163840
2019-03-27 00:24:45.948664: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000502674900 of size 256
2019-03-27 00:24:45.950790: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000502674A00 of size 57344
2019-03-27 00:24:45.954943: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000502682A00 of size 256
2019-03-27 00:24:45.957896: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000502682B00 of size 1024
2019-03-27 00:24:45.960177: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000502682F00 of size 256
2019-03-27 00:24:45.962306: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000502683000 of size 256
2019-03-27 00:24:45.966632: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000502683100 of size 256
2019-03-27 00:24:45.968855: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000502683200 of size 256
2019-03-27 00:24:45.971488: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000502683300 of size 256
2019-03-27 00:24:45.975879: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000502683400 of size 256
2019-03-27 00:24:45.978070: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000502683500 of size 256
2019-03-27 00:24:45.980210: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000502683600 of size 256
2019-03-27 00:24:45.982355: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000502683700 of size 256
2019-03-27 00:24:45.986665: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000502683800 of size 256
2019-03-27 00:24:45.988902: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000502683900 of size 256
2019-03-27 00:24:45.992089: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000502683A00 of size 256
2019-03-27 00:24:45.996070: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000502683B00 of size 256
2019-03-27 00:24:45.998308: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000502683C00 of size 256
2019-03-27 00:24:46.000578: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000502683D00 of size 256
2019-03-27 00:24:46.003125: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000502683E00 of size 256
2019-03-27 00:24:46.007414: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000502683F00 of size 256
2019-03-27 00:24:46.009588: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000502684000 of size 256
2019-03-27 00:24:46.011739: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000502684100 of size 256
2019-03-27 00:24:46.014203: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000502684200 of size 256
2019-03-27 00:24:46.018499: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000502684300 of size 256
2019-03-27 00:24:46.020866: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000502684400 of size 256
2019-03-27 00:24:46.023188: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000502684500 of size 256
2019-03-27 00:24:46.027759: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000502684600 of size 2560
2019-03-27 00:24:46.029929: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000502685000 of size 1536
2019-03-27 00:24:46.032093: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000502685600 of size 512
2019-03-27 00:24:46.034312: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000502685800 of size 65536
2019-03-27 00:24:46.038622: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000502695800 of size 512
2019-03-27 00:24:46.041473: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000502695A00 of size 512
2019-03-27 00:24:46.043815: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000502695C00 of size 512
2019-03-27 00:24:46.046132: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000502695E00 of size 327680
2019-03-27 00:24:46.050398: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005026E5E00 of size 2560
2019-03-27 00:24:46.052578: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005026E6800 of size 1536
2019-03-27 00:24:46.054811: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005026E6E00 of size 65536
2019-03-27 00:24:46.059004: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005026F6E00 of size 512
2019-03-27 00:24:46.061160: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005026F7000 of size 65536
2019-03-27 00:24:46.063374: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000502707000 of size 512
2019-03-27 00:24:46.065971: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000502707200 of size 512
2019-03-27 00:24:46.071322: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000502707400 of size 327680
2019-03-27 00:24:46.073815: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000502757400 of size 2560
2019-03-27 00:24:46.075948: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000502757E00 of size 1536
2019-03-27 00:24:46.079928: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000502758400 of size 65536
2019-03-27 00:24:46.082857: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000502768400 of size 512
2019-03-27 00:24:46.085007: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000502768600 of size 512
2019-03-27 00:24:46.089210: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000502768800 of size 512
2019-03-27 00:24:46.091671: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000502768A00 of size 163840
2019-03-27 00:24:46.093984: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000502790A00 of size 327680
2019-03-27 00:24:46.096148: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005027E0A00 of size 2560
2019-03-27 00:24:46.101289: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005027E1400 of size 1536
2019-03-27 00:24:46.103999: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005027E1A00 of size 65536
2019-03-27 00:24:46.106386: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005027F1A00 of size 256
2019-03-27 00:24:46.110448: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005027F1B00 of size 512
2019-03-27 00:24:46.112773: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005027F1D00 of size 512
2019-03-27 00:24:46.115276: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005027F1F00 of size 512
2019-03-27 00:24:46.119243: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005027F2100 of size 327680
2019-03-27 00:24:46.121445: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000502842100 of size 57344
2019-03-27 00:24:46.123919: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000502850100 of size 2560
2019-03-27 00:24:46.126355: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000502850B00 of size 1536
2019-03-27 00:24:46.131169: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000502851100 of size 65536
2019-03-27 00:24:46.133447: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000502861100 of size 512
2019-03-27 00:24:46.136477: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000502861300 of size 512
2019-03-27 00:24:46.140725: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000502861500 of size 256
2019-03-27 00:24:46.142877: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000502861600 of size 1024
2019-03-27 00:24:46.145039: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000502861A00 of size 25600000
2019-03-27 00:24:46.147457: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005040CBA00 of size 2560
2019-03-27 00:24:46.151582: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005040CC400 of size 2560
2019-03-27 00:24:46.153818: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005040CCE00 of size 2560
2019-03-27 00:24:46.156300: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005040CD800 of size 2560
2019-03-27 00:24:46.160955: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005040CE200 of size 2560
2019-03-27 00:24:46.163096: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005040CEC00 of size 512
2019-03-27 00:24:46.165598: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005040CEE00 of size 512
2019-03-27 00:24:46.169477: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005040CF000 of size 512
2019-03-27 00:24:46.171755: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005040CF200 of size 512
2019-03-27 00:24:46.174348: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005040CF400 of size 512
2019-03-27 00:24:46.177007: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005040CF600 of size 512
2019-03-27 00:24:46.181303: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005040CF800 of size 512
2019-03-27 00:24:46.183508: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005040CFA00 of size 512
2019-03-27 00:24:46.185671: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005040CFC00 of size 512
2019-03-27 00:24:46.187939: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005040CFE00 of size 512
2019-03-27 00:24:46.192038: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005040D0000 of size 512
2019-03-27 00:24:46.194658: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005040D0200 of size 512
2019-03-27 00:24:46.196849: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005040D0400 of size 512
2019-03-27 00:24:46.200755: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005040D0600 of size 512
2019-03-27 00:24:46.202887: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005040D0800 of size 512
2019-03-27 00:24:46.205115: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005040D0A00 of size 1536
2019-03-27 00:24:46.207536: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005040D1000 of size 1536
2019-03-27 00:24:46.212521: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005040D1600 of size 1536
2019-03-27 00:24:46.215067: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005040D1C00 of size 1536
2019-03-27 00:24:46.217470: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005040D2200 of size 1536
2019-03-27 00:24:46.221493: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005040D2800 of size 65536
2019-03-27 00:24:46.224274: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005040E2800 of size 65536
2019-03-27 00:24:46.226516: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005040F2800 of size 65536
2019-03-27 00:24:46.230519: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000504102800 of size 65536
2019-03-27 00:24:46.232876: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000504112800 of size 65536
2019-03-27 00:24:46.235056: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000504122800 of size 65536
2019-03-27 00:24:46.238370: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000504132800 of size 3584
2019-03-27 00:24:46.243036: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000504133600 of size 3584
2019-03-27 00:24:46.245224: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000504134400 of size 3584
2019-03-27 00:24:46.247705: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000504135200 of size 3584
2019-03-27 00:24:46.251627: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000504136000 of size 3584
2019-03-27 00:24:46.254309: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000504136E00 of size 327680
2019-03-27 00:24:46.256792: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000504186E00 of size 327680
2019-03-27 00:24:46.259061: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005041D6E00 of size 327680
2019-03-27 00:24:46.263561: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000504226E00 of size 327680
2019-03-27 00:24:46.266127: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000504276E00 of size 163840
2019-03-27 00:24:46.268358: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 000000050429EE00 of size 57344
2019-03-27 00:24:46.270525: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005042ACE00 of size 256
2019-03-27 00:24:46.275366: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005042ACF00 of size 1024
2019-03-27 00:24:46.277900: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005042AD300 of size 256
2019-03-27 00:24:46.280001: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005042AD400 of size 256
2019-03-27 00:24:46.284373: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005042AD500 of size 256
2019-03-27 00:24:46.286973: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005042AD600 of size 256
2019-03-27 00:24:46.289448: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005042AD700 of size 256
2019-03-27 00:24:46.294282: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005042AD800 of size 256
2019-03-27 00:24:46.296616: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005042AD900 of size 256
2019-03-27 00:24:46.298748: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005042ADA00 of size 256
2019-03-27 00:24:46.300934: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005042ADB00 of size 256
2019-03-27 00:24:46.305078: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005042ADC00 of size 256
2019-03-27 00:24:46.307840: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005042ADD00 of size 256
2019-03-27 00:24:46.310303: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005042ADE00 of size 256
2019-03-27 00:24:46.314791: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005042ADF00 of size 256
2019-03-27 00:24:46.316957: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005042AE000 of size 256
2019-03-27 00:24:46.319096: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005042AE100 of size 256
2019-03-27 00:24:46.321218: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005042AE200 of size 256
2019-03-27 00:24:46.325407: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005042AE300 of size 256
2019-03-27 00:24:46.327587: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005042AE400 of size 256
2019-03-27 00:24:46.329807: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005042AE500 of size 256
2019-03-27 00:24:46.334338: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005042AE600 of size 256
2019-03-27 00:24:46.336518: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005042AE700 of size 256
2019-03-27 00:24:46.338654: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005042AE800 of size 256
2019-03-27 00:24:46.341455: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005042AE900 of size 256
2019-03-27 00:24:46.345686: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005042AEA00 of size 256
2019-03-27 00:24:46.347844: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005042AEB00 of size 256
2019-03-27 00:24:46.349977: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005042AEC00 of size 256
2019-03-27 00:24:46.354303: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005042AED00 of size 256
2019-03-27 00:24:46.357305: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005042AEE00 of size 256
2019-03-27 00:24:46.359521: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005042AEF00 of size 256
2019-03-27 00:24:46.363772: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005042AF000 of size 256
2019-03-27 00:24:46.365991: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005042AF100 of size 256
2019-03-27 00:24:46.368123: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005042AF200 of size 256
2019-03-27 00:24:46.370296: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005042AF300 of size 256
2019-03-27 00:24:46.375104: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005042AF400 of size 256
2019-03-27 00:24:46.377282: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005042AF500 of size 256
2019-03-27 00:24:46.379513: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005042AF600 of size 256
2019-03-27 00:24:46.381668: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005042AF700 of size 256
2019-03-27 00:24:46.385993: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005042AF800 of size 25600000
2019-03-27 00:24:46.388239: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B19800 of size 512
2019-03-27 00:24:46.390777: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B19A00 of size 512
2019-03-27 00:24:46.393028: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B19C00 of size 512
2019-03-27 00:24:46.397423: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B19E00 of size 512
2019-03-27 00:24:46.399601: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B1A000 of size 512
2019-03-27 00:24:46.401744: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B1A200 of size 512
2019-03-27 00:24:46.406111: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B1A400 of size 512
2019-03-27 00:24:46.408609: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B1A600 of size 512
2019-03-27 00:24:46.411150: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B1A800 of size 512
2019-03-27 00:24:46.413310: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B1AA00 of size 512
2019-03-27 00:24:46.417484: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B1AC00 of size 512
2019-03-27 00:24:46.419691: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B1AE00 of size 512
2019-03-27 00:24:46.421913: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B1B000 of size 512
2019-03-27 00:24:46.426117: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B1B200 of size 512
2019-03-27 00:24:46.428702: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B1B400 of size 512
2019-03-27 00:24:46.430840: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B1B600 of size 512
2019-03-27 00:24:46.435708: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B1B800 of size 512
2019-03-27 00:24:46.438041: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B1BA00 of size 512
2019-03-27 00:24:46.440449: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B1BC00 of size 512
2019-03-27 00:24:46.442863: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B1BE00 of size 512
2019-03-27 00:24:46.447199: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B1C000 of size 512
2019-03-27 00:24:46.449356: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B1C200 of size 512
2019-03-27 00:24:46.451512: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B1C400 of size 512
2019-03-27 00:24:46.455950: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B1C600 of size 512
2019-03-27 00:24:46.458822: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B1C800 of size 512
2019-03-27 00:24:46.461397: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B1CA00 of size 512
2019-03-27 00:24:46.465213: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B1CC00 of size 512
2019-03-27 00:24:46.467338: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B1CE00 of size 512
2019-03-27 00:24:46.469619: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B1D000 of size 512
2019-03-27 00:24:46.472302: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B1D200 of size 512
2019-03-27 00:24:46.476815: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B1D400 of size 512
2019-03-27 00:24:46.479027: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B1D600 of size 512
2019-03-27 00:24:46.481231: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B1D800 of size 512
2019-03-27 00:24:46.483377: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B1DA00 of size 512
2019-03-27 00:24:46.487659: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B1DC00 of size 512
2019-03-27 00:24:46.489837: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B1DE00 of size 512
2019-03-27 00:24:46.492007: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B1E000 of size 512
2019-03-27 00:24:46.494226: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B1E200 of size 512
2019-03-27 00:24:46.498384: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B1E400 of size 512
2019-03-27 00:24:46.500586: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B1E600 of size 512
2019-03-27 00:24:46.502794: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B1E800 of size 512
2019-03-27 00:24:46.506903: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B1EA00 of size 512
2019-03-27 00:24:46.509601: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B1EC00 of size 512
2019-03-27 00:24:46.511799: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B1EE00 of size 512
2019-03-27 00:24:46.516145: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B1F000 of size 512
2019-03-27 00:24:46.518506: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B1F200 of size 512
2019-03-27 00:24:46.520654: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B1F400 of size 512
2019-03-27 00:24:46.522900: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B1F600 of size 512
2019-03-27 00:24:46.527577: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B1F800 of size 512
2019-03-27 00:24:46.529703: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B1FA00 of size 512
2019-03-27 00:24:46.531907: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B1FC00 of size 512
2019-03-27 00:24:46.536561: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B1FE00 of size 3584
2019-03-27 00:24:46.538834: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B20C00 of size 3584
2019-03-27 00:24:46.541277: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B21A00 of size 3584
2019-03-27 00:24:46.543514: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B22800 of size 3584
2019-03-27 00:24:46.548639: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B23600 of size 3584
2019-03-27 00:24:46.552471: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B24400 of size 512
2019-03-27 00:24:46.557250: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B24600 of size 256
2019-03-27 00:24:46.559613: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B24700 of size 256
2019-03-27 00:24:46.562156: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B24800 of size 256
2019-03-27 00:24:46.566686: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B24900 of size 256
2019-03-27 00:24:46.568984: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B24A00 of size 256
2019-03-27 00:24:46.571105: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B24B00 of size 256
2019-03-27 00:24:46.574017: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B24C00 of size 256
2019-03-27 00:24:46.578415: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B24D00 of size 256
2019-03-27 00:24:46.580620: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B24E00 of size 256
2019-03-27 00:24:46.582799: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B24F00 of size 256
2019-03-27 00:24:46.584992: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B25000 of size 256
2019-03-27 00:24:46.589184: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B25100 of size 256
2019-03-27 00:24:46.591602: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B25200 of size 256
2019-03-27 00:24:46.593767: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B25300 of size 256
2019-03-27 00:24:46.597899: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B25400 of size 256
2019-03-27 00:24:46.600037: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B25500 of size 256
2019-03-27 00:24:46.602129: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B25600 of size 256
2019-03-27 00:24:46.604285: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B25700 of size 256
2019-03-27 00:24:46.608613: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B25800 of size 256
2019-03-27 00:24:46.611488: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B25900 of size 256
2019-03-27 00:24:46.613712: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B25A00 of size 256
2019-03-27 00:24:46.617575: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B25B00 of size 256
2019-03-27 00:24:46.619721: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B25C00 of size 256
2019-03-27 00:24:46.622370: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B25D00 of size 256
2019-03-27 00:24:46.624708: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B25E00 of size 256
2019-03-27 00:24:46.628821: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B25F00 of size 256
2019-03-27 00:24:46.631065: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505B26000 of size 256
2019-03-27 00:24:46.633706: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Free at 0000000505B26100 of size 4800000
2019-03-27 00:24:46.637665: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505FB9F00 of size 512
2019-03-27 00:24:46.640014: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505FBA100 of size 4096
2019-03-27 00:24:46.642690: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505FBB100 of size 256
2019-03-27 00:24:46.644834: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505FBB200 of size 512
2019-03-27 00:24:46.648950: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505FBB400 of size 256
2019-03-27 00:24:46.651171: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505FBB500 of size 65536
2019-03-27 00:24:46.653418: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505FCB500 of size 256
2019-03-27 00:24:46.657522: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505FCB600 of size 512
2019-03-27 00:24:46.660063: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505FCB800 of size 256
2019-03-27 00:24:46.662273: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505FCB900 of size 512
2019-03-27 00:24:46.664470: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505FCBB00 of size 256
2019-03-27 00:24:46.668641: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505FCBC00 of size 256
2019-03-27 00:24:46.670807: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000505FCBD00 of size 25600000
2019-03-27 00:24:46.673664: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000507835D00 of size 512
2019-03-27 00:24:46.677975: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000507835F00 of size 512
2019-03-27 00:24:46.680133: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000507836100 of size 3584
2019-03-27 00:24:46.682347: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000507836F00 of size 512
2019-03-27 00:24:46.684500: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000507837100 of size 2560
2019-03-27 00:24:46.688564: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000507837B00 of size 512
2019-03-27 00:24:46.691113: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000507837D00 of size 1536
2019-03-27 00:24:46.693362: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000507838300 of size 512
2019-03-27 00:24:46.697743: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000507838500 of size 65536
2019-03-27 00:24:46.699940: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000507848500 of size 256
2019-03-27 00:24:46.702096: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000507848600 of size 512
2019-03-27 00:24:46.704244: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000507848800 of size 256
2019-03-27 00:24:46.709516: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000507848900 of size 512
2019-03-27 00:24:46.711666: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000507848B00 of size 512
2019-03-27 00:24:46.714002: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000507848D00 of size 256
2019-03-27 00:24:46.716227: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000507848E00 of size 512
2019-03-27 00:24:46.720174: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000507849000 of size 512
2019-03-27 00:24:46.722551: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000507849200 of size 327680
2019-03-27 00:24:46.724962: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000507899200 of size 512
2019-03-27 00:24:46.728984: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000507899400 of size 3584
2019-03-27 00:24:46.731106: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 000000050789A200 of size 512
2019-03-27 00:24:46.733248: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 000000050789A400 of size 2560
2019-03-27 00:24:46.735467: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 000000050789AE00 of size 512
2019-03-27 00:24:46.739613: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 000000050789B000 of size 65536
2019-03-27 00:24:46.742579: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005078AB000 of size 1536
2019-03-27 00:24:46.744724: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005078AB600 of size 512
2019-03-27 00:24:46.749748: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005078AB800 of size 65536
2019-03-27 00:24:46.752120: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005078BB800 of size 256
2019-03-27 00:24:46.754379: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005078BB900 of size 512
2019-03-27 00:24:46.758937: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005078BBB00 of size 256
2019-03-27 00:24:46.761069: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005078BBC00 of size 512
2019-03-27 00:24:46.763237: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005078BBE00 of size 256
2019-03-27 00:24:46.765401: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005078BBF00 of size 512
2019-03-27 00:24:46.769536: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005078BC100 of size 512
2019-03-27 00:24:46.771683: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005078BC300 of size 512
2019-03-27 00:24:46.774159: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005078BC500 of size 327680
2019-03-27 00:24:46.778108: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 000000050790C500 of size 512
2019-03-27 00:24:46.780277: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 000000050790C700 of size 3584
2019-03-27 00:24:46.782443: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 000000050790D500 of size 512
2019-03-27 00:24:46.784596: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 000000050790D700 of size 2560
2019-03-27 00:24:46.788858: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 000000050790E100 of size 1024
2019-03-27 00:24:46.791141: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 000000050790E500 of size 1536
2019-03-27 00:24:46.793284: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 000000050790EB00 of size 512
2019-03-27 00:24:46.795430: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 000000050790ED00 of size 65536
2019-03-27 00:24:46.799576: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 000000050791ED00 of size 256
2019-03-27 00:24:46.802078: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 000000050791EE00 of size 163840
2019-03-27 00:24:46.804269: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000507946E00 of size 512
2019-03-27 00:24:46.808255: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000507947000 of size 256
2019-03-27 00:24:46.810403: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000507947100 of size 512
2019-03-27 00:24:46.812577: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000507947300 of size 256
2019-03-27 00:24:46.814817: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000507947400 of size 512
2019-03-27 00:24:46.818912: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000507947600 of size 512
2019-03-27 00:24:46.821058: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000507947800 of size 327680
2019-03-27 00:24:46.823383: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000507997800 of size 512
2019-03-27 00:24:46.825513: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000507997A00 of size 3584
2019-03-27 00:24:46.829487: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000507998800 of size 512
2019-03-27 00:24:46.831622: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000507998A00 of size 256
2019-03-27 00:24:46.833798: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000507998B00 of size 2560
2019-03-27 00:24:46.835977: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000507999500 of size 512
2019-03-27 00:24:46.840423: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000507999700 of size 1536
2019-03-27 00:24:46.842614: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000507999D00 of size 512
2019-03-27 00:24:46.844749: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000507999F00 of size 65536
2019-03-27 00:24:46.849085: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005079A9F00 of size 256
2019-03-27 00:24:46.851255: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005079AA000 of size 512
2019-03-27 00:24:46.853381: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005079AA200 of size 256
2019-03-27 00:24:46.855618: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005079AA300 of size 512
2019-03-27 00:24:46.859745: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005079AA500 of size 256
2019-03-27 00:24:46.861891: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005079AA600 of size 57344
2019-03-27 00:24:46.864112: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005079B8600 of size 512
2019-03-27 00:24:46.867961: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005079B8800 of size 512
2019-03-27 00:24:46.870081: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 00000005079B8A00 of size 327680
2019-03-27 00:24:46.872393: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000507A08A00 of size 512
2019-03-27 00:24:46.874662: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000507A08C00 of size 3584
2019-03-27 00:24:46.878784: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000507A09A00 of size 512
2019-03-27 00:24:46.880920: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000507A09C00 of size 2560
2019-03-27 00:24:46.883121: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000507A0A600 of size 512
2019-03-27 00:24:46.885256: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000507A0A800 of size 1536
2019-03-27 00:24:46.889426: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000507A0AE00 of size 512
2019-03-27 00:24:46.891896: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000507A0B000 of size 256
2019-03-27 00:24:46.894444: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Free at 0000000507A0B100 of size 512
2019-03-27 00:24:46.896560: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000507A0B300 of size 4802560
2019-03-27 00:24:46.900692: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000507E9FB00 of size 51200000
2019-03-27 00:24:46.902908: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 000000050AF73B00 of size 51276800
2019-03-27 00:24:46.905148: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 000000050E05A700 of size 51276800
2019-03-27 00:24:46.907640: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 0000000511141300 of size 51302400
2019-03-27 00:24:46.911857: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 000000051422E300 of size 51302400
2019-03-27 00:24:46.914065: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 000000051731B300 of size 51200000
2019-03-27 00:24:46.916325: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 000000051A3EF300 of size 51200000
2019-03-27 00:24:46.920393: I tensorflow/core/common_runtime/bfc_allocator.cc:632] Chunk at 000000051D4C3300 of size 100401152
2019-03-27 00:24:46.922976: I tensorflow/core/common_runtime/bfc_allocator.cc:638] Summary of in-use Chunks by size:
2019-03-27 00:24:46.925856: I tensorflow/core/common_runtime/bfc_allocator.cc:641] 114 Chunks of size 256 totalling 28.5KiB
2019-03-27 00:24:46.928142: I tensorflow/core/common_runtime/bfc_allocator.cc:641] 126 Chunks of size 512 totalling 63.0KiB
2019-03-27 00:24:46.932282: I tensorflow/core/common_runtime/bfc_allocator.cc:641] 4 Chunks of size 1024 totalling 4.0KiB
2019-03-27 00:24:46.934754: I tensorflow/core/common_runtime/bfc_allocator.cc:641] 1 Chunks of size 1280 totalling 1.3KiB
2019-03-27 00:24:46.936929: I tensorflow/core/common_runtime/bfc_allocator.cc:641] 16 Chunks of size 1536 totalling 24.0KiB
2019-03-27 00:24:46.941068: I tensorflow/core/common_runtime/bfc_allocator.cc:641] 16 Chunks of size 2560 totalling 40.0KiB
2019-03-27 00:24:46.943685: I tensorflow/core/common_runtime/bfc_allocator.cc:641] 16 Chunks of size 3584 totalling 56.0KiB
2019-03-27 00:24:46.945916: I tensorflow/core/common_runtime/bfc_allocator.cc:641] 1 Chunks of size 4096 totalling 4.0KiB
2019-03-27 00:24:46.948111: I tensorflow/core/common_runtime/bfc_allocator.cc:641] 4 Chunks of size 57344 totalling 224.0KiB
2019-03-27 00:24:46.952419: I tensorflow/core/common_runtime/bfc_allocator.cc:641] 19 Chunks of size 65536 totalling 1.19MiB
2019-03-27 00:24:46.954677: I tensorflow/core/common_runtime/bfc_allocator.cc:641] 4 Chunks of size 163840 totalling 640.0KiB
2019-03-27 00:24:46.957198: I tensorflow/core/common_runtime/bfc_allocator.cc:641] 13 Chunks of size 327680 totalling 4.06MiB
2019-03-27 00:24:46.961080: I tensorflow/core/common_runtime/bfc_allocator.cc:641] 1 Chunks of size 4802560 totalling 4.58MiB
2019-03-27 00:24:46.963457: I tensorflow/core/common_runtime/bfc_allocator.cc:641] 4 Chunks of size 25600000 totalling 97.66MiB
2019-03-27 00:24:46.966004: I tensorflow/core/common_runtime/bfc_allocator.cc:641] 3 Chunks of size 51200000 totalling 146.48MiB
2019-03-27 00:24:46.970404: I tensorflow/core/common_runtime/bfc_allocator.cc:641] 2 Chunks of size 51276800 totalling 97.80MiB
2019-03-27 00:24:46.972737: I tensorflow/core/common_runtime/bfc_allocator.cc:641] 2 Chunks of size 51302400 totalling 97.85MiB
2019-03-27 00:24:46.975246: I tensorflow/core/common_runtime/bfc_allocator.cc:641] 1 Chunks of size 100401152 totalling 95.75MiB
2019-03-27 00:24:46.977965: I tensorflow/core/common_runtime/bfc_allocator.cc:645] Sum Total of in-use chunks: 546.43MiB
2019-03-27 00:24:46.982622: I tensorflow/core/common_runtime/bfc_allocator.cc:647] Stats:
Limit: 577778483
InUse: 572977920
MaxInUse: 573142272
NumAllocs: 491
MaxAllocSize: 305406464
2019-03-27 00:24:46.987591: W tensorflow/core/common_runtime/bfc_allocator.cc:271]```