Skip to content

Comments

Upgrade to CircleCi 2.0#5930

Merged
dlang-bot merged 3 commits intodlang:masterfrom
wilzbach:circle
Dec 17, 2017
Merged

Upgrade to CircleCi 2.0#5930
dlang-bot merged 3 commits intodlang:masterfrom
wilzbach:circle

Conversation

@wilzbach
Copy link
Contributor

Looks fairly straight-forward so far.

Note that CircleCi 2.0 allows to use custom Docker images.
So for example, we could built the DMD compiler into it and avoid the network issues during it's download.
Also allows this other customization, e.g. sth. like @ZombineDev's PIE-hardened docker images (https://github.com/ZombineDev/dmd-test-suite-docker) to run all testsuites on a PIE-hardened system.

@dlang-bot
Copy link
Contributor

Thanks for your pull request, @wilzbach!

Bugzilla references

Your PR doesn't reference any Bugzilla issue.

If your PR contains non-trivial changes, please reference a Bugzilla issue or create a manual changelog.

name: Install gdb
- run:
command: ./.circleci/run.sh style_lint
name: Run code style & linter
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently I get the following:

Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
core.exception.OutOfMemoryError@src/core/exception.d(702): Memory allocation failed
----------------
[Inferior 1 (process 2163) exited with code 01]
No stack.

See also: https://circleci.com/gh/wilzbach/phobos/13
Sadly gdb doesn't forward the exit code, it's apparently a well-known bug.

- checkout
- run:
command: cat /proc/meminfo
name: OS information
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you also try cat /sys/fs/cgroup/memory/memory.limit_in_bytes?

@wilzbach
Copy link
Contributor Author

@ZombineDev asked me to print /proc/meminfo, here it's for posterity:

> cat /proc/meminfo
MemTotal:       61831480 kB
MemFree:        10226440 kB
MemAvailable:   58237560 kB
Buffers:          148940 kB
Cached:         45595956 kB
SwapCached:            0 kB
Active:         23696912 kB
Inactive:       24558780 kB
Active(anon):    2538692 kB
Inactive(anon):    18920 kB
Active(file):   21158220 kB
Inactive(file): 24539860 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:             0 kB
SwapFree:              0 kB
Dirty:            345124 kB
Writeback:           280 kB
AnonPages:       2518916 kB
Mapped:           437112 kB
Shmem:             45160 kB
Slab:            2995992 kB
SReclaimable:    2794880 kB
SUnreclaim:       201112 kB
KernelStack:       41648 kB
PageTables:        27448 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    30915740 kB
Committed_AS:   14163388 kB
VmallocTotal:   34359738367 kB
VmallocUsed:           0 kB
VmallocChunk:          0 kB
HardwareCorrupted:     0 kB
AnonHugePages:    479232 kB
CmaTotal:              0 kB
CmaFree:               0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:     6479872 kB
DirectMap2M:    56434688 kB

@wilzbach
Copy link
Contributor Author

Can you also try cat /sys/fs/cgroup/memory/memory.limit_in_bytes?

> cat /sys/fs/cgroup/memory/memory.limit_in_bytes
9223372036854771712

@wilzbach
Copy link
Contributor Author

I logged in via ssh and added ----DRT-gcopt=profile:1:

core.exception.OutOfMemoryError@src/core/exception.d(702): Memory allocation failed
----------------
	Number of collections:  10
	Total GC prep time:  0 milliseconds
	Total mark time:  15 milliseconds
	Total sweep time:  0 milliseconds
	Total page recovery time:  0 milliseconds
	Max Pause Time:  4 milliseconds
	Grand total GC time:  16 milliseconds
GC summary:   65 MB,   10 GC   16 ms, Pauses   15 ms <    4 ms


@echo "Check that Ddoc runs without errors"
$(DMD) $(DFLAGS) -defaultlib= -debuglib= $(LIB) -w -D -Df/dev/null -main -c -o- $$(find etc std -type f -name '*.d') 2>&1 | grep -v "Deprecation:"; test $$? -eq 1
$(DMD) $(DFLAGS) -defaultlib= -debuglib= $(LIB) -w -D -Df/dev/null -main -c -o- $$(find etc std -type f -name '*.d') 2>&1
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DMD built in debug mode always prints: DMD v2.077.1 DEBUG on startup, hence the fragile grep failed.

@wilzbach
Copy link
Contributor Author

So I now get the same stack traces as on CircleCi 1.0:

Program received signal SIGSEGV, Segmentation fault.
0x0000000000a8d3ec in gc.impl.conservative() (this=..., ptop=0x5dd16f0, pbot=0x5dd16b0) at src/gc/impl/conservative/gc.d:1990
1990	            auto p = *p1;
#0  0x0000000000a8d3ec in gc.impl.conservative() (this=..., ptop=0x5dd16f0, pbot=0x5dd16b0) at src/gc/impl/conservative/gc.d:1990
#1  0x0000000000a8dcf1 in gc.impl.conservative() (this=0x7fffffff9cf0, __applyArg0=...) at src/gc/impl/conservative/gc.d:2188
#2  0x0000000000a978bd in rt.util.container.treap() (this=0x7fffffff9cb0, e=...) at src/rt/util/container/treap.d:47
#3  0x0000000000a97c95 in rt.util.container.treap() (dg=..., node=0x2879820) at src/rt/util/container/treap.d:221
#4  0x0000000000a97c72 in rt.util.container.treap() (dg=..., node=0x2951f40) at src/rt/util/container/treap.d:218
#5  0x0000000000a97cc1 in rt.util.container.treap() (dg=..., node=0x5c37e30) at src/rt/util/container/treap.d:224
#6  0x0000000000a97c72 in rt.util.container.treap() (dg=..., node=0x29503a0) at src/rt/util/container/treap.d:218
#7  0x0000000000a97cc1 in rt.util.container.treap() (dg=..., node=0x5c37da0) at src/rt/util/container/treap.d:224
#8  0x0000000000a97cc1 in rt.util.container.treap() (dg=..., node=0x5c37ae0) at src/rt/util/container/treap.d:224
#9  0x0000000000a97cc1 in rt.util.container.treap() (dg=..., node=0x5906650) at src/rt/util/container/treap.d:224
#10 0x0000000000a97cc1 in rt.util.container.treap() (dg=..., node=0x53fa8b0) at src/rt/util/container/treap.d:224
#11 0x0000000000a97cc1 in rt.util.container.treap() (dg=..., node=0x53c7da0) at src/rt/util/container/treap.d:224
#12 0x0000000000a97cc1 in rt.util.container.treap() (dg=..., node=0x53c7c70) at src/rt/util/container/treap.d:224
#13 0x0000000000a97cc1 in rt.util.container.treap() (dg=..., node=0x524bcb0) at src/rt/util/container/treap.d:224
#14 0x0000000000a97cc1 in rt.util.container.treap() (dg=..., node=0x5018220) at src/rt/util/container/treap.d:224
#15 0x0000000000a97cc1 in rt.util.container.treap() (dg=..., node=0x500e5e0) at src/rt/util/container/treap.d:224
#16 0x0000000000a97cc1 in rt.util.container.treap() (dg=..., node=0x25180f0) at src/rt/util/container/treap.d:224
#17 0x0000000000a978f3 in rt.util.container.treap() (this=..., dg=...) at src/rt/util/container/treap.d:52
#18 0x0000000000a9788f in rt.util.container.treap() (this=..., dg=...) at src/rt/util/container/treap.d:47
#19 0x0000000000a8dc70 in gc.impl.conservative() (this=..., nostack=false) at src/gc/impl/conservative/gc.d:2185
#20 0x0000000000a8e736 in gc.impl.conservative() (this=..., nostack=false) at src/gc/impl/conservative/gc.d:2417
#21 0x0000000000a8c879 in gc.impl.conservative() (this=..., bits=8, alloc_size=@0x7fffffff9f88: 256, bin=4 '\004') at src/gc/impl/conservative/gc.d:1711
#22 0x0000000000a8c70a in gc.impl.conservative() (this=..., bits=8, alloc_size=@0x7fffffff9f88: 256, size=129) at src/gc/impl/conservative/gc.d:1676
#23 0x0000000000a8a67d in gc.impl.conservative() (this=0xe9d0d0, ti=0xe442f0 <TypeInfo_AAxC6dparse3ast9Attribute.__init()>, alloc_size=@0x7fffffff9f88: 256, bits=8, size=129) at src/gc/impl/conservative/gc.d:517
#24 0x0000000000a9077e in gc.impl.conservative() (this=0xe9d0d0, _param_3=@0x7fffffff9fa8: 0xe442f0 <TypeInfo_AAxC6dparse3ast9Attribute.__init()>, _param_2=@0x7fffffff9f88: 256, _param_1=@0x7fffffff9fb0: 8, _param_0=@0x7fffffff9fb8: 129) at src/gc/impl/conservative/gc.d:390
#25 0x0000000000a8a71f in gc.impl.conservative() (this=0xe9d0d0, __HID11=0x7fffffff9fd8, ti=0xe442f0 <TypeInfo_AAxC6dparse3ast9Attribute.__init()>, bits=8, size=129) at src/gc/impl/conservative/gc.d:543
#26 0x0000000000a4e617 in gc_qalloc (__HID9=0x7fffffffa028, sz=129, ba=8, ti=0xe442f0 <TypeInfo_AAxC6dparse3ast9Attribute.__init()>) at src/gc/proxy.d:122
#27 0x0000000000a4ddeb in core.memory.GC.qalloc() (__HID2=0x7fffffffa0b0, ti=0xe442f0 <TypeInfo_AAxC6dparse3ast9Attribute.__init()>, ba=8, sz=129) at src/core/memory.d:406
#28 0x0000000000a9324f in rt.lifetime.__arrayAlloc() (__HID23=0x7fffffffa1a0, tinext=0xe1d120 <TypeInfo_AxC6dparse3ast9Attribute.__init()>, ti=0xe442f0 <TypeInfo_AAxC6dparse3ast9Attribute.__init()>, info=..., arrsize=128) at src/rt/lifetime.d:445

@wilzbach
Copy link
Contributor Author

@ZombineDev or others: how about moving forward with this?
Reasons:

  • We need to upgrade to CircleCi 2.0 anyhow at some point
  • As an additional benefit the job run time is a lot faster (6 mins vs 10 mins)
  • CircleCi 2.0 allow a lot more options like e.g. using our own Docker image.
  • This will unbreak our CircleCi failures by removing the fragile DDoc grep (DScanner is still failing and throwing its stack)

build:
working_directory: ~/phobos
docker:
- image: circleci/node:4.8.2
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking forward to replacing this with our custom image ;)

@MartinNowak
Copy link
Member

MartinNowak commented Dec 18, 2017

Interesting, indeed we should use our own build images to leverage caching.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants