Skip to content

Conversation

@spadgett
Copy link
Member

@spadgett spadgett commented Feb 7, 2019

The console build pods are getting out of memory killed. This is happening on almost every PR.

could not wait for build: the build console failed after 6m44s with reason OutOfMemoryKilled: The build pod was killed due to an out of memory condition.

Done in 271.62s.
yarn run v1.7.0
$ rm -rf ./public/dist && NODE_ENV=production ts-node -O '...:"commonjs"}' ./node_modules/.bin/webpack --mode=production
Starting type checking service...
Using 1 worker with 2048MB memory limit

/cc @stevekuznetsov @petr-muller

Let me know if you guys have concerns with this change or if there is a better way to fix it.

@openshift-ci-robot openshift-ci-robot added the size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. label Feb 7, 2019
@spadgett
Copy link
Member Author

spadgett commented Feb 7, 2019

@jcaianirh

Copy link
Member

@jcaianirh jcaianirh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

The console build pods are getting out of memory killed.
@spadgett spadgett force-pushed the console-memory-request branch from 8bc9bc9 to b88d052 Compare February 7, 2019 18:25
@smarterclayton smarterclayton added lgtm Indicates that a PR is ready to be merged. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Feb 7, 2019
@openshift-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

Approval requirements bypassed by manually added approval.

This pull-request has been approved by: jcaianirh, spadgett

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-merge-robot openshift-merge-robot merged commit 337f7bd into openshift:master Feb 7, 2019
@openshift-ci-robot
Copy link
Contributor

@spadgett: Updated the following 2 configmaps:

  • ci-operator-configs configmap in namespace ci using the following files:
    • key openshift-console-master.yaml using file ci-operator/config/openshift/console/openshift-console-master.yaml
  • ci-operator-configs configmap in namespace ci-stg using the following files:
    • key openshift-console-master.yaml using file ci-operator/config/openshift/console/openshift-console-master.yaml
Details

In response to this:

The console build pods are getting out of memory killed. This is happening on almost every PR.

could not wait for build: the build console failed after 6m44s with reason OutOfMemoryKilled: The build pod was killed due to an out of memory condition.

Done in 271.62s.
yarn run v1.7.0
$ rm -rf ./public/dist && NODE_ENV=production ts-node -O '...:"commonjs"}' ./node_modules/.bin/webpack --mode=production
Starting type checking service...
Using 1 worker with 2048MB memory limit

/cc @stevekuznetsov @petr-muller

Let me know if you guys have concerns with this change or if there is a better way to fix it.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@spadgett spadgett deleted the console-memory-request branch February 7, 2019 18:44
@smarterclayton
Copy link
Contributor

After this change I see you using 4.1GB. So it looks like you just slightly hit the limit incrementally.

@jcaianirh
Copy link
Member

@smarterclayton are you suggesting we increment using a smaller value?

@spadgett
Copy link
Member Author

spadgett commented Feb 8, 2019

Yeah, it's much better, although I still see occasional errors in console-e2e tests:

https://openshift-gce-devel.appspot.com/build/origin-ci-test/pr-logs/pull/openshift_console/1173/pull-ci-openshift-console-master-console-e2e/382

ERROR in ./node_modules/lodash/lodash.js
Module build failed: Error: ENOMEM: not enough memory, read
 @ ./node_modules/patternfly-react/dist/esm/components/DualListSelector/DualListControlled.js 14:0-33 111:14-21
 @ ./node_modules/patternfly-react/dist/esm/components/DualListSelector/index.js
 @ ./node_modules/patternfly-react/dist/esm/index.js
 @ ./public/components/error.tsx
 @ ./public/components/app.jsx
 @ multi ./polyfills.js ./public/components/app.jsx

@smarterclayton
Copy link
Contributor

smarterclayton commented Feb 9, 2019 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants