Skip to content

Conversation

@davidamorimcruz
Copy link

cloudstack-usage fails to start throwing Integer exception during PID retrieval, and the service keeps restarting after 10s (as defined in the systemd service definition).

Adding the pid to the java arguments in the systemd service definition makes it stop looping in centos7

@davidamorimcruz
Copy link
Author

proof that this works in centos 7.2 running cloudstack 4.7.1:

# cat /etc/redhat-release 
CentOS Linux release 7.2.1511 (Core) 

# systemctl status cloudstack-usage
● cloudstack-usage.service - CloudStack Usage Server
   Loaded: loaded (/usr/lib/systemd/system/cloudstack-usage.service; enabled; vendor preset: disabled)
   Active: active (running) since wo 2016-02-10 15:53:12 UTC; 5min ago
     Docs: http://www.cloudstack.org/
 Main PID: 1750 (sh)
   CGroup: /system.slice/cloudstack-usage.service
           ├─1750 /bin/sh -ec      export UCP=`ls /usr/share/cloudstack-usage/cloud-usage-*.jar /usr/share/cloudstack-usage/lib/*.jar | tr "\n" ":"`;      export CLASSPATH="$UCP:/etc/cloudstack/usage:/usr/share/java/mysql-connector-java.jar";      /usr/lib/jvm/jre/bin/java -Dpid=$$ -Xms256m -Xmx2048m -cp "$CLASSPATH" $JAVA_CLASS
           └─1754 /usr/lib/jvm/jre/bin/java -Dpid=1750 -Xms256m -Xmx2048m -cp /usr/share/cloudstack-usage/cloud-usage-4.7.1.jar:/usr/share/cloudstack-usage/lib/activation-1.1.jar:/usr/share/cloudstack-usage/lib/annotations-2.0.1.jar:/usr/share/cloudstack-usage/lib/antisamy-1.4.3.jar:/usr/share/cloudstack-usage/lib/aopalliance-1.0.jar:/usr/share/cloudstack-usage/lib/apache-log4j-extras-1.2.17.jar:/usr/share/cloudstack-usage/lib/aspectjweaver-1.8.4.jar:/usr/share/cloudstack-usage/lib/aws-java-sdk-1.10.34.jar:/usr/share/cloudstack-usage/lib/aws-java-sdk-api-gateway-1.10.34.jar:/usr/share/cloudstack-usage/lib/aws-java-sdk-autoscaling-1.10.34.jar:/usr/share/cloudstack-usage/lib/aws-java-sdk-cloudformation-1.10.34.jar:/usr/share/cloudstack-usage/lib/aws-java-sdk-cloudfront-1.10.34.jar:/usr/share/cloudstack-usage/lib/aws-java-sdk-cloudhsm-1.10.34.jar:/usr/share/cloudstack-usage/lib/aws-java-sdk-cloudsearch-1.10.34.jar:/usr/share/cloudstack-usage/lib/aws-java-sdk-cloudtrail-1.10.34.jar:/usr/share/cloudstack-usage/lib/aws-java-sdk-cloudwatch-1.10.34.jar:/usr/share/cloudstack-usage/lib/aws-java-sdk-cloudwatchmetrics-1.10.34.jar:/usr/share/cloudstack-usage/lib/aws-java-sdk-codecommit-1.10.34.jar:/usr/share/cloudstack-usage/lib/aws-java-sdk-codedeploy-1.10.34.jar:/usr/share/cloudstack-usage/lib/aws-java-sdk-codepipeline-1.10.34.jar:/usr/share/cloudstack-usage/lib/aws-java-sdk-cognitoidentity-1.10.34.jar:/usr/share/cloudstack-usage/lib/aws-java-sdk-cognitosync-1.10.34.jar:/usr/share/cloudstack-usage/lib/aws-java-sdk-config-1.10.34.jar:/usr/share/cloudstack-usage/lib/aws-java-sdk-core-1.10.34.jar:/usr/share/cloudstack-usage/lib/aws-java-sdk-datapipeline-1.10.34.jar:/usr/share/cloudstack-usage/lib/aws-java-sdk-devicefarm-1.10.34.jar:/usr/share/cloudstack-usage/lib/aws-java-sdk-directconnect-1.10.34.jar:/usr/share/cloudstack-usage/lib/aws-java-sdk-directory-1.10.34.jar:/usr/share/cloudstack-usage/lib/aws-java-sdk-dynamodb-1.10.34.jar:/usr/share/cloudstack-usage/lib/aws-java-sdk-ec2-1.10.34.jar:/usr/share/cloudstack-usage/lib/aws-java-sdk-ecs-1.10.34.jar:/usr/share/cloudstack-usage/lib/aws-java-sdk-efs-1.10.34.jar:/usr/share/cloudstack-usage/lib/aws-java-sdk-elasticache-1.10.34.jar:/usr/share/cloudstack-usage/lib/aws-java-sdk-elasticbeanstalk-1.10.34.jar:/usr/share/cloudstack-usage/lib/aws-java-sdk-elasticloadbalancing-1.10.34.jar:/usr/share/cloudstack-usage/lib/aws-java-sdk-elasticsearch-1.10.34.jar:/usr/share/cloudstack-usage/lib/aws-java-sdk-elastictranscoder-1.10.34.jar:/usr/share/cloudstack-usage/lib/aws-java-sdk-emr-1.10.34.jar:/usr/share/cloudstack-usage/lib/aws-java-sdk-glacier-1.10.34.jar:/usr/share/cloudstack-usage/lib/aws-java-sdk-iam-1.10.34.jar:/usr/share/cloudstack-usage/lib/aws-java-sdk-importexport-1.10.34.jar:/usr/share/cloudstack-usage/lib/aws-java-sdk-inspector-1.10.34.jar:/usr/share/cloudstack-usage/lib/aws-java-sdk-iot-1.10.34.jar:/usr/share/cloudstack-usage/lib/aws-java-sdk-kinesis-1.10.34.jar:/usr/share/cloudstack-usage/lib/aws-java-sdk-kms-1.10.34.jar:/usr/share/cloudstack-usage/lib/aws-java-sdk-lambda-1.10.34.jar:/usr/share/cloudstack-usage/lib/aws-java-sdk-logs-1.10.34.jar:/usr/share/cloudstack-usage/lib/aws-java-sdk-machinelearning-1.10.34.jar:/usr/share/cloudstack-usage/lib/aws-java-sdk-marketplacecommerceanalytics-1.10.34.jar:/usr/share/cloudstack-usage/lib/aws-java-sdk-opsworks-1.10.34.jar:/usr/share/cloudstack-usage/lib/aws-java-sdk-rds-1.10.34.jar:/usr/share/cloudstack-usage/lib/aws-java-sdk-redshift-1.10.34.jar:/usr/share/cloudstack-usage/lib/aws-java-sdk-route53-1.10.34.jar:/usr/share/cloudstack-usage/lib/aws-java-sdk-s3-1.10.34.jar:/usr/share/cloudstack-usage/lib/aws-java-sdk-ses-1.10.34.jar:/usr/share/cloudstack-usage/lib/aws-java-sdk-simpledb-1.10.34.jar:/usr/share/cloudstack-usage/lib/aws-java-sdk-simpleworkflow-1.10.34.jar:/usr/share/cloudstack-usage/lib/aws-java-sdk-sns-1.10.34.jar:/usr/share/cloudstack-usage/lib/aws-java-sdk-sqs-1.10.34.jar:/usr/share/cloudstack-usage/lib/aws-java-sdk-ssm-1.10.34.jar:/usr/share/cloudstack-usage/lib/aws-java-sdk-storagegateway-1.10.34.jar:/usr/share/cloudstack-usage/lib/aws-java-sdk-sts-1.10.34.jar:/usr/share/cloudstack-usage/lib/aws-java-sdk-support-1.10.34.jar:/usr/share/cloudstack-usage/lib/aws-java-sdk-swf-libraries-1.10.34.jar:/usr/share/cloudstack-usage/lib/aws-java-sdk-waf-1.10.34.jar:/usr/share/cloudstack-usage/lib/aws-java-sdk-workspaces-1.10.34.jar:/usr/share/cloudstack-usage/lib/batik-css-1.7.jar:/usr/share/cloudstack-usage/lib/batik-ext-1.7.jar:/usr/share/cloudstack-usage/lib/batik-util-1.7.jar:/usr/share/cloudstack-usage/lib/bcprov-jdk15-1.46.jar:/usr/share/cloudstack-usage/lib/bcprov-jdk16-1.46.jar:/usr/share/cloudstack-usage/lib/bsh-core-2.0b4.jar:/usr/share/cloudstack-usage/lib/cglib-nodep-3.1.jar:/usr/share/cloudstack-usage/lib/cloud-api-4.7.1.jar:/usr/share/cloudstack-usage/lib/cloud-core-4.7.1.jar:/usr/share/cloudstack-usage/lib/cloud-engine-api-4.7.1.jar:/usr/share/cloudstack-usage/lib/cloud-engine-components-api-4.7.1.jar:/usr/share/cloudstack-usage/lib/cloud-engine-schema-4.7.1.jar:/usr/share/cloudstack-usage/lib/cloud-framework-cluster-4.7.1.jar:/usr/share/cloudstack-usage/lib/cloud-framework-config-4.7.1.jar:/usr/share/cloudstack-usage/lib/cloud-framework-db-4.7.1.jar:/usr/share/cloudstack-usage/lib/cloud-framework-events-4.7.1.jar:/usr/share/cloudstack-usage/lib/cloud-framework-ipc-4.7.1.jar:/usr/share/cloudstack-usage/lib/cloud-framework-jobs-4.7.1.jar:/usr/share/cloudstack-usage/lib/cloud-framework-managed-context-4.7.1.jar:/usr/share/cloudstack-usage/lib/cloud-framework-quota-4.7.1.jar:/usr/share/cloudstack-usage/lib/cloud-framework-rest-4.7.1.jar:/usr/share/cloudstack-usage/lib/cloud-framework-security-4.7.1.jar:/usr/share/cloudstack-usage/lib/cloud-utils-4.7.1.jar:/usr/share/cloudstack-usage/lib/commons-beanutils-1.8.3.jar:/usr/share/cloudstack-usage/lib/commons-beanutils-core-1.7.0.jar:/usr/share/cloudstack-usage/lib/commons-codec-1.10.jar:/usr/share/cloudstack-usage/lib/commons-collections-3.2.1.jar:/usr/share/cloudstack-usage/lib/commons-configuration-1.10.jar:/usr/share/cloudstack-usage/lib/commons-daemon-1.0.15.jar:/usr/share/cloudstack-usage/lib/commons-dbcp-1.4.jar:/usr/share/cloudstack-usage/lib/commons-digester-1.8.jar:/usr/share/cloudstack-usage/lib/commons-fileupload-1.2.jar:/usr/share/cloudstack-usage/lib/commons-httpclient-3.1.jar:/usr/share/cloudstack-usage/lib/commons-io-2.4.jar:/usr/share/cloudstack-usage/lib/commons-lang-2.6.jar:/usr/share/cloudstack-usage/lib/commons-lang3-3.4.jar:/usr/share/cloudstack-usage/lib/commons-logging-1.1.3.jar:/usr/share/cloudstack-usage/lib/commons-net-3.3.jar:/usr/share/cloudstack-usage/lib/commons-pool-1.6.jar:/usr/share/cloudstack-usage/lib/commons-validator-1.4.0.jar:/usr/share/cloudstack-usage/lib/cxf-bundle-jaxrs-2.7.13.jar:/usr/share/cloudstack-usage/lib/ehcache-core-2.6.9.jar:/usr/share/cloudstack-usage/lib/ejb-api-3.0.jar:/usr/share/cloudstack-usage/lib/esapi-2.1.0.jar:/usr/share/cloudstack-usage/lib/geronimo-javamail_1.4_spec-1.7.1.jar:/usr/share/cloudstack-usage/lib/geronimo-servlet_3.0_spec-1.0.jar:/usr/share/cloudstack-usage/lib/gson-1.7.2.jar:/usr/share/cloudstack-usage/lib/guava-18.0.jar:/usr/share/cloudstack-usage/lib/guava-testlib-18.0.jar:/usr/share/cloudstack-usage/lib/httpclient-4.5.jar:/usr/share/cloudstack-usage/lib/httpcore-4.4.jar:/usr/share/cloudstack-usage/lib/jackson-annotations-2.6.0.jar:/usr/share/cloudstack-usage/lib/jackson-core-2.6.3.jar:/usr/share/cloudstack-usage/lib/jackson-databind-2.6.3.jar:/usr/share/cloudstack-usage/lib/jackson-jaxrs-base-2.6.3.jar:/usr/share/cloudstack-usage/lib/jackson-jaxrs-json-provider-2.6.3.jar:/usr/share/cloudstack-usage/lib/jackson-module-jaxb-annotations-2.6.3.jar:/usr/share/cloudstack-usage/lib/jasypt-1.9.2.jar:/usr/share/cloudstack-usage/lib/java-ipv6-0.15.jar:/usr/share/cloudstack-usage/lib/javassist-3.18.2-GA.jar:/usr/share/cloudstack-usage/lib/javax.inject-1.jar:/usr/share/cloudstack-usage/lib/javax.persistence-2.1.0.jar:/usr/share/cloudstack-usage/lib/javax.ws.rs-api-2.0-m10.jar:/usr/share/cloudstack-usage/lib/jetty-continuation-8.1.15.v20140411.jar:/usr/share/cloudstack-usage/lib/jetty-http-8.1.15.v20140411.jar:/usr/share/cloudstack-usage/lib/jetty-io-8.1.15.v20140411.jar:/usr/share/cloudstack-usage/lib/jetty-security-8.1.15.v20140411.jar:/usr/share/cloudstack-usage/lib/jetty-util-8.1.15.v20140411.jar:/usr/share/cloudstack-usage/lib/joda-time-2.2.jar:/usr/share/cloudstack-usage/lib/jsch-0.1.51.jar:/usr/share/cloudstack-usage/lib/jsr305-1.3.9.jar:/usr/share/cloudstack-usage/lib/log4j-1.2.17.jar:/usr/share/cloudstack-usage/lib/mail-1.4.7.jar:/usr/share/cloudstack-usage/lib/nekohtml-1.9.12.jar:/usr/share/cloudstack-usage/lib/not-yet-commons-ssl-0.3.9.jar:/usr/share/cloudstack-usage/lib/objenesis-2.1.jar:/usr/share/cloudstack-usage/lib/opensaml-2.6.1.jar:/usr/share/cloudstack-usage/lib/openws-1.5.1.jar:/usr/share/cloudstack-usage/lib/powermock-core-1.5.3.jar:/usr/share/cloudstack-usage/lib/powermock-module-junit4-1.5.3.jar:/usr/share/cloudstack-usage/lib/powermock-module-junit4-common-1.5.3.jar:/usr/share/cloudstack-usage/lib/powermock-reflect-1.5.3.jar:/usr/share/cloudstack-usage/lib/reflections-0.9.9.jar:/usr/share/cloudstack-usage/lib/serializer-2.7.1.jar:/usr/share/cloudstack-usage/lib/servlet-api-2.5.jar:/usr/share/cloudstack-usage/lib/slf4j-api-1.7.7.jar:/usr/share/cloudstack-usage/lib/slf4j-log4j12-1.7.7.jar:/usr/share/cloudstack-usage/lib/spring-aop-3.2.12.RELEASE.jar:/usr/share/cloudstack-usage/lib/spring-beans-3.2.12.RELEASE.jar:/usr/share/cloudstack-usage/lib/spring-context-3.2.12.RELEASE.jar:/usr/share/cloudstack-usage/lib/spring-core-3.2.12.RELEASE.jar:/usr/share/cloudstack-usage/lib/spring-expression-3.2.12.RELEASE.jar:/usr/share/cloudstack-usage/lib/stax2-api-3.1.4.jar:/usr/share/cloudstack-usage/lib/trilead-ssh2-1.0.0-build217.jar:/usr/share/cloudstack-usage/lib/truth-0.23.jar:/usr/share/cloudstack-usage/lib/velocity-1.7.jar:/usr/share/cloudstack-usage/lib/woodstox-core-asl-4.4.1.jar:/usr/share/cloudstack-usage/lib/wsdl4j-1.6.3.jar:/usr/share/cloudstack-usage/lib/xalan-2.7.1.jar:/usr/share/cloudstack-usage/lib/xercesImpl-2.10.0.jar:/usr/share/cloudstack-usage/lib/xml-resolver-1.2.jar:/usr/share/cloudstack-usage/lib/xmlschema-core-2.1.0.jar:/usr/share/cloudstack-usage/lib/xmlsec-1.5.6.jar:/usr/share/cloudstack-usage/lib/xmltooling-1.4.1.jar:/usr/share/cloudstack-usage/lib/xom-1.2.5.jar::/etc/cloudstack/usage:/usr/share/java/mysql-connector-java.jar com.cloud.usage.UsageServer

feb 10 15:53:15 mgmt1 sh[1750]: 15:53:15,391  INFO UsageManagerImpl:274 - Starting Usage Manager
feb 10 15:53:15 mgmt1 sh[1750]: 15:53:15,442  INFO ComponentContext:132 - Starting org.apache.cloudstack.quota.QuotaStatementImpl_EnhancerByCloudStack_ffecfbc5
feb 10 15:53:15 mgmt1 sh[1750]: 15:53:15,442  INFO QuotaStatementImpl:97 - Starting Statement Manager
feb 10 15:53:15 mgmt1 sh[1750]: 15:53:15,443  INFO ComponentContext:132 - Starting org.apache.cloudstack.quota.QuotaAlertManagerImpl_EnhancerByCloudStack_381d9881
feb 10 15:53:15 mgmt1 sh[1750]: 15:53:15,443  INFO QuotaAlertManagerImpl:128 - Starting Alert Manager
feb 10 15:53:15 mgmt1 sh[1750]: 15:53:15,443  INFO ComponentContext:132 - Starting org.apache.cloudstack.quota.QuotaManagerImpl_EnhancerByCloudStack_e2e40673
feb 10 15:53:15 mgmt1 sh[1750]: 15:53:15,443  INFO QuotaManagerImpl:123 - Starting Quota Manager
feb 10 15:53:15 mgmt1 sh[1750]: 15:53:15,443  INFO ComponentContext:132 - Starting com.cloud.usage.UsageAlertManagerImpl_EnhancerByCloudStack_328874ba
feb 10 15:53:15 mgmt1 sh[1750]: 15:53:15,444 DEBUG DefaultListableBeanFactory:243 - Returning cached instance of singleton bean 'usageManagerImpl'
feb 10 15:53:15 mgmt1 sh[1750]: 15:53:15,444  INFO UsageServer:70 - UsageServer ready...

in /var/log/cloudstack/usage/usage.log:

...
DEBUG [cloud.usage.UsageManagerImpl] (main:null) (logid:) Checking to see if usage.vmops.pid exists.
INFO  [cloud.usage.UsageManagerImpl] (main:null) (logid:) Implementation Version is 4.7.1
DEBUG [cloud.usage.UsageManagerImpl] (main:null) (logid:) Usage stats aggregation time zone: GMT
DEBUG [cloud.usage.UsageManagerImpl] (main:null) (logid:) Execution Time: Wed Feb 10 00:15:00 UTC 2016
DEBUG [cloud.usage.UsageManagerImpl] (main:null) (logid:) Current Time: Wed Feb 10 15:53:15 UTC 2016
...

export UCP=`ls /usr/share/cloudstack-usage/cloud-usage-*.jar /usr/share/cloudstack-usage/lib/*.jar | tr "\\n" ":"`; \
export CLASSPATH="$UCP:/etc/cloudstack/usage:/usr/share/java/mysql-connector-java.jar"; \
${JAVA_HOME}/bin/java -Xms${JAVA_HEAP_INITIAL} -Xmx${JAVA_HEAP_MAX} -cp "$CLASSPATH" $JAVA_CLASS'
${JAVA_HOME}/bin/java -Dpid=${JAVA_PID} -Xms${JAVA_HEAP_INITIAL} -Xmx${JAVA_HEAP_MAX} -cp "$CLASSPATH" $JAVA_CLASS'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM (did not test by creating packages and installing it though)

@Slair1
Copy link
Contributor

Slair1 commented Mar 9, 2016

FYI, this fix also worked for me, Cloudstack 4.8 on Centos7

@ustcweizhou
Copy link
Contributor

LGTM
works fine in our internal version.

@swill
Copy link
Contributor

swill commented Apr 27, 2016

Two people have confirmed this is working and we also have two LGTM. Is this one ready to merge. I don't think running CI is going to add much value beyond the manual checks people have already done. Do you guys agree?

@yadvr
Copy link
Member

yadvr commented Apr 27, 2016

@swill agree

@kiwiflyer
Copy link
Contributor

Tested this in a 4.8 lab. I agree that there is little point in running CI against this, as it's a config change.

LGTM.

@asfgit asfgit merged commit ce18079 into apache:master Apr 28, 2016
asfgit pushed a commit that referenced this pull request Apr 28, 2016
CLOUDSTACK-9283: add pid to java arguments in cloudstack-usage.servicecloudstack-usage fails to start throwing Integer exception during PID retrieval, and the service keeps restarting after 10s (as defined in the systemd service definition).

Adding the pid to the java arguments in the systemd service definition makes it stop looping in centos7

* pr/1409:
  CLOUDSTACK-9283: add pid to java arguments in systemd/cloudstack-usage.service

Signed-off-by: Will Stevens <williamstevens@gmail.com>
nathanejohnson added a commit to myENA/cloudstack that referenced this pull request Nov 8, 2016
apache#1409

Addresses index out of range exceptions
nathanejohnson added a commit to myENA/cloudstack that referenced this pull request Nov 8, 2016
apache#1409

add pid to java arguments in cloudstack-usage.service
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants