adding Huawei OBS as object store. #8359

scsynergy · 2023-12-15T08:38:28Z

Description

This PR adds an object storage plugin for Huawei OBS, but also introduces an error I do not know how to fix when I
execute "mvn -pl :cloud-client-ui jetty:run":
http://192.168.17.252:8080/client/ returns "HTTP ERROR 503 Service Unavailable" because of the following exception:

[WARNING] Failed startup of context o.e.j.m.p.JettyWebAppContext@1df8ea34{/client,file:///opt/cloudstack-huawei-obs/client/target/classes/META-INF/webapp/,UNAVAILABLE}{file:///opt/cloudstack-huawei-obs/client/target/classes/META-INF/webapp/}
java.lang.NullPointerException

When I remove my module from client/pom.xml http://192.168.17.252:8080/client/ is available and I can log into the UI,
but my plugin is not registered as provider.

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
[ x] New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)
build/CI

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
[x ] Minor

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

turns out that following this guide "client/pom.xml" https://docs.cloudstack.apache.org/en/4.18.1.0/developersguide/plugins.html#edit-configuration-files causes an error when Jetty starts up.

added "Huawei OBS" to the UI provider dropdown

added the plugin to the dependencies in client/pom.xml this causes [WARNING] Failed startup of context o.e.j.m.p.JettyWebAppContext@63893cf6{/client,file:///opt/cloudstack-huawei-obs/client/target/classes/META-INF/webapp/,UNAVAILABLE}{file:///opt/cloudstack-huawei-obs/client/target/classes/META-INF/webapp/} java.lang.NullPointerException when doing mvn -pl :cloud-client-ui jetty:run

boring-cyborg · 2023-12-15T08:38:33Z

Congratulations on your first Pull Request and welcome to the Apache CloudStack community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/cloudstack/blob/main/CONTRIBUTING.md)
Here are some useful points:

In case of a new feature add useful documentation (raise doc PR at https://github.com/apache/cloudstack-documentation)
Be patient and persistent. It might take some time to get a review or get the final approval from the committers.
Pay attention to the quality of your code, ensure tests are passing and your PR doesn't have conflicts.
Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Issues, Mailing list and Slack.
Be sure to read the CloudStack Coding Conventions.
Apache CloudStack is a community-driven project and together we are making it better 🚀.
In case of doubts contact the developers at:
Mailing List: dev@cloudstack.apache.org (https://cloudstack.apache.org/mailing-lists.html)
Slack: https://apachecloudstack.slack.com/

…-obs

DaanHoogland · 2024-01-30T14:45:39Z

plugins/pom.xml

    </modules>
    <dependencies>
-        <dependency>
+	<dependency>


tab

Suggested change

<dependency>

<dependency>

DaanHoogland · 2024-01-30T14:47:27Z

plugins/storage/object/huawei-obs/pom.xml

@@ -0,0 +1,44 @@
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">


Suggested change

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">



<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

DaanHoogland · 2024-01-30T14:48:10Z

...main/java/org/apache/cloudstack/storage/datastore/driver/HuaweiObsObjectStoreDriverImpl.java

@@ -0,0 +1,453 @@
+package org.apache.cloudstack.storage.datastore.driver;


Suggested change

package org.apache.cloudstack.storage.datastore.driver;

// Licensed to the Apache Software Foundation (ASF) under one

// or more contributor license agreements. See the NOTICE file

// distributed with this work for additional information

// regarding copyright ownership. The ASF licenses this file

// to you under the Apache License, Version 2.0 (the

// "License"); you may not use this file except in compliance

// with the License. You may obtain a copy of the License at

//

// http://www.apache.org/licenses/LICENSE-2.0

//

// Unless required by applicable law or agreed to in writing,

// software distributed under the License is distributed on an

// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY

// KIND, either express or implied. See the License for the

// specific language governing permissions and limitations

// under the License.

package org.apache.cloudstack.storage.datastore.driver;

DaanHoogland · 2024-01-30T14:48:30Z

...ava/org/apache/cloudstack/storage/datastore/lifecycle/HuaweiObsObjectStoreLifeCycleImpl.java

@@ -0,0 +1,111 @@
+package org.apache.cloudstack.storage.datastore.lifecycle;


Suggested change

package org.apache.cloudstack.storage.datastore.lifecycle;

// Licensed to the Apache Software Foundation (ASF) under one

// or more contributor license agreements. See the NOTICE file

// distributed with this work for additional information

// regarding copyright ownership. The ASF licenses this file

// to you under the Apache License, Version 2.0 (the

// "License"); you may not use this file except in compliance

// with the License. You may obtain a copy of the License at

//

// http://www.apache.org/licenses/LICENSE-2.0

//

// Unless required by applicable law or agreed to in writing,

// software distributed under the License is distributed on an

// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY

// KIND, either express or implied. See the License for the

// specific language governing permissions and limitations

// under the License.

package org.apache.cloudstack.storage.datastore.lifecycle;

DaanHoogland · 2024-01-30T14:48:43Z

.../java/org/apache/cloudstack/storage/datastore/provider/HuaweiObsObjectStoreProviderImpl.java

@@ -0,0 +1,64 @@
+package org.apache.cloudstack.storage.datastore.provider;


Suggested change

package org.apache.cloudstack.storage.datastore.provider;

// Licensed to the Apache Software Foundation (ASF) under one

// or more contributor license agreements. See the NOTICE file

// distributed with this work for additional information

// regarding copyright ownership. The ASF licenses this file

// to you under the Apache License, Version 2.0 (the

// "License"); you may not use this file except in compliance

// with the License. You may obtain a copy of the License at

//

// http://www.apache.org/licenses/LICENSE-2.0

//

// Unless required by applicable law or agreed to in writing,

// software distributed under the License is distributed on an

// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY

// KIND, either express or implied. See the License for the

// specific language governing permissions and limitations

// under the License.

package org.apache.cloudstack.storage.datastore.provider;

DaanHoogland · 2024-01-30T14:49:10Z

.../java/org/apache/cloudstack/storage/datastore/driver/HuaweiObsObjectStoreDriverImplTest.java

@@ -0,0 +1,87 @@
+package org.apache.cloudstack.storage.datastore.driver;


Suggested change

package org.apache.cloudstack.storage.datastore.driver;

// Licensed to the Apache Software Foundation (ASF) under one

// or more contributor license agreements. See the NOTICE file

// distributed with this work for additional information

// regarding copyright ownership. The ASF licenses this file

// to you under the Apache License, Version 2.0 (the

// "License"); you may not use this file except in compliance

// with the License. You may obtain a copy of the License at

//

// http://www.apache.org/licenses/LICENSE-2.0

//

// Unless required by applicable law or agreed to in writing,

// software distributed under the License is distributed on an

// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY

// KIND, either express or implied. See the License for the

// specific language governing permissions and limitations

// under the License.

package org.apache.cloudstack.storage.datastore.driver;

DaanHoogland · 2024-01-30T14:49:22Z

...a/org/apache/cloudstack/storage/datastore/provider/HuaweiObsObjectStoreProviderImplTest.java

@@ -0,0 +1,34 @@
+package org.apache.cloudstack.storage.datastore.provider;


Suggested change

package org.apache.cloudstack.storage.datastore.provider;

// Licensed to the Apache Software Foundation (ASF) under one

// or more contributor license agreements. See the NOTICE file

// distributed with this work for additional information

// regarding copyright ownership. The ASF licenses this file

// to you under the Apache License, Version 2.0 (the

// "License"); you may not use this file except in compliance

// with the License. You may obtain a copy of the License at

//

// http://www.apache.org/licenses/LICENSE-2.0

//

// Unless required by applicable law or agreed to in writing,

// software distributed under the License is distributed on an

// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY

// KIND, either express or implied. See the License for the

// specific language governing permissions and limitations

// under the License.

package org.apache.cloudstack.storage.datastore.provider;

DaanHoogland · 2024-01-30T14:50:43Z

...uawei-obs/src/main/resources/META-INF/cloudstack/storage-object-huawei-obs/module.properties

@@ -0,0 +1,2 @@
+name=storage-object-huawei-obs


Suggested change

name=storage-object-huawei-obs

# Licensed to the Apache Software Foundation (ASF) under one

# or more contributor license agreements. See the NOTICE file

# distributed with this work for additional information

# regarding copyright ownership. The ASF licenses this file

# to you under the Apache License, Version 2.0 (the

# "License"); you may not use this file except in compliance

# with the License. You may obtain a copy of the License at

#

# http://www.apache.org/licenses/LICENSE-2.0

#

# Unless required by applicable law or agreed to in writing,

# software distributed under the License is distributed on an

# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY

# KIND, either express or implied. See the License for the

# specific language governing permissions and limitations

# under the License.

name=storage-object-huawei-obs

DaanHoogland · 2024-01-30T14:51:01Z

...s/META-INF/cloudstack/storage-object-huawei-obs/spring-storage-object-huawei-obs-context.xml

@@ -0,0 +1,12 @@
+<beans xmlns="http://www.springframework.org/schema/beans"


Suggested change

<beans xmlns="http://www.springframework.org/schema/beans"



<beans xmlns="http://www.springframework.org/schema/beans"

codecov · 2024-01-30T14:53:08Z

Codecov Report

Attention: 281 lines in your changes are missing coverage. Please review.

Comparison is base (cfb4d43) 4.38% compared to head (34e1673) 29.29%.

❗ Current head 34e1673 differs from pull request most recent head 9f2cbec. Consider uploading reports for the commit 9f2cbec to get more accurate results

Files	Patch %	Lines
...tastore/driver/HuaweiObsObjectStoreDriverImpl.java	9.19%	243 Missing and 4 partials ⚠️
...e/lifecycle/HuaweiObsObjectStoreLifeCycleImpl.java	8.57%	32 Missing ⚠️
...ore/provider/HuaweiObsObjectStoreProviderImpl.java	84.61%	2 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##              main    #8359       +/-   ##
============================================
+ Coverage     4.38%   29.29%   +24.91%     
- Complexity       0    32112    +32112     
============================================
  Files          361     5344     +4983     
  Lines        28706   375353   +346647     
  Branches      5004    54585    +49581     
============================================
+ Hits          1258   109961   +108703     
- Misses       27309   250717   +223408     
- Partials       139    14675    +14536

Flag	Coverage Δ
simulator-marvin-tests	`22.54% <4.68%> (?)`
uitests	`4.39% <ø> (+0.01%)`	⬆️
unit-tests	`16.49% <9.68%> (?)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

…sitory

DaanHoogland · 2024-01-31T08:19:32Z

...object/huawei-obs/local-huawei-sdk/com/huawei/storage/esdk-obs-java/maven-metadata-local.xml

@@ -0,0 +1,12 @@
+<?xml version="1.0" encoding="UTF-8"?>


Suggested change

<?xml version="1.0" encoding="UTF-8"?>



<?xml version="1.0" encoding="UTF-8"?>

DaanHoogland · 2024-01-31T08:20:42Z

...huawei-obs/local-huawei-sdk/com/huawei/storage/esdk-obs-java/3.23.9/esdk-obs-java-3.23.9.pom

@@ -0,0 +1,9 @@
+<?xml version="1.0" encoding="UTF-8"?>


Suggested change

<?xml version="1.0" encoding="UTF-8"?>



<?xml version="1.0" encoding="UTF-8"?>

DaanHoogland · 2024-01-31T08:23:42Z

...huawei-obs/local-huawei-sdk/com/huawei/storage/esdk-obs-java/3.23.9/esdk-obs-java-3.23.9.jar

@scsynergy , I think we need to not add binary files in this repo. There is a construct for non-opensource files to be added in https://github.com/shapeblue/cloudstack-nonoss and then used if a project needs it.

DaanHoogland · 2024-01-31T08:28:41Z

@scsynergy is your with that jar added? if so you probably want to add it as non-oss. (se my remark above)
Also note I gave your PR a more descriptive title (change if you don´t like it)
Feel free to ask the community anything here or on the mailing lists (or on a range of other platforms)

…sitory added missing dependencies

JoaoJandre

Hey, @scsynergy please merge this PR with main and change the loggers names/imports. #7131 description has a script you can use if you want to (after merging with main).

JoaoJandre · 2024-02-08T14:35:03Z

...ava/org/apache/cloudstack/storage/datastore/lifecycle/HuaweiObsObjectStoreLifeCycleImpl.java

+
+public class HuaweiObsObjectStoreLifeCycleImpl implements ObjectStoreLifeCycle {
+
+    private static final Logger LOG = Logger.getLogger(HuaweiObsObjectStoreLifeCycleImpl.class);


Suggested change

private static final Logger LOG = Logger.getLogger(HuaweiObsObjectStoreLifeCycleImpl.class);

protected Logger logger = LogManager.getLogger(getClass());

Also need to update the log4j import

DaanHoogland · 2024-02-09T13:46:23Z

@scsynergy did you close this on purpose? Are you going to open a new PR?

scsynergy added 5 commits December 13, 2023 10:50

added plugin for Huawei OBS object storage

a8e75d9

added plugin for Huawei OBS object storage

fd1db41

turns out that following this guide "client/pom.xml" https://docs.cloudstack.apache.org/en/4.18.1.0/developersguide/plugins.html#edit-configuration-files causes an error when Jetty starts up.

Merge branch 'apache:main' into main

69b77af

added plugin for Huawei OBS object storage

dc12e08

added "Huawei OBS" to the UI provider dropdown

boring-cyborg bot added component:build component:storage component:UI labels Dec 15, 2023

shwstppr marked this pull request as draft December 15, 2023 11:44

vishesh92 assigned kishankavala Dec 19, 2023

scsynergy added 4 commits January 24, 2024 10:30

Merge branch 'apache:main' into main

007c530

using correct SDK now

bd34cd5

Merge branch 'apache:main' into main

d3453eb

Merge branch 'main' of https://github.com/scsynergy/cloudstack-huawei…

dcfd55a

…-obs

DaanHoogland reviewed Jan 30, 2024

View reviewed changes

plugins/pom.xml

</modules>

<dependencies>

<dependency>

<dependency>

Copy link

Contributor

DaanHoogland Jan 30, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tab

Suggested change

<dependency>

<dependency>

DaanHoogland reviewed Jan 30, 2024

View reviewed changes

DaanHoogland changed the title ~~This is a draft pull request for Kishan~~ adding Huawei OBS as object store. Jan 30, 2024

DaanHoogland assigned vishesh92 and unassigned kishankavala Jan 30, 2024

include JAR file which is not available through any public Maven repo…

34e1673

…sitory

DaanHoogland reviewed Jan 31, 2024

View reviewed changes

scsynergy added 2 commits January 31, 2024 11:10

include JAR file which is not available through any public Maven repo…

5b87cf1

…sitory added missing dependencies

Merge branch 'apache:main' into main

b638c1b

JoaoJandre reviewed Feb 8, 2024

View reviewed changes

Merge branch 'apache:main' into main

9f2cbec

scsynergy closed this Feb 9, 2024

DaanHoogland added this to the unplanned milestone May 7, 2024

		@@ -0,0 +1,44 @@
		<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

-<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+<!--
+  Licensed to the Apache Software Foundation (ASF) under one
+  or more contributor license agreements.  See the NOTICE file
+  distributed with this work for additional information
+  regarding copyright ownership.  The ASF licenses this file
+  to you under the Apache License, Version 2.0 (the
+  "License"); you may not use this file except in compliance
+  with the License.  You may obtain a copy of the License at
+    http://www.apache.org/licenses/LICENSE-2.0
+  Unless required by applicable law or agreed to in writing,
+  software distributed under the License is distributed on an
+  "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+  KIND, either express or implied.  See the License for the
+  specific language governing permissions and limitations
+  under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

		@@ -0,0 +1,453 @@
		package org.apache.cloudstack.storage.datastore.driver;

-package org.apache.cloudstack.storage.datastore.driver;
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package org.apache.cloudstack.storage.datastore.driver;

		@@ -0,0 +1,111 @@
		package org.apache.cloudstack.storage.datastore.lifecycle;

		@@ -0,0 +1,64 @@
		package org.apache.cloudstack.storage.datastore.provider;

		@@ -0,0 +1,87 @@
		package org.apache.cloudstack.storage.datastore.driver;

		@@ -0,0 +1,34 @@
		package org.apache.cloudstack.storage.datastore.provider;

-name=storage-object-huawei-obs
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+name=storage-object-huawei-obs

		@@ -0,0 +1,12 @@
		<beans xmlns="http://www.springframework.org/schema/beans"


		public class HuaweiObsObjectStoreLifeCycleImpl implements ObjectStoreLifeCycle {

		private static final Logger LOG = Logger.getLogger(HuaweiObsObjectStoreLifeCycleImpl.class);

	private static final Logger LOG = Logger.getLogger(HuaweiObsObjectStoreLifeCycleImpl.class);
	protected Logger logger = LogManager.getLogger(getClass());

adding Huawei OBS as object store. #8359

adding Huawei OBS as object store. #8359

Uh oh!

Conversation

scsynergy commented Dec 15, 2023

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

Uh oh!

boring-cyborg bot commented Dec 15, 2023

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

codecov bot commented Jan 30, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

DaanHoogland Jan 31, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

DaanHoogland commented Jan 31, 2024

Uh oh!

JoaoJandre left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

DaanHoogland commented Feb 9, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

codecov bot commented Jan 30, 2024 •

edited

Loading

DaanHoogland Jan 31, 2024 •

edited

Loading