CMake problems when compiling SDK as a shared library

[tete17]

What platform/OS are you using?

Linux/Docker

Which version of the SDK?

1.7.264

What compiler are you using? what version?

gcc 7.4.0

What are your CMake arguments?

• BUILD_ONLY="s3"
• CUSTOM_MEMORY_MANAGEMENT=OFF
• ENABLE_TESTING=OFF
• ENABLE_UNITY_BUILD=ON

Can you provide a TRACE level log? (sanitize any sensitive information)

Log output of the CMake consumer project

-- The C compiler identification is GNU 7.4.0
-- The CXX compiler identification is GNU 7.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found AWS SDK for C++, Version: 1.7.264, Install Root:/usr/local, Platform Prefix:, Platform Dependent Libraries: pthread;crypto;ssl;z;curl
-- Components specified for AWSSDK: s3, application wll be depending on libs: aws-cpp-sdk-s3;aws-cpp-sdk-core
-- Try finding aws-cpp-sdk-core
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Check if compiler accepts -pthread
-- Check if compiler accepts -pthread - yes
-- Found Threads: TRUE  
CMake Error at /usr/local/lib/aws-c-common/cmake/aws-c-common-config.cmake:7 (include):
  include could not find load file:

    /usr/local/lib/aws-c-common/cmake/static/aws-c-common-targets.cmake
Call Stack (most recent call first):
  /usr/share/cmake-3.10/Modules/CMakeFindDependencyMacro.cmake:48 (find_package)
  /usr/local/lib/aws-c-event-stream/cmake/aws-c-event-stream-config.cmake:2 (find_dependency)
  /usr/share/cmake-3.10/Modules/CMakeFindDependencyMacro.cmake:48 (find_package)
  /usr/local/lib/cmake/aws-cpp-sdk-core/aws-cpp-sdk-core-config.cmake:2 (find_dependency)
  /usr/local/lib/cmake/AWSSDK/AWSSDKConfig.cmake:303 (find_package)
  CMakeLists.txt:6 (find_package)


CMake Error at /usr/local/lib/aws-checksums/cmake/aws-checksums-config.cmake:4 (include):
  include could not find load file:

    /usr/local/lib/aws-checksums/cmake/static/aws-checksums-targets.cmake
Call Stack (most recent call first):
  /usr/share/cmake-3.10/Modules/CMakeFindDependencyMacro.cmake:48 (find_package)
  /usr/local/lib/aws-c-event-stream/cmake/aws-c-event-stream-config.cmake:3 (find_dependency)
  /usr/share/cmake-3.10/Modules/CMakeFindDependencyMacro.cmake:48 (find_package)
  /usr/local/lib/cmake/aws-cpp-sdk-core/aws-cpp-sdk-core-config.cmake:2 (find_dependency)
  /usr/local/lib/cmake/AWSSDK/AWSSDKConfig.cmake:303 (find_package)
  CMakeLists.txt:6 (find_package)


CMake Error at /usr/local/lib/aws-c-event-stream/cmake/aws-c-event-stream-config.cmake:8 (include):
  include could not find load file:

    /usr/local/lib/aws-c-event-stream/cmake/static/aws-c-event-stream-targets.cmake
Call Stack (most recent call first):
  /usr/share/cmake-3.10/Modules/CMakeFindDependencyMacro.cmake:48 (find_package)
  /usr/local/lib/cmake/aws-cpp-sdk-core/aws-cpp-sdk-core-config.cmake:2 (find_dependency)
  /usr/local/lib/cmake/AWSSDK/AWSSDKConfig.cmake:303 (find_package)
  CMakeLists.txt:6 (find_package)


-- Found aws-cpp-sdk-core
-- Try finding aws-cpp-sdk-s3
-- Found aws-cpp-sdk-s3
-- Configuring incomplete, errors occurred!
See also "/tmp/build/CMakeFiles/CMakeOutput.log".
See also "/tmp/build/CMakeFiles/CMakeError.log".

Hey Guys. I think I found a problem on the installed cmake files when compiling the library with shared libraries.

Looking at the log output it seems that cmake is not properly communicating to the dependant libraries (i.e. aws-c-common, aws-checksums, etc...) that the installation was done for a shared library.

I have put up a small Docker file for reproducibility purpose

FROM ubuntu:18.04

RUN apt-get update \
        && apt-get install -y \
                ca-certificates \
                curl \
                build-essential \
                curl \
                git \
                cmake \
                libz-dev \
                libssl-dev \
                libcurl4-openssl-dev


RUN git clone --depth 1 --shallow-submodules --branch 1.7.264 https://github.com/aws/aws-sdk-cpp.git \
        && mkdir aws-obj \
        && (cd aws-obj \
                && cmake \
                        -DBUILD_ONLY="s3" \
                        -DCUSTOM_MEMORY_MANAGEMENT=OFF \
                        -DENABLE_TESTING=OFF \
                        -DENABLE_UNITY_BUILD=ON \
                        ../aws-sdk-cpp \
                && make -j$(nproc) install) \
        && rm -r aws-obj aws-sdk-cpp

WORKDIR /tmp
COPY . /tmp/

RUN mkdir build && cd build \
    && cmake .. \

The CMakeLists.txt file looks like this

cmake_minimum_required(VERSION 3.10)
project(aws_bug)

set(CMAKE_CXX_STANDARD 14)

find_package(AWSSDK REQUIRED COMPONENTS s3)

add_executable(aws_bug main.cpp)

Digging through the Cmake files I found that this is communicated with the Cmake variable BUILD_SHARED_LIBS to "ON". Applying this variable to the project seemed to fix the issue as a workaround but i believe is not the intended behaviour. Let me know if you need more input from me

[KaibaLopez]

Hi @tete17 ,
So, this is a kind of "known issue", but one we can't really change.
The error happens because cmake's find_packagewill try and search for the libs to link and if not specified on your cmake arguments it will take the project's default, which in the aws-c-commons case is static.
But the sdk's default is shared, so when you built the sdk, it built the dependencies as shared libraries, and when you try to link them to your application it looks for static libraries and can't find them, thus the error message.
The problem then is that the sdk does not own the dependencies, so we can't change their default, and so we need this "workaround" instead, (you could potentially also do it the other way around and specify -DBUILD_SHARED_LIBS=OFF when building and installing the sdk).
Hope this helps and let me know if you got more questions.

[tete17]

Hi @KaibaLopez

Thanks for the quick response. I think building the sdk as a static library is not an option currently for us but thanks for suggesting it.

I would also like to ask you if this behaviour has changed recently? I haven't tested yet but I am pretty sure I didn't saw this behaviour in previous versions of the library (something like 1.7.198).

I still think there is ways to improve the situation. I see 2 options here at least.

• We could add that define right before the internal find_package this library does on the internal cmake files it instals on the system (i.e. AwsSharedLibSetup.cmake) so that the user does not need to.
• I could submit a MR to aws-c-commons to take a similar approach to what opentracing-cpp which is to expose 2 different targets, in their specific opentracing-static and opentracing. This would still give us the flexibility of choosing your compilation mode but we would be more explicit and correct when deciding which library to use.

Let me know what you think or if you see other options here.

[KaibaLopez]

Hi @tete17 ,
It was somewhat recent that we made the change you are right, we were building them as static libraries before though, so you probably don't want to go back to previous versions.
Could you make a PR for the first option you suggested? It sounds doable but I'm not super familiar with cmake's internal operations to be honest so I'd appreciate the help.