rcp_error with imagecapture request in cpp

[Jaeyoung-Lim]

Bug report

• AirSim Version/#commit: 6e2512e
• UE/Unity version: Unreal 4.25
• autopilot version: Computer vision mode
• OS Version: Ubuntu 18.04

What's the issue you encountered?

While it is possible to use the msr::airlib::MultirotorRpcLibClient for setting poses with .simSetVehiclePose(), when requesting images, it fails with a rpc_error

Waiting for connection - X
Connected!

Client Ver:1 (Min Req:1), Server Ver:1 (Min Req:1)
terminate called after throwing an instance of 'rpc::rpc_error'
  what():  rpc::rpc_error during call

The code snippet I am using for requesting images are as the following

    vector<ImageRequest> request = { ImageRequest("0", ImageType::Scene) };
    const vector<ImageResponse>& response = client_.simGetImages(request);  

where the types are defined separately

typedef msr::airlib::ImageCaptureBase::ImageRequest ImageRequest;
typedef msr::airlib::ImageCaptureBase::ImageResponse ImageResponse;
typedef msr::airlib::ImageCaptureBase::ImageType ImageType;

This is running as part as a ROS node

What I don't understand is that since I am able to control the pose, the link between ros node and airsim is valid. However, I am not able to retrieve images from the server

Settings

{
  "SeeDocsAt": "https://github.com/Microsoft/AirSim/blob/master/docs/settings.md",
  "SettingsVersion": 1.2,
  "PhysicsEngineName":"ExternalPhysicsEngine",
  "SimMode": "ComputerVision"
}

How can the issue be reproduced?

1. Try using the image API in cpp?

Include full error message in text form

Waiting for connection - X
Connected!

Client Ver:1 (Min Req:1), Server Ver:1 (Min Req:1)
terminate called after throwing an instance of 'rpc::rpc_error'
  what():  rpc::rpc_error during call

What's better than filing an issue? Filing a pull request :).

[Jaeyoung-Lim]

Also tried using the HelloDrone example and fails as the following

~/dev/AirSim/HelloDrone/build$ ./HelloDrone 
Waiting for connection - X
Connected!

Client Ver:1 (Min Req:1), Server Ver:1 (Min Req:1)
Press Enter to get FPV image

Exception raised by the API, something went wrong.
rpclib: function 'simGetImages' (called with 3 arg(s)) threw an exception. The exception is not derived from std::exception. No further information available.

@jonyMarino Would you have any ideas if the simGetImages need configuration on settings.json before it works?

[jonyMarino]

Hi Jaeyoung! You are using the ComputerVision mode but I think what you want is the Multirotor one. Your code should work changing that.

[Jaeyoung-Lim]

@jonyMarino Thanks for the reply!

How can I capture images when I am in computer vision mode?

I am trying to use airlib so that it can orient the camera as it is attached on a gimbal, and capture the images through mavlink. Would there be a better way to achieve this without using computer vision mode?

[zimmy87]

Hi @Jaeyoung-Lim, for capturing images in computer vision mode, I think you'll want to use AirSim's RpcLibClientBase instead of MultirotorRpcLibClient. Your call to simGetImages() should be fine, I think you're just using the wrong client.

[jonyMarino]

But if you want to use our ROS wrapper, maybe it is better to use the ExternalPhysicsEngine in Multirotor mode. If you have problems with that, let us know.

[Jaeyoung-Lim]

@zimmy87 @jonyMarino Thanks for the inputs,

I tried using RpcLibClientBase as the client, but the simGetImages() and simGetImage() are both failing with the same errors. The pose can still be successfully set also with the RpcLibClientBase

Waiting for connection - X
Connected!

Client Ver:1 (Min Req:1), Server Ver:1 (Min Req:1)
terminate called after throwing an instance of 'rpc::rpc_error'
  what():  rpc::rpc_error during call

I have also tried disabling computer vision mode and run the HelloDrone example and this does not work either.

Not sure what I am missing here. What I am basically trying to do using airlib is as the following. Would there be a better way to do this?

• Set an arbitrary pose of the camera
• Get the image out of the camera

[jonyMarino]

Jaeyoung. Neither Zimmy nor I were able to reproduce your issue. Are you using the same AirSim version for the airsim client (HelloDrone) and server (your UE AirSim project)?

[Jaeyoung-Lim]

@jonyMarino Oh, I forgot to mention this. I built my airlib with gcc-8 and for the simulation using a landscape mountains binary from the 1.5.0-linux release.

Do you think this might be causing the issue? Since I am building my package with gcc and linking airlib I thought this was what I should do

[Jaeyoung-Lim]

By repackaging the latest Airsim on linux solved the problem. (See the image captured below)

This means that the 1.5.0-linux release is not compatible anymore with the latest Airsim.

Since I am interested in using the gcc compiled Airlib I tried to build 1.5.0 on gcc-8 but the build script caused a lot of problems.

@jonyMarino @zimmy87 Since my problems were caused by a regression on the airlib API, it would be very helpful if there can be a new linux release, so other people do not need to repackage their Airsim with UE4. Would this be something possible?

Thanks!

[jonyMarino]

@Jaeyoung-Lim That is in our plans. @saihv is solving one more issue and then we will start the new release.

[Jaeyoung-Lim]

@jonyMarino @saihv Any chance of a ETA? would be quite helpful to plan my work 😄

[rajat2004]

@Jaeyoung-Lim Re the RPC error with 1.5.0 binary and latest master for compiling the client code, that's mostly due to #3320, I should have thought about and mentioned in the PR at least that it would break binary compatibility with the releases, sorry for that, and something to note at least in the future

[jonyMarino]

@jonyMarino @saihv Any chance of a ETA? would be quite helpful to plan my work 😄

This month!

[jonyMarino]

@Jaeyoung-Lim Here you have!