How can I feed a network with images of various shapes?

Dear all,
The provided data loading API, say ImageRecordIter, need the user to specify data_shape, but the convolutional layer is supposed to handle images of various shapes.
Is there any efficient way to feed the networks with images of various shapes?
If not, any advice on implementing such kind of data iterator?

Thanks in advance.

[piiswrong]

Even if you feed in images of different shape, the network still has fixed shape. You need per example reshape for that, which currently isn't implemented. Also you have to have unified shape within batch in any case

[tornadomeet]

one pr of image segmentation example is you want,please check #975, but currently it only support minibatch=1

[jonbakerfish]

I use the following code to extract images in arbitrary shapes:

import numpy as np
import mxnet as mx
import mxnet.symbol as sym
from mxnet import nd

## load symbol & params for BN-21k
input_shape = (1,3,224,224)
net_symbol = sym.load('./models/inception-21k/Inception-symbol.json')
save_dict = nd.load('./models/inception-21k/Inception-0009.params')
arg_params = {}
aux_params = {}
for k, v in save_dict.items():
    tp, name = k.split(':', 1)
    if tp == 'arg':
        arg_params[name] = v
    if tp == 'aux':
        aux_params[name] = v

## remove fc layer & set output layer
internals = net_symbol.get_internals()
net_symbol = sym.Group([internals['global_pool_output']])
# ## or, for multiple outputs:
# net_symbol = sym.Group([
#                  internals['global_pool_output'],
#                  internals['ch_concat_5b_chconcat_output'],
#                  internals['ch_concat_5a_chconcat_output'],
#                  internals['ch_concat_4e_chconcat_output'],
#                  internals['ch_concat_4d_chconcat_output'],
#                  internals['ch_concat_4c_chconcat_output'],
#              ])
arg_names = set(net_symbol.list_arguments())
arg_params = {k : v for k, v in arg_params.items() if k in arg_names}
aux_names = set(net_symbol.list_auxiliary_states())
aux_params = {k : v for k, v in aux_params.items() if k in aux_names}


## initialize executor
pred_exec = net_symbol.simple_bind(ctx=mx.gpu(), grad_req='null', data=input_shape)
pred_exec.copy_params_from(arg_params, aux_params)


## extract features
for img in img_list: # a list of images in different shapes (1,channels,height,width)
    input_shape = img.shape

    ## reshape & feed image to executor
    pred_exec = pred_exec.reshape(partial_shaping=False, allow_up_sizing=True, **{'data':input_shape})
    input_data = pred_exec.arg_dict['data']
    img_nd = mx.nd.array(img)
    img_nd.copyto(input_data)

    ## forward & get results
    pred_exec.forward(is_train=False)
    out = pred_exec.outputs

The key step is reshape the executor every time before forward. Is there any more efficient way to do this?

However, it might have trouble for some image shapes depending on your models. E.g., when input_shape=(1,3,500,500), the above example gets the following error:

MXNetError: InferShape Error in ch_concat_4e_chconcat: [00:36:15] src/operator/./concat-inl.h:152: Check failed: (dshape[j]) == (tmp[j]) Incorrect shape[2]: (1,512,15,15). (first input shape: (1,960,16,16))

One solution is to reshape the image to a slightly different shape. E.g. (1,3,512,512) where 512%32 == 0. Any better idea?