is there interface to construct a RNNState object by using vector or numpy.ndarray

[zhang-wen]

importing dynet in python, i try to change the intermedia state of lstm by some calculations, but after calculation between some parameters and s.h()[-1].npvalue(), how can i still return a RNNState object to continue to be a input of the lstm.

following is my code to covert state s,

def convert_dec_state(self, ctx_watt, s, bxm=None):
    w_cz = dy.parameter(self.dec_w_cz)
    w_hz = dy.parameter(self.dec_w_hz)
    b_z = dy.parameter(self.dec_b_z)
    w_cr = dy.parameter(self.dec_w_cr)
    w_hr = dy.parameter(self.dec_w_hr)
    b_r = dy.parameter(self.dec_b_r)

    w_ch = dy.parameter(self.dec_w_ch)
    w_hh = dy.parameter(self.dec_w_hh)
    b_h = dy.parameter(self.dec_b_h)

    # dy.colwise_add(w_cz * ctx_watt, b_z)
    logger.debug('w_cz: {} {}'.format(type(w_cz), w_cz.npvalue().shape))
    logger.debug('attention {} {}'.format(type(ctx_watt), ctx_watt.npvalue().shape))
    print (w_cz * ctx_watt).npvalue().shape
    z = dy.logistic(w_cz * ctx_watt + w_hz * s + b_z)    # sigmoid
    r = dy.logistic(w_cr * ctx_watt + w_hr * s + b_r)    # sigmoid
    c_h = w_ch * ctx_watt

    hidden = dy.tanh(dy.dot_product((w_hh * s + b_h), r) + c_h)
    hidden = dy.dot_product(s, z) + dy.dot_product((1. - z), hidden)

    bxm = dy.transpose(bxm)
    if bxm is not None:
        hidden = bxm[:, None] * hidden + (1. - x_m)[:, None] * s.npvalue()
    hshape = hidden.shape     # (src_len, batch)
    hidden_dy = dy.matInput(hshape[0], hshape[1])
    hidden_dy.set(hidden.flatten())
    return hidden_dy

i try to invoke

s = self.convert_dec_state(ctx_with_attend, s, bxm)

to change the s, but failed, because the returned hidden_dy is just a <type '_gdynet.Expression'>, not a <_gdynet.RNNState> which should be returned, i do not how to change hidden_dy into type
<_gdynet.RNNState>, any reply will be appreciated.

[yoavg]

I do not understand exactly what you are trying to achieve and what is the problem. Can you perhaps send a smaller, minimal and complete code fragment that demonstrates it?

[zhang-wen]

sorry about unclear description, i actually want to change the state of lstm layer, please have a look at following simple code:

from _gdynet import *
m = Model()
builder = LSTMBuilder(1, 10, 10, m)
s0 = builder.initial_state()
s = s0.add_input(vecInput(10,))
ws = parameter(m.add_parameters((5, 10)))

s1 = s.add_input(s.output())  # no problem here

new_s = ws * s     # problem here ...
s1 = new_s.add_input(s.output())

there are problems about the last two lines, because i hope the type of new_s is RNNState, can you understand my issue ? if unclear, please let me know, thanks.

[yoavg]

If you are trying to set the state vector within the RNNState, could get_h and set_h help?

[yoavg]

If you are trying to set the state vector within the RNNState, could get_h and set_h help?

[zhang-wen]

exactly, you are right, this is what i want.
but i h() actually return outputs of each layer, i need set_s i think. but there are not set_s function for RNNState yet.

[yoavg]

Yes, this needs to be implemented, also in the C++ part. Should be easy though. Do you feel like sending a pull request?

[zhang-wen]

sure, but i am not very familiar with C++, i can try. and another problems, it is not reasonable that the set_h() function return a RnnState,
i must invoke like this:

new_s = s.set_h([(expr,)])

i think s should change even after invoking:

s.set_h([(expr,)])

but now, after invoking s.set_h(), the state s do not change.

[pmichel31415]

Hey I can try and handle that. If I understand correctly set_h has a common interface for all builders/states, i.e. same for RNN and LSTMs.
Currently for LSTM it only updates h and not c (the cell state). I'll try to change this in the C++ code and add set_s in cython (never used cython before though but will try).

[zhang-wen]

correct, i have looked at the c++ code, RNNBuilder is the father struct of SimpleRNNBuilder, GRUBuilder and LSTMBuilder, the set_h function has been implement in RNNBuilder, but not set_s function found, we just need to implement it in rnn.h and add set_h_impl function for each child struct, i think.
but i am not sure about the storage of s and h, if the layer size is 2,
for some step i:

s_i = [h1, h2, o1, o2]
h_i = [o1, o2]

for step i+1:

s_{i+1} = [h2, h3, o2, o3]
h_{i+1} = [o2, o3]

is my understanding right ?

[yoavg]

Thanks @pmichel31415 , that'd be great.
The cython part should be easy if you just follow the structure of the existing set_h functions in the .pxd and .pyx files.

[zhang-wen]

new_o = vecInput((3))
s1n = s1.set_h((new_o,))

i got a new state s1n, however
s1n.s() will produce following error:

terminate called after throwing an instance of 'std::bad_alloc'
  what():  std::bad_alloc

print s1n.s()[0].npvalue() produce following error:
terminate called after throwing an instance of 'std::invalid_argument'
what(): Bad dimensions for AffineTransform: [{3} {3,3} {3} {3,3} {3} {3,3} {3,3}]

s1n is indeed RNNState type, what is the problem ?

[pmichel31415]

Hey, I think the C++ implementation of set_h was faulty because it didn't update the cell state of the lstm.
I tried to correct this, can you check that it works (cf #127, including last commits)?

For me the following code works now :

from dynet import *
m = Model()
builder = LSTMBuilder(1, 10, 10, m)
s0 = builder.initial_state()
s = s0.add_input(vecInput(10,))
ws = parameter(m.add_parameters((5, 10)))

s1 = s.add_input(s.output())  # no problem here
new_o = vecInput((3))
s1n = s1.set_h((new_o,))
print s1n.h()[0].npvalue()
print s1n.s()[0].npvalue()`

(also set_s should also work now)

[zhang-wen]

ok, i have not tried your code yet, however, i modified the code locally, i try to add some other function. but thank you very much. i am implementing an attention model by using dynet, there are lots of problem, hope we can keep in touch with each other. 👍

[zhang-wen]

and i am trying to learn the c++ and Cython code, maybe i need to change them.