"InvalidArgumentError (see above for traceback): indices[6,25] = 25 is not in [0, 25)"

[lisi-github]

Hi policeme, thank you for sharing the code!
But i have some trouble in running the code，while the vocab size of the training equals that in predicting. And other parameters are the same with yours. Did you have the same problem with me. I'm pending for your solution.

Full Error Trace is as follows:
"""""

hparams

2019-09-01 19:12:35.633634: I T:\src\github\tensorflow\tensorflow\core\platform\cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
Traceback (most recent call last):
File "D:\python\python3\lib\site-packages\tensorflow\python\client\session.py", line 1278, in _do_call
return fn(*args)
File "D:\python\python3\lib\site-packages\tensorflow\python\client\session.py", line 1263, in _run_fn
options, feed_dict, fetch_list, target_list, run_metadata)
File "D:\python\python3\lib\site-packages\tensorflow\python\client\session.py", line 1350, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.InvalidArgumentError: indices[6,25] = 25 is not in [0, 25)
[[Node: predict/decode_5/positional_encoding/embedding_lookup = GatherV2[Taxis=DT_INT32, Tindices=DT_INT32, Tparams=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"](predict/decode_24/positional_encoding/Const, predict/decode_5/positional_encoding/Tile, predict/decode_5/positional_encoding/embedding_lookup/axis)]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "D:/work/nlg/code/transformer-nlg-1/train.py", line 75, in
train()
File "D:/work/nlg/code/transformer-nlg-1/train.py", line 40, in train
m.encoder_input: article_id, m.decoder_input: title_id, m.decoder_output: target_id
File "D:\python\python3\lib\site-packages\tensorflow\python\client\session.py", line 877, in run
run_metadata_ptr)
File "D:\python\python3\lib\site-packages\tensorflow\python\client\session.py", line 1100, in _run
feed_dict_tensor, options, run_metadata)
File "D:\python\python3\lib\site-packages\tensorflow\python\client\session.py", line 1272, in _do_run
run_metadata)
File "D:\python\python3\lib\site-packages\tensorflow\python\client\session.py", line 1291, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: indices[6,25] = 25 is not in [0, 25)
[[Node: predict/decode_5/positional_encoding/embedding_lookup = GatherV2[Taxis=DT_INT32, Tindices=DT_INT32, Tparams=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"](predict/decode_24/positional_encoding/Const, predict/decode_5/positional_encoding/Tile, predict/decode_5/positional_encoding/embedding_lookup/axis)]]

Caused by op 'predict/decode_5/positional_encoding/embedding_lookup', defined at:
File "D:/work/nlg/code/transformer-nlg-1/train.py", line 75, in
train()
File "D:/work/nlg/code/transformer-nlg-1/train.py", line 34, in train
m = Transformer(hp)
File "D:\work\nlg\code\transformer-nlg-1\model.py", line 49, in init
logits = self.decode(self.encoder_input, dec_input, memory, False)
File "D:\work\nlg\code\transformer-nlg-1\model.py", line 134, in decode
dec += positional_encoding(dec, self.hp.maxlen2)
File "D:\work\nlg\code\transformer-nlg-1\modules.py", line 278, in positional_encoding
outputs = tf.nn.embedding_lookup(position_enc, position_ind)
File "D:\python\python3\lib\site-packages\tensorflow\python\ops\embedding_ops.py", line 310, in embedding_lookup
transform_fn=None)
File "D:\python\python3\lib\site-packages\tensorflow\python\ops\embedding_ops.py", line 133, in _embedding_lookup_and_transform
result = _clip(array_ops.gather(params[0], ids, name=name),
File "D:\python\python3\lib\site-packages\tensorflow\python\ops\array_ops.py", line 2659, in gather
return gen_array_ops.gather_v2(params, indices, axis, name=name)
File "D:\python\python3\lib\site-packages\tensorflow\python\ops\gen_array_ops.py", line 3761, in gather_v2
"GatherV2", params=params, indices=indices, axis=axis, name=name)
File "D:\python\python3\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "D:\python\python3\lib\site-packages\tensorflow\python\util\deprecation.py", line 454, in new_func
return func(*args, **kwargs)
File "D:\python\python3\lib\site-packages\tensorflow\python\framework\ops.py", line 3155, in create_op
op_def=op_def)
File "D:\python\python3\lib\site-packages\tensorflow\python\framework\ops.py", line 1717, in init
self._traceback = tf_stack.extract_stack()

InvalidArgumentError (see above for traceback): indices[6,25] = 25 is not in [0, 25)
[[Node: predict/decode_5/positional_encoding/embedding_lookup = GatherV2[Taxis=DT_INT32, Tindices=DT_INT32, Tparams=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"](predict/decode_24/positional_encoding/Const, predict/decode_5/positional_encoding/Tile, predict/decode_5/positional_encoding/embedding_lookup/axis)]]

Process finished with exit code 1

[lisi-github]

I have found my fault ! But there is another error.
I'm confused with your "pred.py" code:
(1) _init_graph:
self.memory = self.model.encode(self.xs, False)[0]
self.logits = self.model.decode(self.xs, self.ys, self.memory, False)[0]
(2) predict:
memory = self.sess.run(self.memory, feed_dict={self.input_x: [input_x]})
return self.bs.search(self.sess, input_x, memory[0])

when i running the code ,there is always an error as:

ValueError: Layer dense_634 expects 1 inputs, but it received 4 input tensors. Inputs received: [<tf.Tensor 'predict/beam_search/strided_slice_3:0' shape=(512,) dtype=float32>, <tf.Tensor 'predict/beam_search/strided_slice_3:0' shape=(512,) dtype=float32>, <tf.Tensor 'predict/beam_search/strided_slice_3:0' shape=(512,) dtype=float32>, <tf.Tensor 'predict/beam_search/strided_slice_3:0' shape=(512,) dtype=float32>]

[juvien]

Excuse me,How could i predict the test.csv?

[lisi-github]

Excuse me,How could i predict the test.csv?

normally just give the decoder_input a start flag

[juvien]

oh. . like this?
python test.py --logdir myLog --batch_size 32 --test myTest ? could you give a specific sample，thanks!