Gated recurrent unit（ GRU） is an improved long short term memory model（ LSTM） architecture

it is effective to improve training time-consuming features of LSTM. Performance of some activation functions such as sigmoid tanh

rectified linear units（ ReLU） are compared and researched on the basis of GRU architecture and analyze their advantages and disadvantages in detail. Propose a novel activation function named tanh linear unit（ TLU）. The experiment shows that the new activation function can not only speed up training speed of deep neural networks

but also effectively reduce training error.