The first entry is here.
The second entry is here.

 In the 1st and 2nd entries, we showed that the authors of the specification could be estimated with high precision in the neural network. However, in the 1st and 2nd we only changed the input data. In the development of the neural network, the structure of the neural network is considered to be a big theme. When we develop the neural network, the structure of the neural network such as the number of layers and the number of nodes will be the subject of consideration first. But as is well known, it is important to change the initial value and use various optimization methods (The first and the second were all SGD). Various methods are conceivable, but first of all it is possible to make the layer deep. Since there are only 18 items (the number of input nodes) in this development, it can be anticipated that a large number of layers will not be required.

 For the time being, we learned the hidden layer as two layers (13 nodes, 8 nodes) by using training data and test data of “Both habit of document expression and Habit of flow of logic”. The result is 98% correct answer rate. About 90 cases of test data it means 2 incorrect answers. It is a very high accuracy rate. For this matter, we can say that a deep neural network is not necessary. It is understandable. We have too few input nodes. The sample was inappropriate to simulate the development of the deep neural network. Well, even though the sample was inappropriate, I think that the high accuracy rate is accomplished because the input data was suitable for estimation of the author.

 Initially, I wanted to improve the rate of correct answers gradually while using modern technic like a self-encoder (ten years have passed since self-encoder became a topic). In addition, not only simple neural networks, but also the complicated structure of the network, such as inputting the output of one node to the other node in the same layer, there are various ingenuity in the structure of the network. However, judging from the results so far, it seems that there is not enough sample to be able to judge the structure of the neural network and the usefulness of various techniques in the current sample. Although there may be room for further improvement, even if the correct answer rate becomes 100%, I cannot conclude the reason of high accuracy rate is achieved by the structure change of the neural network. It can be just a coincidence. Therefore, I would like to try development with a self-encoder etc. in another project in the future (I think that it is good to experience the development with word 2vec and Q learning at present).

 Now, since I first experienced the development process related to the neural network in the 1st to 3rd entries, I would like to think about the patenting of the invention which can occur in the development process next time.

The fourth entry is here.