Abstract:
Having a patent document, associating discrete semantic annotations has emerged as
an exciting research area. Text annotation aids patent practitioners, such as; examiners
and attorneys in quickly identifying the important arguments of any invention, allowing
for quick marking of the patent text. In the manual patent analysis process, recognizing
the semantic information by marking paragraphs is commonly used to improve readabil ity. To overcome this time-consuming and laborious task, we have used Deep Learning
models on the dataset of size 150k patent samples. Prior researchers have used differ ent Machine Learning models but we approached this problem as deep learning task
and investigate the merits of Deep learning models for highlighting patent paragraphs.
To raise the level of performance of the base models of Patent Sentiment Analysis, we
trained five different deep learning-based models on the Patent dataset: Simple Feed forward Neural Network, Long Short-Term Memory (LSTM), BERT, RoBERTa, and
DistilBERT. We have trained different Feed Forward Neural Networks (FFNN) with
different hyperparameters and logged our precision, recall and f1-score. We used pre trained GloVe embeddings as wordembeddings. Sentence embeddings were calculated
as the element-wise mean of all word vectors of that sentence. We have trained FFNN
for 500 epochs and log the loss and test the accuracy of the model after each epoch. The
loss and accuracy graphs are smoothed using exponential moving averages so we can
see the overall trend of the model. This work uses Deep Learning to assist patent prac titioners in automatically highlighting semantic information and creating a sustainable
and efficient patent analysis.
