Java Deep Learning Cookbook

Train neural networks for classification, NLP, and reinforcement learning using Deeplearning4j

Rahul Raj

• Publisher: Packt Publishing
• Year: 2019
• ISBN:
  9781788999472
• Edition: 1
• Topic: Computer Science
• Subtopic: Natural Language Processing
• Index: Computer Science

Use Java and Deeplearning4j to build robust, scalable, and highly accurate AI models from scratch

Key Features

• Install and configure Deeplearning4j to implement deep learning models from scratch
• Explore recipes for developing, training, and fine-tuning your neural network models in Java
• Model neural networks using datasets containing images, text, and time-series data

Book Description

Java is one of the most widely used programming languages in the world. With this book, you will see how to perform deep learning using Deeplearning4j (DL4J) – the most popular Java library for training neural networks efficiently.

This book starts by showing you how to install and configure Java and DL4J on your system. You will then gain insights into deep learning basics and use your knowledge to create a deep neural network for binary classification from scratch. As you progress, you will discover how to build a convolutional neural network (CNN) in DL4J, and understand how to construct numeric vectors from text. This deep learning book will also guide you through performing anomaly detection on unsupervised data and help you set up neural networks in distributed systems effectively. In addition to this, you will learn how to import models from Keras and change the configuration in a pre-trained DL4J model. Finally, you will explore benchmarking in DL4J and optimize neural networks for optimal results.

By the end of this book, you will have a clear understanding of how you can use DL4J to build robust deep learning applications in Java.

What you will learn

• Perform data normalization and wrangling using DL4J
• Build deep neural networks using DL4J
• Implement CNNs to solve image classification problems
• Train autoencoders to solve anomaly detection problems using DL4J
• Perform benchmarking and optimization to improve your model's performance
• Implement reinforcement learning for real-world use cases using RL4J
• Leverage the capabilities of DL4J in distributed systems

Who this book is for

If you are a data scientist, machine learning developer, or a deep learning enthusiast who wants to implement deep learning models in Java, this book is for you. Basic understanding of Java programming as well as some experience with machine learning and neural networks is required to get the most out of this book.

Implementing Natural Language Processing

In this chapter, we will discuss word vectors (Word2Vec) and paragraph vectors (Doc2Vec) in DL4J. We will develop a complete running example step by step, covering all the stages, such as ETL, model configuration, training, and evaluation. Word2Vec and Doc2Vec are natural language processing (NLP) implementations in DL4J. It is worth mentioning a little about the bag-of-words algorithm before we talk about Word2Vec.

Bag-of-words is an algorithm that counts the instances of words in documents. This will allow us to perform document classification. Bag of words and Word2Vec are just two different types of text classification. Word2Vec can use a bag of words extracted from a document to create vectors. In addition to these text classification methods, term frequency–inverse document frequency (TF-IDF) can be used to judge the topic/context of the document. In the case of TF-IDF, a score will be calculated for all the words, and word counts will be replaced with this score. TF-IDF is a simple scoring scheme, but word embeddings may be a better choice, as the semantic similarity can be captured by word embedding. Also, if your dataset is small and the context is domain-specific, then bag of words may be a better choice than Word2Vec.

Word2Vec is a two-layer neural network that processes text. It converts the text corpus to vectors.

Note that Word2Vec is not a deep neural network (DNN). It transforms text data into a numerical format that a DNN can understand, making customization possible.

We can even combine Word2Vec with DNNs to serve this purpose. It doesn't train the input words through reconstruction; instead, it trains words using the neighboring words in the corpus.

Doc2Vec (paragraph vectors) associates documents with labels, and is an extension of Word2Vec. Word2Vec tries to correlate words with words, while Doc2Vec (paragraph vectors) correlates words with labels. Once we represent documents in vector formats, we can then use these formats as an input to a supervised learning algorithm to map these vectors to labels.

In this chapter, we will cover the following recipes:

• Reading and loading text data
• Tokenizing data and training the model
• Evaluating the model
• Generating plots from the model
• Saving and reloading the model
• Importing Google News vectors
• Troubleshooting and tuning Word2Vec models
• Using Word2Vec for sentence classification using CNNs
• Using Doc2Vec for document classification

Technical requirements

The examples discussed in this chapter can be found at https://github.com/PacktPublishing/Java-Deep-Learning-Cookbook/tree/master/05_Implementing_NLP/sourceCode/cookbookapp/src/main/java/com/javadeeplearningcookbook/examples.

After cloning our GitHub repository, navigate to the directory called Java-Deep-Learning-Cookbook/05_Implementing_NLP/sourceCode. Then, import the cookbookapp project as a Maven project by importing pom.xml.

To get started with NLP in DL4J, add the following Maven dependency in pom.xml:

<dependency>
 <groupId>org.deeplearning4j</groupId>
 <artifactId>deeplearning4j-nlp</artifactId>
 <version>1.0.0-beta3</version>
 </dependency>

Data requirements

The project directory has a resource folder with the required data for the LineIterator examples:

For CnnWord2VecSentenceClassificationExample or GoogleNewsVectorExampleYou, you can download datasets from the following URLs:

• Google News vector: https://deeplearning4jblob.blob.core.windows.net/resources/wordvectors/GoogleNews-vectors-negative300.bin.gz
  
• IMDB review data: http://ai.stanford.edu/~amaas/data/sentiment/aclImdb_v1.tar.gz
  

Note that IMDB review data needs to be extracted twice in order to get the actual dataset folder.

For the t-Distributed Stochastic Neighbor Embedding (t-SNE) visualization example, the required data (words.txt) can be located in the project root directory itself.

Reading and loading text data

We need to load raw sentences in text format and iterate them using an underlined iterator that serves the purpose. A text corpus can also be subjected to preprocessing, such as lowercase conversion. Stop words can be mentioned while configuring the Word2Vec model. In this recipe, we will extract and load text data from various data-input scenarios.

Getting ready

Select an iterator approach from step 1 to step 5 depending on what kind of data you're looking for and how you want to load it.

How to do it...

1. Create a sentence iterator using BasicLineIterator:

File file = new File("raw_sentences.txt");
SentenceIterator iterator = new BasicLineIterator(file);

For an example, go to https://github.com/PacktPublishing/Java-Deep-Learning-Cookbook/blob/master/05_Implementing_NLP/sourceCode/cookbookapp/src/main/java/com/javadeeplearningcookbook/examples/BasicLineIteratorExample.java.

2. Create a sentence iterator using LineSentenceIterator:

File file = new File("raw_sentences.txt");
SentenceIterator iterator = new LineSentenceIterator(file);

For an example, go to https://github.com/PacktPublishing/Java-Deep-Learning-Cookbook/blob/master/05_Implementing_NLP/sourceCode/cookbookapp/src/main/java/com/javadeeplearningcookbook/examples/LineSentenceIteratorExample.java.

3. Create a sentence iterator using CollectionSentenceIterator:
   

List<String> sentences= Arrays.asList("sample text", "sample text", "sample text");
SentenceIterator iter = new Collect...

Table of contents

• Title Page
• Copyright and Credits
• Dedication
• About Packt
• Contributors
• Preface
• Introduction to Deep Learning in Java
• Data Extraction, Transformation, and Loading
• Building Deep Neural Networks for Binary Classification
• Building Convolutional Neural Networks
• Implementing Natural Language Processing
• Constructing an LSTM Network for Time Series
• Constructing an LSTM Neural Network for Sequence Classification
• Performing Anomaly Detection on Unsupervised Data
• Using RL4J for Reinforcement Learning
• Developing Applications in a Distributed Environment
• Applying Transfer Learning to Network Models
• Benchmarking and Neural Network Optimization
• Other Books You May Enjoy