Tensorflow

---

---

Tensorflow is an open sourse software library for numerical computation. It’s designed to work with Deep Learning (DL) and Machine Learning (ML) problems. However the system is general enough to deal with problems from other domains as well.

It provides High Level APIs and Low Level APIs. The High Level APIs is useful to try already build models but it is difficult to customize such models. On the other side, by using the Low Level API the user has freedom to easily build its own models,train and run them.

---

Tensorflow Main Concepts

---

---

The main component of Tensorflow is the tensor. Everything in Tensorflow works around tensors.

A tensor consists of a set of primitive values shaped into an array of any number of dimensions. It has two main properties:

• rank: number of dimensions
• shape: tuple of integers specifying the array’s length along each dimension

In a nutshell, any Tensorflow program consist into a serie of operation that consume and produce new tensors.

Tensorflow workflow is divided into two main sections/parts:

1. Building a computational graph
2. Running the computational graph

---

Graph

The graph is just a series of operations that work over tensors arranged into a graph. The graph is composed by two type of objects:

• Operations: Nodes of the graph. Consume and produce tensors
• Tensors: Edges of the graph

---

Tensors

There are different types of tensors:

• tf.Variable: This is the only tensor that is not immutable (i.e., its value can change)
• tf.constant: constant tensor
• tf.placeholder: placeholder for a tensor that will be always fed. It will produce an error if it is evaluated. It must be fed using the feed_dict optional argument when evaluated.
• tf.SparseTensor: sparse tensor represented as three separeate dense tensors: indices, values and dense_shape

---

Operations

There are a wide range of operation provided by Tensorflow:

• add
• substract
• multiply
• negative
• abs
• conv2d
• relu
• max_pool
• …

Although we can aplly custom functions to the tensors it is important and recommended to use the operations provided by the Tensorflow library. Tensorflow operations are implemented efficiently taking advantadge of how the framework works in such a way that when we execute the graph the operation are arranged and ran efficiently.

---

Building the graph

---

At time of building the graph what we do is to design what or model is gonna do, which calculations it will perform, etc. It’s like a blueprint, we are just designing the workflow of our model but not running it.

Thus, we don’t need values at designing time.

As an example we could design a graph that computes the equation:$$2x^2+3$$.

The tensorflow code to build that graph is:

import tensorflow as tf

two = tf.constant(2,tf.float32,name='2')
three = tf.constant(3,tf.float32,name='3')
x = tf.placeholder(tf.float32,shape=(),name='x')
x_x = tf.multiply(x,x,name='xx')
x_x_2 = tf.multiply(x_x,two,name='2xx')
y = tf.add(x_x_2,three,name='y')

If we want to see the resulting graph Tensorflow provides a tool called Tensorboard. Tensorboard allows us to see the designed graph:

This graph is just a design of what we want to compute but nothing has been computed yet. Tensorflow will arrange efficiently all the computations when we evaluate the graph.

Once we have built the graph that describes our model its time to execute/evaluate it.

---

Executing the graph

---

In order to evaluate the tensors we need to instantiate a Tensorflow Session (tf.Session). If we look at the code above we can see that we have different tensors and ops. To evaluate them we can pass to the session the desired tensor/operation to evaluate. The Tensorflow inference engine will manage how to make the computations internally.

We create the session:

sess = tf.Session()

For example if we want to evaluate the tensor two we will write:

print(two)
result = sess.run(two)
print('result: ', result)

That will output:

Tensor("2:0", shape=(), dtype=float32)
('result: ', 2.0)

Here we can see that two is a Tensor and its value when evaluated is the float 2.0. Moreover if we evaluate y:

print(y)
result = sess.run(y)
print('result: ', result)

The output is:

Tensor("y:0", shape=(), dtype=float32)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 895, in run
    run_metadata_ptr)
...

We can see that y is a tensor. However, when we evaluate y an exception is raised. That is because we are evaluating a tf.placehodler that has not been fed. We need to initialize it with the optional argument feed_dict.

result = sess.run(y,feed_dict={x:3})
print('result: ', result)

Now that we gave the value of 3 to the x tensor the result of the equation is prompt:

('result: ', 21.0)

---

Deep Learning Approach

---

---

In order to build and train Deep Neural Networks (DNN) the steps to follow are essentially the same listed above. Furthermore, the design of the graph and the tensors and operations to use are more complex. But the main concepts remain the same!

In this hands-on we are going to provide encapsulated functions that will make the DNN building code easier and we will focus into the feeding phase, that is how we build the input pipeline to feed the DNN.

When we have built a DNN we need to feed them in order to train it. DNN are built for a variety of tasks. Taking as an example a Convolutional Neural Network (CNN) that classifies images it learns during the training from the examples given from our dataset. Tensorflow provides the Dataset API to build our Dataset and feed the NN with its examples.

Dataset API

---

---

In the Tensorflow context a Dataset is composed by a serie of elements with the same structure. An element is composed by one or more tensors called components having each of them its own type and shape.

1. In order to define an input pipeline to our model (the Dataset) we need to start defining a source from which build the Dataset. The nature of the source could be different:
   • data in memory
   • TFrecord format files in disk
   • csv files
   • txt files
   • …
2. However, we can further create a new Dataset by applying one of more transformations involving different datasets (e.g., join two datasets, …).
3. Finally, in order to consume the values of the Dataset to be fed into the model an Iterator must be used.

We are going to see two main cases:

• We can load the whole dataset in memory
• Our dataset does not fit in memory

---

Dataset fits in memory

---

If our dataset fits in memory means that we will have all the elements from the dataset in some sort structrure:

• List of images: [[128,128,3],[128,128,3],…]
• Dict: {‘name’: [‘Pep’,’Maria’,…],’age’: [14,29,…],’height’: [166,170,…]}

Let’s say that we have our features in a variable called features (does not matter if it is a list or a dictionary) and the corresponding labels to that features into a variable called labels. Notice that their dimensions must be the same (if we have features for 10 examples we need to have 10 labels as well).

Then to build our Dataset we just need to use the function slice_from_tensors:

dataset = tf.data.Dataset.from_tensor_slices((features,labels))

Now we have our dataset source. If we have two datasets dataset_1 and dataset_2 containing instances with the same structure we can build a new dataset by applying a transformation:

new_dataset = tf.data.Dataset.zip((dataset_1,dataset_2))

Tensorflow provides a wide set of transformation to apply to the Dataset instances.

---

Dataset does not fit in memory

---

Usually in DL the bigger the data the better the training of the DNN. Thus, often it’s not possible to load the whole dataset into memory. We will focus on that case.

In those cases we need to have our files stored at disk in some sort of format. Tensorflow recommends to use its own binary format called TFRecord. Another common case is that we have the raw images stored in our filesystem and a txt or csv file describing those images (name, label, etc.).

We assume that we have a csv file where each line describes one instance of our dataset:

Name,path,label
image_1,/path/to/image/image_1.png,0
image_2,/path/to/image/image_2.png,0
image_3,/path/to/image/image_3.png,0
image_4,/path/to/image/image_4.png,0
...

Tensorflow provides a subclass of Dataset called TextLineDataset to deal with this kind of input. Thus, the code will look like:

dataset = tf.data.TextLineDataset(csv_path).skip(1)

At this point we just have built a Dataset object being each line of the csv file an element. Notice that we skipped the first line cause is just the header. We want however a Dataset where the elements are composed just by two components: the image and the label. To accomplish that we can apply a transformation to the current Dataset. The Dataset API provides a map function in order to apply an specific function to each of its elements. In our case we will apply a parsing function.

dataset = dataset.map(_parse_line)

So the function _parse_line will be something like:

def _parse_line(line):
    # Decode the line into its fields
    image_name,image_path,label = tf.decode_csv(line, record_defaults=[tf.string, tf.string, tf.int32])
    image = tf.decode_image(tf.read_file(image_path), channels=3)
    return image, label

Thus, by applying _parse_line the resulting dataset will be composed by an image and the corresponding label. You may think that if we do that to every image our Dataset will not fit in memory. The key point here is how we consume tha data from the Dataset.

As commented before, to consume data from the Dataset we need to use Iterators.

---

Iterators

---

Tensorflow provides 4 iterators:

• one-shot: suited for most of the cases
• initializable: when we have parameters involved in the creation of the Dataset
• reinitializable: allows to be used with multiple Dataset’s
• feedable: allows more parametrization

We will focus on the one-shot iterator since it is suitable for most of the cases. So to create the iterator we just:

iterator = dataset.make_one_shot_iterator()
next_element = iterator.get_next()

Above we have created the iterator and one operation that when evaluated in the current Session will pop the next element. Thus, when we evaluate next_element we will obtain a Tensor with the image and the label*. However, when we train a DNN we usually use batches of elements instead of a single element.

The Dataset API provides a serie of functionalities to deal with that situation. We are going to see some of them:

• shuffle: shuffles the dataset in such a way that the order in which the DNN sees the elements is randomized
• repeat: allows us to walk through the Dataset multiple times
• batch: batches the Dataset in such a way that when we evaluate next_element we will obtain a batch instead of a single instance

For example, if we want to train our network for 10 epochs (sees the whole Dataset 10 times),with the Datased shuffled and with batces of 32 images the code will look like:

dataset = dataset.shuffle()
dataset = dataset.batch(32)
dataset = dataset.repeat(10)
iterator = dataset.make_one_shot_iterator()
next_element = iterator.get_next()

That will make the work. The only remaining thing is to take care of how we design the input of our DNN.

In the given example where we design a graph that computes$$y=2x^2+3$$we saw that the input placeholder is just a float number so its shape its simple. Now, we have as an input a tensor with 32 instances of images ([32,128,128,3]) and the 32 labels ([32,1]). Thus we have to prepare the placeholders for that data:

images_placeholder = tf.placeholder(tf.float32, shape=(32,128,128,3))
labels_placeholder = tf.placeholder(tf.int32, shape=(32))

And feed the training with the feed_dict argument:

# we evaluate the next element
images,labels = sess.run(next_element)
# we built the dict that we will use to feed the input
feed_dict={images_placeholder:images,labels_placeholder:labels}
# we will have a Tensorflow operation for the training called train_op
sess.run(train_op,feed_dict=feed_dict)

We will run this code as many times as needed until we consume the whole Dataset (10 times in this case). We will know thay by catching the exception OutOfRange that is raised when the Dataset is consumed.

 

 

 

Tensorboard

---

---

In order to visualize the graph that we designed through Tensorflow code we just have to add the following line:

tf.summary.FileWriter('/path/to/folder',sess.graph)

Notice that we need to have initialized a Session (sess in this case). It is important to put this line below the code where we design our computational graph. That will produce a file called eventsXXXX in the specified folder.
After that, we open a terminal, navigate to the specified folder and run tensorboard:

cd /path/to/folder
tensorboard --logdir ./

Then we open a browser and navigate to localhost:6006.

 

 

 

Playground

---

---

Inhere we provide the reader with some code snipets to play around Tensorflow. Such snipets lack of some parts of the code that have to be fullfilled by the reader.

Those parts are the related to the building phase of the Dataset and how the NN has to be fed. Although we provide a pre-defined NN to work with there is also another blank space where the user can define its own NN designs.

---

Files to Download

---

• Dataset Creation Snippet
• dlho2018.py
• tf_layers.py
• training.py
• evaluating.py

Everything

---

Dummy Dataset Creation

---

In order to build a dataset we provide dataset_creation.py in order to generate an artificial Dataset with the desired number of instances. This code allows to generate images containing a square or a triangle.

In order to use the snippet we call it giving four parameters:

• shape: {square,triangle} to define the shape to draw
• num_images: how many images we want to generate
• output_directory: where to store the images
• size: size of the images (squared images)

Example:

python dataset_creation.py --shape square --num_images 10000 --output_directory ./squares --size 128

That will create a folder in the current directory called squares containing 10000 images of 128x128x3 of squares. Moreover, a csv describing the generated images is stored as well.
We do the same for the triangles. We join the csv files:

cat squares/info.cv triangles/info.csv > dataset.csv

Thus, we will have the csv file describing our dataset.

---

Provided Code

---

We have 5 files:

• dlho2018.py: provides some functions needed to facilitate the work of the hands-on
• dlho2018_nns.py: file where to put the different NN designs. It contains one pre-defined
• tf_layers.py: file containing the code of the layers
• training.py: file with the code to train a NN (NEED TO BE COMPLETED)
• evaluating.py: filw with the code to evaluate the trained NN (NEED TO BE COMPLETED)

training.py and evaluating.py are partially implemented. They need to be fullfilled with some code explained before (Tensorlow tab). Moreover, the dlho2018_nn.py contains an already defined NN but we provide some abstraction to let the reader build its own models. The layers provided (implemented in tf_layers) are:

• j_fc_layer: fully connected layer
• j_conv_layer: convolutional layer
• j_dropout_layer: dropout layer
• j_logits: logits layer
• j_flatten_layer: flatten layer

Let’s check the existing NN:

flat = j_flatten_layer(images,'flat')
fc1 = j_fc_layer(flat,128,'fc1')
fc2 = j_fc_layer(fc1,32,'fc2')
fc3 = j_fc_layer(fc2,64,'fc3')
logits = j_logits(fc3,2,'logits')

Notice that by using this code we just need to write the layer to add taking into account that the input of each of them is the output of the previous one. However the first one is the images given to the function and the last one has to be logits which is the one used to compute the classification output.

 

 

 

Setup

---

---

In order to setup the environment is recommended to use a virtual environment. We create it through the command virtualenv

virtualenv env
source env/bin/activate

Now we are logged to the environment that we just created. From now on, everything that we intall will no affect our system, it will be installed just inside the virtual environment.

We install tensorflow and Pillow.

pip install tensorflow
pip install Pillow

Notice that we need to use the code that we will provide from the terminal where we have logged into the virtual environment.