# The Keras Functional API

The Keras API makes creating deep learning models fast and easy. While the sequential API allows you to create models layer-by-layer it is limited in that it does not allow you to create models that share layers or have multiple inputs or outputs. The functional API is an alternate way of creating models that offers a lot more flexibility, including creating more complex models.

## Setup

The following installs pydot and graphviz. You can use brew install graphviz on Mac as well.

We need Tensorflow, of course. The clear_session ensures that the state is reset (e,,g, when using this code in a notebook):

## Example

The advantages of the functional API are:

• re-use of layers across multiple models
• enabling models with complex (non-linear) topologies
• enabling models with multiple inputs and/or outputs

Let’s see how a basic model can created with the functional API:

The input layer defines how the 16-dimensional vectors are digested:

The batch size is always omitted, we only specify the shape of each sample. You can check the input layer like so:

The functional aspect shows when adding a new layer which calls the input layer:

The “layer call” action is like drawing an arrow from “inputs” to this layer we created.
We’re “passing” the inputs to the dense layer, and out we get x.

Let’s add a few more layers to our graph of layers:

At this point, we can create a Model by specifying its inputs and outputs in the graph of layers:

The model’s summary can be checked:

We can also plot the model as a graph (this is where graphviz is used):

And optionally display the input and output shapes of each layer in the plotted graph:

## Training, evaluation, serialization and inference

The functional API only affects how you define your mode. All other aspects remain precisely the same. Including serialization. For example, this saves and loads the model and it would be the same for a model defined via the sequence API:

## Using the same graph of layers to define multiple models

In the functional API, models are created by specifying their inputs
and outputs in a graph of layers. That means that a single graph of layers
can be used to generate multiple models.

In the example below, we use the same stack of layers to instantiate two models:
an encoder model that turns image inputs into 16-dimensional vectors,
and an end-to-end autoencoder model for training.

Note that we make the decoding architecture strictly symmetrical to the encoding architecture,
so that we get an output shape that is the same as the input shape (28, 28, 1).
The reverse of a Conv2D layer is a Conv2DTranspose layer, and the reverse of a MaxPooling2D
layer is an UpSampling2D layer.

## All models are callable, just like layers

You can treat any model as if it were a layer, by calling it on an Input or on the output of another layer.
Note that by calling a model you aren’t just reusing the architecture of the model, you’re also reusing its weights.

Let’s see this in action. Here’s a different take on the autoencoder example that creates an encoder model, a decoder model,
and chain them in two calls to obtain the autoencoder model:

As you can see, model can be nested: a model can contain submodels (since a model is just like a layer).

A common use case for model nesting is ensembling.
As an example, here’s how to ensemble a set of models into a single model that averages their predictions:

## Manipulating complex graph topologies

### Models with multiple inputs and outputs

The functional API makes it easy to manipulate multiple inputs and outputs.
This cannot be handled with the Sequential API.

Here’s a simple example. Let’s say you’re building a system for ranking custom issue tickets by priority and routing them to the right department.

You model will have 3 inputs:

• Title of the ticket (text input)
• Text body of the ticket (text input)
• Any tags added by the user (categorical input)

It will have two outputs:

• Priority score between 0 and 1 (scalar sigmoid output)
• The department that should handle the ticket (softmax output over the set of departments)

Let’s built this model in a few lines with the Functional API.

Let’s plot the model:

When compiling this model, we can assign different losses to each output.
You can even assign different weights to each loss, to modulate their
contribution to the total training loss.

Since we gave names to our output layers, we could also specify the loss like this:

We can train the model by passing lists of Numpy arrays of inputs and targets:

When calling fit with a Dataset object, it should yield either a
tuple of lists like ([title_data, body_data, tags_data], [priority_targets, dept_targets])
or a tuple of dictionaries like
({'title': title_data, 'body': body_data, 'tags': tags_data}, {'priority': priority_targets, 'department': dept_targets}).

## A toy resnet model

In addition to models with multiple inputs and outputs, the Functional API makes it easy to manipulate non-linear connectivity topologies. That is, models where layers are not connected sequentially. This also cannot be handled with the Sequential API.

A common use case for this is residual connections.

Let’s build a toy ResNet model for CIFAR10 to demonstrate this.

Let’s plot the model:

It can be trained like any other neural topology:

## Sharing layers

Another good use for the functional API are models that use shared layers. Shared layers are layer instances that get reused multiple times in a same model: they learn features that correspond to multiple paths in the graph-of-layers.

Shared layers are often used to encode inputs that come from similar spaces (say, two different pieces of text that feature similar vocabulary), since they enable sharing of information across these different inputs, and they make it possible to train such a model on less data. If a given word is seen in one of the inputs, that will benefit the processing of all inputs that go through the shared layer.

To share a layer in the Functional API, just call the same layer instance multiple times. For instance, here’s an Embedding layer shared across two different text inputs:

## Extracting and reusing nodes in the graph of layers

Because the graph of layers you are manipulating in the Functional API is a static datastructure, it can be accessed and inspected. This is how we are able to plot Functional models as images, for instance.

This also means that we can access the activations of intermediate layers (“nodes” in the graph) and reuse them elsewhere. Let’s look at an example. This is a VGG19 model with weights pre-trained on ImageNet:

And these are the intermediate activations of the model, obtained by querying the graph datastructure:

We can use these features to create a new feature-extraction model, that returns the values of the intermediate layer activations — and we can do all of this in 3 lines.

This comes in handy when implementing neural style transfer, among other things.

## Extending the API by writing custom layers

Keras has a wide range of layers but if you don’t find what you need, it’s easy to extend the API by creating your own layers.

All layers subclass the Layer class and implement:
– A call method, that specifies the computation done by the layer.
– A build method, that creates the weights of the layer (note that this is just a style convention; you could create weights in __init__ as well).

Here’s a simple implementation of a Dense layer:

If you want your custom layer to support serialization, you should also define a get_config method,
that returns the constructor arguments of the layer instance:

Optionally, you could also implement the classmethod from_config(cls, config), which is in charge of recreating a layer instance given its config dictionary. The default implementation of from_config is:

## When to use the Functional API

How to decide whether to use the Functional API to create a new model, or just subclass the Model class directly? In general, the Functional API is higher-level, easier & safer to use, and has a number of features that subclassed Models do not support. However, Model subclassing gives you greater flexibility when creating models that are not easily expressible as directed acyclic graphs of layers (for instance, you could not implement a Tree-RNN with the Functional API, you would have to subclass Model directly).

Here are the strengths of the Functional API:

The properties listed below are all true for Sequential models as well (which are also data structures), but they aren’t true for subclassed models (which are Python bytecode, not data structures).

#### It is less verbose.

No super(MyClass, self).__init__(...), no def call(self, ...):, etc.

Compare:

With the subclassed version:

#### It validates your model while you’re defining it.

In the Functional API, your input specification (shape and dtype) is created in advance (via Input), and every time you call a layer, the layer checks that the specification passed to it matches its assumptions, and it will raise a helpful error message if not.

This guarantees that any model you can build with the Functional API will run. All debugging (other than convergence-related debugging) will happen statically during the model construction, and not at execution time. This is similar to typechecking in a compiler.

#### Your Functional model is plottable and inspectable.

You can plot the model as a graph, and you can easily access intermediate nodes in this graph — for instance, to extract and reuse the activations of intermediate layers, as we saw in a previous example:

#### Your Functional model can be serialized or cloned.

Because a Functional model is a data structure rather than a piece of code, it is safely serializable and can be saved as a single file that allows you to recreate the exact same model without having access to any of the original code.

### Here are the weaknesses of the Functional API:

#### It does not support dynamic architectures.

The Functional API treats models as DAGs of layers. This is true for most deep learning architectures, but not all: for instance, recursive networks or Tree RNNs do not follow this assumption and cannot be implemented in the Functional API.

#### Sometimes, you just need to write everything from scratch.

When writing advanced achitectures, you may want to do things that are outside the scope of “defining a DAG of layers”: for instance, you may want to expose multiple custom training and inference methods on your model instance. This requires subclassing.

## Mix-and-matching different API styles

Choosing between the Functional API or Model subclassing isn’t a binary decision that restricts you to one category of models. All models in the tf.keras API can interact with each, whether they’re Sequential models, Functional models, or subclassed Models/Layers written from scratch.

You can always use a Functional model or Sequential model as part of a subclassed Model/Layer:

Inversely, you can use any subclassed Layer or Model in the Functional API as long as it implements a call method that follows one of the following patterns:

• call(self, inputs, **kwargs) where inputs is a tensor or a nested structure of tensors (e.g. a list of tensors), and where **kwargs are non-tensor arguments (non-inputs).
• call(self, inputs, training=None, **kwargs) where training is a boolean indicating whether the layer should behave in training mode and inference mode.
• call(self, inputs, mask=None, **kwargs) where mask is a boolean mask tensor (useful for RNNs, for instance).
• call(self, inputs, training=None, mask=None, **kwargs) — of course you can have both masking and training-specific behavior at the same time.

In addition, if you implement the get_config method on your custom Layer or Model, the Functional models you create with it will still be serializable and clonable.

Here’s a quick example where we use a custom RNN written from scratch in a Functional model:

Tags: