Gist

The TF estimators are high-level pre-made blocks which help you perform quickly some learning tasks:

  • you can run Estimator-based models on a local host or on a distributed multi-server environment without changing your model. Furthermore, you can run Estimator-based models on CPUs, GPUs, or TPUs without recoding your model.
  • estimators simplify sharing implementations between model developers.
  • you can develop a state of the art model with high-level intuitive code. In short, it is generally much easier to create models with Estimators than with the low-level TensorFlow APIs.
  • estimators are themselves built on tf.keras.layers, which simplifies customization.
  • estimators build the graph for you.
  • estimators provide a safe distributed training loop that controls how and when to:
    • build the graph
    • initialize variables
    • load data
    • handle exceptions
    • create checkpoint files and recover from failures
    • save summaries for TensorBoard

When writing an application with Estimators, you have to separate the data input pipeline from the model, which simplifies experiments with different data sets.

Below is a straightforward example of a linear regression with estimators. It highlights the general workflow.

You need of course to import some namespaces

import tensorflow as tf
tf.enable_eager_execution()
assert tf.executing_eagerly()
import tensorflow.contrib.eager as tfe
# too much info otherwise
tf.logging.set_verbosity(tf.logging.ERROR)

import numpy as np
import matplotlib.pyplot as plt
import seaborn as sea
sea.set()
sea.set

%matplotlib inline

and define some data

X = np.arange(0,100)
np.random.shuffle(X)
# set a linear y = ax+b
y = 3*X +2 + np.random.normal(size=[100])

The feature columns go into the definition of the estimator:

column =  tf.feature_column.numeric_column('x')
lin_reg = tf.estimator.LinearRegressor(feature_columns=[column])

and the usage of input functions allows you to decouple input from model

train_input = tf.estimator.inputs.numpy_input_fn(
    x={"x": X},
    y=y, shuffle=False,num_epochs=2500,batch_size=1)
lin_reg.train(train_input)

To predict things you use input functions as well

predict_input = tf.estimator.inputs.numpy_input_fn(
     x={"x": X},
     num_epochs=1, shuffle=False)
results = list(lin_reg.predict(predict_input))

fig, ax = plt.subplots()
sea.regplot(X,y, ax=ax)
ax2 = ax.twinx()
sea.regplot(X, np.array([x["predictions"][0] for x in list(results) ]) , ax=ax2, color='r')

and it’s clear that estimators are the fastest road to results in terms of API mechanics. Next to Keras and the low-level API the estimators is a third API layer in TensorFlow. Whether this approach is the best depends on your context:

  • the checkpoints mechanism related to model restoration should, in particular, be considered in contrast to the non-estimator approaches
  • the (dis)advantages of creating custom estimators 
  • the flexibility of Keras versus the flexibility of estimator pipelines (also in the context of serving models)

Estimators definitely turn TensorFlow into something close to Scikit-Learn but whether you should use TF as a replacement for sklearn. Well, probably not.

The whole estimator stack is also available in R and works similarly, maybe even easier:

library(tfestimators)

response <- function() "Species"
features <- function() setdiff(names(iris), response())

# split into train, test datasets
set.seed(123)
partitions <- modelr::resample_partition(iris, c(test = 0.2, train = 0.8))
iris_train <- as.data.frame(partitions$train)
iris_test  <- as.data.frame(partitions$test)

# construct feature columns
feature_columns <- feature_columns(
  column_numeric(features())
)

# construct classifier
classifier <- dnn_classifier(
  feature_columns = feature_columns,
  hidden_units = c(10, 20, 10),
  n_classes = 3
)

# construct input function 
iris_input_fn <- function(data) {
  input_fn(data, features = features(), response = response())
}

# train classifier with training dataset
train(classifier, input_fn = iris_input_fn(iris_train))

# valuate with test dataset
predictions <- predict(classifier, input_fn = iris_input_fn(iris_test))
evaluation <- evaluate(classifier, input_fn = iris_input_fn(iris_test))

The TensorFlow API for R is actually very nicely documented and complements well the Python docs.