TF LinearRegression estimator – Orbifold Consulting

The TF estimators are high-level pre-made blocks which help you perform quickly some learning tasks:

you can run Estimator-based models on a local host or on a distributed multi-server environment without changing your model. Furthermore, you can run Estimator-based models on CPUs, GPUs, or TPUs without recoding your model.
estimators simplify sharing implementations between model developers.
you can develop a state of the art model with high-level intuitive code. In short, it is generally much easier to create models with Estimators than with the low-level TensorFlow APIs.
estimators are themselves built on tf.keras.layers, which simplifies customization.
estimators build the graph for you.
estimators provide a safe distributed training loop that controls how and when to:
- build the graph
- initialize variables
- load data
- handle exceptions
- create checkpoint files and recover from failures
- save summaries for TensorBoard

When writing an application with Estimators, you have to separate the data input pipeline from the model, which simplifies experiments with different data sets.

Below is a straightforward example of a linear regression with estimators. It highlights the general workflow.

You need of course to import some namespaces

import tensorflow as tf
tf.enable_eager_execution()
assert tf.executing_eagerly()
import tensorflow.contrib.eager as tfe
# too much info otherwise
tf.logging.set_verbosity(tf.logging.ERROR)

import numpy as np
import matplotlib.pyplot as plt
import seaborn as sea
sea.set()
sea.set

%matplotlib inline

and define some data

X = np.arange(0,100)
np.random.shuffle(X)
# set a linear y = ax+b
y = 3*X +2 + np.random.normal(size=[100])

The feature columns go into the definition of the estimator:

column =  tf.feature_column.numeric_column('x')
lin_reg = tf.estimator.LinearRegressor(feature_columns=[column])

and the usage of input functions allows you to decouple input from model

train_input = tf.estimator.inputs.numpy_input_fn(
    x={"x": X},
    y=y, shuffle=False,num_epochs=2500,batch_size=1)
lin_reg.train(train_input)

To predict things you use input functions as well

predict_input = tf.estimator.inputs.numpy_input_fn(
     x={"x": X},
     num_epochs=1, shuffle=False)
results = list(lin_reg.predict(predict_input))

fig, ax = plt.subplots()
sea.regplot(X,y, ax=ax)
ax2 = ax.twinx()
sea.regplot(X, np.array([x["predictions"][0] for x in list(results) ]) , ax=ax2, color='r')

and it’s clear that estimators are the fastest road to results in terms of API mechanics. Next to Keras and the low-level API the estimators is a third API layer in TensorFlow. Whether this approach is the best depends on your context:

the checkpoints mechanism related to model restoration should, in particular, be considered in contrast to the non-estimator approaches
the (dis)advantages of creating custom estimators
the flexibility of Keras versus the flexibility of estimator pipelines (also in the context of serving models)

Estimators definitely turn TensorFlow into something close to Scikit-Learn but whether you should use TF as a replacement for sklearn. Well, probably not.

The whole estimator stack is also available in R and works similarly, maybe even easier:

library(tfestimators)

response <- function() "Species"
features <- function() setdiff(names(iris), response())

# split into train, test datasets
set.seed(123)
partitions <- modelr::resample_partition(iris, c(test = 0.2, train = 0.8))
iris_train <- as.data.frame(partitions$train)
iris_test  <- as.data.frame(partitions$test)

# construct feature columns
feature_columns <- feature_columns(
  column_numeric(features())
)

# construct classifier
classifier <- dnn_classifier(
  feature_columns = feature_columns,
  hidden_units = c(10, 20, 10),
  n_classes = 3
)

# construct input function 
iris_input_fn <- function(data) {
  input_fn(data, features = features(), response = response())
}

# train classifier with training dataset
train(classifier, input_fn = iris_input_fn(iris_train))

# valuate with test dataset
predictions <- predict(classifier, input_fn = iris_input_fn(iris_test))
evaluation <- evaluate(classifier, input_fn = iris_input_fn(iris_test))

The TensorFlow API for R is actually very nicely documented and complements well the Python docs.

You might also like