Most of the applied problems we get asked about are not images. They are rows: sensor readings joined to maintenance logs, claims joined to provider attributes, transactions joined to account history. Tabular data is also the one domain where a deep network is most likely to lose — to a gradient-boosted tree that trains in ninety seconds on a laptop.
So the first deliverable on a tabular project is not a Keras model. It is a number the Keras model has to beat.
Step 1: the gradient-boosting baseline
Before any layers, train LightGBM or XGBoost on the same split, with the same target and the same metric. Ordinal-encode the categoricals, leave the missing values alone (both libraries handle them natively), spend twenty minutes on num_leaves and learning_rate, and record the result.
That number does three things. It tells you whether the signal is there at all. It caps how much a neural network can plausibly be worth. And it gives you the fallback you ship if the network underdelivers — which, on small and medium tables with heterogeneous columns, it frequently does. The published comparisons are consistent on this point, and our engagements match them: below roughly 10k rows, boosted trees usually win outright; between 10k and a few hundred thousand, it depends on the feature mix; above that, and especially with high-cardinality categoricals, free text, embeddings from another model, or multi-task targets, the neural net starts to earn its keep.
Skipping this step is how teams end up defending a 12-layer MLP that is 0.004 AUC behind a model with no GPU dependency.
Step 2: get the target and the split right
Tabular leakage is subtler than image leakage, and it is the reason most "great offline, dead in production" models die.
- Time. If predictions will be made on future rows, split by time, not at random. A random split lets the model see next month to predict this month.
- Entity. If the same machine, patient, or account appears in many rows, split by entity. Otherwise the model memorizes the entity, not the phenomenon.
- Feature availability. For every column, ask what its value is at prediction time. Aggregates computed over the full history, fields backfilled after the outcome is known, and status flags updated by the event you are predicting are all leakage. This audit is tedious and it is the highest-value hour on the project.
Write the split rule down in the run record. It is the thing reviewers will want to re-derive six months later.
Step 3: FeatureSpace, so preprocessing ships with the model
The classic failure mode of tabular deep learning is not the architecture. It is that preprocessing lives in a notebook: a fitted scaler here, a category dictionary there, a get_dummies call whose column order depends on the order rows arrived. Then serving re-implements it, slightly differently, and the model degrades for reasons nobody can find.
Keras's keras.utils.FeatureSpace exists to fix exactly this. You declare each column's type, adapt it once on the training data, and get back a preprocessing layer you can put inside the served graph.
import keras
from keras.utils import FeatureSpace
feature_space = FeatureSpace(
features={
# numeric, z-scored
"temp_c": FeatureSpace.float_normalized(),
"pressure_kpa": FeatureSpace.float_normalized(),
# numeric, but the relationship is not monotonic -> bin it
"hours_since_service": FeatureSpace.float_discretized(num_bins=16),
# low-cardinality categoricals
"line_id": FeatureSpace.string_categorical(num_oov_indices=1),
"shift": FeatureSpace.string_categorical(num_oov_indices=1),
# high-cardinality categorical -> learned embedding
"part_sku": FeatureSpace.string_categorical(
max_tokens=20000, num_oov_indices=1, output_mode="int"
),
},
crosses=[
FeatureSpace.cross(["line_id", "shift"], crossing_dim=64),
],
output_mode="concat",
)
train_ds_no_labels = train_ds.map(lambda x, y: x)
feature_space.adapt(train_ds_no_labels)
Two details worth stating plainly.
Always set num_oov_indices=1. Production will send you a category you have never seen. Without an out-of-vocabulary bucket, that row either raises or silently maps to index 0, which is some other real category. Both are worse than a bucket that says "unknown."
Adapt on training rows only. Adapting on the full table computes normalization statistics and vocabularies over your validation and test data. That is leakage, it will inflate your offline numbers, and it is easy to do by accident when the adapt call sits above the split in a notebook.
For best throughput, apply the feature space in the tf.data pipeline during training, then attach it to the model at export so the served artifact takes raw values:
# fast training: preprocess in the pipeline, async and prefetched
train_prep = train_ds.map(
lambda x, y: (feature_space(x), y), num_parallel_calls=tf.data.AUTOTUNE
).prefetch(tf.data.AUTOTUNE)
# serving: raw dict in, prediction out -- one artifact, no skew
raw_inputs = feature_space.get_inputs()
encoded = feature_space.get_encoded_features()
outputs = trained_core(encoded)
inference_model = keras.Model(raw_inputs, outputs)
inference_model.export("export/1")
That last block is the whole point. One artifact holds the vocabularies, the normalization constants, the crosses, and the weights. There is nothing for a serving team to reproduce.
Step 4: an architecture that is deliberately boring
Start with an MLP: two or three Dense layers of 128–512 units, ReLU (or GELU), dropout around 0.1–0.3, and either BatchNorm or LayerNorm. Adam at 1e-3, cosine decay, EarlyStopping(restore_best_weights=True).
x = keras.layers.Dense(256, activation="relu")(encoded)
x = keras.layers.Dropout(0.2)(x)
x = keras.layers.Dense(128, activation="relu")(x)
x = keras.layers.Dropout(0.2)(x)
outputs = keras.layers.Dense(1, activation="sigmoid")(x)
On tabular problems, embedding width for high-cardinality columns and preprocessing choices (bin that skewed count? cross those two IDs?) move the metric more than depth does. Tune those first. Reach for FT-Transformer, TabNet, or NODE only after the MLP has beaten the tree baseline, or when the MLP is close and you have a concrete reason to think attention over features will close the gap. In our experience that reach pays off maybe one engagement in four, and it always costs training time and serving latency.
Step 5: calibration, not just ranking
Most tabular models feed a threshold or an expected-value calculation, so the probabilities have to mean something. Check a reliability curve and Brier score, not only AUC. Neural nets trained with class weighting or heavy oversampling come out badly calibrated almost by construction — the ranking can be fine while the probabilities are systematically high. Prefer fixing the loss (focal loss, or plain cross-entropy with a tuned decision threshold) over resampling the data, and if you do distort the class balance, calibrate afterward on a held-out slice and keep that calibrator inside the served artifact.
What the decision looks like
At the end of a tabular engagement we put four numbers on one page: the tree baseline, the Keras model, the latency and infrastructure cost of each, and the maintenance surface of each. A neural net that wins by 0.002 AUC while adding a GPU dependency and a preprocessing graph is a loss dressed as a win. A neural net that wins by three points of recall at fixed precision, absorbs free-text and embedding features the trees cannot use, and serves preprocessing inside its own artifact is worth shipping.
Deep learning on tables is a real tool. It is just one that has to earn the slot, and the honest baseline is what lets you tell the difference.