Convergent Training and Inference

You have constructed every foundational component of modern machine learning:

1. The Linear Neuron: Computes weighted combinations of environmental signals.
2. The Activation Function: Introduces non-linear squashing into bounded probabilities.
3. The Loss Function: Quantifies prediction error with mathematical rigor.
4. The Optimization Step: Shifts weights downhill along the loss slope.
5. The Training Epoch: Uses the chain rule to backpropagate errors across all data samples.

In this final stage, you bring all these components together to build the complete training and inferenceThe stage where a trained model processes new input data to make classificationsView official documentation engine.

Consider the truth table for the logical AND gate:

• Inputs $[0, 0] \implies 0$
• Inputs $[1, 0] \implies 0$
• Inputs $[0, 1] \implies 0$
• Inputs $[1, 1] \implies 1$

At the beginning, we initialize all weights to $0$ ($w_1 = 0, w_2 = 0$) and bias to $0$ ($b = 0$). With these initial values, the neuron knows nothing: for any input it computes $z = 0$, giving a sigmoid output of $\sigma(0) = 0.5$ (maximum uncertainty).

However, as the optimization loop runs through 200 epochs:

1. During each epoch, the model compares its predictions against the truth table targets.
2. Backpropagation computes the average gradients across all samples.
3. The weights gradually increase while the bias becomes negative.
4. By the end of training, the parameters converge such that the weighted sum only clears the threshold when both inputs are active.

You have just trained a machine to discover a boolean rule purely from examples!

When standard input receives "op": "train-and-predict", process:

• epochs: number of full training epochs to run (positive integer)
• learning_rate: learning rate $\alpha$
• samples: training dataset containing pairs of {"x": [...], "y": 0 or 1}
• test_inputs: list of feature vectors to classify after training

Step-by-Step Procedure

1. Initialization:

   • Determine input dimension $d$ from len(samples[0]["x"]).
   • Initialize weights to zeroes: weights = [0.0] * d.
   • Initialize bias to zero: bias = 0.0.
   • Batch size: $N = \text{len}(samples)$.

2. Training Loop (Repeat epochs times):

   • Initialize gradient accumulators: grad_w = [0.0] * d and grad_b = 0.0.
   • For each sample $s$ in samples:
     • $z = \sum_{j=1}^{d} (w_j \cdot s.x_j) + b$.
     • $\hat{y} = \frac{1}{1 + e^{-z}}$.
     • $\delta = (\hat{y} - s.y) \cdot \hat{y} \cdot (1 - \hat{y})$.
     • Accumulate: grad_w[j] += delta * s.x[j] and grad_b += delta.
   • Update weights and bias by subtracting the mean gradient scaled by learning_rate: $$w_j \leftarrow w_j - \alpha \cdot \frac{\text{grad\_w}[j]}{N}$$ $$b \leftarrow b - \alpha \cdot \frac{\text{grad\_b}}{N}$$

3. Inference Phase:

   • For each feature vector $x_{\text{test}}$ in test_inputs:
     • Evaluate the neuron using the finalized weights and bias: $$z = \sum_{j=1}^{d} (w_j \cdot x_{\text{test}, j}) + b$$ $$p = \frac{1}{1 + e^{-z}}$$
     • Apply the standard classification threshold: $$\text{prediction} = \begin{cases} 1 & \text{if } p \ge 0.5 \\ 0 & \text{if } p < 0.5 \end{cases}$$

• Python Pattern:

  dim = len(samples[0]["x"])
  w = [0.0] * dim
  b = 0.0
  n = len(samples)
  
  # Phase 1: Training
  for _ in range(epochs):
      grad_w = [0.0] * dim
      grad_b = 0.0
      for s in samples:
          x, y = s["x"], s["y"]
          z = sum(xi * wi for xi, wi in zip(x, w)) + b
          y_hat = 1.0 / (1.0 + math.exp(-z))
          delta = (y_hat - y) * y_hat * (1.0 - y_hat)
          for j in range(dim):
              grad_w[j] += delta * x[j]
          grad_b += delta
      for j in range(dim):
          w[j] -= lr * (grad_w[j] / n)
      b -= lr * (grad_b / n)
  
  # Phase 2: Inference (Predictions)
  predictions = []
  for x in test_inputs:
      z = sum(xi * wi for xi, wi in zip(x, w)) + b
      y_hat = 1.0 / (1.0 + math.exp(-z))
      predictions.append(1 if y_hat >= 0.5 else 0)
  
  return {"predictions": predictions}
  

• TypeScript Pattern:

  const dim = samples[0].x.length;
  const n = samples.length;
  const w = new Array(dim).fill(0);
  let b = 0;
  
  // Phase 1: Training
  for (let ep = 0; ep < epochs; ep++) {
    const gradW = new Array(dim).fill(0);
    let gradB = 0;
    for (const s of samples) {
      let z = b;
      for (let i = 0; i < dim; i++) z += s.x[i] * w[i];
      const yHat = 1 / (1 + Math.exp(-z));
      const delta = (yHat - s.y) * yHat * (1 - yHat);
      for (let i = 0; i < dim; i++) gradW[i] += delta * s.x[i];
      gradB += delta;
    }
    for (let i = 0; i < dim; i++) w[i] -= lr * (gradW[i] / n);
    b -= lr * (gradB / n);
  }
  
  // Phase 2: Inference
  const predictions = test_inputs.map((x) => {
    let z = b;
    for (let i = 0; i < dim; i++) z += x[i] * w[i];
    const yHat = 1 / (1 + Math.exp(-z));
    return yHat >= 0.5 ? 1 : 0;
  });
  
  return { predictions };
  

ML Engineering Insight

Notice the sharp separation between phases:

1. Training: Model weights mutate along loss gradients guided by ground-truth labels y.
2. Inference: Weights are completely frozen; the network only executes the forward pass on unseen data without knowing true answers or computing gradients.

1. If samples is empty or epochs <= 0, write error to stderr and exit with code 1.
2. Output a JSON object containing the array of predicted integers:

   { "predictions": [0, 0, 0, 1] }