ReLU Activation Function Graph Calculator

ReLU Activation Function Graph Calculator

Emerging AI Skill PRO

Enter Calculation

Formula

f(xk)=max⁡(0,xk),xk=Start x+k·Step,k=0,1,…,Count−1f(x_k) = \max\left(0,\; x_k\right), \quad x_k = {{\text{Start }x}} + k \cdot \text{Step}, \quad k = 0,1,\dots,\text{Count}-1
Usage stats

Advertisement

Results

ReLU sweep generated
101
points · x = -5 … 5
End x5
Min x-5
Max x5
x f(x) -5 5 5
x f(x)
-5 0
-4.9 0
-4.8 0
-4.7 0
-4.6 0
-4.5 0
-4.4 0
-4.3 0
-4.2 0
-4.1 0
-4 0
-3.9 0
-3.8 0
-3.7 0
-3.6 0
-3.5 0
-3.4 0
-3.3 0
-3.2 0
-3.1 0
-3 0
-2.9 0
-2.8 0
-2.7 0
-2.6 0
-2.5 0
-2.4 0
-2.3 0
-2.2 0
-2.1 0
-2 0
-1.9 0
-1.8 0
-1.7 0
-1.6 0
-1.5 0
-1.4 0
-1.3 0
-1.2 0
-1.1 0
-1 0
-0.9 0
-0.8 0
-0.7 0
-0.6 0
-0.5 0
-0.4 0
-0.3 0
-0.2 0
-0.1 0
0 0
0.1 0.1
0.2 0.2
0.3 0.3
0.4 0.4
0.5 0.5
0.6 0.6
0.7 0.7
0.8 0.8
0.9 0.9
1 1
1.1 1.1
1.2 1.2
1.3 1.3
1.4 1.4
1.5 1.5
1.6 1.6
1.7 1.7
1.8 1.8
1.9 1.9
2 2
2.1 2.1
2.2 2.2
2.3 2.3
2.4 2.4
2.5 2.5
2.6 2.6
2.7 2.7
2.8 2.8
2.9 2.9
3 3
3.1 3.1
3.2 3.2
3.3 3.3
3.4 3.4
3.5 3.5
3.6 3.6
3.7 3.7
3.8 3.8
3.9 3.9
4 4
4.1 4.1
4.2 4.2
4.3 4.3
4.4 4.4
4.5 4.5
4.6 4.6
4.7 4.7
4.8 4.8
4.9 4.9
5 5

Report an error on this page

What is the ReLU activation function?

The Rectified Linear Unit, or ReLU, is one of the most widely used activation functions in modern neural networks. It is defined as \(f(x) = \max(0, x)\): for any positive input it returns the input unchanged, and for any negative input it returns zero. This simple piecewise-linear rule introduces non-linearity into a network while being extremely cheap to compute and easy to differentiate.

Graph of the ReLU function showing a flat line at zero for negative x and a rising diagonal line for positive x
The ReLU function outputs zero for negative inputs and the input itself for positive inputs.

How to use this calculator

Enter three values: the initial value of x (where the sweep begins), the increment or step size (how much x changes each iteration), and the number of repetitions (how many points to generate). The tool evaluates ReLU at each x value, builds a complete (x, f(x)) data table, and draws a line graph showing the characteristic flat-then-rising shape with its corner at the origin.

The formula explained

ReLU is \(f(x) = \max(0, x)\), which is equivalent to "x if x > 0, else 0". The sweep is generated by the rule $$x_k = \text{startX} + k \cdot \text{stepX}, \quad k = 0,1,\dots,\text{iterations}-1$$ so the final x value (endX) equals \(\text{startX} + (\text{iterations} - 1) \cdot \text{stepX}\). The function is flat at zero across all negative inputs, then rises with a constant slope of 1 for positive inputs.

Advertisement
Diagram showing the two pieces of the ReLU formula: zero region and identity region split at the origin
ReLU is piecewise: f(x)=0 when x is negative and f(x)=x when x is positive.

Worked example

Using the defaults startX = -5, stepX = 0.1, and iterations = 101, the sweep covers x from -5 up to \(-5 + 100 \cdot 0.1 = +5\) across 101 inclusive points. At x = -2.0, \(f = \max(0, -2.0) = 0\). At x = 0, \(f = 0\). At x = 0.1, \(f = 0.1\). At x = 2.5, \(f = 2.5\). At x = 5.0, \(f = 5.0\). The plotted curve sits at zero throughout the negative range, then climbs linearly to (5, 5).

FAQ

Is ReLU differentiable at zero? No. ReLU has a corner (kink) at x = 0, so it is not differentiable there. Its derivative is 0 for x < 0 and 1 for x > 0; by convention the derivative at 0 is often taken as 0.

Can the step be negative? Yes. A negative step sweeps x downward. A step of zero makes every x equal to the starting value (a degenerate constant column).

Why is ReLU so popular? It avoids the vanishing-gradient problem of sigmoid and tanh for positive inputs, is computationally trivial, and tends to produce sparse activations, which often speeds up training.

Last updated:

Most popular in Math and Statistics

View all Math and Statistics calculators