What is the Softplus function?
The Softplus function, \(f(x) = \ln(1 + e^{x})\), is a smooth, differentiable approximation of the ReLU (rectified linear unit) activation used in neural networks. Unlike ReLU, which has a sharp corner at the origin, Softplus is smooth everywhere and always strictly positive. This calculator builds a table of x, f(x) and its first derivative over a range you choose, and plots both curves so you can see the characteristic gentle S-to-ramp shape.
How to use it
Enter three values: the Initial value of x (the first abscissa), the Increment (spacing between points), and the Number of repetitions (how many rows to generate). For example, an initial value of -5, increment 0.1 and 101 repetitions produce x from -5.0 to +5.0. The result is a scrollable table plus a graph of Softplus and its derivative.
The formula explained
Softplus is $$f(x) = \ln\!\left(1 + e^{x}\right).$$ Its derivative is $$f^{\prime}(x) = \frac{e^{x}}{1 + e^{x}} = \frac{1}{1 + e^{-x}},$$ which is exactly the logistic sigmoid. As x grows large and positive, \(f(x)\) approaches \(x\) and \(f^{\prime}(x)\) approaches 1; as x goes large and negative, \(f(x)\) approaches 0 and \(f^{\prime}(x)\) approaches 0. To avoid overflow for large x, the tool uses the numerically stable form $$f(x) = \max(x, 0) + \ln\!\left(1 + e^{-|x|}\right).$$
Worked example
At x = 0: \(f(0) = \ln(2) = 0.693147\) and \(f^{\prime}(0) = 0.5\). At x = 1: $$f(1) = \ln(1 + 2.718282) = 1.313262$$ and $$f^{\prime}(1) = \frac{1}{1 + e^{-1}} = 0.731059.$$ At x = -1: \(f(-1) = 0.313262\) and \(f^{\prime}(-1) = 0.268941\). Notice the identity \(f(x) - f(-x) = x\), e.g. \(1.313262 - 0.313262 = 1\).
FAQ
Why use Softplus instead of ReLU? Softplus is smooth and has a non-zero gradient everywhere, which can help gradient-based optimization, though ReLU is cheaper to compute.
Is the output always positive? Yes. \(\ln(1 + e^{x}) > 0\) for every finite x because \(1 + e^{x} > 1\).
What does the derivative represent? It is the slope of the Softplus curve and equals the logistic sigmoid, ranging monotonically from 0 to 1 with value 0.5 at x = 0.
