to

activation function intro

Sparisoma Viridi
2 mins read ·

Short info about activation function

intro

In neural network, or NN, activation function decides whether a neuron should be activated or not by calculating the weighted sum and futher adding bias to it, where its purpose is to introduce non-linearity into the output of a neuron 1. The functions ensure that algorithmic networks, e.g. neural network, deep learning, artificial intelligence, machine learning, convolutional neural networks, etc. focus on priority problems by splitting or segregating the inputs, then processing power is being used most effectively 2. There are at least three types of activation functions for NN, which are binary step function, linear activation function, and nonlinear activation functions 3. How to choose the activation function for the ouput layer is based on the type of prection problem that the NN is solving, e.g regression requires linear activation function, while binary classification requires sigmoid activation function 4.

notes

  1. Sakshi Tiwari, “Activation functions in Neural Networks”, GeeksforGeeks, 17 Feb 2023, url https://www.geeksforgeeks.org/activation-functions-neural-networks/ [20240416]. ↩︎

  2. Nikolaj Buhl, “Activation Functions in Neural Networks: With 15 examples”, Encord, 25 Jul 2023, url https://encord.com/blog/activation-functions-neural-networks/ [20240416]. ↩︎

  3. Pragati Baheti, “Activation Functions in Neural Networks [12 Types & Use Cases]”, V7Labs, 27 May 2021, url https://www.v7labs.com/blog/neural-networks-activation-functions [20240416]. ↩︎

  4. Jason Brownlee, “How to Choose an Activation Function for Deep Learning”, Machine Learning Mastery, 22 Jan 2021, url https://machinelearningmastery.com/choose-an-activation-function-for-deep-learning/ [20240416]. ↩︎

Tags: