Frequent Questions
992 questions
99
votes
3
answers
87k
views
What is self-supervised learning in machine learning?
What is self-supervised learning in machine learning? How is it different from supervised learning?
- 42.4k
11
votes
1
answer
1k
views
Can supervised learning be recast as reinforcement learning problem?
Let's assume that there is a sequence of pairs $(x_i, y_i), (x_{i+1}, y_{i+1}), \dots$ of observations and corresponding labels. Let's also assume that the $x$ is considered as independent variable ...
- 903
8
votes
3
answers
18k
views
What is the difference between a stochastic and a deterministic policy?
In reinforcement learning, there are the concepts of stochastic (or probabilistic) and deterministic policies. What is the difference between them?
- 42.4k
66
votes
4
answers
17k
views
Are neural networks prone to catastrophic forgetting?
Imagine you show a neural network a picture of a lion 100 times and label it with "dangerous", so it learns that lions are dangerous.
Now imagine that previously you have shown it millions ...
- 2,258
43
votes
5
answers
17k
views
How should I handle invalid actions (when using REINFORCE)?
I want to create an AI which can play five-in-a-row/Gomoku. I want to use reinforcement learning for this.
I use the policy gradient method, namely REINFORCE, with baseline. For the value and policy ...
- 712
34
votes
1
answer
52k
views
How does the (decoder-only) transformer architecture work?
How does the (decoder-only) transformer architecture work which is used in impressive models such as GPT-4?
- 2,780
5
votes
2
answers
3k
views
Do convolutional neural networks perform convolution or cross-correlation?
Typically, people say that convolutional neural networks (CNN) perform the convolution operation, hence their name. However, some people have also said that a CNN actually performs the cross-...
- 42.4k
5
votes
4
answers
3k
views
What is the fundamental difference between an ML model and a function?
A model can be roughly defined as any design that is able to solve an ML task. Examples of models are the neural network, decision tree, Markov network, etc.
A function can be defined as a set of ...
- 4,062
92
votes
6
answers
104k
views
What's the difference between model-free and model-based reinforcement learning?
What's the difference between model-free and model-based reinforcement learning?
It seems to me that any model-free learner, learning through trial and error, could be reframed as model-based. In ...
- 1,021
22
votes
3
answers
6k
views
Why doesn't Q-learning converge when using function approximation?
The tabular Q-learning algorithm is guaranteed to find the optimal $Q$ function, $Q^*$, provided the following conditions (the Robbins-Monro conditions) regarding the learning rate are satisfied
$\...
- 42.4k
11
votes
1
answer
8k
views
What is the time complexity of the forward pass algorithm of a feedforward neural network?
How do I determine the time complexity of the forward pass algorithm of a feedforward neural network? How many multiplications are done to generate the output?
- 113
32
votes
3
answers
19k
views
Where can I find the proof of the universal approximation theorem?
The Wikipedia article for the universal approximation theorem cites a version of the universal approximation theorem for Lebesgue-measurable functions from this conference paper. However, the paper ...
- 485
18
votes
3
answers
3k
views
Which explainable artificial intelligence techniques are there?
Explainable artificial intelligence (XAI) is concerned with the development of techniques that can enhance the interpretability, accountability, and transparency of artificial intelligence and, in ...
- 42.4k
7
votes
2
answers
3k
views
How to estimate the capacity of a neural network?
Is it possible to estimate the capacity of a neural network model? If so, what are the techniques involved?
- 308
12
votes
2
answers
7k
views
What exactly is averaged when doing batch gradient descent?
I have a question about how the averaging works when doing mini-batch gradient descent.
I think I now understood the general gradient descent algorithm, but only for online learning. When doing mini-...
- 455