Question

1)Which of the following are correct statements about the Bellman Equation?

A) Bellman equation represents the value of a state in terms of the value of successor states.

B) Bellman equation represents the expected value of successor states.

C) Bellman equation can be written for a state or a state-action pair.

D) Bellman equation is based on an approximation of the value of the current state.

2) Which of the following describes what a backup diagram represents? (Please explain)

a)Shows the current state and all possible subsequent actions and states, and the expected value of a state can be computed by ‘backing-up’ over the values of subsequent states in the diagram.

b)Shows all possible paths to arrive at the current state, and can be used to compute the expected values of the predecessor states by ‘backing up’ over the values of these predecessor states.

c)Shows all possible paths to arrive at the current state, and can be used to compute the expected values of the current state by ‘backing up over the state values in the diagram.

d)Shows the current state and all possible subsequent actions and states, and the expected value of a predecessor state can be computed by ‘backing-up’ over the values of the states in the diagram.

Answer 1

1) option :- B

Reason:-
   It is named after Richard E.Bellman.It is mainly used for Dynamic programming.It writes the (value) of a decision problem at a certain point in time in terms of payoff from some initial choices and the (value) of the remaining decision problem that results from those initial choices.
Terms related to this topic is reward,return,cost function,state etc.
The importance of Bellman equations is that they let us express values of states as values of other states.

Applications:-
  
1) Reinforcement Learning
2) Dynamic programming(Princple of optimality).

2) option :- C