[Help-glpk] existence of the solution for the LP formulation of Markov decision problem

[Philippe Preux]

Hi,

I am interesting in Markov decision problems, let's say in the discrete 
setting, infinite horizon with accumulated discounted reward. I assume 
rewards are bounded. In this case, we know from the theory of MDPs that 
the value function always exists.
The formulation of this problem as an LP problem is well-known.
We know from MDP theory that, since the value function exists, is 
unique, and is bounded, and the optimal policy exists too, that the 
primal and dual of the equivalent LP formulation are bounded feasible.
Now, my question is: let us suppose we only have the problem under its 
LP guise and we do not know it is bounded feasible; would there be a way 
to know by simply looking at it (not trying to solve it), I mean, just 
knowing A and b, that this problem is bounded, feasible in the primal 
and the dual?

Thanks for any hint

Philippe