I was knowing that Finite Automata can have only one input states but several output states. Recently, I've come across an example of removing ^-moves that has many input states.
Please help!
Deterministic finite automata are equivalent to nondeterministic finite automata with epsilon/lambda transitions. Having multiple input/start/initial states is equivalent to having a single input/start/initial state with epsilon/lambda transitions to the desired input/start/initial states. It may not be "standard" or "usual" to speak of finite automata with multiple input/start/initial states but it adds no expressive or computational power.
Related
how can you show a) Deterministic pushdown automata (DPDA) are more powerful than finite automata and less powerful than a non-determinstic pushdown automata?
(1) First, show that any language that can be accepted by a finite automaton can also be accepted by a deterministic pushdown automaton. Recall that any language accepted by a finite automaton is accepted by a deterministic finite automaton, and a deterministic pushdown automaton can simulate a deterministic finite automaton simply by doing nothing interesting with its stack. Next, show there is a non-regular language accepted by a DPDA. 0^n 1^n is a good candidate. Prove this language is non-regular using the pumping lemma or Myhill-Nerode theorem, thenshow the DPDA that pushes on 0s and then switches to popping on 1s works.
(2) First, note that NPDAs can accept any language accepted by DPDAs since all DPDAs are also NPDAs that happen not to make use of nondeterminism. Next, find a language that has an NPDA but no DPDA. Even-length palindromes over the alphabet {0, 1} might work. There is a simple NPDA for this that nondeterministically guesses when the first half of the input has been read and switches from pushing to popping. To show there is no DPDA is more challenging. Perhaps you could argue as follows: suppose there were a DPDA. Then, in any configuration of the DPDA, only one transition would be possible. If string w leads to an accepting state in the DPDA and empties the stack, x00 may lead either to an accepting or non-accepting state (since x00 either may or may not be an even-length palindrome). This is a contradiction, though, so our DPDA does not exist. The same argument fails for NPDAs, by the way, because there can be multiple paths through, so one failed choice means nothing.
NDPA is more powerful than DPDA because we can add more transitions to it. It is possible for every language to add a transition. For some languages, we can construct DPDA there exist an NPDA but there are some languages that are accepted by NPDA but are not by DPDA. This is said to be powerful when it accepts more sets of languages than other automata.
In fact, it is more powerful than DFA(Deterministic finite automata) and NFA(Non-deterministic finite automata) also because, In the case of DFA and NFA, they are equivalent in power. So for every language accepted by DFA there exist an NFA and Vice-Versa. There is not any language for which we construct NFA but not DFA. Hence, we can’t convert NPDA to DPDA always and we can convert NFA to equivalent dfa.
What are the major issues essential to consider when constructing a finite state machine representing a given language? I know that finite state machines take strings as inputs, and that as each element of the string is read, the machine state changes until the EOF is reached. If once the string has been read completely the machine is in one of the final states the string is accepted. What I don't understand is what considerations need to be made when constructing the FSA (other than the string it should accept, and the definition of each transition function.)
One thing you'll want to consider is the number of states. There are many equivalent ways to define the machine but the fewer number of states would typically be preferred because the same result is achieved with less complexity and space.
The representation of a computational automaton requires space proportional to the number of states so optimizing the spacial complexity through state reduction is desirable or necessary.
I am implementing a SARSA(lambda) model in C++ to overcome some of the limitations (the sheer amount of time and space DP models require) of DP models, which hopefully will reduce the computation time (takes quite a few hours atm for similar research) and less space will allow adding more complexion to the model.
We do have explicit transition probabilities, and they do make a difference. So how should we incorporate them in a SARSA model?
Simply select the next state according to the probabilities themselves? Apparently SARSA models don't exactly expect you to use probabilities - or perhaps I've been reading the wrong books.
PS- Is there a way of knowing if the algorithm is properly implemented? First time working with SARSA.
The fundamental difference between Dynamic Programming (DP) and Reinforcement Learning (RL) is that the first assumes that environment's dynamics is known (i.e., a model), while the latter can learn directly from data obtained from the process, in the form of a set of samples, a set of process trajectories, or a single trajectory. Because of this feature, RL methods are useful when a model is difficult or costly to construct. However, it should be notice that both approaches share the same working principles (called Generalized Policy Iteration in Sutton's book).
Given they are similar, both approaches also share some limitations, namely, the curse of dimensionality. From Busoniu's book (chapter 3 is free and probably useful for your purposes):
A central challenge in the DP and RL fields is that, in their original
form (i.e., tabular form), DP and RL algorithms cannot be implemented
for general problems. They can only be implemented when the state and
action spaces consist of a finite number of discrete elements, because
(among other reasons) they require the exact representation of value
functions or policies, which is generally impossible for state spaces
with an infinite number of elements (or too costly when the number of
states is very high).
Even when the states and actions take finitely many values, the cost
of representing value functions and policies grows exponentially with
the number of state variables (and action variables, for Q-functions).
This problem is called the curse of dimensionality, and makes the
classical DP and RL algorithms impractical when there are many state
and action variables. To cope with these problems, versions of the
classical algorithms that approximately represent value functions
and/or policies must be used. Since most problems of practical
interest have large or continuous state and action spaces,
approximation is essential in DP and RL.
In your case, it seems quite clear that you should employ some kind of function approximation. However, given that you know the transition probability matrix, you can choose a method based on DP or RL. In the case of RL, transitions are simply used to compute the next state given an action.
Whether is better to use DP or RL? Actually I don't know the answer, and the optimal method likely depends on your specific problem. Intuitively, sampling a set of states in a planned way (DP) seems more safe, but maybe a big part of your state space is irrelevant to find an optimal pocliy. In such a case, sampling a set of trajectories (RL) maybe is more effective computationally. In any case, if both methods are rightly applied, should achive a similar solution.
NOTE: when employing function approximation, the convergence properties are more fragile and it is not rare to diverge during the iteration process, especially when the approximator is non linear (such as an artificial neural network) combined with RL.
If you have access to the transition probabilities, I would suggest not to use methods based on a Q-value. This will require additional sampling in order to extract information that you already have.
It may not always be the case, but without additional information I would say that modified policy iteration is a more appropriate method for your problem.
If we have a finite automaton with no final/accepting states. So that F is empty. How do you minimize it?
I got this to a test in which I was asked to minimize an automaton but it had empty F and I didn't know how to approach the problem because the automaton had no accepting states. Is a single initial state with all the transitions into itself the correctly minimized automaton?
I thought that if two automatons A and B for any possible input, A returns the exactly same output as B, they must be equivalent. Thus if an automaton has no final state, then it accepts no input or no input is valid, so it must be equivalent with any other automaton that behaves this way.
If the definition of Finite Automata enforces non empty state set, a single initial state without any transition will do.
Minimization of a finite automata helps in reducing the compile time, as it removes identical operations.
When we minimize an expression,we tend to remove the unused states or we merge two or more states into a single equivalent state which are likely to produce same output. Merging states like this should produce a smaller automaton that accomplishes exactly the same task as our original one. (Here the unreachable states are removed and the states are checked whether they are distinguishable or not. If undistinguishable the states are merged else the states are uniquely represented).
As it reduces the compilation time,results in increasing the processing speed of the program. Running a program which can run a second or two faster than other programs, can vastly increase the scope of the program.
As the question states:
I am trying to understand automata. Can every regular language have a linear bounded automaton?
Yes, it is possible to have Linear Bounded Automata for each and every regular languages. Regular languages are proper subset of Context Sensitive Languages. CSL is the language associated with Linear Bounded Automata (LBA). For more information on hierarchy of classes of formal grammars and languages read about Chomsky Hierarchy.
http://en.wikipedia.org/wiki/Chomsky_hierarchy