I'm looking for an alternative to JMegahal that is just as simple, and easy to use, but yields better results. I know JMegahal uses Markov chains to generate new strings, and I know that they're not necessarily the best. I was pointed towards Bayesian Network as the best conceptual solution to this problem, but I cannot find any libraries for Java that are easy to use at all. I saw WEKA, but it seemed bloated, and hard to follow. I also saw JavaBayes, but it was almost completely undocumented (their javadocs contained little to no information, and the variables were poorly named) and the library was blatantly written in C-style, making it stand out in Java.
You might want to consider extending JMegahal to filter the input sentences. Back in the mid-90s, Jason Hutchens had written a C version of this 4th-order Markov strings algorithm (it was probably used as inspiration for the JMegahal implementation actually). At that time, Jason has added filters to improve on the implementations (by replacing 'you' by 'I', etc...). By doing some basic string manipulation meant to change the subject from the speaker to the system, the output became a lot more coherent. I think the expanded program was called HeX.
Reference 1
Reference 2
Related
the custom_federated_algorithms_2 tutorial presents a local_train function using tff.federated_computation.
There a comment saying "while we could have implemented this logic entirely in TensorFlow, relying on tf.data.Dataset.reduce...":
regarding this comment:
I didn't manage to actually convert the code to using tf.data.Dataset.reduce seems non-trivial and the debug comments really don't help
I wonder what is the motivation of using federated_computation in cases like this, I looked all over the guides and real did find an explanation for what is going on here and when should we use it.
thank's!
Addressing these two in order:
It may not be trivial to adapt the code given directly to use tf.data.Dataset.reduce; that comment is intended to call out that the logic expressed here is also expressible using the dataset-reduce primitive, as effectively it only represents a local reduction, there are no communications across placements happening here.
There are at least two distinct purposes of this demonstration. One is to show that TFF as a language does not necessarily rely on the in-graph looping constructs of TensorFlow; another is to demonstrate the ability to "capture" values using the federated computation decorator. This could be used to natively capture something like learning rate decay in TFF, by evaluating a function of the round number and closing over it in the manner above, though there are other ways to implement similar functionality, as demonstrated here for example.
I personally find this pattern a little confusing; reading into the question behind the question a little, I agree that it is confusing to use a federated_computation decorator where there is no communication happening. When writing TFF, I generally express all my local computation in TensorFlow proper (usually in a functional manner), and let TFF handle the communication only. The purpose of the second tutorial is to show that TFF proper is actually much more flexible than indicated by restricting oneself to using the pattern just described.
I look for spell checker that could use language model.
I know there is a lot of good spell checkers such as Hunspell, however as I see it doesn't relate to context, so it only token-based spell checker.
for example,
I lick eating banana
so here at token-based level no misspellings at all, all words are correct, but there is no meaning in the sentence. However "smart" spell checker would recognize that "lick" is actually correctly written word, but may be the author meant "like" and then there is a meaning in the sentence.
I have a bunch of correctly written sentences in the specific domain, I want to train "smart" spell checker to recognize misspelling and to learn language model, such that it would recognize that even thought "lick" is written correctly, however the author meant "like".
I don't see that Hunspell has such feature, can you suggest any other spell checker, that could do so.
See "The Design of a Proofreading Software Service" by Raphael Mudge. He describes both the data sources (Wikipedia, blogs etc) and the algorithm (basically comparing probabilities) of his approach. The source of this system, After the Deadline, is available, but it's not actively maintained anymore.
One way to do this is via a character-based language model (rather than a word-based n-gram model). See my answer to Figuring out where to add punctuation in bad user generated content?. The problem you're describing is different, but you can apply a similar solution. And, as I noted there, the LingPipe tutorial is a pretty straightforward way of developing a proof-of-concept implementation.
One important difference - to capture more context, you may want to train a larger n-gram model than the one I recommended for punctuation restoration. Maybe 15-30 characters? You'll have to experiment a little there.
I'm working on a project to generate questions from sentences. Right now, I'm at a point where I can generate questions like:
"Angela Merkel is the chancelor of Germany." -> "Angela Merkel is who?"
Now, of course, I want the questions to look like "Who is...?" instead. Is there any easy way to do this that I haven't thought of yet?
My current idea would be to train an English(not quite question) -> English(question) translator, maybe using existing machine translation engines like moses. Is this overkill? How much data would I need? Are there corpora that address this or a similar problem? Is using a general translation engine even appropriate for this task?
Check out Michael Heilman's dissertation Automatic Factual Question Generation from Text for background on question generation and to see what his approach to this problem looks like. You can find more by searching for research on "question generation". He mentions a corpus from Microsoft: the Microsoft Research Question-Answering Corpus.
I don't think that an approach based solely on (current) statistical machine translation approaches is going to work that well, since you're usually going to need a deeper syntactic analysis of the source sentence to do a good job of generating an appropriate question. For simple questions like your example, it's pretty easy to design syntactic tree transformations to generate the question, but it gets much trickier as soon as the sentences get a little more complicated.
Off the top of my head, if you restrict yourself to relatively simple questions, you could do a parse, and then flip around the elements to get the question. How do you decide the question word though? Who, What, Where, Why... for this you'll need a classifier that looks at the elements of a sentence. Angela Merkel should be easy to classify as a person/name, so she gets s 'Who', Berlin should be in a dictionary of geos, so it gets 'Where'.
I'm not sure about specific software, but I'd probably do it with NLTK, using a dependency parse and then whatever classification scheme you feel like.
Ultimately your success depends on how big your input and output space is. I'd go for the absolute simplest possible problem first.
I've been studying Ruby on Rails for a class project. I keep hearing "everything in Ruby is an object". What I am not sure I understand is WHY that is a good thing, or maybe IF that is a good thing?
A counterexample would be that in Java Integer is an object but int is not, which means different operations apply to both (admittedly in recent Java there is automatic conversion to/from the object version, but this can introduce unexpected performance issues). Objects are a little slower due to indirection, but more flexible; everything being an object means everything behaves consistently. Again Java would be an example: an array is not an object, and ArrayIterator is something that is bolted on after the fact (with multiple third party implementations, even) and therefore not quite consistent with the way collection class iterators work.
It makes Ruby very flexible. Numbers and other primitive types can be altered or extended.
This can also result in quite elegant syntax:
5.times { print "print this 5 times" }
Everything in Ruby is not an object (Yes, I know what people generally mean when saying that statement, but its still not totally true). It is more appropriate to say "everything in Ruby evaluates to an object". This is an interesting insight to make, and for the more proper elaboration, I'll simply cite David Black. It's a good read:
http://rubylearning.com/blog/2010/09/27/almost-everything-is-an-object-and-everything-is-almost-an-object/
Like other people have pointed out already, there are primitives other than objects in other languages like Java. For the compiler and computer, it's a good thing to get the most efficient code, however, programmers need to use different functions and methods based on which one they are dealing with.
Since Ruby is designed for humans, not computers, sacrificing a bit of computational resource for the sake of human's productivity is considered good. Thus, Ruby has never had the distinction between objects and primitives. It definitely lowers the learning curve for novices, too. Internally, Ruby is using a technique called tagged pointers and the performance penalty by the lack of primitives is not as bad, as far as I know.
Another thing worth noting is that in Ruby, classes are also objects, which means you can modify classes and their behaviors with ease even when the code is running. This dynamic nature gives programmers so much power and the Ruby code tends to look quite terse. Ruby on Rails is taking full advantage of this dynamic aspect of the Ruby language.
I'm still new to OOP, and the way I initially perceived it was to throw alot of procedural looking code inside of objects, and think I'd done my job. But as I've spent the last few weeks doing alot of thinking, reading, and coding (and looking at good code, which is a hugely under-rated resource), I believe I'm starting to grasp the different outlook. It's really just a matter of clarity, simplicity, and organization once you get down to it.
But now I'm starting to look at things as objects that are not as black and white a slamdunk case for being an object. For example, I have a parser, and usually the parser returns some strings that I have to deal with. But it has one specialized case where it has to return an array, and what goes in that array and how it's formatted has specialized rules. This only amounts to two lines plus one method of code, but this code sticks out to me as not being cleanly fitting in the Parser class, and I want to turn it into its own "ActionArray" object.
But is it going to far? Has OOP become a hammer that is making me look at everything like a nail? Is it possible to go too far with turning things into objects?
It's your call, but you should think of objects as real life objects.
Take for example a car. You could describe a car with different objects:
Engine
Wheels
Chassis
Or you could describe a car with just one object:
Engine
You can keep it simple and stupid or you can spread the dependency to different objects.
As a general guideline, I think Sesame Street says it best: you need an new object when "one of these things is not like the others".
Listen to your code. If it is telling you that your objects are becoming polluted with non-essential state and behavior (and thus violating the "Single Responsibility Principle"), or that one part of your object has a rate of change that is different from the rest, and so on, it is telling you that you are missing an object.
Do the simplest thing that could possibly work. When that no longer works, do the next simplest thing. And so on. In general, this means that a system tends to move from fewer, larger objects to more, smaller objects; but not always.
There are a number of great resources for OO design. In addition to the ones already mentioned, I highly recommend Smalltalk Best Practice Patterns and Implementation Patterns by Kent Beck. They use Smalltalk and Java examples, respectively, but I find the principles translate quite well to other OO languages.
Design patterns are your friend. A class rarely exists in a vacuum. It interacts with other classes, and the mechanisms by which your classes are coupled together is going to directly affect your ability to modify your code in the future. With poor class design, a change that you make in one class may ripple down and force changes in other classes, which cause you to have to change other classes, etc.
Design patterns force you to think about how classes relate to each other. For example, your Parser class might choose to implement the Strategy design pattern to abstract out the mechanism for parsing. You might decide to create your Parser as a Template design pattern, and then have each actual instance of the Parser complete the template.
The original book on Design Patters (Design Patterns: Elements of Reusable Object-Oriented Software is excellent, but can be dense and intimidating reading if you are new to OOP. A more accessible book (and specific to Ruby) might be Design Patterns in Ruby, which has a nice introduction to design patterns, and talks about the Ruby way of implementing those patterns.
Object oriented programming is a pretty tricky tool. Many people today are getting into the same conflict, by forgetting the fundamental OOP purpose, which is improving code maintainability.
You can always brainstorm about your future OO code reusability and maintainability, and decide yourself if it's the best way to go. Take look at this interesting study:
Potok, Thomas; Mladen Vouk, Andy Rindos (1999). "Productivity Analysis of Object-Oriented Software Developed in a Commercial Environment"