How to determine if a given line in a file is a valid English phrase?

I need to determine if the given "string" in the file is an English sentence or not. I am using Python. An approximate answer. I understand that this is a question of NLP, but is there an easy tool that provides a reasonable approximation? I do not want to use the full-featured NLP toolkit for this, although if this is the only way, then this is fine.

If the NLP toolkit is the answer, then the one I'm reading is the Natural Language Toolkit . If someone has a simple example of how to quickly find an offer, give me one.

+3
source share
5 answers
+4

, Punkt Tokenizer nltk, . , ( Acron)

+2

, . , , Amazon Mechanical Turk. , , , , .

, Mechanical Turk API - , , Python, .

+1

, ?

this ?

0

You can use Python Reverend . It has less than 400 lines of code. See how you can use it:

>>> from thomas import Bayes

>>> guesser = Bayes()

>>> guesser.train("en" u"a about above after again against all am an and any are aren't as at be because been before being below between both but by can't cannot could couldn't did didn't do does doesn't doing don't down during each few for from further had hadn't has hasn't have haven't having he he'd he'll he her here here hers herself him himself his how how i i'd i'll i'm i've if in into is isn't it it its itself let me more most mustn't my myself no nor not of off on once only or other ought our ours    ourselves out over own same shan't she she'd she'll she should shouldn't so some such than that that the their theirs them themselves then there there these they they'd they'll they're they've this those through to too under until up very was wasn't we we'd we'll we're we've were weren't what what when when where where which while who who whom why why with won't would wouldn't you you'd you'll you're you've your yours yourself yourselves")
>>> guesser.train("pt" u"último é acerca agora algmas alguns ali ambos antes apontar aquela aquelas aquele aqueles aqui atrás bem bom cada caminho cima com como comprido conhecido corrente das debaixo dentro desde desligado deve devem deverá direita diz dizer dois dos e ela ele eles em enquanto então está estão estado estar   estará este estes esteve estive estivemos estiveram eu fará faz fazer fazia fez fim foi fora horas iniciar inicio ir irá ista iste isto ligado maioria maiorias mais mas mesmo meu muito muitos nós não nome nosso novo o onde os ou outro para parte pegar pelo pessoas pode poderá    podia por porque povo promeiro quê qual qualquer quando quem quieto são saber sem ser seu somente têm tal também tem tempo tenho tentar tentaram tente tentei teu teve tipo tive todos trabalhar trabalho tu um uma umas uns usa usar valor veja ver verdade verdadeiro você")
>>> guesser.train("es" u"un una unas unos uno sobre todo también tras otro algún alguno alguna algunos algunas ser es soy eres somos sois estoy esta estamos estais estan como en para atras porque por qué estado estaba ante antes siendo ambos pero por poder puede puedo podemos podeis pueden fui fue fuimos fueron hacer hago hace hacemos haceis hacen cada fin incluso primero  desde conseguir consigo consigue consigues conseguimos consiguen ir voy va vamos vais van vaya gueno ha tener tengo tiene tenemos teneis tienen el la lo las los su aqui mio tuyo ellos ellas nos nosotros vosotros vosotras si dentro solo solamente saber sabes sabe sabemos sabeis saben ultimo largo bastante haces muchos aquellos aquellas sus entonces tiempo verdad verdadero verdadera     cierto ciertos cierta ciertas intentar intento intenta intentas intentamos intentais intentan dos bajo arriba encima usar uso usas usa usamos usais usan emplear empleo empleas emplean ampleamos empleais valor muy era eras eramos eran modo bien cual cuando donde mientras quien con entre sin trabajo trabajar trabajas trabaja trabajamos trabajais trabajan podria podrias podriamos podrian podriais yo aquel")

>>> guesser.guess(u'what language am i speaking')
>>> [('en', 0.99990000000000001)]
>>> guesser.guess(u'que língua eu estou falando')
>>> [('pt', 0.99990000000000001)]
>>> guesser.guess(u'en qué idioma estoy hablando')
>>> [('es', 0.99990000000000001)]

You must be very careful in choosing the best training data for your needs. Just to give you an idea, I am collecting some stopping words from English , Portuguese and Spanish .

0
source

All Articles