Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 6 years ago.
Improve this question
I am developing a Ruby on Rails application where I want to detect the number of physical objects (bottles and food packets) in an image.
I just explored Google Vision API (https://cloud.google.com/vision/) to check whether this is possible or not. I uploaded a photo which has some cool drink bottles and got the below response.
{
"responses" : [
{
"labelAnnotations" : [
{
"mid" : "\/m\/01jwgf",
"score" : 0.77698487,
"description" : "product"
},
{
"mid" : "\/m\/0271t",
"score" : 0.72027034,
"description" : "drink"
},
{
"mid" : "\/m\/02jnhm",
"score" : 0.51373237,
"description" : "tin can"
}
]
}
]
}
My concern here is, it is not giving the number of cool drink bottles available in the image, rather it returning type of objects available in the photo.
Is this possible in Google Vision API or any other solution available for this?
Any help would be much appreciated.
This problem unfortunately is not a problem that is fully solved. You can go with some object detection algorithms like Faster RCNN and YOLO. They can give you the objects up to a bounding box if they are included in ImageNet dataset; however, of course you can train your own classifier with them. I recommend YOLO which is really easy to use and nicely documented.
Also, you can deploy a DIGITS object detection server which includes Faster RCNN. It gives you a really nice user interface to use those models.
I've made a simple command line program that detects faces and replaces them with emojis using OpenCV through JRuby. It's an absolute pain to set up, but once it is done it is a beauty to write in. I also made a small script to create OpenCV JRuby projects that can be executed with the required command line arguments in a shell script, which alleviates most, if not all, of the pain when setting up.
Later on when I'm at my computer I'll upload both the project and the script to GitHub and link them here if you want me to, but for now I can direct you to this project as an example.
EDIT
Here are the links to the JRuby OpenCV project and script:
JRuby OpenCV Project
Project Creation Script
Related
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 4 years ago.
Improve this question
I am trying to build my own training corpus for Named Entity Recognition, but I don't know if there is already an existing tool for this or if I have to implement one myself.
Basically, what I need to do is take a corpus and manually tag it word by word, which is pretty tedious, but it has to be done.
Can anyone tell me if there is already an existing one and where to get it?
I had a good experience working with BRAT.
GATE is also a very complex tool for annotating, steeper learning curve.
We had a nice experience using DataTurks . They provide nice intuitive UI which allows to add collaborator, insights into data, leaderboard for annotators and some other funky features.
https://dataturks.com
For online annotation of text or HTML corpus of relatively short documents I also recommend BRAT. You will have to go under the hood of the python web application if you want to do anything custom. It also failed to work for me on large HTML documents (100 or so pages).
I have also used stand-alone apps:
Protege + Knowtator: a bit cumbersome to setup / use, but it
works;
Gate: also cumbersome, and it somewhat works. Backup
your annotations at regular intervals as you might get
surprised by a stacktrace that also wiped or corrupted your annotated
corpus (which is just serialized Java objects).
If you are dealing with PDF documents, we built a web-based PDF Annotation Tool: NOTA. It accepts anything printed to PDF, including scans. We do commercial OCR on our end to recover text from images. There is a REST API to create color-coded annotation schemas and pre-populate documents with annotations, as well as a REST API for exporting formatted text and annotation offsets. There is also a JS API you can use to customize any annotation workflows, add metadata to annotations, etc. Relationships are not supported out of the box. Large documents, 200+ pages are supported. Email us and we can give you an API key to try it out. Details and documentation links can be found here. It is free for small research projects.
Here is a screenshot of what the annotations looks like :
I co-develop myself the web-based text annotation tool: tagtog.net
There is nothing to install, and you can define the type of entities you want to annotate. Additionally you can annotation relationships, document labels, and much more. You can upload your documents in many different formats, including PDF or markdown. You can annotate together with your team collaboratively. We have put great care in making the interface easy and beautiful. It looks like this:
You can start right away with a free account. Also I would be happy to help you with any doubt or issue you may have; just ping me or write us an email to the address shown on the website, tagtog.net.
Our annotation tool Prodigy is very scriptable, and is designed for active learning. It integrates especially well with our NLP library spaCy.
We've paid particular attention to the Named Entity Recogntion (NER) annotation workflows, as entity recognition can otherwise be very slow. I have a tutorial video on this:
https://www.youtube.com/watch?v=l4scwf8KeIA
There is this tool called, Dataturks is super simple to use, fully online NLP annotation tool, so that I even can easily push my teammates to complete datasets for our projects.
try TagEditor ,
It is a desktop application designed to annotate text for training with spaCy library.
You can tag Named Entities, Dependencies, Parts of speech, text categories
and print json file.
Example
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 1 year ago.
Improve this question
I am currently using PDF Box to parse a pdf and I am trying to figure out how to retrieve data about the text such as the font (bold, size, etc) and the location of the font.
Any suggestions?
After poking around the (hard to find) PDFBox docs, I found this little gem.
Apparently one of the examples shows exactly how to do everything you asked. Basically, you subclass PdfTextStripper and override the processTextPosition method. There, you query the TextPosition for whatever information you need.
For future reference, you can find the javaDoc here: http://pdfbox.apache.org/apidocs/index.html
Edit 2018-04-02: original link is dead, but example can be found in the SVN repo here.
One of the best things for text extraction from PDFs is TET, the text extraction toolkit. TET is part of the PDFlib.com family of products.
PDFlib.com is Thomas Merz's (the author of the "PostScript and PDF Bible") company.
TET's first incarnation is a library. That one can probably do everything you want, including to positional information about each text element on the page. Oh, and it can also extract images. It recombines+merges images which are fragmented into pieces.
pdflib.com also offers another incarnation of this technology, the TET plugin for Acrobat. Obviously you'd need Acrobat as well to make use of this.
And the third incarnation is the PDFlib TET iFilter. This is a standalone tool for user workstations. Both this is free (as in beer) to use for private, non-commercial purposes.
Lastly, TET also comes with a commandline interface.
TET is really powerful. Way better than Adobe's own text extraction. It extracted text for me where other tools (including Adobe's) do spit out garbage only.
A few months ago I tested their desktop standalone tool, and what they say on their webpage is true. It has a very good commandline. Some of my "problematic" PDF test files the tool handled to my full satisfaction.
This thing is my recommendation for every sophisticated and challenging PDF text extraction requirements.
TET is simply awesome. It detects tables. Inside tables, it identifies cells spanning multiple columns. It identifies table rows and contents of each table cell separately. It deals very well with hyphenations: it removes hyphens and restores complete words. It supports non-ASCII languages (including CJK, Arabic and Hebrew). When encountering ligatures, it restores the original characters...
Give it a try.
The GetPageText function with extract option 3 or 4 in Quick PDF Library returns a CSV string for the selected page which includes the text (either individual words or a piece of text) and the related font name, text color, text size and co-ordinates on the page.
Note: it is a commercial library and I work for the company that sells it.
PDF files can be parsed with tabula-py, or tabula-java.
I made a full tutorial on how to use tabula-py on this article. You can tabula in a web-browser too as long as you have installed Java.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 23 days ago.
The community reviewed whether to reopen this question 23 days ago and left it closed:
Original close reason(s) were not resolved
Improve this question
I'm trying to find a webservice that will allow me to get a County name (not Country) for a specific Lat/Long. I would be performing the lookup within a server application (likely a Java application). It doesn't have a be a webservice if there is some library out there I suppose, but I would like up-to-date information. I have looked around quite a bit for an API that supports this, but so far I haven't been able to find one that works. I have tried the Yahoo APIs like so:
http://where.yahooapis.com/geocode?q=39.76144296429947,%20-104.8011589050293
But it doesn't populate the address information. I've tried with some of the "flags" options there too to no avail.
I've also looked around at Googles APIs as well, but I've read multiple places that they don't populate the County.
So does anyone know of any APIs that will take a Lat/Long and return the County associated with that location? And if you have any examples, that would be great.
I'd also like to know which APIs allow for use in a commercial application. A lot of the data I've found says that you can't use the data to make money. I might be reading those wrong, but I'm looking to build a service that I'd likely charge for that would use this data. So I'd need options. Maybe free services while I'm exploring options, and pay services down the road.
Just for completeness, I found another API to get this data that is quite simple, so I thought I'd share.
https://geo.fcc.gov/api/census/
The FCC provides an Block API for exactly this problem and it uses census data to perform the look up.
Their usage limit policy is (From developer#fcc.gov)
We do not have any usage limits for the block conversion API, but we do ask that you try to spread out your requests over time if you can.
Google does populate the county for your example,
http://maps.googleapis.com/maps/api/geocode/json?latlng=39.76144296429947,-104.8011589050293&sensor=false
In the response, look under the key address_components which contains this object representing "Adams" county,
{
long_name: "Adams"
short_name: "Adams"
-types: [
"administrative_area_level_2"
"political"
]
}
Here's from the Geocoding API's docs,
administrative_area_level_2 indicates a second-order civil entity below the country level. Within the United States, these administrative levels are counties. Not all nations exhibit these administrative levels.
Another option:
Download the cities database from http://download.geonames.org/export/dump/
Add each city as a lat/long -> Country mapping to a spatial index such as an R-Tree (some DBs also have the functionality)
Use nearest-neighbour search to find the country corresponding to the closest human settlement for any given point
Advantages:
Does not depend on aa external server to be available
Much faster (easily does thousands of lookups per second)
Disadvantages:
May give wrong answers close to borders, especially in sparsely populated areas
You may want to have look at Tiger data and see if it has polygons containing the county name in an attribute. If it does the Java Geotools API lets you work with this data. You will be performing point in polygon queries for the county polygons followed by a feature attribute look-up.
Maybe this is a great solution.It is in a json format.I always use this in my projects.
http://maps.google.com/maps/geo?ll=10.345561,123.896932
And simply extract the information using php.
$x = file_get_contents("http://maps.google.com/maps/geo?ll=10.345561,123.896932");
$j_decodex = json_decode($x);
print_r($j_decodex);
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 3 years ago.
Improve this question
I'd like to see some online vs. offline image optimizers comparison numbers, namely Yahoo! Smush.it vs. OptiPNG or pngcrush.
How those things differ in speed and resulting image size, and what is the best choice?
Very detailed and comprehensive comparison — with lots of tools and results on many different types of PNGs and optimizations:
http://css-ig.net/png-tools-overview
I think it's a much better source than PunyPNG's small comparison showing that their tool is best [partly at converting image formats rather than optimizing existing format] :)
I really don't know how reliable the information on this site is because they have their own compression service but take a look at the comparison in the URL: http://punypng.com/about/comparison
I copied the following image:
And installed two of the tools you mentioned offline:
brew install optipng pngcrush
And compared image sizes using default settings with an online tool called reSmush.it:
879K feat-social-awareness.original.png
712K feat-social-awareness.optipng.png
700K feat-social-awareness.pngcrush.png
205K feat-social-awareness.resmushit.png
Speed of each tool was not measured for the above test. Subjectively they all felt about the same.
Comparing the images visually I was unable to see the difference between the original and the optimized versions created using the offline tools. In the case of reSmush.it, however, there was a noticeable loss in image fidelity which can be easily reproduced using their API (see example).
As a result, the above sizes are not an apples-to-apples comparison. More like apples-to-gorillas. So I went back and increased the reSmush.it quality to 100 by setting qlty=100 as specified in their API docs and got back the same lossy PNG as with the default settings.
So what's the best choice? Well, it depends…
If compute resources are a major constraint consider using reSmush.it.
If image fidelity is a concern don't use reSmush.it.
If you use OptiPNG you're likely going to lose your original files (it overwrites by default).
If you use pngcrush you're getting better compression compared to optipng without a noticeable loss in image fidelity.
If you want lossy optimization similar to reSmush.it in an offline tool try pngquant.
And if serving images over the wire under heavy bandwidth constraints consider a different image format altogether, such as Fabrice Bellard's BPG Image format.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 8 years ago.
Improve this question
Is there a free (as in "cheers"), linux-compatible, interactive visual editor for graphviz or other graphs? aptitude seems to be drawing a blank.
edit: "cheers" means both "beer" and "speech". meta-edit: I guess it should be "free as in beach".
edit 2: Maybe a suitable svg editor would be a more realistic goal. I basically want something that can be used to conveniently create a collection of labeled shapes and lines which connect them. Actually it would probably make more theoretical sense to extract the graph from this data, since it includes both semantic data (the graph) and presentation data (the way it's arranged on the screen, the colours used, etc). Is there a way to lay out labeled shapes conveniently with inkscape or some other free vector graphics editor? I really need rearranging of the nodes, and (re)flowing of the text in them, to happen with maximum convenience.
I've also realized that this is really a superuser question. I was going to repost it over there when I found an existing question that seems likely to provide me with an answer: dia.
edit 3: dia seems useful except that it doesn't seem to be possible to get the textual contents of node objects to wrap in any useful manner (ie any way other than by inserting manual line breaks). This is kind of a dealbreaker, since it screws most of the convenience factor that's my incentive to do things this way rather than with a text editor or a pen and paper. But it supports some sort of event model and Python-based scripting, so I'm going to dig around a bit and see if I can use python to wrap the text in response to content changes. Unless one of you lovely people has a better idea..? Basically I want to have the option to explicitly set the node size via GUI interaction, and have the contents wrap and rescale (within a certain range of font sizes) to fit it. Rich text would be pretty useful.
In other words, this is actually a valid SO question at this point, since it appears to require coding.
To save time those eager to try existing programs handling DOT graphs:
dotty can display DOT graphs and with little luck you can move its nodes with a mouse, nothing more, and you can easily segfault as a bonus (I tried latest stable graphviz)
lefty is only a special-purpose language interpreter used by dotty, nothing to look at
KGraphEditor is an empty wishful project (a QT window and a few buttons)
gvedit is not really a graph editor: it provides a simple text editor and you hit F5 to run a layout tool and open a picture; you can actually get more functionality from configuring your own favourite text editor
grappa is an abandoned java applet, which I failed to run
interestingly, dia can export to DOT ("PyDia DOT Export"), but due to its buggy printing, you have to post-process the files to use them
graphedit can read in DOT a graph and you can move its nodes around and change their colors
Eclipse people started working on DOT support in GEF4, so it can display DOT graphs
GraphUI has a very interesting demonstration video, but beware: although it might seem that the graph is being created by clicking and dragging, in reality all editing happens through the keyboard, using shortcuts. On the plus side, contextual instructions are always available showing which shortcuts do what.
DotEditor claims a tree editor, modifying node attributes/color/shape with mouse.
The graph editors mentioned in other answers, yEd (a Java application) and JointJS/Rappid (a JaveScript thing) apparently have nothing to do with DOT (tried both).
I believe there exist no working DOT-handling graph editor out there at all.
JointJS is a Javascript graph editing library based on Backbone : http://www.jointjs.com/
The author also provides Rappid, an online graph editor which might suit your needs, I don't know about dot files import though.