Parsing search results from website, compojure/clojure - parsing

Question

For some time I'm working on a simple clojure project for movies, so I'm trying to parse search results from a particular web site, in my case imdb.com. Not sure If I'm on the right track for this so I'm hoping someone would help me out.
Homepage will look simple enough, with text-field where you would enter movie name and submit button named "Search". I'll try to be as much deatailed as possible:
1.This is the main page:
(defn view-input []
(view-layout
[:h2 "Find your Movie"]
[:body {:style "font: 14pt/16pt helvetica; background-color: #F2FB78; padding-top:100px; text-align: center" }
(form-to [:post "/"]
[:br]
[:br]
(text-field {:placeholder "Enter movie name" } :a) [:br]
(submit-button "Search")
)]
))
2.These are the functions that I've been using:
(defn create-flick-url [a]
(str "http://www.imdb.com/search/title?title=" a "&title_type=feature"
))
(defn flick-vec [categories a]
(vec (let [flick-url (create-flick-url a)
flick-names (print-flick-name-content flick-url)]
(mapper-gen4 flick-names
(get-image-content flick-url)
))) )
(defn view-output2 [categories a]
(view-layout
[:h2 "Search results"]
[:form {:method "post" :action "/"}
(interleave
(for [flick (flick-vec categories a)]
(label :title (:name flick)))
(for [flick-name (flick-vec categories a)]
[:br])
(for [flick-image (flick-vec categories a)]
[:img {:src (:image flick-image)}])
(for [flick (flick-vec categories a)]
[:br]))
]))
3.And this is the GET/POST part in the same class, where I'm using the view-output and view-output2 functions :
(defroutes main-routes
(GET "/" []
(view-input))
(POST "/" [categories a]
(view-output2 categories a))
4.Also, these are the functions that previous ones are using:
(defn print-flick-name-content
[url]
(vec (flatten (map :content (h3+table url)))))
(defn get-image-content
[url]
(vec (flatten (map #(re-find #"http.*jpg" %)
(map :style (map :attrs (h3+table2 url)))))))
(defn get-page
"Gets the html page from passed url"
[url]
(html/html-resource (java.net.URL. url)))
(defn h3+table
"Return seq of <h3> and table tags, where content of the <h3> tag meet defined condition"
[url]
(html/select (get-page url)
[:td (html/attr= :class "title") :h3 :a]))
(defn h3+table2
"Return seq of <h3> and table tags, where content of the <h3> tag meet defined condition"
[url]
(html/select (get-page url)
[:td (html/attr= :class "image")]))
5.And here's the last one, function defined in another class which deals with map:
(defn mapper-gen4
[names images] (sort-by :name (map #(hash-map
:name %1 :image %2) names images)))
I know it's a bit much, but this way someone will see where the problem is, so far the Search result page shows no results, nor errors, only blank page with h2 Search Results title. Thanks in advance!

Answer 1

I would start at the form handling route:
(POST "/" [categories a]
(view-output2 categories a))
inserting a humble print statement:
(POST "/" [categories a]
(do
(println "CAT" categories "A" a)
(view-output2 categories a)))
Make sure your handler includes wrap-reload so you can refresh the page and check the console. You might see that categories and a are nil, in which case you might next try something like this:
(POST "/" req
(do
(println "REQ" req)
(view-output2 *hard-coded-categories* *hard-coded-a*)))
Replace hard-coded-categories and hard-coded-a with the data structure you are expecting to see. This will test:
You will see in the request where the parameters are.
You will see whether your rendering code does what you expect with the right data.
If indeed the problem is that categories and a are nil, it might just be that you forgot a middleware handler (see why this matters).
If they contain the data you expect in the structure you expect, then it is time to drill down into your other functions. For this I recommend using a REPL session and calling your top level function with the hard-coded values you are expecting from the form, however if you are using wrap-reload you could also just resubmit the form. For example you could add printouts of the inputs and outputs of your mapper-gen4 function and get-page function.
Finally after playing with some values and results, copy these from your REPL into a test file so that you have some permanent assertions about how your code behaves.
If all else fails, posting a link to your github project will get you better help - or if it is private, create a minimal example project so that people can help you more precisely.