Hacker News client in clojure
Some time ago Hacker News released official API. What better reason to learn to write http client in clojure using HTTP Kit.
Top stories
Let's start with getting top stories.
(ns hn-cljient.core
(:require [org.httpkit.client :as http]))
(defn get-top-stories []
(let [{:keys [status headers body error] :as response} @(http/get "https://hacker-news.firebaseio.com/v0/topstories.json")]
body))
This returns something like:
"[8438690,8438728,8438515,8437671,8438904,8438063,8437367,8439018,8436659,8437403,8438920,8438157,8437230,8438586,8437737,8437026,8438590,8438109...]"
Thats not very useful. Looks like we got just a list of ids. Let's modify the function so it returns the actual articles.
(ns hn-cljient.core
(:require [org.httpkit.client :as http]
[clojure.data.json :as json]))
(def item-url "https://hacker-news.firebaseio.com/v0/item/")
(def top-stories-url "https://hacker-news.firebaseio.com/v0/topstories.json")
(defn item-body [item]
(clojure.walk/keywordize-keys (json/read-str (:body item))))
(defn get-top-story-ids []
(let [{:keys [status headers body error] :as response} @(http/get top-stories-url)]
(json/read-str body)))
(defn get-item [id]
(let [url (str item-url id ".json")]
(http/get url)))
(defn get-top-stories []
(let [ids (get-top-story-ids)
futures (map get-item ids)
stories (map deref futures)]
(mapv (fn [x] (item-body x)) stories)))
Previous get-top-stories function is renamed to get-top-story-ids. It returns a vector of ids. Get-item returns a future that will realize once the item is fetched. Result of futures are put into stories. JSON-structures are read into clojure maps and their keys are transformed from strings to symbols. Result will be vector of maps similar to this:
{:text "",
:score 104,
:title "Ride Sharing by the Numbers",
:by "ignatiusjr",
:kids
[8439034 8438977 8439178 8439065 8439173 8439133 8439214 8439829],
:url "http://blog.whatsthefare.com/2014/10/it-pays-to-compare.html",
:type "story",
:time 1412961253,
:id 8438904}
Item comments
In the previous json-snippet one can see kids-key that has list of ids. Those are comments of the item. Here we get list of kids as full items.
(defn get-item-comments [id]
(let [future (get-item id)
item (item-body @future)]
(mapv (fn [x] (-> x get-item deref)) (:kids item))))
Notice that it gets only first level of comments. This could probably be used to get full threads if applied recursively to kids of kids and so on.
Conclusion
Here is a very simple implementation for a client using Hacker new API. The performance did not seem very good to me. It might be http-kit library or more likely something about my code. Feel free to drop a comment if you know how to make it better.
Software professional with a passion for quality. Likes TDD and working in agile teams. Has worked with wide-range of technologies, backend and frontend, from C++ to Javascript. Currently very interested in functional programming.
