Movable Type Powered 'Trendolizer' Ready for the U.S. Elections

| 2 Comments | No TrackBacks |

In my spare time I've been working on http://www.trendolizer.com, a site that automatically finds and posts stories, videos, images etc. that are trending on Facebook.  I recently redesigned the site using the Bootstrap framework, and today seems to be an excellent time to point out that there is also a 'politics' section at http://politics.trendolizer.com where you can follow what is trending regarding the U.S. presidential election.  Read on if you want to know more about the technical details.

The core of the system is formed by a number of perl scripts that collect huge amounts of fresh links from a variety of sources: Reddit, Twitter, RSS feeds...  The scripts then use the Facebook API and memcached to store the current number of likes/shares for each link.  At regular intervals this information is checked again and compared to the original number.  The difference tells us how much a link is being shared around right now.

If a link is deemed sufficiently trending, Movable Type's Perl API is used to create a new entry, along with an optional image asset.  Title, description and image are taken from the OpenGraph tags found at the URL, if possible.  In other words, much like what happens when you share a link on Facebook.

Publish queue and Bob the Rebuilder then take care of getting the site updated as fast as possible.  Meanwhile, the perl scripts keep tracking the sharing rate of the links in question, and if it goes higher still the custom field holding that info for the entry gets updated as well.

New stories are tweeted out automatically, with an optional hashtag based on which 'subtrendolizer' they are posted on (there are about a dozen sections, for example booksmovies, tech...)

Using the sharing rate the main page displays both a chronological listing of trending stories and a 'Hot list' that shows the most shared items.

The individual entry templates are extremely lightweight but show a lot of extra content fetched in via various javascript APIs: tweets and reddit submissions about the item, a graph showing the evolution of shares/likes over time and Facebook comments.

Since this is a site with an international audience, dates and times are pretty meaningless unless the time zone is included.  But nobody will ever recalculate to their local time anyway, so on the main page relative dates are used ("1 hour, 20 minutes ago").  Since the main page is republished every few minutes anyway, these stay 'up to date'.  But the individual entries normally aren't republished frequently, so there the GMT date/time is used but it is converted to a relative date/time using moment.js.

Internally in Movable Type, Trendolizer is a website with about a dozen blogs below it.  Each blog uses the same theme, and most of the theme templates basically include modules that live at the website level.  This makes changing a particular template (for example the header or the sidebar) very easy to do.

The entire system runs fully-automated with no human editors at all.  And in my experience, it is quite good at catching breaking news stories and viral trends right when they start taking off.  But don't take my word for it: have a look for yourself (and check back a few times during the day).


No TrackBacks

TrackBack URL: http://www.movabletips.com/cgi-bin/mt/mt-tb.cgi/3539

2 Comments

Very interesting idea.
I assume that this could be easily modified to track stories on LinkedIn.

Could it be personalized to one's own LI network? To find out what stories are trending only among your 1st degree connections?

Thanks.

Very nice update of the design.

Really needed, because the previous design was really oldfashioned, 90's style.

Leave a comment