Use your imagination

Monday, March 27, 2006

Add Digg counts to your RSS feed and blog posts using FeedFlare

I recently completed my first FeedFlare unit which will add your digg counts as links on your RSS feeds and blog posts. I am hosting this service and am opening it up to the general public as well. I am allowing 1000 hits or requests per host name and after that number is exceeded for an individual host the links will start forwarding to an ad page. If your host passes the 1000 hits mark you have the option of subscribing to the service which will remove the ad page. I am doing this ad page/subscription model so that my server does not get overloaded and to help pay the hosting bills should I start getting a large number of requests.

You can start using the service now if you have a FeedBurner account. Just login to FeedBurner and add a custom FeedFlare unit pointing to http://flarexml.referpals.com/DiggCountFlare.xml.

For more information see the FeedFlare documentation on the FeedBurner site and also see the page I created for the service. Please post any issues or questions to the comments of this post or e-mail them to feedflare[at]referpals[dot]com.

[Update 9/10/2006] I now have a dynamic flare for showing Netscape votes and comments as well. Get the details on this one here http://ipbs.blogspot.com/2006/09/new-netscape-flare-shows-vote-and.html

Tuesday, March 21, 2006

Regular Expressions for the Azureus RSS Feed plugin

Relating to my post on "How to get your missed TV shows with Azureus and Bittorrent", quite a few people have been showing interest in the regular expressions that I use in my Azureus RSS Feed plugin. One in particular that people seem to have interest in is the regular expression to download all first season, first episodes. I created one that works quite well. The only current caveat is that it doesn't work with the smart history feature, so using it does require some manual intervention now and again. Here is the regular expression I use and how I use it:

Let me break it down for a better understanding of how it works. There are basically six parts to this expression that you can see if you take out the groupings that are contained in parens. The first five of them for different variations of season and episode one and the last excludes certain terms. The first five parts look like (?=.*?\bs01e01\b) separated by a pipe (OR) symbol. If you wish to create another search term simply add in another grouping prepended with the pipe symbol. The new text to search for should be placed between the \b elements like so |(?=.*?\bfind this\b).

The last part of the expression that excludes terms is of the form (?!french|german|swedish).. I am attempting to exclude content in languages that I do not understand. If you wish to add another language, simply append it to the list with a pipe and the string just before the closing paren like so (?!french|german|swedish|klingon)..

Of course this regular expression is not only limited to searching for first seasons and first episodes. You could use it to search for whatever else you want like all HD content.

Caveats: If you use this as one of your RSS Feed filters then you will need to uncheck or turn off the "Smart History" feature. This is because this feature currently watches to make sure you are downloading incremental episodes. Some methods I've tried to avoid duplicates is to first create "Fail" filters for the items I do not want after the item was initially downloaded. I do this by checking the download history tab in the RSS Feed plugin and finding the item in question to see which feed it came from. Then I go to the feed on the status tab and look for the item, it'll have a Matches value of "Done", and then right click and "Create Filter". Set the filter mode to "Fail" and then move it to the top of the filters list. Finally remove the torrent and data from the "My Torrents" view.

The second method I use is to go to the "My Torrents" view and stop the torrent download or seed and then wait for a day or two. The reason for waiting is for the torrent to be pushed off the RSS feed so that it won't be picked up again. After waiting I then delete the torrent and data from my system. This method works for most torrents, but it will take up precious disk space while it is on your system and if the torrent is ever reseeded by someone it'll show up again.

Leave some comments to let everyone know what expressions you use that may also be handy. Keep it clean and don't give show or title specific expressions as this is not a "My Favorite Show" list.

For a good regular expression testing tool check this out:
Rex V: an AJAX Regular EXpression eValuator

Saturday, March 18, 2006

Digg counts are now real-time

If you haven't noticed, I now have the Digg counts coming through real-time on this blog. If you read my last post on getting your Digg counts to show for your posts you may have noticed that the counts were delayed due to RSS feed caching. I decided to bypass the RSS feeds and created my own ASP page which will grab my dugg posts from digg and update the numbers on my blog in real-time. If anyone is interested in the ASP code and more details to do this just let me know in the comments and if enough people want to know then I'll take the time to post the code and details.

Happy Digging!

Friday, March 10, 2006

How to get your Digg counts displayed on your blog

[Update 5/1/2006] This is now easier if you use FeedBurner you can use my service as described here http://ipbs.blogspot.com/2006/03/add-digg-counts-to-your-rss-feed-and.html

I recently took a little time to figure out how to display Digg counts for every post on this blog and thought others might like to know how to do it as well. I searched around for other people doing this and couldn't find anything, so if there is an easier or better way, please let me know.

It is a bit of pain currently to do it, but after it is setup it seems to work well so far in the browsers I've tested which are IE6+ and Firefox 1.5+. I have not done any extensive testing yet, so your mileage may vary.

The reason it is such a pain is partially due to the way the RSS search feed on digg currently does NOT work. I first attempted to do a search on the digg website by typing in my blog domain "ipbs.blogspot.com" and the web site search returned exactly what I was looking for. Digg was also kind enough to place an RSS link for the search along with those search results, but the problem is it doesn't seem their RSS code is as polished as the on site search code. From their other RSS links, like for user Diggs, they have started to return some useful data, like Digg counts, so hopefully in the future they'll fix their RSS search to work with domain searches then we could skip the next step.

This bug lead me to use an online html to RSS scraping utility in order to turn those search results I want into an RSS feed. The reason I want to get it as an RSS feed is first to aleviate as much server load on Digg as I can by caching the feed elsewhere and not having to execute the search on Digg for every request. Second to be able to easily process the results. Third because attempting to use AJAX or something to directly pull the results from the Digg search page brings up other issues when accessing across domains.

So the html to RSS scraper I chose to use is at http://rssgenerator.plech.net/. This one was pretty simple to get working with a regular expression that should work universally on the Digg search. Here is a screenshot and the text version of the parameters I used:

Source page: http://digg.com/search?search=[insert your blog domain]&submit=Submit

Regular Expression: <h3><a href="(.*?)".*<\/h3>(.*\r?\n)*?.*<ul class="news-digg">(.*\r?\n)*?.*(<a .*(<\/a>)?)

Items Title Submatch: 4

Items Desc. Submatch: 1

Result Caching (hours): 1

Just replace the bracketed portion of the "Source page" URL and you should be good to go. The regular expression picks out the Digg story link, News/Blog story link, and the digg count. After clicking the "Generate" button this can all be seen in the results. If you aren't getting any items in your results try going to the Digg search page and make sure something is being returned for your search. To complete this step copy the URL from the page that appeared after you clicked the "Generate" button.

Now that we have a URL for an RSS feed of our dugg stories as well as Digg counts we need to get them to our blog somehow. As I said before, I decided not to use an AJAX method due to cross domain communication issues, so I decided to get the feed returned to me as HTML by going through another online service that I've been using for other parts of my blog page. This service is Feed Digest. In case you don't know, what this service does is takes a group of RSS feeds and turns them into a combined format that can then be transformed to HTML or whatever you decide. I used this on my sidebar to get a feed of some of the top sites I like to keep an eye on in my "Other BS" section. However for my current purpose I am just going to use it to transform my RSS feed into some JavaScript.

On my first attempt to use the RSS generator URL in Feed Digest it didn't work due to the URL being to long. My quick and dirty solution to this is to use one of the many online URL shortening services at http://compactURL.com. Pick whichever one you may favor since any of them should work.

Now with shortened URL in hand, goto Feed Digest and enter it to create a new digest. If you don't already have an account you'll need to signup. You get 5 digests for free which is more than we'll need for this. On the feed info and parameters page the important part that we'll need to modify in order to return some useful JavaScript to our blog is the "Digest Layout/Template" section. Scroll to it and click the link stating ">> Click here to edit HTML templates manually (for advanced users)". If you aren't familiar with JavaScript then I suggest you enter the info exactly as I have it below. Otherwise, feel free to experiment and let me know if you do anything cool.

Per Item Template:

myDiggs[myDiggs.length] = new Digg("%DESCRIPTIONPLAIN%", "%URL%", "%TITLE%");

Header Template:

<script type="text/javascript">
var myDiggs = new Array();
function Digg(blogURL, diggURL, diggs)
this.blogURL = blogURL;
this.diggURL = diggURL;
this.diggs = diggs;

function lookupDigg(storyURL)
for(var i = 0; i < myDiggs.length; i++)
aDigg = myDiggs[i];
if (aDigg.blogURL == storyURL)
return aDigg;
return null;

function ipbs_addOnLoadEvent(func) {
var oldonload = window.onload;
if (typeof window.onload != 'function') {
window.onload = func;
} else {
window.onload = function() {

Footer Template:

function showMyDiggs()
var postDivs = document.getElementsByName("diggCount");
for(var i = 0; i < postDivs.length; i++)
var postDiv = postDivs[i];
var aDigg = lookupDigg(postDiv.attributes["link"].value);
if (aDigg != null)
postDiv.innerHTML = "<a href='" + aDigg.diggURL + "' target='_blank'><strong>" + aDigg.diggs.replace(" ", "</strong> ") + "</a>" + postDiv.innerHTML;
postDiv.title = "Digg this item!";
postDiv.innerHTML = "<a href='http://digg.com/submit?phase=2&url=" + encodeURIComponent(postDiv.attributes["link"].value) + "' target='_blank'><strong>Digg</strong> This!</a> " + postDiv.innerHTML;
postDiv.style.display = "inline";


Now with that all done save the changes and test the digest by clicking the view HTML link. View the source of what is returned, probably a blank page, and make sure a JavaScript exists that looks like it is building an array from the RSS items. The next step is to get the script executing on your blog. Click the "Get code" link for your digest and copy the JavaScript code to add to your blog. Add the code anywhere on your main blog template. You'll also need to modify your blog template for interaction with the script that will execute.

The script is expecting to find a DIV for each story or post on the page with the following attributes defined for the tag:

<div id="diggCount" name="diggCount" class="digg-count" title="Count is delayed at least 1 hour. Click to Digg this post!" link="<$VariableThatGivesLinkToSpecificStory$>"><div>

Here is the exact way I defined the DIV for my blogger blog. I have it setup so that the Digg count appears next to the title of each post on the Main or Archive page or a "Digg this!" link if the post hasn't already been dugg. On an individual post or item page I have it set so the digg count stays in the bottom left corner of the browser (in Firefox) and I'll get more into how these styles are configured in a bit, but here is the Blogger lines I'm using for my post titles.

<div id="diggCount" name="diggCount" class="digg-count" title="Count is delayed at least 1 hour. Click to Digg this post!" link="<$BlogItemPermalinkUrl$>"></div><h3 class="post-title"><a href="<$BlogItemPermalinkUrl$>"><$BlogItemTitle$></a>

The script is also expecting the DIVs to be hidden by setting the "display" style to "none" via CSS. You can style yours however you wish, but I tried to model mine after the Digg count found on Digg itself. Here are the styles I added to my blog template. These need to go inside a style tag, which can usually be found near the top of your blog template:

.digg-count {
display: none;
margin-right: 2px;
float: left;
position: fixed;
left: 2px;
bottom: 2px;
width: 54px;
height: 54px;
background-color: transparent;
background-attachment: scroll;
-moz-background-clip: -moz-initial;
-moz-background-origin: -moz-initial;
-moz-background-inline-policy: -moz-initial;
color: rgb(147, 136, 63);
text-align: center;

.digg-count a, .digg-count a:visited, .digg-count a:hover, .digg-count a:link, .digg-count a:active {
color: rgb(80, 71, 13);
text-decoration: none;
padding-top: 0.5em;
padding-right-value: 0pt;
padding-bottom: 0.5em;
padding-left-value: 0pt;
padding-left-ltr-source: physical;
padding-left-rtl-source: physical;
padding-right-ltr-source: physical;
padding-right-rtl-source: physical;
text-decoration: none;
margin-bottom: 0pt;
font-size: 8pt;
font-weight: normal;
text-align: center;
vertical-align: middle;
display: block;

.digg-count a strong{
display: block;
font-size: 12pt;
margin-top: 5px;

Note the ItemPage tags which I use to move the Digg count block down to the bottom left corner when viewing an individual post.

That's it for now. Let me know if you think I've missed something or you found this useful. I hope in the future this will be easier to achieve with some tools that Digg creates, but for now it looks like we need to take the long road.

<--- Don't forget to Digg this post;)

Friday, March 03, 2006

TiddlyWiki - a reusable non-linear personal web notebook

While searching for a standalone wiki app today I came across this little gem called TiddlyWiki. Seems like a great concept and my experience with it so far is excellent. It is basically a personal wiki all written in a single html file. It makes use of JavaScript for all of it's functionality. It even allows saving and modifying of pages and saves a backup of the old page when modified (You need to accept the security alert). There is even an option to generate an RSS feed file when saving. I'll be using this for documenting a few of the projects I'm working on as well as collecting ideas and thoughts. I'll just create a copy of the starter page for each project or category. Another bonus to this is that if one ever wants to publish one of the wikis to the web it is not hard since one just uploads the file(s) to a site. Kudos to the developers for a great job!