Posts Tagged ‘feeds’

The FriendFeed API

Thursday, May 29th, 2008

First off, let’s define aggregation as taking a bunch of things and putting them all in one place.

Let’s try something.  I’m going to read off a bunch of web services.  Which ones are you a member of?

  • Digg
  • Google Reader
  • Google Talk
  • Gmail
  • Mixx
  • Reddit
  • del.icio.us
  • Furl
  • Ma.gnolia
  • StumbleUpon
  • Jaiku
  • Pownce
  • Twitter
  • Seesmic
  • Vimeo
  • YouTube
  • Flickr
  • Picasa
  • SmugMug
  • Zooomr
  • Tumblr
  • iLike
  • Last.fm
  • Pandora
  • Goodreads
  • LibraryThing
  • Amazon
  • Disqus
  • LinkedIn
  • Netflix Queue
  • Netvibes
  • SlideShare
  • Upcoming
  • Yelp
  • or any blog
  • or anything else putting off an RSS feed

Each of these can be pulled into one feed by FriendFeed.com .

How many of those services are you a member of?  What do you do when you want visitors to see all your posts in one place?  Think about using FriendFeed and harnessing it’s API .  We’ll use it to create a single central feed out of the chaos.

So how does FriendFeed work?  Each member loads up their various accounts from around the web – Google Reader shared articles, Twitter posts, blog entries, YouTube videos, Flickr photos, del.icio.us bookmarks, all that stuff I listed earlier – and members can have discussions around each entry.  It’s a furious stream of information and comments coming from all directions. blogs, news, pictures, etc – all flying by, with comments.  It’s the daily web posting activity of thousands of people in one place.

Facebook already has a nice FriendFeed app so your friends can see your FriendFeed-aggregated posts.

Otherwise showing it on another web page (like a very cool business’ web page) means we’re off to the FriendFeed API.

First thing we need is a FriendFeed account.  They’re free.  Go to friendfeed.com and follow the instructions.

Then load all of your web memberships into FriendFeed so you’ll have stuff in your feed.

Then we need to download the API from http://code.google.com/p/friendfeed-api/ .  Documentation and forums are available at http://friendfeed.com/api/ .

Then, in order to authenticate with the API, we get our new account’s Remote Key from http://friendfeed.com/remotekey .

That’s all the official FriendFeed stuff we need.  Now on to our application.  I called it FriendFeedFormatter.

It is not affiliated with FriendFeed.  Use it all at your own risk. FriendFeed has stated that the API is young and could change at any time.  That disclaimer out of the way…

My code does all the work of taking our nickname and remote key and producing an HTML snippet from our FriendFeed feed that we can then have included in a PHP page.  Download example.zip from the show notes.  It’s got all the code that I’ve written using the API.

http://www.davidvanvickle.com/friendfeed/example.zip

In the show notes is a link to see sample output.  That will give you an idea of what to expect, and you can begin to think of how to embellish it.

http://www.davidvanvickle.com/friendfeed/build.php

There are only two files in example.zip.  One is my FriendFeedFormatter PHP class.  The other is build.php, which is the file you’ll customize and run.

Now find the FriendFeed API you downloaded.  In order for my custom class to work, you’ll need to include the friendfeed.php file from the API in the same folder.

So when you’re done there will be 3 files uploaded to a folder on your PHP enabled web server.  friendfeed.php from the API, plus FriendFeedFormatter.php and build.php.

Before we try to run anything we need to change a couple things inside of build.php.  So open that file in a text editor.

Enter your FriendFeed nickname where it says “MY_NICKNAME”.  Then enter your remote key where it says “MY_REMOTEKEY”.  That’s it.  There are more things we could toggle but let’s start here.

So save and close build.php and upload the 3 files to the web server.  Now you should be able to hit your build.php with a web browser and see what happens.

If all is well then you’ll see the HTML table that the FriendFeedFormatter produced using the API.  Now look in the webserver folder where you uploaded the 3 files.  You should now see a 4th file.  That file is what you’ll be including in your normal web pages.  It is JUST the HTML table displaying your feed.  Just a little piece of a web page, not the whole thing.  Perfect for inserting into a page that already exists.

Why don’t I just write a Javascript snippet like everyone else?  Well because I don’t want to slow my web pages with a call to a remote site.  The FriendFeed servers are pretty fast, but I still don’t want that dependancy if I can help it.  As long as I can run that build.php file on a regular basis, I will have a new enough include file for my site.  And it will load right away.

So how do I run build.php on a regular basis?  Good question.  If you want to automate that, I have half an answer.  If you’re familiar with running cron jobs, or if you want to learn how, cron would be a good way.  Cron allows your server to perform a task – such as calling build.php – according to a schedule, like every few minutes, instead of waiting for a web user to come along and trigger it.  Cron is usually running already.  You don’t have to install it.  Just add a crontab to call lynx, wget, or on older servers, the php cgi executable and pass it build.php and how often it should run.

Truth be told, I don’t have much experience with cron, but it seems to be where people go for scheduling scripts.  I included some examples in the show notes that I scraped from the web to get you started.  I’m sorry that I don’t have more tested examples.

Run every Monday morning at 4:41AM.  (minute hour day month weekday command)

41 04 * * 1 /path/to/lynx http://www.mydomain.com/path/to/your/cron.php
or
41 04 * * 1 /path/to/wget http://www.mydomain.com/path/to/your/cron.php
or
41 04 * * 1 /path/to/php /path/to/your/cron.php

Samples from:
http://www.hackernotcracker.com/2007-04/run-php-scripts-with-crond-and-crontab.html
http://www.modwest.com/help/kb.phtml?cat=2&qid=105

More at
http://www.webmasterworld.com/forum40/1258.htm

Otherwise just keep build.php bookmarked and hit it every time you do something that updates your feed.

Fortunately for server performance, but unfortunate for us, is that FriendFeed is on it’s own schedule.  Just because I update my blog, doesn’t mean FriendFeed is immediately aware of that post.  It may take a few minutes for it to show up in the FriendFeed feed.  (That’s a great reason to figure out how to make the cron way work)

The code to pull in the include file in your PHP page is the same code in build.php that displays the output in the browser.  The FriendFeedFormatter->get_output() method.

Once you see the output, I’m sure you’ll want to customize it more.  Go for it.

I built in a few enhancements.

One enhancement is date formatting.  The date may come to you a couple different ways.  One is in ISO8601 format.  That looks like this. “2008-05-21T15:02:22Z”

Another format would be as a UNIX timestamp, or a long number that represents the number of seconds since January 1, 1970.

You should be able to look in the date column, see which kind of date you have, and change the value of the date format variable in build.php.

$ff_date_format = ‘ ISO8601’; // or ‘unix’

On one server I got one format, on another I got the other format.  I think that’s based on the PHP version I had running but I’m not sure.  Anyway, that’s why my code has two ways to format dates.Another customization is the title of the table.  I recommend making that a link to your Friendfeed page.  The current set up has a link to your FF page, plus a link to  subscribe to your feed.

The formatter has a couple features users may appreciate.  One is inline Youtube videos.  First the user sees a JPG image of the video, but when it is clicked, it turns into a playable video.

Another feature is link detection, which is programmed to happen with Twitter entries.  The static URL’s become clickable.

The last feature is, when Flickr is detected, it lists the thumbnails of the images.

These are things I expected to happen from the API, but they weren’t there, so I added them to my formatter.

Open the formatter and notice how I can treat each service’s entries differently.  I can detect Twitter, for instance, and scan the entry for URL’s to convert into links.  Or I can detect Youtube and make the inline play feature happen.

The important thing is just to hack away to get going using FriendFeed to aggregate your public web activity.

Share and Enjoy:
  • Digg
  • del.icio.us
  • Facebook
  • Google Bookmarks
  • E-mail this story to a friend!
  • LinkedIn
  • MySpace
  • Reddit
  • Slashdot
  • StumbleUpon
  • FriendFeed
  • Turn this article into a PDF!
  • Ping.fm
  • Twitter

Making RSS Feeds

Wednesday, April 16th, 2008

You’ve heard the saying, “It’s better to give than to receive”. If that’s true we’re doing great today. We’re going to be on the giving end of RSS feeds. Being on the receiving end is easy, just fire up your feed reader/aggregator like google.com/reader and go down the list of new stories. Giving can be easy to, but sometimes it means getting into the code to produce exactly what you want.

Maybe you’re wondering what a feed is. Well they’re all over the internet. It is how the internet syndicates it’s content. Feeds are flying out of blogs, news web sites, and any other tuned in site with frequent content updates.

It’s basically just an xml file on a web server, like a web page or an image is a file on a web server. People use a dedicated tool to view the file contents. For a web site, you use a web browser, for a feed, you use a feed reader. I prefer the free web based reader at google.com/reader. Some email programs and web browsers are capable of being readers as well. As usual, if a tool is dedicated to a task, it is a better experience, that’s why I use a dedicated feed reader.

The Internet has lost a lot of it’s “geeks-only” feel since the web is full of stuff everyone can use. Web feeds still feel geek-ish, but that doesn’t make them any less useful. Reading feeds is like reading email. A message with a time stamp comes in and you read it. An email is addressed to a specific person or group, while a feed is open to the world. Just subscribe. Your feed reader becomes the starting point for all your internet news and updates.

So let’s go deeper. What’s in a feed technically? It’s just a text file with readable content organized using markup called XML, like a web page uses HTML. It’s editable with any basic text editor. Notepad, Textedit, everyone’s got an editor. Word to the wise – don’t use Word. Word wants to add it’s own formatting. You don’t want to change the formatting.

Editing XML means getting familiar with tags so you can give the content meaning. What’s a title? What’s a link? Tags distinguish these things. There are specific tags for feeds. There are different sets of these tags depending on which feed format you’re using. The one I always see is RSS 2, but there is also Atom, RSS 1 and RDF. An RSS 2 file uses the tags rss, channel, item, title, link, description, pubdate, and guid.

Everything is wrapped in an rss tag. In that is the channel tag and a bunch of item tags. The channel describes the feed. The feed has a title and copy, for example, which would go into the title and description tags respectively. Other tags describe the feed’s publication date and URL that would point to the feed’s associated web page.

The feed is hopefully full of stories and site updates. Each of these gets its own item tag. Some of the tags in the item tag are the same as the ones in the channel tag. title, link, description, pubdate. For every new blog entry or news story a new item tag is made. The title goes in the title tag, the body copy goes in the description tag, the web site link goes in the link tag, the date the item should appear in readers goes in the pubDate tag, etc.

Hey, you know this podcast? You may have received it because you’re subscribed to its feed using your podcast feed reader, iTunes. Well this podcast feed has got some extra tags in the channel and in each item. Apple Inc has declared it’s own feed namespace. These are tags starting with the word itunes followed by a colon followed by a descriptive word like owner, name, email, image, category, author, subtitle, summary, duration, or keywords. Some of these are a little redundant to the main feed tags, but they’re used by iTunes to get the info it needs from a feed.

<itunes:image href="http://www.davidvanvickle.com/images/seriesface2_300.png"/>

Podcasts may not have survived if it wasn’t for Apple . Podcasts certainly wouldn’t be on their way to thriving as they are now. Apple is making the complex simple. So buy an iPod, connect it to iTunes, get podcasts for free. Feeds are knitting all that together. So we’ll do what Apple says and include these tags in our podcast feed along with the others.

Anyway, we actually have one more tag to look at. It is the enclosure tag, which determines where the audio file will come from. Audio media type and file size are attributes of this tag.

So let’s say we’ve just Notepadded the heck out of an RSS 2 XML file using all these tags. What are the chances something isn’t right? The chances are high. So let’s upload the file to a web site, then wander over to a feed validator to debug it. feedvalidator.org is an awesome free service for debugging feeds. Go there, type the URL of your feed, and it will tell you how to fix any issues. Doing this beats the possibility of upsetting your feed reader and not having your content syndicated. Better to broadcast something that works.

feedvalidator.org even tells you if the http headers are wrong. If they are, modify your server to serve files with that extension as “Content-Type=text/xml”. Similarly, since you’re declaring the served content as xml, the first line of that xml file should be the first line of every good xml file.

<?xml version="1.0" encoding="iso-8859-1"?>

Okay eventually we survive validation. We just manually edited a feed.

Sometimes we can edit the feed directly like this, sometime we can’t. Sometimes someone else is producing the feed. If you have a blog, you already have a feed. Just keep on bloggin, and that will automatically update your feed.

Let’s go back to situations where we make it ourself. What are other methods besides Notepad? Do we want automated generation or manual generation? For automated, we could use a server side scripting language to crank through a database and produce our XML. For manual generation, we could get an off the shelf desktop application that outputs feeds. At least we’d get away from Notepad that way.

It is good when feeds are automatic – the product of some other activity, like a blog. Or maybe I’m a system admin and I want to get notifications from my servers when something goes wrong. The last thing I want is more email, especially from a non-human. Maybe I could write a program that puts system logs into a feed. Then anyone with access could opt in to see what’s up with the server.

Maybe I want to convert my feed from one with MP3 files to one with M4A files? I could read the feed and replace the enclosure tags, maybe add some iTunes tags.

Maybe I upload the school newsletter every week, and every time I do, the system updates the feed with a link to the new newsletter for subscribers.

Maybe I’m a viagra salesman, gaaa, and I want to crap on a feed by inserting advertising. If I was that kind of person and a programmer – highly unlikely – I could suck in a feed, and make every fifth item be an ad, or at the end of every item description I could put an ad. At least I could make it a targeted ad based on the subject matter of the feed.

The tools to make these things happen might be a web scripting language and a database. There are many approaches. PHP MySQL anyone? Each entry is a new database record. The feed is generated every time someone visits a URL by a web script which loops through the database records to produce the feed items. It serves xml instead of html content, as we learned.

What if I’m a sys admin? I am concerned with the feed being somewhat private. First I might deliver feed via SSL https instead of http. That would encrypt the transmission and most feed readers are okay with this. After I get the SSL going, I make sure users have to call the URL with an MD5 feed key instead of a guessable numeric database primary key. This would also facilitate multiple feeds from the same web script based on which MD5 key was passed in.

Ok let’s say I’m not a programmer. I want a desktop program that can make the feed for me. Adding an entry is a manual process. I use a program called Feeder for this podcast feed. Learn more at reinventedsoftware.com/feeder.

Here are some features you’ll want. Of course you want fast programmer-less edits. You want it to use itunes tags when podcasting. You want it to publish the feed for you, like via FTP.

You want it to “ping” services that have to load your feed. iTunes, technorati and feedburner are three examples of services you may want to ping if you have accounts set up with them. If you want them to always have your latest and greatest, you’ll want to ping them. iTunes can be ping-ed with a basic GET request that includes you’re podcast ID, however most pinging happens via an XML POST that contains the ping command. Of course, a good desktop app will keep that behind the scenes.

So I mentioned feedburner.com . They are good for a couple things – reporting and redirecting. They have other services but let’s focus on these 2 free things

Basically you give people a feedburner URL instead of your feed’s direct URL. By doing that, you are free to change your feed’s direct URL without leaving subscribers with a dead subscription. So now feedburner is getting a bunch of requests that they can analyze and produce usage reports for you. Great reporting.

Without feedburner we’re responsible for returning http redirect headers when the feed’s direct URL changes.

When you are forced to put your feed at a different URL, you want to think about redirecting existing subscribers to the new URL. You may not care and your subscribers will get a 404 response. You may care a little and give your subscribers a 410 “gone” response. At least you’re leaving a note, even if it’s “screw you guys”. Don’t leave nasty notes.

A good response is 301 with a header location redirect. Since they ultimately got the feed they were looking for, they may not notice that you delivered it from left field instead of from straight ahead.

If you’re on Apache you can use .htaccess files for redirecting. In the folder with the old feed file, load a file named .htaccess and insert a “redirect permanent” line like

Redirect permanent /oldfeed.xml http://www.domain.com/newfeed.xml

If you just can’t get enough XML, you could return an XML redirect. That would be a file in the old location that only contains redirect and newlocation tags.

Don’t be afraid to dig into the XML flying all around us.

Share and Enjoy:
  • Digg
  • del.icio.us
  • Facebook
  • Google Bookmarks
  • E-mail this story to a friend!
  • LinkedIn
  • MySpace
  • Reddit
  • Slashdot
  • StumbleUpon
  • FriendFeed
  • Turn this article into a PDF!
  • Ping.fm
  • Twitter

Friends with RSS Feeds

Saturday, March 29th, 2008

The only thing shorter than my list of friends is my list of friends with RSS feeds. I’m lucky to have smart friends. Enjoy.

  1. Lorianne Tibbets – Will Write for Coffee (feed)
  2. Patrick Shen – my immortality project (feed)
Share and Enjoy:
  • Digg
  • del.icio.us
  • Facebook
  • Google Bookmarks
  • E-mail this story to a friend!
  • LinkedIn
  • MySpace
  • Reddit
  • Slashdot
  • StumbleUpon
  • FriendFeed
  • Turn this article into a PDF!
  • Ping.fm
  • Twitter

Top 5 Podcasts for March

Saturday, March 29th, 2008

Previous Podcast Post

Podcasts come and go like the beautiful underfunded whims they are. Here is the current morning line up.

(Links go directly to iTunes.)

  1. Mac OS Ken
  2. (weeklies) Cute With Chris, or No Agenda, or Amtrekker, or TWIT
  3. Front Page (of the New York Times)
  4. Tech5
  5. Mahalo Daily

Get an iPod, or better, an iPhone. Connect it directly into your car stereo. Start enjoying your commutes.

Share and Enjoy:
  • Digg
  • del.icio.us
  • Facebook
  • Google Bookmarks
  • E-mail this story to a friend!
  • LinkedIn
  • MySpace
  • Reddit
  • Slashdot
  • StumbleUpon
  • FriendFeed
  • Turn this article into a PDF!
  • Ping.fm
  • Twitter

5 Podcasts to Improve Your Daily Commute

Sunday, February 3rd, 2008

If you’re like me, you’ve grown tired of radio and want something with fewer commercials and better content for your daily commute. Explore podcasts. Improve the quality of your forced idle time by subscribing to podcasts from the iTunes podcast directory, hooking your iPod to your car stereo and hitting play. (Of course there are other MP3 players, and other media sources, but this is mine.)

Here are my 5 commuting faves – quick ones, 5 to 10 minutes each.

1. Mac OS Ken – A witty Apple news program by Ken Ray. It’s concise and worth a listen. My commute always starts with this one. Weekdays. 10 to 15 minutes.

2. Front Page – Get the world headlines with this daily New York Times snapshot. Every day. 5 minutes.

3. Tech5 – A daily tech news review. The pessimistic host, John C. Dvorak, will grow on you. Best to take him lightly. Weekdays. 5 minutes.

4. Tekzilla – Practical geek tips, often presented by listeners and more applicable. It’s a video program, so if you’re driving, just listen while you watch the road. Weekly 45 minutes, plus quicker 5 minute episodes.

5. Podictionary – My vocabulary sucks, but nevertheless I’m fascinated by word origins. Ironic. Good thing there are informed people like Charles Hodgson to help. 5 minutes.

The type of commute may dictate the best podcast format to listen to or watch – long or short, audio or video. If you take the train, video podcasts would be reasonable. Your eyes are safely distracted. Otherwise stick to audio podcasts for the commute. If your commute involves lots of stoplights, like mine, shorter audio podcasts work. Red lights provide frequent breaks to safely proceed to the next podcast. If your commute involves the freeway, longer audio podcasts would be safest.

The important thing is, just like with cell phones, I’m careful not to let my device distract me from the primary task of driving.

Feel free to comment with your podcasts! I’d love to check them out. There are so many out there, something for everyone.

Share and Enjoy:
  • Digg
  • del.icio.us
  • Facebook
  • Google Bookmarks
  • E-mail this story to a friend!
  • LinkedIn
  • MySpace
  • Reddit
  • Slashdot
  • StumbleUpon
  • FriendFeed
  • Turn this article into a PDF!
  • Ping.fm
  • Twitter