Saturday, November 22, 2008

rcFeedMe - A Fully Automated Podcatcher



rcFeedMe.zip (v1.44 - 11/8/2011) (source is included)

rcFeedMe is now Open Source (Apache License, Version 2.0) and the latest source is located here :
https://bitbucket.org/arogan/rcfeedme/overview

- So I've been using Juice for a while as my podcatcher. It works ok most of the time but there were a few things that annoyed me: Sometimes downloads would just hang, no way to group feeds into different download directories, and sometimes something about a feed would change and all of the files would download. That last one might not be Juice's fault but still there are ways to solve this problem in the podcatcher.
- So I went looking to see if there was a better free alternative. Also, I'm one of those people who refuses to install itunes. I tried a few from this site:
http://www.podcatchermatrix.org/ but most of them had more issues than Juice with interfaces that seemed to make the whole process of configuring your feeds a chore.
- So I did what any programmer would do. I wrote my own. rcFeedMe is a podcatcher - basically it reads rss feeds and downloads them unattended all automatically on a schedule. It is NOT a rss reader. I use google reader for that. Once you set it up you shouldn't have to touch it anymore except when you want to add/remove feeds.
- Tip: If you find a show that only has an itunes feed and not a regular rss xml one read this article. I found it quite helpful.

Design goals/features:
- Quick and easy interface. It might not be that pretty but I wanted something that I could quickly enter settings for all aspects of the program.
- OPML import/export.
- Complete automation. Once setup, it just runs in the tray and checks the feeds for new downloads at an interval you set. No annoying popups or notifications. I just want it to silently do its thing. Downloads have to be as robust as possible and retry automatically if the download stalls.
- Maintain a history of every file downloaded so you will never get flooded with downloads when a feed changes or download something you've previously downloaded.
- Flexible system to determine where feed downloads are saved.
- Multi threaded with the ability to set the number of threads to use.
- Some basic abilities to sync to a mp3 player. I want to keep this program pretty focused on just downloading.
- Clean install. You just unzip the files and run so it's completely self contained and portable. All persisted data is stored in clear xml files.
- Source included

Developer Notes:
- Written in Visual Studio 2008 - c# targeting .net 3.5 framework. It took about a week to write the core functionality. It took another week to flesh out a few features and bug fixes.
- Utilizes Windows Communication Foundation (WCF) SyndicationFeed. I coded around their publish date parsing bug. Try it with ask a ninja and you'll see. SyndicationFeed isn't a very robust class and there were several feeds I had to massage the xml before it would load into a SyndicationFeed.
- I started with a pretty clean OO design but it broke down toward the end with me taking all sorts of shortcuts especially with presentation elements getting mixed in (mostly to help performance). This also means the memory footprint grew a bit too though cpu utilization is low.
- All persisted data is in clear xml files.
- Thread management is a bit of a mess. I'm sure there is a much better/cleaner way to do things.
- Overall life cycle of a file: First you subscribe to a feed (class Feed), I refresh that feed and get a list of all files (MediaFeed which is a keyed collection of Media; attribute on Feed). At this point in time I figure out what the filename will be and verify that a good link exists pointing to the file that will be downloaded. Next, the queue (class Queue which is a collection of Media) is built taking into account check keep counts and history. I then process the queue downloading all the files trying to spread download across different sites (this algorithm can be improved, it's very simplified right now). Once the entire queue is finished (or the user stops the downloads) I cleanup the queue. All successfully downloaded files are moved to History (collection of Media), all the ones stopped or errored are moved to errors (temporary collection of Media, never persisted to disk). Then I trim files based on keep count. It's a very safe and conservative trim. I walk the save paths resolved from Feeds (collection of Feed), I then search history by filename, if I find a match in history then it's considered for deletion based on save flag and keep count. Keyed collections were used instead of dictionaries because I needed serialization.
- I don't check free space anywhere so ummm yeah manage your disk space. The idea is keep count should keep space usage pretty constant.
- CleanupConfig.xml. This file is used to help cleanup the raw xml that causes issues with SyndicationFeed class. This has two sections in it. One is for search and replace, the other removes text between the start and end strings inclusive. In the future, as more problematic feeds are reported and fixed, hopefully I can just release an updated CleanupConfig.xml file instead of doing a new build. There is no gui for this file so if you want to manually edit it just use notepad.
- Other code used:
http://www.asp101.com/articles/christopher/filesystem/default.asp
http://www.brunofigueiredo.com/post/c-OPML-Parser.aspx (make sure you apply the fixes mentioned in the comments)
http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=176890&SiteID=1 - DataGridView progress bar
http://www.codeproject.com/KB/grid/DataGridViewColumnOrder.aspx - Save DataGridView column widths and position. Modified to persist settings in appSettings instead of user.config to keep rcFeedMe portable.
http://www.codeproject.com/KB/grid/CheckBoxHeaderCell.aspx?fid=452379 - Checkbox in DataGridView column header
http://developer.novell.com/wiki/index.php/TagLib_Sharp - ID3 Tag Library
http://www.veryicon.com/pack/aqua-smiles/ (icons)
- Thanks to coolmoedee345 for helping with the testing who listens and watches more podcasts than I think is humanly possible which of course made him the ideal tester.


Requirements:
- Windows XP, Vista, or Windows 7 (I've tested in both 32bit and 64bit)
- .net framework 3.5

Quick Start:
- Import an opml file or enter the url for the feeds you are interested in.
- Update the path on the settings tab to where you want to save your downloads.
- Click Start.

Usage:
- All the labels on the settings tab and many headers on the grids have tooltip text. That should help clarify some things.
- Command line parameters:
"1" or "auto" - (without quotes) to start in automatic mode. This means the app will minimize and the scheduler will start. This is useful as a shortcut in your startup group.
"2" or "lite" - The same as 1 except it puts it into "lite mode" which uses less memory but doesn't let you interact much at all with the program. Memory usage will still be pretty high when it's actually processing the download queue, but I try to free up as much resources as possible when rcFeedMe is idle and waiting for the next scheduled run.
"3" or "once" - Run Once: It's just like 1 except after finishing the run it will automatically exit. Useful if you would rather schedule rcFeedMe with windows scheduler and not have it running in the tray all the time.
"4" or "hidden" - This is exactly like run once but you never see a window. The only thing you will see is the system tray icon while it is running. This should be a lot less annoying than run once. Note: The gui is completely disabled in this mode so double clicking on the system tray icon does nothing.
- Right click on the grids for additional functionality. You can multi select rows in most grids by clicking the white space to the left of the row. Hold down ctrl to select multiple rows, shift for ranges of rows, and click the square in the top left corner to select all rows.
- Grids can be sorted by clicking on the header of the column.
- It should use proxy information that is setup for IE automatically though I haven't tested this.
- While a file is in the process of being downloaded it will have an extension of .rcFeedMeTemp. This extension is removed once the file has successfully completed downloading.

Main Menu:
File | Import opml: Import an opml file. Feeds with a url that already exists in the grid will be skipped.
File | Export opml: Export feeds to an opml file.
About: Shows the current version and a link to this post.
Debug | Save Settings: Not usually needed since Refresh All feeds saves all settings. Settings are also automatically saved when you close the application.
Debug | Load Settings: Not usually needed since Refresh All feeds saves all settings. Settings are automatically loaded on application start up.
Debug | Dump Raw XML Feeds: If you have a feed giving you problems (no files showing up/ERROR status when refreshing) you might want a raw dump of the rss feed. This will save the file in the same directory as the rcFeedMe.exe. It also dumps a fixed version if it's a problematic feed that I had to massage a bit to get working.
Debug | Sync Count: Count how many files that need to be synced and display it on the Sync button as Sync (n) where n = number of files that still need to be synced.

Toolbar:
Start: Start the scheduler.
Stop: Stop the scheduler.
Save then Reload Settings: Click this if you changed something and want to save it immediately but don't want to refresh all the feeds.
Refresh All Feeds: Saves all Settings, re-loads all settings, and refreshes all feeds. This is the button you want to click after you've added/removed feeds.
Process All Feeds: Does a Refresh All feeds, builds the queue, and processes the queue. This is the same thing that the scheduler runs.
Sync: Syncs files to a path of your choice (usually to a mp3 player or something). Paths are defined just like Save Paths but instead use the Sync Path fields (in feeds, groups, and settings). Sync is really a one way copy from where the file was downloaded to the Sync Path. Once a file has been copied, it is flagged as Synced in History so it won't sync again even if you delete the file on the mp3 player. You can always uncheck Synced in the history if you want to sync the file again. One small difference is sync paths are created if they do not exist (this includes even base directories). After the downloads are finished it will count how many files that need to be synced and display it on the Sync button as Sync (n) where n = number of files that still need to be synced. This lets me know in the morning if I have new podcasts waiting for me to be synced to my mp3 player. You can also get a count immediately with the Sync Count menu item under Debug.
Search History: Type in the textbox any substring you want to search in your history on Feed Name, Title, and File Name.

Status bar:
- First area is just general information on the last action.
- The next grey label area is the current overall status of rcFeedMe.
Idle - not much is happening.
Feed - Feeds are in the process of being refreshed.
Queue - Queue is being processed. Files are being downloaded.
Sync - Files are being copied to your mp3 player.
You will see the word [Lite] next to the status if you are running in low memory usage Lite mode.
Depending on the status, certain functions might not be available.
- Progress Bar shows status of either the feeds refreshing or the download queue. Download progress is only updated every percentage change.
- The tray icon will show teeth while rcFeedMe is being fed (downloading). Actually, it's anytime it is not idle.

Feeds Tab:

At the top is a grid displaying all your rss feeds and a variety of parameters to customize how files are downloaded. At the bottom displays all the files associated with the feed you clicked on in the top grid. Note: You must click "Refresh All Feeds" before you will see anything in the lower grid.
- How the save path of a file is determined: If a path is specified at the feed level, then that exact path will be used. Next, if a group is specified, then the group path will be used as the base path, a sub directory with the feed name will be created in this base path, and then the file will be saved in that sub directory. Finally, if a feed has no save path defined, group is set to (None), then the Save Path on the Settings Tab will be used as the base path. The same sub directory logic is used as in the group scenario. Note: Sync Path follows the same logic.
- Please make sure all base directories are created first or the program will throw an error.

Feeds Grid (top):
- Name: The name of the feed. Please be careful when editing this field. It's best not to touch it once you've begun downloading files for the feed since I use feed name + file name as the key to determine if something has already been downloaded (history).
- Url: The address that points to the feed. If you enter this first and leave Name blank then it will try and fetch the name of the feed for you. It's not quite as robust as refreshing the feed so if this fails, try typing the feed name manually and then refreshing the feed. There is a chance it still might work.
- Group: Any groups defined on the group tab will be in this drop down. Assign this feed to a group or leave it set to (None).
- Keep Count: This number determines two things. 1) It determines the maximum number of possible files that can be downloaded for the feed. So if it is set to 5 then at most 5 of the most recent files will be downloaded for this feed. 2) It also determines the maximum number of shows to keep on your hard drive. So if you download the latest 5 files, then a new one shows up in the rss feed, then the new show will be downloaded and the oldest existing file will be deleted. This is the primary method of managing disk space. By specifying keep counts your disk space used shouldn't change much. Set it to 0 to disable the feed. Enter -1 if you want to download and keep everything. I've seen other podcatchers use number of days or something like that which I find inconvenient since different feeds publish new shows at different or irregular intervals.
- Include Extension: Pipe(|) delimited list of file extensions(include period(.)) to match against for downloads. Sometimes undesirable file types get included into the feed (say a .aspx or something) or the feed is a consolidated feed with the same show in different file formats (say wmv, mov, avi, etc). This lets you filter down the files to download by a list of extensions. Example: if you only want wmv files to download from the feed then enter ".wmv" (with the period but no quotes, search is case insensitive). If you wanted wmv and mov files then enter ".wmv|.mov".
- Include Text: pipe(|) delimited list of substrings that are compared to the filenames for download. This is similar to extension but is a substring search of the entire filename. If it finds a matches then the file will be downloaded. This is useful if a feed contains many different versions of the same show in the same file format but at different quality levels/bit rates. Example: a feed has Show01High.wmv,Show01Med.wmv,Show01Low.wmv. If you wanted just the high quality ones you would enter "High" (without the quotes, search is case insensitive).
- Save Path: Enter the complete path where the podcasts for this feed will be saved. This path overrides all other paths (groups and global settings). If you use this feature please make sure you specify a unique path here that is NOT shared with any other feed. Having more than one feed save to the same path will cause unpredictable results.
- Prepend Date? Prepend the Publish Date to filename. Useful to make the filenames unique or help with sorting. Date format used is YYYYMMDD so it sorts propery. If both Prepend Date and Prepend Title are checked, then Date comes first.
- Prepend Title?: Prepend title to filename. Useful to make the filenames unique if only a generic filename is used in the feed. Take a look at the GameTrailers feeds. All the filenames are just download.mp4. Checking this will take the title of the show and prepend it to the filename. So download.mp4 will become "show title - download.mp4".
- Sync Path: Just like save Path but defines where files are copied to when you click the Sync button.
- Sync?: Check if you want files from this feed to sync to your mp3 player.
- Apply Cache Fix?: Appends a fake query string to the end of the url when making the request for the feed. This guarantees a cached copy will not be used but also causes some sites to throw an error. Most sites don't need this. If you do check this, then do a refresh and make sure the feed still works. To be honest I'm not sure if this "fix" really helps anything. I notice on a few feeds (like hdtv podcast) where there is a bit of a lag before the latest show appears in the feed. What is odd is if you view the feed in a browser (and therefore the xslt is applied) you see the latest version of the rss feed. If you instead grab the raw xml (I use webClient but trying httpwebrequest doesn't make a difference) it's the previous version and takes a while before it's updated. If I use WebBrowser class then yeah I get the latest data but it's after the xslt is applied so it's html and no longer rss xml. It's pretty odd. It only happens on a few feeds, and it's not a big deal since the raw rss xml eventually does get updated. If any programmers out there can explain to me what is happening, why, and maybe how to fix it that would be great.
- Username: Enter a user name if the site requires login credentials.
- Password: Enter a password if the site requires login credentials. Once you save, this field will show only the encrypted version (I also save it encrypted in the feeds.xml file). Type over it to change it or just erase it if you no longer need a password.
- Priority: Used to help sort mp3's during playlist creation. See the playlist section below for more information. Leave it blank to exclude the podcast from the playlist.
- Decode?: Apply url decode to filename
- Prepend Short Date? Just like Prepend Date but a shorter format: M/D
- Post DL Exe: This command will be executed after every single media that is downloaded from this feed. You can also specify a .cmd batch file if you wish. This process is launched asynchronously when a download is completed successfully (user aborts and failures do not count). Do NOT use double quotes even if the path contains a space.
- Post DL Args: Include any arguments needed to run the above exe. Please use double quotes around anything that might contain spaces (even tokens).
- Pre Sync Exe: Like the post except this executes right before the copy is done during a sync. This is executed synchronously.
- Pre Sync Args: arguments for pre sync exe.
- Suppress Sync: This suppresses the actual copy command that Sync usually does but it still does all the other things that sync normally does like updating the sync flag in history. This is very useful if the pre sync exe will actually be doing the copying for you.

available substitution tokens:
%filename% - just the file name of what was downloaded
%fullname% - complete path and filename
%savepath% - where the file is located
%groupname% - group name (if one exists)
%syncpath% - sync path
%feedname% - feed name
%date% - publish date in the same format as prepend date (yyyymmdd)

example: I want my mp3's split into 20 min chunks when I sync to my usb thumb drive.
First, you need a command line utility that will split mp3's. I like mp3split.
Next, I would setup my feed as follows:
feed name: This is Only a Test
Sync path: d:\portableDrive
post dl exe: blank
post dl args: blank
pre sync exe: C:\Apps\mp3splt_2.4_i386\mp3splt.exe
pre sync args: -t 20.0 -d "%syncpath%\%feedname%" "%fullname%"
Suppress Sync: checked

After token substitution it would generate the following:
Pre Sync exe = C:\Apps\mp3splt_2.4_i386\mp3splt.exe
Pre Sync args = -t 20.0 -d "d:\portableDrive\This is Only a Test" "c:\podcasts\This is Only a Test\thisisonlyatest-90-20111103.mp3"

Notice how you can combine tokens but just be careful where you place your double quotes.
What this would do instead of just copying the mp3 to the sync path it would instead execute the above which would chop up the mp3 into 20min chunks and output them to the destination directory d:\portableDrive\This is Only a Test.

- Right Click menu lets you abort refreshing feeds.

Files Grid (bottom)
- Add to Queue: Manually add the file to the download queue. The file will be automatically flagged as "Keep". See History Tab for more information.
- In Queue?: Read only - checked if the file is already in the queue.
- Title: Show title. This is the text used in the filename if Prepend Title? was checked above.
- Summary: Show summary
- Publish Date: Date the show was published. This is the date used in determining the latest shows to download. By default the grid is sorted descending on this field.
- File Name: Filename that will be used when downloading and saving the file.
- Link: The url pointing to the actual file that will be downloaded.

Download Queue Tab:

- I attempt to spread the downloads across different feeds to help maximize speed and thread efficiency.
- Feed Name: Name of the feed.
- Title: Show Title
- Link: The url pointing to the actual file that will be downloaded.
- File Name: Complete path and filename of what is being downloaded and saved.
- Publish Date: Date the show was published.
- Status:
Queued - In the download queue but has not started downloading yet.
Downloading - Download is in progress.
Downloaded - File was downloaded successfully.
Error - File was not downloaded properly. Some sort of error occurred. Partially downloaded files in error status are automatically deleted.
Stopped - User right clicked on the Queue grid and selected "Stop All Downloads". Any downloads in progress will be stopped immediately and have this status. The partially downloaded file will be deleted.
Bytes - Number of bytes currently downloaded/Total Bytes.
Progress - % downloaded
Attempt - How many times rcFeedMe tried to download this file. Once attempts hits the download retry maximum (set in the Settings tab) then the file will be set to Error status.

Download Queue Context Sensitive Menu (right click):
- Build Queue: Not used often unless you want manual control over some of the steps that "Process All Feeds" does. Click this after doing a Refresh All Feeds. It will build the queue but not actually start downloading yet.
- Process Queue: Begins downloading the files in the queue.
- Stop all Downloads: Immediately stop all downloads and halts the processing of the queue.
- Remove Selected Rows: Remove the files from the download queue.
- Move Selected Rows to History. This is more for debug but it might come in handy. After build queue you can move rows from the queue to History. This will trick the program into thinking these files were already downloaded. Useful in some situations where you want to pre-populate the history. Maybe there is a specific show you just don't want to download.

Download History Tab:

- After the queue is completely finished processing, all successfully downloaded files are moved here. The history will continue to grow over time. It might be a good idea to trim it down once in a while by sorting by date and then selecting multiple rows of the oldest data and deleting it.
- I won't go through all the columns since I've already describe most of them by now. Three columns are editable on this grid: Title, File Name, and Keep?.
- I allow editing of Title and File Name since these combined is the primary key I use to determine if a show has already been downloaded. This is just in case somebody wants to tweak something in the history.
- Keep?: Checking this prevents rcFeedMe from deleting the file. This goes along with Keep Count back on the feed tab. Any files you added manually from the files grid on the feeds tab will have this already checked. It makes sense because you added the file manually (and not by the scheduling engine) so the scheduler shouldn't delete it. Kept shows are excluded from the keep count calculation.
- Right click | purge history: This will clear the entire history file (a backup is created though just in case). Useful if you want to reset and re-download
everything.
- Synced?: If checked it means the file has already been synced to your mp3 player so it won't get synced again in the future. I you wish to sync the file again for some reason just uncheck this box.
- Make sure you save any changes made before leaving this tab since this tab gets refreshed whenever you click on it.

Recent Errors Tab:

- After the queue is completely finished processing, all the files that failed to download are moved here. This only shows files that error out on the most recent run. The data in this tab is not persisted anywhere so if you close rcFeedMe and then restart it, it will be blank. If you want to see a running history of all the errors from this tab, please see Errors Log Tab.

Group Defaults Tab:

- Save Path: Specify the download path for the group. The groups you define here appear in the drop down on the Feeds tab.
- Sync Path: Default path to sync to for the group.
- Playlist Path: If a path is specified then a playlist will be created at the end of the batch run for all podcasts in this group with a priority specified.
- Playlist Cutoff Days: Go back this many days to figure out how many podcasts to include. This is based on the podcast publish date.

Settings Tab:

These are your global preferences.
- Frequency: This is how often the scheduler should refresh all feeds and download any new files. Default is 360 minutes (every 6 hrs).
- Default Save Path: If a feed doesn't have a path specified and group is set to none, then this path will be used as the base path. See Feeds Tab for a more detailed explanation of how the download path is determined for a file.
- Default Sync Path: If a feed doesn't have a path specified and group is set to none, then this path will be used as the base Sync path.
- Keep Count: Default value used to populate the Keep Count field on the feeds tab when importing an opml file. Default is 5. See Feeds Tab for more information.
- Threads used to Scan: Number of threads to use when refreshing the rss feeds. Default is 6. Maximum allowed is 16.
- Threads used to download: Number of threads to use when downloading files. I would recommend 2. Any higher and some sites limit the number of simultaneous downloads and could produce false failed downloads. I try to minimize this by spreading out simultaneous downloads across different feeds. Maximum allowed is 6.
- Scan Stall: Number of seconds that elapse of inactivity while a feed is being refreshed. If the feed refresh stalls for this many seconds, it will be considered an error.
- Download Stall: Number of seconds that elapse of inactivity while a file is being downloaded. If the download stalls for this many seconds, it will be considered an error.
- Scan Timeout: If the total time for a feed refresh takes longer than this many seconds, it is considered an error.
- Download Timeout: If the total time for a file download takes longer than this many seconds, it is considered an error.
- Scan Retry: When an error occurs during a feed refresh, retry this many times before giving up.
- Download Retry: When an error occurs during a file download, retry this many times before giving up.
- Time Zones (Advanced): Used to help parse Publish Date. Sometimes it helps to strip out the time zone.
- Date Formats (Advanced): Used to help parse Publish Date. Specify the .net Custom Date and Time Format string. see: http://msdn.microsoft.com/en-us/library/8kb3ddd4.aspx

Log Tab:

- A very detailed log of everything that happens in the program. The log self trims so it won't grow in size forever. The log is refreshed every time you go to this tab or you can right click and refresh immediately. The log will grow until it hits 1.5MB. After that the oldest entries will be deleted and the log will be automatically trimmed down to 1MB. This cycle just keeps repeating. Log size check and trimming occurs when you process your queue.

Error Log Tab:

- Since the Recent Errors Tab isn't saved anywhere I decided to create a separate error log. The information is also buried in the complete log but this log only contains information that was in the recent errors tab. This log also self trims.

Playlist Creation
- This is a new feature I added basically to use with subsonic, isub, and my iphone and it's a new way for me to listen to podcasts on my commute without ever having to use itunes/sync again.
- This is all optional. You start by specifying a full path and filename in Group Defaults tab. So say I want a .m3u playlist generated for my audio podcasts. I would enter something like:
c:\subsonicPlaylists\AudioPodcasts.m3u.
For this to work properly with subsonic/iSub I setup in subsonic|settings|general|Playlist Folder to:
C:\subsonicPlaylists
- Next, you put in a playlist cutoffdays value in group defaults. This tells how far back to include podcasts. So if I use 14 that means only podcasts with a publish date within the last two weeks would be included in the playlist.
- Finally, on the feeds tab you need to give a priority number for each podcast in that group you want included in the playlist. Leave priority blank if you want that podcasts excluded from the playlist.
example:
The Instance - 5
GWJ - 10
Maximum PC - 15
TWIT - 20
I like to number by 5's so it's easier if I need to re-prioritize later.
- At the end of a process all feeds (which happens in any schedule run or run once) I generate the playlist as follows:
For any groups with a playlist path and cut off days defined, find all podcasts with a priority. I start with the lowest priority first. I get the most recent podcast (within cutoff days) and add that to the playlist first. Then I get the next lowest priority and the most recent podcast for that feed, and so on. Once I've walked all the feeds with priorities defined, I start over again at the top and pick up any remaining podcasts that fall within the cutoff days and throw them at the bottom of the playlist.
- NOTE: I retag the mp3 files when I do this with a short date: M/D-title. I have to do it b/c this is how I want to see it in iSub and it allows me the quickest way to scan for new podcasts from the feeds I care about the most. The original tag is written to the ID3 comment field with a prefix of [ORI].
- NOTE: Only file extensions with .mp3 are supported.
- Then in iSub I go to playlists|server and pick AudioPodcasts. With the bookmarking system in iSub this works out really well and lets me listen to podcasts on my commute but music while at the office.
- So what you end up with all of this is a constantly refreshing (however you schedule rcFeedMe to run) server playlist that I can hit with iSub, quickly glance over and see any new podcasts with the ones I care about most bubbling to the top of the list (instead of a strict latest date sort), all streaming without ever having to sync to my iPhone. I've been listening to podcasts this way for the last few weeks and it's been working great. Of course this helps if you have an unlimited data plan which I do.

Upgrading from a previous version
- Usually you only need to copy over the following files:
rcFeedMe.exe
rcCommmon.dll
gfoidl.DataGridView.dll
taglib-sharp.dll
cleanupconfig.xml (if you have added your own entries in this file you'll have to manually merge them)

Hopefully you find this program useful. Leave any questions or suggestions in the comments, and I'll try to respond.

UPDATE: If you have issues with a specific feed give http://feedvalidator.org/ a try. It's quite helpful in pointing out where the problem might be.

Version History:

- I've been making some updates to this release. It is now on version 1.44f. If you aren't running the latest version please re-download.
- Fixed a bug where you couldn't erase a playlist path from the groups tab.
- Known issue: adding a new feed by entering a url and them immediately checking any checkbox will throw an error. The work around is simply to either leave the row first or save your settings before checking any of the checkboxes.
- There is a new setting: Log duplicate errors. By default these are not logged since they are mostly harmless and just clutter up the log. This happens when a feed happens to list the exact same file twice.
- Maximum log size has been increased from 1.5 MB to 10 MB.
- Added 5 new columns to feeds: Post DL Exe, Post DL Args, Pre Sync Exe, Pre Sync Args, Suppress Sync. Please see the above section Feeds Tab for more details. This adds the ability to hook in your own post download process or pre sync process.
- Added new command line option 4 (or "hidden"). This is just like run once but you will never see a window (only the system tray icon).
- Made a preliminary fix to an obscure error when syncing (or getting sync count): "An item with the same key has already been added."
- Properly set status to idle before calling create play list to prevent the busy error during a batch run.
- NOTE: The source has been updated to visual studio 2010 (though it still targets the .net 3.5 framework).

v1.43 (8/22/2010)
- Added Url Decode option on filename
- Added ability to generate playlists. See above for more Information.
- Note: I now change last modified date to equal publish date after the file has been downloaded.

v1.42 (4/4/2010)
- Added a fix for EA podcast in the cleanupconfig.xml
- Added preliminary support for userid/password for sites that require it. I wasn't really able to really test it because I don't have any podcasts that require a login. See the Feeds Tab section above for more information.

v1.41 (11/5/09)
- This is mainly user interface tweaks. All checkbox columns now have a checkbox in the column header so you can easily toggle the checkboxes for all rows.
- All grid column positions and widths are now saved. Even the split location on the first tab is saved. All these settings are saved in rcFeedMe.exe.Config. So if you want to reset just delete the appSettings section.

v1.40 (11/2/09)
- added httpWebRequest useUnsafeHeaderParsing="true" to rcFeedMe.exe.config. This fixed an issue with a specific feed. You can read more about the issue here.
- cleanupconfig.xml is now a lot more flexible. You can now add fixes that only apply to a particular feed url instead of globally to all feeds. This allows me to add specific feed fixes without the risk of breaking some other feed. Just add the optional attribute feedUrl to either ReplaceStruct or RemoveStruct. If you leave feedUrl attribute off then it still applies globally.
- Added Restore to the right click menu on the tray icon. It does the same thing as double clicking the tray icon.

v1.39 (4/21/09)
- updated CleanupConfig.xml to handle another problem feed. Fixed a rare issue where if a filename differed only in case it would hang the feed refresh process.

v1.38 (2/22/09)
- Fixed a bug where I was never using DL stall time from the config (I was accidentally using scan stall). Also, rcFeedMe no longer shows up in the alt-tab list when minimized to the tray.

v1.37 (2/16/09)
- Fixed a problem with idle thumbs feed. Added better error logging when there is an xml parsing error. I now try and extract the xml fragment that was causing the problems and save that into the log.

v1.36 (1/27/09)
- Some feeds are missing the type attribute on the enclosure tag. I've added some code that should work around this issue.

v1.35 (1/11/09)
- added new Error Log tab. Anything that ever shows up in the Recent Errors tab is archived in this log. Sure the same information is also captured in the compete Log but this is easier to read if you just want to look at your failed downloads. This log also self trims.

v1.34 (1/7/09)
- added a size check. When media is finished downloading, I check received bytes against total bytes expected. If it received less then it is considered an error.
- latest CleanupConfig.xml is rolled into this release

v1.33 (12/12/08)
- fixed a tiny bug where I forgot to reset the tray icon tool tip text and form title after doing a sync.

v1.32 (12/11/08)
- Files that are too long are now trimmed down. This can happen if you prepend the Title and it happens to be a very long title. I've set maximum file name length to be 240. It will be less than this since I subtract out the full path. Any filenames that needed trimming will show up in the log.
- Fixed a bug when deleting rows out of the Groups tab.

v1.31 (12/10/08)
- Improved error messages in the log if a download fails.
- Sync Count now also displays in the window title and the tooltip text for the tray icon.

v1.30 (12/03/08)
- added a new feature: Sync Count. After the downloads are finished it will count how many files that need to be synced and display it on the Sync button as Sync (n) where n = number of files that still need to be synced. This lets me know in the morning if I have new podcasts waiting for me to be synced to my mp3 player. You can also get a count immediately with the Sync Count menu item under Debug.

v1.29 (12/03/08)
- fixed a bug introduced in the new feature in v1.27 and made worse in v1.28: Crash bug when editing existing feeds.

v1.28 (12/02/08)
- fixed a bug where if you had no group assigned to the feed, clicking add to queue button would cause the app to blow up.
- fixed a crash bug where feed name was null when it is trying to fetch the name.
- added a new xml file: CleanupConfig.xml. This file is used to help cleanup the raw xml that causes issues with SyndicationFeed class. This has two sections in it. One is for search and replace, the other removes text between the start and end strings inclusive. In the future, as more problematic feeds are reported and fixed, hopefully I can just release an updated CleanupConfig.xml file instead of doing a new build. There is no gui for this file so if you want to manually edit it just use notepad.

v1.27 (12/01/08)
- When subscribing to new feeds, if you enter the url first and leave the Name blank, then it will try and fetch the name of the feed for you.

v1.26 (11/24/08)
- fixed some problem feeds that use dc:date instead of pubDate tag in the rss xml.

v1.25 (11/21/2008)
- Initial public release

Monday, November 17, 2008

Little Big Planet (PS3)


- I've only spent about 4-5 hrs with the game so far.
- I don't think I've seen a console game this hyped in a long time. In a lot of way it reminds me of Spore. Not the game itself but how it was perceived and received. There is this huge hype machine surrounding it, how the user created content was going to change the world, make the platform really take off, etc. Then, when it came out it was a good solid game but not nearly as ground breaking as we expected.
- At it's core it's a very capable pretty physics based mostly 2D platformer with some nice 3d graphics. It does have a certain quirky charm about it thanks to the narrator.
- It's very family friendly as far as not having anything offensive in the game. The game play itself is a different story.
- The difficulty ramps up and rather quickly. It didn't take long before my 6yr old was having problems keeping up. It wasn't soon after before I found I was having problems finishing a level. It uses the old lives system. What's worse are the lives pool is shared among all the players. So if you are playing with players with a large range of skills, it's possible that the less skilled players could drain all the lives rather quickly. When you run out of lives you have to start the whole level over again. So even though there are ample number of continue points, the whole lives thing kind of ruins things.
- I also found the camera kind of troublesome. I tries to focus on the player ahead. If another player lags too far behind, they will warp to the lead player when they cross the next check point. The problem is sometimes the camera would focus on the lagging player instead of the leader. I think I would have preferred a zoom in/out and dynamically split the screen when needed. I guess it would get pretty busy if the screen had to split 4 ways and then merge again when the players came together.
- I didn't mess with the content creating side of things yet. I really don't have the time right now but it does seem interesting.
- Browsing user levels seemed a bit cumbersome with levels arranged on planets that you zoom in and out of. I played a few that would earn you 7 trophies in 30 seconds or whatever. I did find a few original "real" levels that were quite fun.
- I'm a bit burnt out on traditional 2d platformers. OK, I was never really that much into them in the first place.
- My kids still really like this game. So it might be ok if you don't mind sticking to easier user generated content or the first few easier levels included in the game.

This game kind of reminds me of the first Viva Pinata. It was a cutesy game tied to a kids cartoon with pretty deep game play. Here you have what a appears to be a very family friendly cutesy platformer but with a punishing difficulty curve and lives system. Add to that a very deep and elaborate creation system that I'm sure the hard core will gobble up. Who are they targeting with this title? I can see myself having some co-op fun with it but I can't recommend it at full price as a family game (maybe <= $30). It will probably be worth full price if you are really interested in creating levels.

Sunday, November 16, 2008

Gears of War 2 (XBOX 360)



WARNING: Potential minor non-story related spoilers though there will be some Gears 1 spoilers as I make some comparisons.
- Finished the single player campaign on normal which was a mistake. It was way too easy. I really should have played it on hardcore. I think it would have been much more enjoyable. It feel a bit longer than Gears 1.
- There are several welcome new features: new weapons, a ability to stick grenades to surfaces to act as proximity mines, meat shields, new execution moves, and an improved cover system (less sticky) though I still had some issues where I would side roll instead of mantle over a ridge when in low cover. This wasn't that big of an issue except for one level where a roll instead of mantle meant instant death.
- Continue points are spaced out nicely keeping frustration to a minimum.
- Graphics and art work is even more impressive than the first game (which was already amazing). Some of the underground vistas are just breathtaking.
- I dig the new soundtrack.
- The new Horde co-op mode is a lot better than it first sounded like on paper. The on the fly tactics, and the true need of teamwork really shines. There are 50 waves. After every 10 it kind of resets but the enemy gets stronger. I spent maybe 6 hrs in this mode, and I feel like I've just scratched the surface. I can see myself playing this mode for a long time. The new weapons, and enemies work really well in horde mode. There's a good variety and really encourages the team to cover all the different engagement aspects.
- Bots have been added to the adversarial modes. I haven't tried the multi player much yet beyond Horde.
- It's gears, more of what you would expect with the core mechanics enhanced and refined.
- Reavers, Corpsers, and Brumaks oh my: Overall, I was a bit disappointed with the single player campaign. I actually thought Gears 1 was better. It had more memorable set piece battles, better boss battles, and better use of the environment for unique game play scenarios (Though gears 2 did have one that I thoroughly enjoyed, there just wasn't enough of that). Things in Gears 1 just felt more weighty and had a bigger impact. I still remember when you first fought a berserker, how you had to use sound, the first time the Hammer of Dawn was introduced, the first time you fought a boomer, seeder, reaver, or corpser. Each was a memorable and epic set piece battle. In Gears 2 it seems to happen a lot less often. Weapons and enemies are introduced with little hoopla. All the big bosses from the first game are just kind of thrown in like cannon fodder with nothing adequate to replace them. Also, the split paths seemed to have more impact in the first game (like when you had to keep the light on your partner or the Kryll would tear them a part). The final boss is probably the most disappointing in that it really wasn't this epic battle I was expecting. I liked how in Gears 1 it combined several things you learned throughout the game into a single tough boss. Also, just a personal preference thing: I like gears 1 "destroyed beauty" setting more than the large portions of underground areas in Gears 2. I guess in the end it just felt like I was going through the motions to reveal more of the pretty ridiculous story that I really didn't buy. I felt the characters were all acting rather irrational. It just felt like more thought and care went into crafting the gears 1 campaign than this one. Again, a lot of this might be because I made the mistake of playing it on normal.
- The first vehicle level is a bit frustrating and doesn't control all that great. The later on rails levels are pretty exciting and even give you limited movement control to dodge attacks (sort of like Panzer Dragoon...ok maybe just a little bit).
- Instead of just collection cog tags, this time you are collecting a variety of items like journal entries and letters which add a bit to the story.

Highly recommended. Sure I had some gripes about the campaign content, but the game play mechanics, excellent multi player, amazing production values still make it a must have. And don't get me wrong. I still really enjoyed the campaign for the most part just not as much as the first Gears of War.

Saturday, November 15, 2008

Mirror's Edge (XBOX 360)



- Finished in about 7 hrs. After that there are speed runs and time trials. The later has a neat ghost mode which shows your best run or anybody on the leaderboards. It's a neat way to extend game play and find new short cuts.
- Music, sound design, graphical style, all create a really original atmospheric setting.
- I found the unique perspective, platforming, exploration, melee combat, and just the right amount of gun play to create a really enjoyable experience that I've never had before.
- Great sense of vertigo educing scale, and even speed when you go into a full sprint.
- Framerate is very smooth all of the time.
- I enjoyed the story and characters. The ending was pretty satisfying if a bit short.
- I'm not as crazy about the art style in the cut scenes. I think I would have preferred them in engine.
- There is a lot of trial and error to the gameplay. Some parts of the game requires unforgiving timing and jumping skills. The continue points are spaced fine so at least you won't have to replay much to get back to where you were but expect to die....a lot....over and over again.
- The controls are well thought out. It's just they are different than most other games so it takes a bit of time to get use to.
- This isn't quite an open world game like crackdown. There is some flexibility in how you approach navigating the environment but there is basically a specific path you must take. It's a pretty wide path with a few different options, but basically it's a path. I'm fine with this since you rarely get lost. If you do, just hold B and the camera will look in the direction you must go....most of the time. It doesn't always work perfectly.
- Game does a great job mixing up the gameplay and pacing. Sometimes your are being chased. Other times you are chasing others. Then there are times where there isn't that time pressure but you have some more difficult level design puzzle to figure out.
- There was one part where I had to jump from a vertical pipe to a catwalk. I couldn't quite aim and/or push my left stick (or is it right stick too) just right so it would "inviso" lock to make the jump. I must have just let go and dropped to my death like 10 times. I'm not sure why I was having such a problem in this one spot.
- Sometimes jumping up and hanging on something and then trying to look and nudge the game to inviso lock on a 90 degree platform next to you is tough. This also happened in just one spot so far.
- I'm getting much better at the soft roll landing now. Jump reverse jump is becoming more natural too.
- Melee combat is kind of tough especially when there are a lot of blues clumped together. Sometimes hiding separates them, sometimes not. I was trying not to shoot and kill people but oh well there goes that achievement. I usually like to start with a slide kick since that makes them double over. It seems like a more effective stun than the jumping kick.
- I'm enjoying my quiet reading time in the elevators.
- There was one spot on chapter 7 (vents working my way up) I was stuck on for quite a while. The timing required to do what was needed was a bit crazy. There was one tricky jumping part in chapter 8 too. Ending wasn't bad but pretty short. Wait through the credits for a bit more story in a voice over. By the end you'll be really good at wall running -> jumping, wall running -> reverse -> jumping -> wall climb -> reverse -> jumping, etc. Disarming timing is like crazy precise without the slow- mo. So ummm yeah use the slow-mo.
- I tried some speed runs and time trials (ghosts are pretty cool). I might work on playing on hard and getting some more achievements like not shooting anybody (which would seem quite difficult on the later levels). Overall I would say it was 6.5hrs of incredible gaming and 30-60min of hair pulling frustration. I still loved it but man a bit more play testing and a slight loosing of timing/precise jumping in just a few spots would have gone a long way to making this an even more enjoyable experience. Maybe it was done on purpose to lengthen gameplay but this wouldn't be the ideal way. This game just felt so original and refreshing in so many ways.
- Oh I played around 6 hrs straight today and no motion sickness. It's odd because I've gotten sick with all the star breeze games (must be something about their engine). I finished both Riddick and the Darkness but I couldn't play more than one hour at a time. Even MGS4 made me a bit ill but Mirror's Edge I was fine.
- I'm impressed with some of the risks EA has been taking recently with original IP's like this. It's even more impressive that this came from Dice the developers of the Battlefield series since this game is so different. I was surprised to learn this was built with Unreal Engine 3 since it looks/feels so different than any other unreal engine game. That really shows how flexible that engine is.
- It's pretty family friendly so I didn't mind playing it while the kids were still up. I would consider it a very mild Teen rating.
- I'll give one hint to anybody who is stuck. Learn to love wall running and jumping (with or without a reverse in there). Learn to spot where you can do it (and it's not always immediately obvious). Getting enough speed, getting good height on the wall run, and timing that jump exactly perfect at the apex of your wall run to get maximum distance and height is CRITICAL to completion of this game.

Highly recommended. This is a gamer's game. It doesn't feel watered down for a mainstream audience. Having said that the learning curve is pretty steep. Stick with it, and you will be rewarded. If you are willing to put up with a bit of frustration, there is a really great and original game that feels and looks like nothing you've played before.