After testing cables, updating drivers (and downgrading to previous releases), adding more cooling, removing cooling (guessing it might be interference) I came across an interesting tool.
DPC Latency Checker:
http://www.thesycon.de/deu/latency_check.shtml
From their site:
If any kernel-mode device driver in your Windows system is implemented improperly and causes excessive latencies of Deferred Procedure Calls (DPCs) then probably drop-outs will occur when you use real-time audio or video streaming applications.
When run, this tool shows a little graph:

Ideally, your system should green all the way across. Note that if your system loading programs it will bounce around a bit.
Anyway...
My htpc had some terrible red peaks at a consistent interval (similar to the image above), according to the tool this indicates that the system would have issues with streaming audio/video. In fact, each time my audio crackled I could see the chart peak a second later.
As a comparison, I have a gaming system (i7 based) that I ran the tool on and the graph stays green, smooth and low.
The tool's designer recommends that you disable devices until the peaks go away.
Unfortunately, I haven't been able to find a device that is causing my peaks yet (I even removed an ATI tuner that I am no longer using). However, I did find a way to make the peaks less pronounced.
HPET - High Precision Event Timer:
http://en.wikipedia.org/wiki/High_Precision_Event_Timer
Apparently, this thing is the devil for people using their computers as DAW (digital audio workstations). Even though it sounds fancy, it seems to cause latency in windows systems.
My Fix:
In my bios, there is an option to enable/disable HPET and set it to 32 or 64bit.
I disabled it, fired up my computer, and my crackling was gone and my hdpvr tunerage in media center feels much improved.
DPC checker also now shows peaks (when idling) that stay in the yellow range.
I think, because the HDPVR seems to be so sensitive to latency, this tool might be a way to make it run more reliably.
Next Steps:
- I still need to track down why I get peaks at all every 3-4 seconds when my other system does not, I'd like to see it green all the way across.
- I need to see if MKV (matroska) high bitrate video playback is improved in media center. I believe Microsoft admitted that high bitrate video can stutter sometimes in media center (stopping and resuming can fix) but I'm hoping I can improve reliability.
I'd love to hear if anyone else notices improvements when disabling HPET (if they have that option in their bios), or if they have any success with DPC in tracking down a device that is introducing latency into their system.
