We just had this setup last week at our shop. There is a metric in the metric browser that reports a 1 if the agent is available. This metric exists for both machine and app agents. Just set up a health rule that alerts if this metric is less than 1.
Application Infrastructure Performance|APP.AvailableLoads|Agent|Machine|Availability
You can set it up by node or tier, depending on how you want to be alerted. Hope this helps.