Definition: As Morningstar says, "Alpha is a measure of the difference between a portfolio's actual returns and its expected performance, given its level of risk as measured by beta." In translation, alpha gives an investor an expectation of how well a mutual fund may perform in relation it's risk level.
Benefits: Alpha is a good tool to measure the value added or subtracted by the mutual fund manager. For example, a positive alpha would indicate that the fund manager has done a good job in the past of outperforming a benchmark, such as the S&P 500, at a similar or lower risk level than that of the benchmark. Put simply, if a fund has high relative risk as measured by beta but the returns are not also high, the alpha will be lower or negative. In other words, higher risk should translate into higher returns. Ultimately, an investor wants an average to low risk fund with above average returns.
Alpha is not useful with index funds because these funds seek to mimic the returns of the benchmark--they are passively managed. Therefore alpha will often be near zero.