While m.att may not have much effect on cast time of spells, it has a noticeable effect on Focus charge speeds, so it's best not to completely exclude it from consideration.
http://vindictusdb.com/item-comparer?i=staff_thor;staff_feather;staff_rakum
Between the three of them, the Rakum only tops the chart in weight, int and physical att; weight and physical att are worthless aspects to consider, and the int really is offset by the other two staves' higher magic att. Personally I think the Dreamwalker is the best option. While it may have the lowest m.att out of the three, it also has the highest balance, att speed, and the second highest crit rate, which offsets that fact (by significantly bumping up its dps).
Of course, if you manage to find a *5 Dreamwalker from Thor then all your problems would be solved, lol.