The most important thing for me is that they have a good website for bill-paying. Some bills are paid automatically every month, and others are paid by me filling in the blanks in a form. It should be fast, reliable and easy.
It would be great if they would export my data to a format that I could write a script to process. XML or JSON would be excellent. I doubt if any of them do this, though. Banks appear to be a couple of decades behind the leading edge.
I tried looking up credit unions. It looks like I could join the one that's affiliated with NYU. But I don't think they actually run it. What do we know about the people who run credit unions? I have no clue which are the good ones. For all I know I could be moving my money from one evil institution to an even worse one. At least I know the one I use now manages to pay the bills I ask them to, with the caveat that they won't let me give money to causes they don't like (which is absolutely a deal-stopper, it should be illegal imho).