You will have to stop your old 0.94 cluster completely to upgrade. If you are replicating between clusters, both clusters will have to go down to upgrade. Make sure it is a clean shutdown so there are no WAL files laying around (TODO: Can 0.96 read 0.94 WAL files?). Make sure zookeeper is cleared of state. All clients must be upgraded to 0.96 too.
The API has changed in a few areas; in particular how you use coprocessors (TODO: MapReduce too?)
TODO: Write about 3.4 zk ensemble and multi support