I found a kind-of-close blowup of the parts. The plate that I can move with my hands, even though the bolts are tight, is the part called the "retainer" on this photo.
http://i195.photobucket.com/albums/z41/coffee_brake/85toyotashifter_zps598b1f2c.jpg
I got the shifter off. I don't know if I'm supposed to go any further than this. The plate in the photos, that has the bolts in it, that has slop in it. It wiggles a good 1/8" just by holding it with my fingers. The bolts were plenty tight, with lock-tite on them, but the plate was still able to...
I think I've got it for now at least.
I took the non-bleeding shoes off and just cleaned everything up and put it all back together. Everything finally bled out right and the brakes, while still not stellar, are much better and the pedal doesn't plunge to the floor like it did.
The e-brake is...