I try to keep some hardware on hand. I hate having to go to the hardware store for a nut or bolt in the middle of a project. As it turns out I still have to go the hardware store a lot....
It is also cheaper to buy in bulk sometimes. When I needed some kreg blue screws this past summer I got 1000 size box. I figured that should be a couple year supply unless I really get busy. Things like 1/4"-20 bolts are another thing I use pretty often but 100 is enough at a time. Then comes lengths, I know what sizes I use the most of and stock more of those then the ones I don't use often.
Then like you mentioned there are left over bits and pieces and I save them all. If I think back at all the stuff I've taken apart and saved the bits from and thrown them in drawers and cans...... but every once in a while I need one special screw or washer I dump out the can and find that oddity. Going to a hardware store is useless at these time. I'm a pack-rat at heart so for me this is just how it is.
For storage I mush have around 100 large coffee cans, lots of peanut butter jars, and similar containers. I have plastic partitioned trays with covers, plastic trays that slide in cabinets, stuff still in boxes....
This past year I have been working on a new way of doing some of my storage. I've been buying bins like this one:
http://www.amazon.com/gp/product/B004EH ... UTF8&psc=1
I have started to organize some of my larger collections of frequently used items this way. So far I'm like this version better then a couple of other companies versions. I also like the pull out bins that you can take with you to where you are working. In the next year I plan to add more of these ending up with say 20 of them and building a roll-a-round shelf stacked up with shelves of the containers. At this point I must have 7 or 8 of them so I still have a ways to go.
I'm on the heavy side of this storage situation, you might not want to come to far in this direction as it takes up a lot of space.
Ed