I generally structure my files like so:
/Documents
:: this folder contains files that I, or someone on my team, created. I try to keep the root folder empty except for the following subfolders:
/Documents/Clients (each client gets their own folder under this one)
:: for context, I am a software consultant; each clients folder looks like this:
./docs/requirements
./docs/design
./code
./status (project management stuff -- status reports, etc)
/Documents/Personal
:: Stuff that I keep for myself - each major project usually gets its own folder, then depending on the type of project, those subfolders might be further subdivided. For example, if it is a software project, it will have subfolders for requirements, source code, binaries, 3rd-party libraries, etc; whereas, my "Writing" folder is flat.
/Documents/MyCompany
:: Stuff that I do for work, but is not specific to a client. This is flat, with the exception of I have a 'Team' folder which contains a subfolder for each person that reports to me.
/eBooks
:: Usually PDF's; information downloaded from external sources
/Downloads
/Music
/Videos
/Pictures
:: self-explanatory
This is probably a lot more folder-happy than a lot of people will recommend.
I've tried it both ways, but I've discovered that I am much more efficient when I am not dependent on a search tool to find everything. Also, for certain things (software projects, for example), a well-defined folder hierarchy is crucial. Finally, a flat folder structure might get you in trouble with file naming if you don't pay careful attention to that "Do you want to overwrite that file?" dialog.
Windows Vista apparently has a new "Smart Folders" feature where you can save your searches as if they were actual folders on your hard drive, but I have not played with it yet. I imagine Mac has something similar (Apple seems to always be ahead of Microsoft in regard to this kind of stuff). This might be a useful way to simulate a folder structure without actually having to create and manage one; a possible downside would be that you have less fine-grained control over the contents.
Since OS X was built on top of unix, you should also have the capability of creating symbolic links to your files (similar to windows shortcuts, but WAY better). This would allow you to keep your files wherever you want, then you can create folders than just contain links to the files. When you finish the project, you can delete the folders but the original files will stay where they were.
Hope something here helps.