OSX default directories are well designed. Almost everybody could keep their stuff organized like that. Desktop represents the slate where files get processed. Downloads go into Download: texts, images, movies, everything. When processed, they go into documents' repository, or the desktop, if not directly into trash. Finally, Applications are for applications: there are security reasons to keep it that way.
That said, Documents is a bit tougher to defend. It's polluted with some applications' data. There is a diffence between archived documents and working documents. And Images, Music, and Photos are documents too.
So here are three directories of my own: Archives, Banks, and Notes. Archives are for closed archives. Banks is for data; one could also name it Databases, but my music and my photos go there too. Notebook are for notes. In french, Notebooks are in fact called "Cahiers", which gives me three folders starting with A, B, C.
Unless necessary, stick to default directories.