In my case, I often work with the paper turned in different directions and without a horizon line or a roof or a head that's being referenced, the decision becomes purely aesthetic. Here's the little square painting turned each possible way. And to me they don't all look the same.
To me, numbers 1 and 2 look more architectural. One is like a building with a chimney and 2 is like a strong arch. In number 3 my attention goes more to yellow stripe along the bottom and its echo above, and in number 4 I see more the the brown area in the centre.
Now here's another example and it gets even more complicated when you're working with two paintings. These two are on panels sized 1 by 3 feet. I painted them separately but used similar colours. When I'd finished I realized that the two pieces actually can have a conversation between them. Although I'd painted them with a horizontal orientation, I decided to hang them vertically, and close enough together that they could be seen together and have some interaction.
Intuitively I placed the panels so the eye would travel across the grey at the top of the painting on the LHS to the same grey at the top of the painting on the RHS, and the white lines move across the centre of both pieces.
But when Bill Porteous looked at them together he had me turn the one on the right 180 degrees.
This is what they look like with this change. I actually think he's right, that this orientation makes them visually more interesting. What do you think?