Brian D. Ripley
Ripley brings together two crucial ideas in pattern recognition: statistical methods and machine learning via neural networks. He brings unifying principles to the fore, and reviews the state of the subject. Ripley also includes many examples to illustrate real problems in pattern recognition and how to overcome them.
Software like Google Picasa demonstrate very well that software can figure out which way a photo was taken without Exif-Data as it is not available in every camera.
Is there a documented algorithm which outputs whether or not an image has to be rotated or not? I want to find out the rotation without the use of EXIF Data. If at all possible I would want to do this with ImageMagick.
I don't know a ready made solution for this problem but it is a classification problem and there are many classic algorithms that can be used. Pattern Recognition and Neural Networks by B.D. Ripley is a good read on the subject.
openCV has a machine learning module that can be used for this.
The solution will probably involve heuristics like 1-3 in Yann Ramin's answer but quantified as numbers between 0 and 1 and put in a vector. You can use imags with exif data about orientation to make a training set for the classifier.