Before the advent of digital photography, the physical characteristics of the recording medium (film) had much to do with the physical characteristics of the final print. There was the ability to crop an image in the darkroom, but in most cases, the aspect ratio of the final print matched the aspect ratio of the negative. Today, we have no such constraints. Many digital cameras provide the option of setting a handful of different aspect ratios in-camera, and of course, cropping in post is no more than a one-click operation.
This freedom to determine the aspect ratio of every image on the fly puts tremendous pressure on the photographer. Failing to choose an aspect ratio prior to composing an image is like trying to design a page layout without knowing the size or shape of the page. Whether it's the native aspect ratio of the digital sensor, or some other aspect ratio chosen for aesthetic reasons, it's good practice to choose an image shape before bringing the viewfinder up to the eye. Doing so provides a solid framework for setting up a composition, while failing to do so changes the process of image composition from one of careful consideration to one of mostly luck and happenstance.
The 120 film format was popular in the 1960s and is still available today. Cameras that take 120 film are often called Medium Format cameras. 120 film allows several aspect ratios, including (but not limited to) 1.35:1, 1:1, and 1.50:1. A number of my favorite photographers from that era shot 120 film in the 6x6 (1:1 aspect ratio) square format. Their influence is one of the reasons I choose to shoot square images on a digital camera with a rectangular sensor.
The other reason I shoot square is related to my work as a graphic artist. In the world of graphic design--particularly print design--artists are working in rectangles nearly 100% of the time. After doing this for decades, you naturally get pretty good at parsing out rectangles. For me, the square format provides a fresh, more interesting challenge that requires a higher degree of intuitive thinking. It also frees me up from adherence to the classical rules of composition that are so deeply ingrained in the rectangle.