Obesity is associated with a higher risk of developing breast cancer and with worse disease outcomes for women of all ages. The composition, density, and organization of the breast tissue stroma are also known to play an important role in the development and progression of the disease. However, the connections between obesity and stromal remodeling are not well understood. We sought to characterize detailed organization features of the collagen matrix within healthy and cancerous breast tissues acquired from mice exposed to either a normal or high fat (obesity inducing) diet. We performed second-harmonic generation and spectral two-photon excited fluorescence imaging, and we extracted the level of collagen-associated fluorescence (CAF) along with metrics of collagen content, three-dimensional, and two-dimensional organization. There were significant differences in the CAF intensity and overall collagen organization between normal and tumor tissues; however, obesity-enhanced changes in these metrics, especially when three-dimensional organization metrics were considered. Thus, our studies indicate that obesity impacts significantly collagen organization and structure and the related pathways of communication may be important future therapeutic targets.