Proper estimation of the number of endmembers is imperative for proper and accurate spectral unmixing of hyperspectral image. The presence of noise and perturbation poses a challenge in estimation and leads to underestimation or overestimation of the number of endmembers. We propose a noise invariant tensor-based rank estimation approach, which avoids reshaping of the data. We perform a noise-robust PARAFAC tensor decomposition using a generalized low-rank factorization in the initial stage and record the core consistency and reconstruction error for a different number of components. In the next stage, we identify the lower and upper bounds of the number of components. Next, we propose a cost function containing both core consistency value and reconstruction error and identify the number of components from the minima of the cost function. The proposed estimation approach is effective for high, mid, and low signal to the noise level. We analyze the performance of the proposed estimation approach on several synthetic as well as real image experiments and verify the proficiency of our proposed estimation even at higher levels of noise.