The success of nonlinear system identification and characterization from experimental time-series often depends on the appropriate pre-processing of the data. This pre-processing can, in many instances, be achieved through linear and/or ''nonlinear'' principal component analysis. In recent years, neural network based techniques as a means to perform the nonlinear principal component (NLPC) analysis have gained increased attention. In this contribution, we address an inherent shortcoming of these techniques: the self-consistency problem. We present a modification to the usual feedforward neural network architecture used to extract the NLPCs that results in attenuation of this problem. The proposed modification may significantly reduce representation errors in many other applications for which NLPC analysis is a powerful tool, such as feature extraction and image processing.