A non-Hermitian operator H defined in a Hilbert space with inner product ⟨ · | · ⟩ may serve as the Hamiltonian for a unitary quantum system if it is η -pseudo-Hermitian for a metric operator (positive-definite automorphism) η . The latter defines the inner product ⟨ · | η · ⟩ of the physical Hilbert space H η of the system. For situations where some of the eigenstates of H depend on time, η becomes time-dependent. Therefore, the system has a non-stationary Hilbert space. Such quantum systems, which are also encountered in the study of quantum mechanics in cosmological backgrounds, suffer from a conflict between the unitarity of time evolution and the unobservability of the Hamiltonian. Their proper treatment requires a geometric framework which clarifies the notion of the energy observable and leads to a geometric extension of quantum mechanics (GEQM). We provide a general introduction to the subject, review some of the recent developments, offer a straightforward description of the Heisenberg-picture formulation of the dynamics for quantum systems having a time-dependent Hilbert space, and outline the Heisenberg-picture formulation of dynamics in GEQM.