Inspired by recent analyses of the finite element heterogeneous multiscale method and the reduced basis technique for nonlinear problems, we present a simple and concise finite element algorithm for the reliable and efficient resolution of elliptic or parabolic multiscale problems of nonmonotone type. Solutions of appropriate cell problems on sampling domains are selected by a greedy algorithm in an offline stage and assembled in a reduced basis (RB). This RB is then used in an online stage to solve two-scale problems at a computational cost comparable to the single-scale case. Both the offline and the online cost are independent of the smallest scale in the physical problem. The performance and accuracy of the algorithm are illustrated on 2D and 3D stationary and evolutionary nonlinear multiscale problems.