This thesis is divided in two parts. First, it presents a new criterion for model selection which is shown to be particularly well suited in "sparse" settings which we believe to be common in many research fields. Our selection procedure is developed for linear regression models, smoothing splines, autoregressive and mixed linear models. These developments are then applied in Biostatistics. The second part presents a new estimation method for the parameters of a time series model. The proposed estimation method offers an alternative to maximum likelihood estimation, that is straightforward to implement and often the only feasible estimation method with complex models. We derive the asymptotic properties of the proposed estimator for inference and perform an extensive simulation study to compare our estimator to existing methods. Finally, we apply our method in engineering to calibrate inertial sensors and demonstrate that it represents a considerable improvement compared to benchmark methods.