This Thesis explores the notion of Krylov complexity as a probe of quantum chaos and as a candidate for holographic complexity. The first Part is devoted to presenting the fundamental notions required to conduct research in this area. Namely, an extensive introduction to the Lanczos algorithm, its properties and associated algebraic structures, as well as technical details related to its practical implementation, is given. Subsequently, an overview of the seminal references and the main debates regarding Krylov complexity and its relation to chaos and holography is provided. The text throughout this first Part combines review material with original analyses which either intend to contextualize, compare and criticize results in the literature, or are the fruit of the investigations leading to the publications on which this Thesis is based. These research projects are the subject of the second Part of the manuscript. In them, methods for the efficient implementation of the Lanczos algorithm in finite many-body systems were developed, allowing to compute numerically the Krylov complexity of models like SYK or the XXZ spin chain up to time scales exponentially large in system size. It was observed that the operator Krylov complexity profile in SYK, a paradigmatic low-dimensional chaotic system with a holographic dual, agrees with holographic expectations, while in the case of integrable models like XXZ complexity is affected by a novel localization effect in the so-called Krylov space which hinders its growth. Finally, an exact, analytical, correspondence between the Krylov complexity of the infinite-temperature thermofield double state in the low-energy regime of the double-scaled SYK model and bulk length in the theory of JT gravity is established.