The Swiss metals sector, comprised of manufacturing basic metals and fabricated metal products, is responsible for nearly 14% of total final energy (TFE) demand of the Swiss industry, making it one of the major energy consuming industrial sectors. This study investigates the current potential for energy efficiency (EE) improvement and CO2 abatement in the Swiss metals sector by means of bottom-up cost curves. Based on the comparison of Specific Energy Consumption for the processes applied in the Swiss metals sector with those of the best available techniques, the maximum technical EE potential for Swiss metals sector is estimated at 19%, while the economic EE potential is in the range of 11%–15%. The corresponding economic CO2 abatement potential is estimated at 6%. Among all measures, EE improvement of the rolling process was found to have the largest potential for TFE savings in the Swiss metals sector. The detailed insight into the EE gap and techno-economically feasible EE improvement solutions for the Swiss metals sector identified in this study will help to overcome the techno-economic barriers in implementing best practices in the high value-added metals sector.