Active and abandoned mine activities constitute the sources of deterioration of water and soil quality in many parts of the world, particularly in the African Copperbelt regions. The accumulation in soils and the release of toxic substances into the aquatic ecosystem can lead to water resources pollution and may place aquatic organisms and human health at risk. In this study, the impact of past mining activity (i.e., abandoned mine) on aquatic ecosystems has been studied using ICP-MS analysis for trace metals and Rare Earth Elements (REE) in sediment samples from Lubumbashi River (RL) and Tshamilemba Canal (CT), Katanga, Democratic Republic of the Congo (DRC). Soil samples from surrounding CT were collected to evaluate trace metal and REE concentrations and their spatial distribution. The extent of trace metal contamination compared to the background area was assessed by Enrichment Factor (EF) and Geoaccumulation Index (Igeo). Additionally, the trace metal concentrations probable effect levels (PELs) for their potential environmental impact was achieved by comparing the trace metal concentrations in the sediment/soil samples with the Sediment Quality Guidelines (SQGs). Spearman's Rank-order correlation was used to identify the source and origin of contaminants. The results highlighted high concentrations of trace metals in surface sediments of CT reaching the values of 40152, 15586, 610, 10322, 60704 and 15152 mg kg−1 for Cu, Co, Zn, Pb, Fe and Mn, respectively. In the RL, the concentrations reached the values of 24093, 2046, 5463, 3340, 68290 and 769 mg kg−1 for Cu, Co, Zn, Pb, Fe and Mn, respectively. The ΣREE varied from 66 to 218 and 142–331 mg kg−1 for CT and RL, respectively. The soil samples are characterized by variable levels of trace metals. The EF analysis showed “extremely severe enrichment” for Cu and Co. However, no enrichment was observed for REE. Except for Mo, Th, U, Eu, Mo, Ho and Tm for which Igeo is classified as “moderately polluted and/or unpolluted”, all elements in different sites are classified in the class 6, “extremely polluted”. The trace metal concentrations in all sampling sites largely exceeded the SQGs and the PELs for the Protection of Aquatic Life recommendation. Cu and Co had positive correlation coefficient values (r = 0.741, P < 0.05, n = 14). This research presents useful tools for the evaluation of water contamination in abandoned and active mining areas.