"This was terrorism, terrorism against us all," Scholz told reporters in the western city after laying a flower at the site of the attack in memory of the victims.
The attack, which investigators believe was carried out by a suspected Islamic State member from Syria, has fuelled political tensions over asylum and deportation rules as well as violent crime ahead of three state elections next month.
"We will have to do everything we can to ensure that those who cannot and are not allowed to stay in Germany are repatriated and deported," Scholz said.
He described the suspected attacker as a "Dublin case", referring to the European Union rules under which asylum seekers must lodge their applications in the first EU country they arrive in.
Authorities had planned to deport the suspect in Friday's knife attack, a 26-year-old Syrian man, to Bulgaria last year, according to German media.
However, the deportation was unsuccessful because the man had not been at his refugee accommodation when authorities tried to carry out the measure, the reports said.
(Reporting by Rachel More, Editing by Friederike Heine)

(L-R) Interior Minister of the State of North Rhine-Westphalia Herbert Reul, North Rhine-Westphalia state premier Hendrik Wuest, German Chancellor Olaf Scholz, Solingen Major Tim Kurzbach and Minister for Economic Affairs, Industry, Climate Protection and Energy of North Rhine-Westphalia Mona Neubaur lay down flowers near the crime scene in Solingen, Germany, 26 August 2024. A man stabbed passers-by at random with a knife during the city festival in Solingen on 23 August evening. Three people have been killed and eight others injured, five of them seriously, in the knife attack, police said. EPA-EFE/CHRISTOPHER NEUNDORF / POOL