Officers arrested the knifeman after he was seen running from the Sainsbury's.
Another eyewitness said: "Everyone was shouting 'get out, run'".
The police confirmed that the unidentified man, wielding a knife, was arrested. "However, we do not believe this to be related to any terror threat".
The alarm was raised at around noon when staff at Sainsbury's, which was packed with people doing their weekend shopping, told everybody to get out. Road closures were in place while forensic investigators conducted enquiries.
The man was detained after armed officers and a police helicopter were deployed to the area.
No members of the public were injured during the incident and road closures remain in place at Halesowen Street, John Street and Union Street.
Shoppers abandoned their trolleys and bags and fled after witnesses reported hearing "an explosion of some sort" at the supermarket in Blackheath, in the West Midlands, which was evacuated shortly after midday. Then it came on the tannoy, evacuation everybody out.
Ambulances on standby at Toys R Us.
"Everybody made an orderly departure".
Force Incident Manager, Chief Inspector Paul Minor said: "The store remains closed while we examine the scene and assess the cause of the fire".
"The man is in custody and his motives remain unclear".
A Sainsbury's spokesman added: "We can confirm there was an incident at our Blackheath store, which is now closed".
Armed police are on the scene as they investigate further.