Victims were given several doses of naloxone, an antidoe for narcotic overdoses, both on the scene and at the hospital.
No deaths have been reported, but two individuals had severe symptoms.
At least one arrest was made on the green. Paramedics were seen sprinting from patient to patient as they began to show symptoms including vomiting, hallucination and unconsciousness.
A new report estimated a record 72,000 Americans died a year ago from drug overdoses, a rise of around 10 percent, the Centres for Disease Control said on Wednesday.
Dozens of people suffering from apparent overdoses were transported to local hospitals from the New Haven Green, according to Rick Fontana, director of the city's Office of Emergency Operations.
Alston says the substance appears to be some type of synthetic cannabis, but authorities are not sure.
In New Haven, Fire Chief John Alston Jr told reporters the opiate problem is a far-reaching one. 'However higher concentrations of it in the emergency room proved effective'.
The Drug Enforcement Administration is assisting with the investigation and will send drug samples to a federal lab for analysis.
Authorities said the patients had smoked "K2", a synthetic cannabinoid, mixed with the powerful opioid fentanyl.
"Today New Haven was on the front lines of a coast-to-coast struggle to combat the public health menace of illicit distribution and use of what appear to be tainted street drugs", said New Haven Mayor Toni Harp in a statement.
Officer David Hartman was quoted by the newspaper as saying the patients were being treated for overdose-related respiratory illnesses. The additives boost the high, but can create a risky, sometimes deadly cocktail.
Fontana said the city's 911 center started receiving calls just after 8am about possible overdoses on the green.
The incident comes in the wake of a new report that found a record 72,000 Americans died of overdoses in 2017.
Police are working to find the source of the synthetic marijuana, which is plant material sprayed with chemicals that has been blamed for overdoses nationwide.
The exact cause of the overdoses has not been confirmed, but officials suspect that synthetic cannabinoids, also called K2 or spice, are the culprit.
Drugs, particularly fentanyl, are now deadlier in the United States than either HIV, vehicle crashes or guns, the report said.