Late-night fights, aggressive panhandling and garbage littering the street is giving Ottawa's ByWard Market a bad name, according to market vendors.

"Some people, they get in people's faces. They'll hit them, harass them, tell them if you can't give them money give me your food," said Christine Generoux.

"[They] get mad, get angry, swear, throw things, hit people, you name it."

Generoux has sold syrup for 13 years and said she had to call 9-1-1 four times within the last month.

"I actually had to call 9-1-1 two weeks ago because there was a guy with a half-empty bottle of liquor yelling in people's faces," she said.

A new report shows there were 147 cases of aggressive panhandling last summer.

ByWard Market Ambassadors patrol the area to try and control various issues.

"We want make sure that everyone who comes to the ByWard market has a good experience but they also feel safe and secure when they're down here visiting the market," said Jasna Jennings, director of the ByWard Market BIA.

"Sometimes they're right at you before you even know it. Like give me your money now. And sometimes they'll even grab at you to get things," said Matthew Denison who lives in the area.

"It's like leave me alone, I don't have any money," he said.

Frank Johnson says another problem is self-designated "parking valets" on York Street.

"I've had people attempt to park me and want some money for it. And I've seen them do it to other people and people feel obligated," said Johnson.

Many said because the area is a premiere tourist destination, these issues need to be handled better.

"When tourists come there's a lot of homeless and it can leave a negative impact on our ByWard market and Parliament buildings because just steps away you see a lot of drug addicts and homeless people causing a lot of mischief, said Denison.