The Borough area in London - safety concerns

Hi!

I have recently moved to London from Sydney through work with my wife, and I have found a place in Borough to rent. The location is close to the intersection of Webber St and Great Suffolk St. I have now been in the area for a few weeks, and found the area does have a big mix of low, medium and high income earners, as well as students and beggars. Being in an old, highly urban setting the area is very interesting but has my raised concerns about safety, particularly walking west along Webber St or Great Suffolk St towards Southwark Underground, and walking around at night (even after 9pm on weekends). 

In the few weeks I've been here, I have become more settled and feel more comfortable day by day, and even excited to explore the area, but the safety concern is always in the back of my head, especially knowing Elephant and Castle is just a 15 mins walk down the road.

Could I please get some advice about this specific area in terms of safety, especially later at night? Are normal precautions (not carrying anything expensive, being alert, etc..) enough to stay safe here, or do more precautions need to be taken (don't go out after 9 or 10pm, stay away from certain estates and streets. etc...?

Thanks in advance.

A okay.