Revisit default ports for clients decision

[gkumbhat]

Description

Currently we are using default port for all types of clients. While it is nice to have things work, it can sometimes create confusions if things seem to work magically. Additionally, people can always be explicit about the port which would make it more clear configuration setting! This story is to revisit the default port decision and remove it if deemed confusing.