It appears as if the original position of the Newport Road / Four Elms Lane milestone was on the north side of Newport Road, shown here on a map from around 1900, before Roath Library was constructed.
1900 OS Map – note milestone was then over the other side of road – Copy
Roath Local History Society 
