Structures shall be located fifty (50) feet or more from any street right-of-way which is fifty (50) feet or greater in width or seventy-five (75) feet or more from the center of any street right-of-way less than fifty (50) feet in width; however, no building need be set back more than the average of the setback of the immediately adjoining structures on either side. A vacant lot may be assumed to be occupied by a building having a minimum setback.