Ask Mr. Smarty Plants

According to this USDA Plant Profile, Viburnum rufidulum (rusty blackhaw) grows natively in Travis County, so you have the right plant in the right place. An article from Floridata on Blackhaw Viburnum includes this information:

"Rusty black-haw is really a small tree rather than a shrub because it usually grows with a single trunk. It has a slender trunk, an open, irregular or rounded crown, and starts branching close to the ground."

This article goes on to say that sometimes it will put out suckers from roots and can even develop into a thicket. Perhaps the smaller stems are really suckers and that is why that area is not developing as well. You can choose to trim off those suckers and make your viburnum a single-leader tree, the way it is apparently meant to be, but we don't think there is any harm in them. 

Returning to the falling of the leaves last summer, we all remember what a very difficult summer last year's was, with extreme heat and drought and watering restrictions. The fact that the main leader of your plant has bloomed and put out leaves normally leads us to believe that it was just reacting to environmental stresses. Here are the Growing Conditions for this plant from our Native Plant Database:

"Water Use: Low
Light Requirement: Part Shade
Soil Moisture: Dry
Cold Tolerant: yes
Soil Description: Dry soils. Limestone-based, Sandy, Sandy Loam, Medium Loam, Clay Loam, Clay
Conditions Comments: Slow-growing. Hard to propagate. With its waxy leaves, rusty blackhaw presents excellent fall hues of red, lavender, pink, and orange. Tiny clusters of flowers bloom in spring. In Manual of the Vascular Plants of Texas, Correll and Johnston noted that the fruit tastes similar to raisins. Rusty blackhaw is good for understory plantings. Birds appreciate the fruit."

As well as we can determine, this is a native tree surviving in its own way, and doing okay at it.  You might also read this USDA Forest Service article Viburnum rufidulum. Toward the end of that article, it details some pests and diseases of this plant that you should look out for. 

From our Native Plant Image Gallery:

Viburnum rufidulum

Viburnum rufidulum

Viburnum rufidulum

Viburnum rufidulum