Ask Mr. Smarty Plants

Based on your description of the dark spots on the plums with tiny worms inside, my best guess is that you are dealing with the plum curculio, a weevil that infests plums, apples, blueberries and other fruit.  The dark spots are from the crescent shaped wounds created when the females lay their eggs in the flesh of the fruit, where the worms hatch and tunnel.  If you see these spots on a plum, it likely has worms inside.  The worms later drop to the soil underneath the trees where they pupate into adults.  To keep the next generation of weevils at bay you must remove and destroy all the infested fruit from the tree and the ground around it so that the worms inside won't get a chance to transform into adults.  (This would be a good time to have a neighbor who keeps chickens!)   Other things you can try are mulching the soil about a foot from the trunk out to the drip line, pruning for an open canopy so that birds will have access to the weevils and releasing beneficial nematodes.   Above all else, keeping the soil underneath the tree clean of fallen fruit is the most important thing you can do to lessen the weevil population and get healthy plums next year.