emarginate

Having a notched margin or edge. Often used in reference to a caudal fin with a slightly concave shape.