A Devon family has said it would be “devastated” and possibly forced to move if a 200-year-old oak tree is removed from their garden.