Alex Semenoff has lost his management job at his family-owned quarry, after concerns over work expenses for overseas travel and safety breaches.