We propose a simple U(2) model of flavor compatible with an SU(5) GUT structure. All hierarchies in fermion masses and mixings arise from powers of two small parameters that control the U(2) breaking. In contrast to previous U(2) models this setup can be realized without supersymmetry and provides an excellent fit to all SM flavor observables including neutrinos. We also consider a variant of this model based on a D6 × U(1)F flavor symmetry, which closely resembles the U(2) structure, but allows for Majorana neutrino masses from the Weinberg operator. Remarkably, in this case one naturally obtains large mixing angles in the lepton sector from small mixing angles in the quark sector. The model also offers a natural option for addressing the Strong CP Problem and Dark Matter by identifying the Goldstone boson of the U(1)F factor as the QCD axion.