Dislocation-grain boundary interaction plays a key role in the plasticity of polycrystalline materials. Capturing the effect of discrete dislocations interacting with a grain boundary in continuum models is not yet achieved. To date several approaches exist, but they have shortcomings in capturing the influence of dislocation--dislocation interaction across a grain boundary and the parameters which control grain boundary yield are phenomenologically motivated. In this work we show that grain boundary yielding is not inherently connected to physical dislocation transmission and that a realistic model needs to incorporate the interaction of dislocations across grain boundaries to capture the true strain distribution in the individual grains. By comparing discrete dislocation dynamics simulations of a single crystal with an artificial grain boundary to continuum dislocation dynamics results, a clear influence on the strain profile from the elastic interaction of dislocations belonging to different grains is shown. Our results demonstrate that continuum models like gradient plasticity need to extend their grain boundary modeling to incorporate dislocation interactions because a single yield criterion is not sufficient.