Rares and mythics from packs have duplicate protection. Which means, that you WILL get a rare from the set that you do not have, guaranteed. (Well, some packs do contain a rare or mythic wild card instead.) The only way to get gems in the rare slot is, if you already have ALL rares from the set, that can be obtained from packs. Same for mythics. (You cannot upgrade normal packs to mythic packs by collecting all rares, though, I've tried.) There are also many rares and mythics in the various sets, that you cannot get from packs. Those are craft only, so you have to spend wildcards on them. Or get the special bundles that contain them.

Darfting rares and mythics that you already have a full set of does give you gems, but Arena does not give you a notification for that. Check your gem count, when you do so. It will increase as soon as you pick the last card of the draft.