Can I be 'immune' to drug addiction?

I have been using diazepam for 2 years about 20 mg's a day and stop it 4 months ago, with no need to use it and no withdrawal symptoms also I bruised a nerve on my spine running down my leg and was treated with oxycodone which i used for 5 weeks. Really felt good with oxycodone but after I stopped taking it, the same scenario happened. This seems to happen in my family too. I must admit i did use them both recreationally, but when I ran out/stopped I was full of energy and what seemed like the exact opposite of withdrawal symptoms. I could go into detail but i think ive covered the basics. So can you actually be immune to drug dependence/withdrawals?