This article about using two compatible forms of contraception should be relevant to your interests.
If you're taking your birth control properly and he knows how to pull out properly and is able to do so properly and consistently, then your protection against pregnancy is about 99.98%, equivalent to if you were perfectly using both the pill and a diaphragm OR both the pill and female condoms. As opposed to 99.7% if you're just using the pill with perfect use.
If the protection against pregnancy and wanting peace of mind on that front is the main factor here, then you can also look into diaphragms, cervical caps, spermicides, and contraceptive sponges as alternatives to condoms for extra protection to work with your hormonal birth control. If you've only been using male condoms so far, you could also consider looking into female condoms, especially if his main reason for wanting to forego condoms is the level of pleasure he gets from sex with condoms vs. not using condoms. (That said, he should probably make sure he's measured his penis properly, is using the right size of condoms, is following best practices for using condoms and maximizing pleasure while doing so, and has tried out non-latex condoms such as the SKYN line.)
Level of protection aside, if you've both been tested at the appropriate testing windows after the last time you had sex with anyone other than one another and you're going to be sexually exclusive, then a condom isn't strictly needed if you're using other forms of contraception.
Of course, if you don't believe him on the sexual exclusivity or don't want sexual exclusivity, that's another thing.
Admittedly, his reports about his sexual habits in the past such as not using condoms except for on two occasions are a bit sketchy, unless he was only having sex within the context of a monogamous relationship up until now.
So it seems like I have a choice between either no sex or sex without a condom.
Remember that if you don't like the deal that someone is offering, you can always look to see what other people are offering. Especially as a 20 year old woman.
Also, there are other things you can do aside from PIV that carry no pregnancy risk that you could discuss if you reach an impasse when it comes to PIV sex.
Anal sex isn't for everyone, and it requires a fair amount of preparation, but generally speaking an awful lot of people don't mind using condoms in that context and there's a pretty reduced risk of pregnancy compared to vaginal intercourse even if it's not a form of birth control.