Is It A Sin To Refuse Sex To Your Spouse?

Is it a sin to reject your spouse for sex according to the Bible? Does the Bible say it is a sin to withhold sex from your spouse?