As a business owner I take it you have a cell phone. Why not just put a reminder for the day before into your cell phone and let it remind you. Along with the note there is also a memo line that you can put the amount. As for quarterly payments to the government you can also put a post dated payment into the on-line banking up to a year ahead for most banks and you can put an amount. If the money is not in the account you can always change the amount or the date as needed before the payment comes out. You may also want to set up a separate account where you throw the money into this account on a weekly or monthly basis based on what your accounting program is telling you is due.
Most banks let you pay just about every payment that is needed for Canada. American payments have to actually be sent at this point but they are working on changing this. This also serves as a 7 year proof that the payment was made and no need for redundant paperwork. I also use one-note to “print” everything to so that I don’t actually need a paper copy of things like this but it can be printed off as a paper copy whenever it is needed. I also use E-Post for paperless bill payments and this lets me put a not about payments and I just archive them every month along with the transaction number so that I don’t have to look elsewhere. I only have 3 bills that are due every month. The other things are set up for automatic debit for the same day or the following Monday so it helps me keep track of everything. If you apply for overdraft it only costs you a few cents if you misjudge and the money comes out anyway. Most banks are willing to provide this service to keep a customer. 