diff --git a/conservancy_beancount/reconcile/statement_reconciler.py b/conservancy_beancount/reconcile/statement_reconciler.py index 0cc69c4fe667dce12f02a98a24b8ff2c3198590b..68d5cb89ce9a11b09dd467b4ea216881b61fe66a 100644 --- a/conservancy_beancount/reconcile/statement_reconciler.py +++ b/conservancy_beancount/reconcile/statement_reconciler.py @@ -1,22 +1,19 @@ """Compare a bank CSV statement with the books. -This tool takes an AMEX or First Republic CSV statement file and -compares it line-by-line with the Beancount books to make sure that -everything matches. This is designed for situations where transactions -are entered into the books directly, rather than being imported from a -statement after the fact. - -The reconciler will attempt to match transactions based on date, -amount, check number and payee, but is forgiving to differences in -dates, the absensce of check number and inexact matches on -payee. Matches are ranked, so where there is only one decent match for -an amount/date this is accepted, but if there are multiple similar -candidates it will refuse to guess. - -The reconciler will also attempt to identify where a single statement -entry has been split out into multiple Beancount postings, such as a -single bank transfer representing health insurance for multiple -employees. +This tool takes an AMEX or First Republic CSV statement file and compares it +line-by-line with the Beancount books to make sure that everything matches. This +is designed for situations where transactions are entered into the books +directly, rather than being imported from a statement after the fact. + +The reconciler will attempt to match transactions based on date, amount, check +number and payee, but is forgiving to differences in dates, the absensce of +check number and inexact matches on payee. Matches are ranked, so where there is +only one decent match for an amount/date this is accepted, but if there are +multiple similar candidates it will refuse to guess. + +The reconciler will also attempt to identify where a single statement entry has +been split out into multiple Beancount postings, such as a single bank transfer +representing health insurance for multiple employees. Run it like this: @@ -28,71 +25,74 @@ $ statement_reconciler \ Background: -Beancount users often write importers to create bookkeeping entries -direct from a bank statement or similar. That approach automates data -entry and reconciliation in one step. In some cases though, it's -useful to manually enter transactions and reconcile them later -on. This workflow helpful in cases like writing a paper check when -there's a time lag between committing to making a payment and the -funds being debited. That's the workflow we're using here. - -Conservancy currently enter data by hand rather than using Beancount -importers. This tool is still somewhat like an importer in that it -needs to extract transaction details from a third-party -statement. Instead of creating directives, it just checks to see that -similar directives are already present. This is a bit like diff-ing a -statement with the books (though we're only interested in the presence -of lines, not so much their order). - -Paper checks are entered in the books when written (a.k.a. "posted"), -but may not be cashed until months later sometimes causing -reconciliation differences that live beyond a month. It's worth noting -that there are really two dates here - the posting date and the -cleared date. Beancount only allows us to model one, which is why -carrying these reconciliation differences between months feels a bit -awkward. +Regular Beancount users often write automated importers to create bookkeeping +entries direct from a bank statement or similar. That combines data entry and +reconciliation in one step. Conservancy uses a different approach; they manually +entering transactions and reconciling them later on. This workflow is helpful in +cases like writing checks (see below). This is the workflow implented by this +tool. + +That said, this tool *is* still somewhat like an importer in that it needs to +extract transaction details from a third-party statement. Instead of creating +directives, it just checks to see that similar directives are already +present. This is a bit like diff-ing a statement with the books (though we're +only interested in the presence of lines, not so much their order). + +Paper checks are entered into the books when written (a.k.a. "posted"), but may +not be cashed until months later sometimes causing reconciliation differences +that live beyond a month. It's worth noting that there are really two dates here +- the posting date and the cleared date. Beancount only allows us to model one, +which is why carrying these reconciliation differences between months feels a +bit awkward. Problems in scope: - - errors in the books take hours to find during reconciliation, - requiring manually comparing statements and the books and are - succeptible to mistakes, such as not noticing when there are two - payments for the same amount on the statement, but not in the books - (as Bradley likes to quote, "you're entering a world of pain") + - errors in the books take hours to find during reconciliation, requiring + manually comparing statements and the books and are succeptible to mistakes, + such as not noticing when there are two payments for the same amount on the + statement, but not in the books (as Bradley likes to quote, "you're entering + a world of pain") - - adding statement/reconciliation metadata to books is/was manual and - prone to mistakes + - adding statement/reconciliation metadata to books is/was manual and prone to + mistakes - - jumping to an individual transaction in a large ledger isn't - trivial - Emacs grep mode is the current best option + - jumping to an individual transaction in a large ledger isn't trivial - Emacs + grep mode is the current best option - - Pam and other staff don't use Emacs + - not all staff use Emacs - - auditors would prefer Bradley didn't perform reconciliation, - ideally not Rosanne either + - auditors would prefer Bradley didn't perform reconciliation, ideally not + Rosanne either - reconciliation reports are created by hand when there are mismatches Other related problems we're not dealing with here: - - after updates to the books files, beancount must be restarted to - reflect updates + - after updates to the books files, beancount must be restarted to reflect + updates - - updates also invalidate the cache meaning restart takes several - minutes + - updates also invalidate the cache meaning restart takes several minutes - balance checks are manually updated in svn/Financial/Ledger/sanity-check-balances.yaml - - transactions are entered manually and reconciled after the fact, - but importing from statements may be useful in some cases + - transactions are entered manually and reconciled after the fact, but + importing from statements may be useful in some cases + +Future possibilities: + + - allow the reconciler to respect manually-applied metadata - not clear how + this would work exactly + + - allow interactive matching where the user can specifiy a match + - consider combining this with helper.py into one more complete tool that both + reconciles and summarises the unreconciled transactions """ # TODO: # - entry_point seems to swallow errors # - extract the magic numbers -# - consider merging in helper.py import argparse import collections