Changeset - 93d102539a7e
[Not reviewed]
0 1 0
Ben Sturmfels (bsturmfels) - 16 months ago 2023-02-11 03:51:13
ben@sturm.com.au
reconciler: Re-wrap module docstring at 80 chars
1 file changed with 59 insertions and 59 deletions:
0 comments (0 inline, 0 general)
conservancy_beancount/reconcile/statement_reconciler.py
Show inline comments
 
"""Compare a bank CSV statement with the books.
 

	
 
This tool takes an AMEX or First Republic CSV statement file and
 
compares it line-by-line with the Beancount books to make sure that
 
everything matches. This is designed for situations where transactions
 
are entered into the books directly, rather than being imported from a
 
statement after the fact.
 

	
 
The reconciler will attempt to match transactions based on date,
 
amount, check number and payee, but is forgiving to differences in
 
dates, the absensce of check number and inexact matches on
 
payee. Matches are ranked, so where there is only one decent match for
 
an amount/date this is accepted, but if there are multiple similar
 
candidates it will refuse to guess.
 

	
 
The reconciler will also attempt to identify where a single statement
 
entry has been split out into multiple Beancount postings, such as a
 
single bank transfer representing health insurance for multiple
 
employees.
 
This tool takes an AMEX or First Republic CSV statement file and compares it
 
line-by-line with the Beancount books to make sure that everything matches. This
 
is designed for situations where transactions are entered into the books
 
directly, rather than being imported from a statement after the fact.
 

	
 
The reconciler will attempt to match transactions based on date, amount, check
 
number and payee, but is forgiving to differences in dates, the absensce of
 
check number and inexact matches on payee. Matches are ranked, so where there is
 
only one decent match for an amount/date this is accepted, but if there are
 
multiple similar candidates it will refuse to guess.
 

	
 
The reconciler will also attempt to identify where a single statement entry has
 
been split out into multiple Beancount postings, such as a single bank transfer
 
representing health insurance for multiple employees.
 

	
 
Run it like this:
 

	
...
 
@@ -28,71 +25,74 @@ $ statement_reconciler \
 

	
 
Background:
 

	
 
Beancount users often write importers to create bookkeeping entries
 
direct from a bank statement or similar. That approach automates data
 
entry and reconciliation in one step. In some cases though, it's
 
useful to manually enter transactions and reconcile them later
 
on. This workflow helpful in cases like writing a paper check when
 
there's a time lag between committing to making a payment and the
 
funds being debited. That's the workflow we're using here.
 

	
 
Conservancy currently enter data by hand rather than using Beancount
 
importers.  This tool is still somewhat like an importer in that it
 
needs to extract transaction details from a third-party
 
statement. Instead of creating directives, it just checks to see that
 
similar directives are already present. This is a bit like diff-ing a
 
statement with the books (though we're only interested in the presence
 
of lines, not so much their order).
 

	
 
Paper checks are entered in the books when written (a.k.a. "posted"),
 
but may not be cashed until months later sometimes causing
 
reconciliation differences that live beyond a month. It's worth noting
 
that there are really two dates here - the posting date and the
 
cleared date. Beancount only allows us to model one, which is why
 
carrying these reconciliation differences between months feels a bit
 
awkward.
 
Regular Beancount users often write automated importers to create bookkeeping
 
entries direct from a bank statement or similar. That combines data entry and
 
reconciliation in one step. Conservancy uses a different approach; they manually
 
entering transactions and reconciling them later on. This workflow is helpful in
 
cases like writing checks (see below). This is the workflow implented by this
 
tool.
 

	
 
That said, this tool *is* still somewhat like an importer in that it needs to
 
extract transaction details from a third-party statement. Instead of creating
 
directives, it just checks to see that similar directives are already
 
present. This is a bit like diff-ing a statement with the books (though we're
 
only interested in the presence of lines, not so much their order).
 

	
 
Paper checks are entered into the books when written (a.k.a. "posted"), but may
 
not be cashed until months later sometimes causing reconciliation differences
 
that live beyond a month. It's worth noting that there are really two dates here
 
- the posting date and the cleared date. Beancount only allows us to model one,
 
which is why carrying these reconciliation differences between months feels a
 
bit awkward.
 

	
 
Problems in scope:
 

	
 
 - errors in the books take hours to find during reconciliation,
 
   requiring manually comparing statements and the books and are
 
   succeptible to mistakes, such as not noticing when there are two
 
   payments for the same amount on the statement, but not in the books
 
   (as Bradley likes to quote, "you're entering a world of pain")
 
 - errors in the books take hours to find during reconciliation, requiring
 
   manually comparing statements and the books and are succeptible to mistakes,
 
   such as not noticing when there are two payments for the same amount on the
 
   statement, but not in the books (as Bradley likes to quote, "you're entering
 
   a world of pain")
 

	
 
 - adding statement/reconciliation metadata to books is/was manual and
 
   prone to mistakes
 
 - adding statement/reconciliation metadata to books is/was manual and prone to
 
   mistakes
 

	
 
 - jumping to an individual transaction in a large ledger isn't
 
   trivial - Emacs grep mode is the current best option
 
 - jumping to an individual transaction in a large ledger isn't trivial - Emacs
 
   grep mode is the current best option
 

	
 
 - Pam and other staff don't use Emacs
 
 - not all staff use Emacs
 

	
 
 - auditors would prefer Bradley didn't perform reconciliation,
 
   ideally not Rosanne either
 
 - auditors would prefer Bradley didn't perform reconciliation, ideally not
 
   Rosanne either
 

	
 
 - reconciliation reports are created by hand when there are mismatches
 

	
 
Other related problems we're not dealing with here:
 

	
 
 - after updates to the books files, beancount must be restarted to
 
   reflect updates
 
 - after updates to the books files, beancount must be restarted to reflect
 
   updates
 

	
 
 - updates also invalidate the cache meaning restart takes several
 
   minutes
 
 - updates also invalidate the cache meaning restart takes several minutes
 

	
 
 - balance checks are manually updated in
 
   svn/Financial/Ledger/sanity-check-balances.yaml
 

	
 
 - transactions are entered manually and reconciled after the fact,
 
   but importing from statements may be useful in some cases
 
 - transactions are entered manually and reconciled after the fact, but
 
   importing from statements may be useful in some cases
 

	
 
Future possibilities:
 

	
 
 - allow the reconciler to respect manually-applied metadata - not clear how
 
   this would work exactly
 

	
 
 - allow interactive matching where the user can specifiy a match
 

	
 
 - consider combining this with helper.py into one more complete tool that both
 
   reconciles and summarises the unreconciled transactions
 
"""
 

	
 
# TODO:
 
#  - entry_point seems to swallow errors
 
#  - extract the magic numbers
 
#  - consider merging in helper.py
 

	
 
import argparse
 
import collections
0 comments (0 inline, 0 general)