NPO-Accounting/import2ledger Changeset - 0eb014aeeb2e

Changeset - 0eb014aeeb2e

Parent rev.

Child rev.

[Not reviewed]

0 4 1

Brett Smith - 6 years ago 2018-07-10 15:36:07
brettcsmith@brettcsmith.org

patreon: New importer for payouts.

5 files changed with 64 insertions and 1 deletions:

README.rst

import2ledger/importers/patreon.py

setup.py

tests/data/PatreonPayouts.csv

tests/data/imports.yml

0 comments (0 inline, 0 general)

README.rst

➞

Show inline comments

@@ ... / @@ -108,192 +108,207 @@ Amazon Affiliate Program @@
   Imports one transaction per month, summarizing all earnings over the month.  Generated from Amazon's "Fee Earnings" report CSV.
 Benevity
 ^^^^^^^^
 ``benevity donations ledger entry``
   Imports one transaction per row in Benevity's donations report CSV.
   This template can use these variables:
   ================ ===========================================================
   Name             Contents
   ================ ===========================================================
   comment          The comment from the donor
   ---------------- -----------------------------------------------------------
   corporation      The name of the participating corporation
   ---------------- -----------------------------------------------------------
   disbursement_id  The ID of the dirbursement from Benevity that includes this
                    donation
   ---------------- -----------------------------------------------------------
   donation_amount  The amount donated by the individual donor named
   ---------------- -----------------------------------------------------------
   frequency        The frequency of this donation as indicated in the report
   ---------------- -----------------------------------------------------------
   match_amount     The amount of the donation match by the participating
                    corporation
   ---------------- -----------------------------------------------------------
   project          The Project named in this row
   ---------------- -----------------------------------------------------------
   transaction_id   The ID of this specific donation
   ================ ===========================================================
 BrightFunds
 ^^^^^^^^^^^
 ``brightfunds donorreport ledger entry``
   Imports one transaction per row in donor report XLS files that BrightFunds mails each month to recipients.
   This template can use these variables:
   ================ ===========================================================
   Name             Contents
   ================ ===========================================================
   company_name     The company name as reported in the spreadsheet
   ---------------- -----------------------------------------------------------
   corporation      The company name as detected by the importer (this is
                    usually what you want)
   ---------------- -----------------------------------------------------------
   donor_name       The donor name as reported in the spreadsheet (usually you
                    want to use ``payee`` instead)
   ---------------- -----------------------------------------------------------
   donor_email      The donor's e-mail address as reported in the spreadsheet
   ---------------- -----------------------------------------------------------
   on_behalf_of     From the corresponding spreadsheet column
   ---------------- -----------------------------------------------------------
   fund             From the corresponding spreadsheet column
   ---------------- -----------------------------------------------------------
   type             From the corresponding spreadsheet column
   ================ ===========================================================
 O'Reilly Media
 ^^^^^^^^^^^^^^
 ``oreilly payments ledger entry``
   Imports one transaction per payment.  Generated from CSV export of O'Reilly's payment history.
   This template can use these variables:
   ================ ===========================================================
   Name             Contents
   ================ ===========================================================
   paid_date        From the corresponding spreadsheet column.  This is usually
                    the end of the month, while the actual transaction date
                    might be slightly off.
   ---------------- -----------------------------------------------------------
 ``oreilly royalties ledger entry``
   Imports one transaction per royalty period (usually a month, historically a quarter).  Generated from CSV export of O'Reilly's royalties statement.
   This template can use these variables:
   ================ ===========================================================
   Name             Contents
   ================ ===========================================================
   start_date       From the corresponding spreadsheet column
   ---------------- -----------------------------------------------------------
   paid_date        From the corresponding spreadsheet column.  This
                    corresponds to a row in the payment history as well.
   ---------------- -----------------------------------------------------------
 Patreon
 ^^^^^^^
 ``patreon income ledger entry``
   Imports one transaction per patron per month.  Generated from Patreon's monthly patron report CSVs.
 ``patreon payout ledger entry``
   Imports one transaction per month for that month's payout.  Generated from Patreon's payout report CSV.
   This template can use these variables:
   =============== ===========================================================
   Name            Contents
   =============== ===========================================================
   pledges_amount  A decimal with the amount paid out by Patreon to pledges
                   you've made
   --------------- -----------------------------------------------------------
   transfer_amount A decimal with the amount paid out by Patreon to your bank
                   account
   =============== ===========================================================
 ``patreon cardfees ledger entry``
   Imports one expense transaction per month for that month's credit card fees.  Generated from Patreon's earnings report CSV.
 ``patreon servicefees ledger entry``
   Imports one expense transaction per month for that month's Patreon service fees.  Generated from Patreon's earnings report CSV.
 ``patreon vat ledger entry``
   Imports one transaction per country per month each time Patreon withheld VAT.  Generated from Patreon's VAT report CSV.
   This template can use these variables:
   ============== ============================================================
   Name           Contents
   ============== ============================================================
   country_name   The full name of the country VAT was withheld for
   -------------- ------------------------------------------------------------
   country_code   The two-letter ISO country code of the country VAT was
                  withheld for
   ============== ============================================================
 Stripe
 ^^^^^^
 ``stripe payment ledger entry``
   Imports one transaction per payment.  Generated from Stripe's payments CSV export.
   This template can use these variables:
   ============== ==========================================================
   Name           Contents
   ============== ==========================================================
   customer_id    The id assigned to this customer by Stripe
   -------------- ----------------------------------------------------------
   customer_email The customer's e-mail address
   -------------- ----------------------------------------------------------
   description    The description given to the payment when it was created
   -------------- ----------------------------------------------------------
   fee            The amount of fees charged by Stripe for this payment, as
                  a plain decimal number
   -------------- ----------------------------------------------------------
   payment_id     The id assigned to this payment by Stripe
   -------------- ----------------------------------------------------------
   payout_id      The id of the associated Stripe payout
   -------------- ----------------------------------------------------------
   tax            The amount of tax withheld by Stripe for this payment, as
                  a plain decimal number
   ============== ==========================================================
 ``stripe payout ledger entry``
   Imports one transaction per payment.  Generated from Stripe's payouts CSV export.
   This template can use these variables:
   ========================== ==============================================
   Name           Contents
   ========================== ==============================================
   adjustment_count           Decimal from the corresponding CSV column
   -------------------------- ----------------------------------------------
   adjustment_gross           Decimal from the corresponding CSV column
   -------------------------- ----------------------------------------------
   adjustment_fees            Decimal from the corresponding CSV column
   -------------------------- ----------------------------------------------
   adjustment_net             Decimal from the corresponding CSV column
   -------------------------- ----------------------------------------------
   balance_txid               Stripe ID of the balance transaction
   -------------------------- ----------------------------------------------
   collected_fee_count        Decimal from the corresponding CSV column
   -------------------------- ----------------------------------------------
   collected_fee_gross        Decimal from the corresponding CSV column
   -------------------------- ----------------------------------------------
   collected_fee_refund_count Decimal from the corresponding CSV column
   -------------------------- ----------------------------------------------
   collected_fee_refund_gross Decimal from the corresponding CSV column
   -------------------------- ----------------------------------------------
   destination_id             Stripe ID of the payout destination account
   -------------------------- ----------------------------------------------
   failure_txid               Stripe ID of the failure balance transaction
   -------------------------- ----------------------------------------------
   payment_count              Decimal from the corresponding CSV column
   -------------------------- ----------------------------------------------
   payment_gross              Decimal from the corresponding CSV column
   -------------------------- ----------------------------------------------
   payment_fees               Decimal from the corresponding CSV column
   -------------------------- ----------------------------------------------
   payment_net                Decimal from the corresponding CSV column
   -------------------------- ----------------------------------------------
   payout_id                  Stripe ID of this payout
   -------------------------- ----------------------------------------------
   refund_count               Decimal from the corresponding CSV column
   -------------------------- ----------------------------------------------
   refund_gross               Decimal from the corresponding CSV column
   -------------------------- ----------------------------------------------
   refund_fees                Decimal from the corresponding CSV column
   -------------------------- ----------------------------------------------
   refund_net                 Decimal from the corresponding CSV column
   -------------------------- ----------------------------------------------

import2ledger/importers/patreon.py

➞

Show inline comments

 import pathlib
 import re
 from . import _csv
 from .. import strparse
 class IncomeImporter(_csv.CSVImporterBase):
     NEEDED_FIELDS = frozenset([
         'FirstName',
         'LastName',
         'Pledge',
         'Status',
     ])
     COPIED_FIELDS = {
         'Pledge': 'amount',
+    }
     ENTRY_SEED = {
         'currency': 'USD',
+    }
     def __init__(self, input_file):
         super().__init__(input_file)
         match = re.search(r'(?:\b|_)(\d{4}-\d{2}-\d{2})(?:\b|_)',
                           pathlib.Path(input_file.name).name)
         if match:
             self.entry_seed['date'] = strparse.date(match.group(1), '%Y-%m-%d')
     def _read_row(self, row):
         if row['Status'] != 'Processed':
             return None
         else:
             return {
                 'payee': '{0[FirstName]} {0[LastName]}'.format(row),
+            }
 class PayoutImporter(_csv.CSVImporterBase):
     AMOUNT_KEY = 'Total funds deducted from creator balance'
     PLEDGE_KEY = 'Funds used for pledges to other creators'
     TRANSFER_KEY = 'Funds transferred to you'
     NEEDED_FIELDS = frozenset([
         'Month',
         TRANSFER_KEY,
         PLEDGE_KEY,
         AMOUNT_KEY,
     ])
     ENTRY_SEED = {
         'currency': 'USD',
         'payee': 'Patreon',
+    }
     def _read_row(self, row):
         amount = strparse.currency_decimal(row[self.AMOUNT_KEY])
         if not amount:
             return None
         else:
             return {
                 'amount': amount,
                 'date': strparse.date(row['Month'], '%Y-%m'),
                 'pledges_amount': strparse.currency_decimal(row[self.PLEDGE_KEY]),
                 'transfer_amount': strparse.currency_decimal(row[self.TRANSFER_KEY]),
+            }
 class FeeImporterBase(_csv.CSVImporterBase):
     ENTRY_SEED = {
         'currency': 'USD',
         'payee': "Patreon",
+    }
     def _read_row(self, row):
         return {
             'amount': row[self.AMOUNT_FIELD],
             'date': strparse.date(row['Month'], '%Y-%m'),
+        }
 class ServiceFeesImporter(FeeImporterBase):
     AMOUNT_FIELD = 'Patreon Fee'
     NEEDED_FIELDS = frozenset(['Month', AMOUNT_FIELD])
 class CardFeesImporter(FeeImporterBase):
     AMOUNT_FIELD = 'Processing Fees'
     NEEDED_FIELDS = frozenset(['Month', AMOUNT_FIELD])
 class VATImporter(FeeImporterBase):
     AMOUNT_FIELD = 'Vat Charged'
     NEEDED_FIELDS = frozenset(['Month', AMOUNT_FIELD])
     COPIED_FIELDS = {
         'Country Code': 'country_code',
         'Country Name': 'country_name',
+    }

setup.py

➞

Show inline comments

 #!/usr/bin/env python3
 import sys
 from setuptools import setup, find_packages
 REQUIREMENTS = {
     'install_requires': [
         'babel',
         'enum34;python_version<"3.4"',
     ],
     'setup_requires': ['pytest-runner'],
     'extras_require': {
         'brightfunds': ['xlrd'],
         'nbpy2017': ['beautifulsoup4', 'html5lib'],
     },
+}
 all_extras_require = [
     req for reqlist in REQUIREMENTS['extras_require'].values() for req in reqlist
+]
 REQUIREMENTS['extras_require']['all_importers'] = all_extras_require
 REQUIREMENTS['tests_require'] = [
     'pytest',
     'PyYAML',
     *all_extras_require,
+]
 setup(
     name='import2ledger',
     description="Import different sources of financial data to Ledger",
-    version='0.5',
+    version='0.6',
     author='Brett Smith',
     author_email='brettcsmith@brettcsmith.org',
     license='GNU AGPLv3+',
     packages=find_packages(include=['import2ledger', 'import2ledger.*']),
     entry_points={
         'console_scripts': ['import2ledger = import2ledger.__main__:main'],
     },
     **REQUIREMENTS,
+)

tests/data/PatreonPayouts.csv

➞

Show inline comments

 new file 100644
 Month,Funds transferred to you,Funds used for pledges to other creators,Total funds deducted from creator balance
 -03,$0,$0,$0
 -04,$123.45,$0,$123.45
 -05,"$2,345.67",$0,"$2,345.67"

tests/data/imports.yml

➞

Show inline comments

 - source: PatreonPatronReport_2017-09-01.csv
   importer: patreon.IncomeImporter
   expect:
     - payee: Alex Jones
       date: !!python/object/apply:datetime.date [2017, 9, 1]
       amount: !!python/object/apply:decimal.Decimal ["1500.00"]
       currency: USD
     - payee: Dakota Doe
       date: !!python/object/apply:datetime.date [2017, 9, 1]
       amount: !!python/object/apply:decimal.Decimal ["12.00"]
       currency: USD
 - source: PatreonEarnings.csv
   importer: patreon.ServiceFeesImporter
   expect:
     - payee: Patreon
       date: !!python/object/apply:datetime.date [2017, 9, 1]
       amount: !!python/object/apply:decimal.Decimal ["61.73"]
       currency: USD
     - payee: Patreon
       date: !!python/object/apply:datetime.date [2017, 10, 1]
       amount: !!python/object/apply:decimal.Decimal ["117.03"]
       currency: USD
 - source: PatreonEarnings.csv
   importer: patreon.CardFeesImporter
   expect:
     - payee: Patreon
       date: !!python/object/apply:datetime.date [2017, 9, 1]
       amount: !!python/object/apply:decimal.Decimal ["52.47"]
       currency: USD
     - payee: Patreon
       date: !!python/object/apply:datetime.date [2017, 10, 1]
       amount: !!python/object/apply:decimal.Decimal ["99.47"]
       currency: USD
 - source: PatreonVat.csv
   importer: patreon.VATImporter
   expect:
     - payee: Patreon
       date: !!python/object/apply:datetime.date [2017, 9, 1]
       amount: !!python/object/apply:decimal.Decimal ["2.00"]
       currency: USD
       country_code: AT
       country_name: Austria
     - payee: Patreon
       date: !!python/object/apply:datetime.date [2017, 9, 1]
       amount: !!python/object/apply:decimal.Decimal ["3.30"]
       currency: USD
       country_code: BE
       country_name: Belgium
     - payee: Patreon
       date: !!python/object/apply:datetime.date [2017, 10, 1]
       amount: !!python/object/apply:decimal.Decimal ["0.40"]
       currency: USD
       country_code: BG
       country_name: Bulgaria
     - payee: Patreon
       date: !!python/object/apply:datetime.date [2017, 10, 1]
       amount: !!python/object/apply:decimal.Decimal ["6.05"]
       currency: USD
       country_code: CZ
       country_name: Czech Republic
 - source: PatreonPayouts.csv
   importer: patreon.PayoutImporter
   expect:
     - payee: Patreon
       date: !!python/object/apply:datetime.date [2018, 4, 1]
       amount: !!python/object/apply:decimal.Decimal ["123.45"]
       pledges_amount: !!python/object/apply:decimal.Decimal ["0"]
       transfer_amount: !!python/object/apply:decimal.Decimal ["123.45"]
       currency: USD
     - payee: Patreon
       date: !!python/object/apply:datetime.date [2018, 5, 1]
       amount: !!python/object/apply:decimal.Decimal ["2345.67"]
       pledges_amount: !!python/object/apply:decimal.Decimal ["0"]
       transfer_amount: !!python/object/apply:decimal.Decimal ["2345.67"]
       currency: USD
 - source: StripePayments.csv
   importer: stripe.PaymentImporter
   expect:
     - payee: Dakota Smith
       date: !!python/object/apply:datetime.date [2017, 11, 8]
       amount: !!python/object/apply:decimal.Decimal ["100.00"]
       fee: !!python/object/apply:decimal.Decimal ["3"]
       tax: !!python/object/apply:decimal.Decimal ["0"]
       currency: USD
       payment_id: ch_oxuish6phae2Raighooghi3U
       customer_id: cus_DohSheeQu8eng3
       customer_email: one@example.org
       payout_id: po_aeYees2ahtier8ohju7Eeyie
       description: "Payment for invoice #102"
     - payee: Dakota Jones
       date: !!python/object/apply:datetime.date [2017, 10, 28]
       amount: !!python/object/apply:decimal.Decimal ["50.00"]
       fee: !!python/object/apply:decimal.Decimal ["1.4"]
       tax: !!python/object/apply:decimal.Decimal ["0"]
       currency: USD
       payment_id: ch_hHee9ef1aeyee1ruo7ochee9
       customer_id: cus_iepae2Iecae8Ei
       customer_email: two@example.org
       payout_id: po_aeYees2ahtier8ohju7Eeyie
       description: "Payment for invoice #100"
 - source: StripePayouts.csv
   importer: stripe.PayoutImporter
   expect:
     - payee: Stripe
       date: !!python/object/apply:datetime.date [2017, 11, 30]
       amount: !!python/object/apply:decimal.Decimal ["-50"]
       currency: USD
       payment_count: !!python/object/apply:decimal.Decimal ["0"]
       payment_gross: !!python/object/apply:decimal.Decimal ["0"]
       payment_fees: !!python/object/apply:decimal.Decimal ["0"]
       payment_net: !!python/object/apply:decimal.Decimal ["0"]
       refund_count: !!python/object/apply:decimal.Decimal ["1"]
       refund_gross: !!python/object/apply:decimal.Decimal ["-50"]
       refund_fees: !!python/object/apply:decimal.Decimal ["0"]
       refund_net: !!python/object/apply:decimal.Decimal ["-50"]
       collected_fee_count: !!python/object/apply:decimal.Decimal ["0"]
       collected_fee_gross: !!python/object/apply:decimal.Decimal ["0"]
       collected_fee_refund_count: !!python/object/apply:decimal.Decimal ["0"]
       collected_fee_refund_gross: !!python/object/apply:decimal.Decimal ["0"]
       adjustment_count: !!python/object/apply:decimal.Decimal ["0"]
       adjustment_gross: !!python/object/apply:decimal.Decimal ["0"]
       adjustment_fees: !!python/object/apply:decimal.Decimal ["0"]
       adjustment_net: !!python/object/apply:decimal.Decimal ["0"]
       validation_count: !!python/object/apply:decimal.Decimal ["0"]
       validation_fees: !!python/object/apply:decimal.Decimal ["0"]
       retried_payout_count: !!python/object/apply:decimal.Decimal ["0"]
       retried_payout_net: !!python/object/apply:decimal.Decimal ["0"]
       total_count: !!python/object/apply:decimal.Decimal ["1"]
       total_gross: !!python/object/apply:decimal.Decimal ["-50"]
       total_fees: !!python/object/apply:decimal.Decimal ["0"]
       total_net: !!python/object/apply:decimal.Decimal ["-50"]
       payout_id: po_faegh6aeghishuethuoSoT2i
       destination_id: ba_chu0Woop5queewi2Ea1Aibah
       balance_txid: txn_EiKahrazei3aeMohk7EeDigh
       failure_txid: ""
     - payee: Stripe
       date: !!python/object/apply:datetime.date [2017, 11, 29]
       amount: !!python/object/apply:decimal.Decimal ["146.50"]
       currency: USD
       payment_count: !!python/object/apply:decimal.Decimal ["2"]
       payment_gross: !!python/object/apply:decimal.Decimal ["150"]
       payment_fees: !!python/object/apply:decimal.Decimal ["3.5"]
       payment_net: !!python/object/apply:decimal.Decimal ["146.5"]
       refund_count: !!python/object/apply:decimal.Decimal ["0"]
       refund_gross: !!python/object/apply:decimal.Decimal ["0"]
       refund_fees: !!python/object/apply:decimal.Decimal ["0"]
       refund_net: !!python/object/apply:decimal.Decimal ["0"]
       collected_fee_count: !!python/object/apply:decimal.Decimal ["0"]
       collected_fee_gross: !!python/object/apply:decimal.Decimal ["0"]
       collected_fee_refund_count: !!python/object/apply:decimal.Decimal ["0"]
       collected_fee_refund_gross: !!python/object/apply:decimal.Decimal ["0"]
       adjustment_count: !!python/object/apply:decimal.Decimal ["0"]
       adjustment_gross: !!python/object/apply:decimal.Decimal ["0"]
       adjustment_fees: !!python/object/apply:decimal.Decimal ["0"]
       adjustment_net: !!python/object/apply:decimal.Decimal ["0"]
       validation_count: !!python/object/apply:decimal.Decimal ["0"]
       validation_fees: !!python/object/apply:decimal.Decimal ["0"]
       retried_payout_count: !!python/object/apply:decimal.Decimal ["0"]
       retried_payout_net: !!python/object/apply:decimal.Decimal ["0"]
       total_count: !!python/object/apply:decimal.Decimal ["2"]
       total_gross: !!python/object/apply:decimal.Decimal ["150"]
       total_fees: !!python/object/apply:decimal.Decimal ["3.5"]
       total_net: !!python/object/apply:decimal.Decimal ["146.5"]
       payout_id: po_Do9pathoo9Pu8jaePhahJa0e
       destination_id: ba_chu0Woop5queewi2Ea1Aibah
       balance_txid: txn_ahsaixiene6Thie1aiti3tuo
       failure_txid: ""
 - source: nbpy2017a.html
   importer: nbpy2017.InvoiceImporter

0 comments (0 inline, 0 general)