This Travel and Reimbursable Expense Policy (“Policy”) applies to all +Conservancy Member Projects (“Projects”) of Software Freedom Conservancy +(“Conservancy”) and has been created to memorialize Conservancy’s +reimbursement policies relating to travel and other business expenses +incurred by Conservancy staff, Project Leadership Committee (“PLC”) +members, and project volunteers while engaged in business on behalf of, or +at the behest of Conservancy and/or a Project (“Travelers”).
+This Policy includes an Easy Reference Guide that can be used as a +template for most of the travel covered under this Policy. When in doubt, +refer to the more detailed sections below.
+Conservancy must maintain effective control of business-related expenses +in order to maintain its financial viability and tax exempt status. +Conservancy and each Project is also accountable to our donors to ensure +that we manage their contributions wisely and maximize our ability to +pursue our charitable mission. As such, Conservancy expects Travelers to +use good judgment and to claim reimbursement for only those expenses that +are necessary and reasonable. Excessive expenses, including but not +limited to luxury accommodations and services unnecessary for, or unrelated +to the furtherance of Conservancy’s charitable mission are not eligible for +reimbursement.
+Any travel expense that adheres to this Policy is considered In-Policy +and does not require special approval, so long as the trip itself +has been approved in writing by Conservancy’s Executive Director or +by a Project’s Leadership Committee (“PLC”) in a regular and documented +PLC vote. Conservancy and/or a PLC can limit allowable travel expenses +to an amount less than what would otherwise be considered acceptable +according to this Policy. If so, the smaller budget is the maximum +allowed expense.
+PLC’s may, in fact, have their own travel policy that is more restrictive +than this one. Please consult the PLC for your Conservancy project before +incurring an expenses to ensure you understand what expenses can be +reimbursed.
+Travelers should adhere to the following guidelines to stay In-Policy.
+Before you buy tickets, save a screenshot of a flight search that + shows the least expensive fare available from multiple airlines for + the dates you need to travel.
+Book at least 14 days in advance.
+Fares within $100 of that lowest fare you found are In-Policy, even if + you book on a different site or for different dates.
+Fares over $750 domestically or $1,650 internationally require + Conservancy’s pre-approval.
+Hotels are In-Policy as long as the average nightly rate you pay + (excluding taxes and fees) is under the maximum rate for lodging for your + destination. See the “Rates” section below for details.
+Be sure the receipt from your hotel includes the dates of your + stay, not just a total.
+Keep and submit PDFs of the following, as applicable:
+Your fare search (i.e., a screen shot)
+Your plane and rail tickets (e-confirmation of purchase is sufficient)
+Your hotel/lodging invoice
+Use your per diem to cover meals and incidental expenses.
+Your total per diem allowance follows the rates published for your + destination. See the “Rates” section below for details.
+Try to be frugal: per diem rates are rather generous.
+When attending a conference where food is included, Do The Right + Thing and claim a lower per diem.
+To receive reimbursement, send transportation and hotel receipts, + along with a list of per diem days and totals, to + accounting@sfconservancy.org. Refer to the Expense Report + section below for details.
+Your complete reimbursement request must be submitted to + accounting@sfconservancy.org within 90 days of the last date of + travel. Untimely requests will not be reimbursed.
+Throughout this document, we refer to rates reported by other parties.
+For travel in the United States, we follow the maximum rates for lodging and +M&IE per diem set by the +US General Services Administration.
+For travel outside the United States, we follow the maximum rates for lodging and +M&IE per diem set by the +US Department of State.
+We calculate the total per diem allowance for a trip using the same method +as the GSA. Travelers may request up to 100% of the listed rate for each +full day of travel, plus 75% of the listed rate for each partial day of +travel. For example, if you fly to a conference on Monday, spend Tuesday +through Thursday at the conference, and return home on Friday, and the per +diem rate for the conference city is $80, you may request up to $360: $80 +for each day Tuesday through Thursday, plus $60 for each day you flew.
+When we convert currencies (e.g., to determine whether a hotel paid in Euros +was within the maximum lodging rate), we use the final rate published by +Open Exchange Rates on the date we received +the reimbursement request. Please do not do your own currency conversions +in your reimbursement requests. Simply report expenses in their original +currency/ies, and we will convert appropriately. If you have questions or +concerns about our rates, just ask, and we’ll be happy to provide details +before we send you final payment.
+Conservancy handles reimbursements on a NET-30 basis, starting from the date +that complete materials are received. If this is an issue, Conservancy is +available to prepurchase expensive items like airline tickets on your +behalf, so that you don’t need to be reimbursed.
+If you seek to be reimbursed for Conservancy Project expenses, please send +the following, in a self contained email (with attachments as necessary), +cc’ing your Project Leadership Committee address (PROJECT@sfconservancy.org) +for Project approval:
+A brief paragraph explaining what was accomplished for the project + during your travel and/or with the funds being reimbursed. This can + be informal; it’s just for our records to confirm the travel advanced + the not-for-profit mission of both Conservancy and the project.
+A brief report listing the items to be reimbursed.
+Full receipt(s) for everything, unless the travel policy does not + require receipts (e.g., for per diem expenses).
+If your receipts are in a different currency than your preferred one + for reimbursement, include documentation of the rate conversion (e.g., a + redacted credit card statement in your preferred currency). Otherwise, + Conservancy will use the prevailing rate for the date of the expense for + conversion.
+Please verify that the receipts that you submit are within the attached + travel policy requirements. Note, however, that your Project Leadership + Committee may have set a stricter budget than what the general + Conservancy policy allows.
+How you’d liked to be reimbursed. The four payment options, in order of + Conservancy’s preference, are:
+PayPal. For this, we need (a) the email address that is registered to + the PayPal account that will receive reimbursement and (b) the preferred + currency. (Please verify that PayPal supports transactions in + your preferred currency before choosing one.) + Note that sometimes PayPal charges fees. Usually we’re taking funds + from our PayPal balance, which means no fees appear on our side, but + they may appear on yours. In our experience, USA PayPal account holders + physically in the USA are not typically charged fees; but we do advise + you to check PayPal’s fee schedule before choosing this method.
+NOTE: Typically, when choosing PayPal as a payment option, amounts owed + in USD will be paid in USD, regardless of your preferred currency + selection.
+Issue you a check in USD from a USA bank, sent to you via post. + For this option, I just need your postal address.
+Wire the amount in your local currency to your bank account in your + country. For this option, I need as much of the following information + as it’s possible for you to collect.
+Required information
+Full Name of the Account Holder
+Full Address of the Account Holder
+Account Number
+Preferred Currency
+Bank Name
+Bank Address
+Bank ACH or ABA routing number (for banks in the US); + or SWIFT and/or BIC code (for banks outside the US)
+Additional required information by country
+All countries inside the EU: IBAN
+Australia: BSB or “Bank Code”
+Brazil: IABN; Tax ID of the Account Holder (your 14-digit CNPJ + or 11-digit CPF); Phone Number of the Account Holder; + and Bank Agency Code
+Colombia: Tax ID of the Account Holder; and Phone Number + of the Account Holder
+India: IFSC Code
+Kenya: The name of the local branch of your bank where you hold your + account
+Qatar: IBAN
+Russian Federation: Beneficiary INN; Patronymic Name of Beneficiary; + and VAT/VO Code of Beneficiary Bank
+Ukraine: BSB or “Bank Code”; Tax ID of the Account Holder; + and Phone Number of the Account Holder
+United States: Phone Number of the Account Holder
+Additional information we can use
+Banks outside the US will often designate a Correspondent Bank when + receiving funds from the US. If you can give us the Correspondent + Bank Name and ACH, we can specify that in the wire. Your bank will + usually take the fewest fees when receiving wires from its + Correspondent Bank, so this will mean the most money for you.
+A check in your local currency, sent to you via post. For this option, we + just need your postal address and what currency you want. Note that this + is a more involved option to process and will usually take at least 30 + days to issue payment. We do not recommend this method. Please choose + it only if you absolutely cannot receive a wire transfer for some reason.
+Project Leadership Committees: when you see emails of this nature, please +be sure to have your designated Representative review the materials and +send an approval message to Conservancy.
+Conservancy foresees the need for periodic reasonable exceptions to +this Policy. Persons working on behalf of a specific Project seeking +an exception to this Policy must petition their PLC to obtain written +approval from Conservancy authorizing the exception. Persons working +directly on behalf of Conservancy seeking an exception to the +Policy must obtain written approval from Conservancy authorizing the +exception.
+PLCs are responsible for creating procedures for requesting exceptions, +and submitting to Conservancy reimbursement requests associated with +their respective Projects. PLCs are also responsible for making available +a list of required response times for inquiries, including but not +limited to, the following two cases
+a specific number of days to respond to regular reimbursement requests, +and
+a specific number of days to respond to pre-authorization requests.
+PLCs are also responsible for monitoring the available balance in their +Project Fund, and for granting or refusing approval for travel expense +requests based on an assessment of the funds available and of any +outstanding contracts payable. PLCs are not to approve travel expense +requests when their Project does not have sufficient funds to cover the +expense. If a PLC has any questions regarding whether their Project has +sufficient funds to cover a Traveler’s expense request, the PLC should +contact Conservancy.
+Domestic transportation costs greater than US$750 requires Conservancy +approval prior to booking, even if all other Policy conditions have been +met. International transportation costs greater than US$1,800 requires +Conservancy approval prior to booking, even if all other Policy conditions +have been met.
+Tickets for travel by air or rail (excluding commuter train and subway) +should be booked at least 14 days in advance; any travel booked less than +14 days in advance requires written pre-authorization by Conservancy. +Tickets for travel by air or rail beyond 365 days in advance also require +written pre-authorization by Conservancy.
+Coach and/or Economy Airfare is the only acceptable class for all flights +(domestic and international) unless a PLC provides a special exception and +a valid reason (such as a need for business class due to a documented +medical reason) to Conservancy for written approval. Travelers may select +their airline of choice (e.g., for the purpose of collecting airline miles +and rewards), provided that the resulting fare otherwise meets the +requirements of this Policy. Travelers should not book out-of-Policy trips +(and thus pay a higher fare) in order to qualify for a mileage upgrade.
+Air travel should be booked at least 14 days in advance; any travel booked +less than 14 days in advance requires written pre-authorization by +Conservancy. Flights beyond 365 days in advance also require written +pre-authorization by Conservancy.
+Conservancy aims to balance cost savings with time savings and convenience. +Budgets for flights are set based on their travel time compared to the +flight with the lowest available fare. Flights with fares that are within +budget are in-Policy.
+To find the lowest available fare, run a flight search that meets these +criteria, and save the results:
+The search must include fares from multiple airlines. Any + widely-recognized airfare search site that lists results from multiple + airlines is acceptable, including sites such as + Orbitz, Kayak or + Hipmunk.
+The search must cover only the dates of relevant travel. For example, if + you’re attending a conference that runs Monday through Friday, the search + must have you arriving no earlier than Sunday, and leaving no later than + Saturday.
+Except as allowed by Policy, the search must not use filters that might + exclude the least expensive fare. For example, you may filter out + flights with two or more connections, since Conservancy does not + consider those reasonable. However, you may not filter out specific + airlines, or flights without WiFi.
+Save the results of this search. A PDF printout of the first page of +results from your browser is ideal. A screenshot can work too. Just make +sure the output shows the search criteria and the lowest available fare. +When you send your reimbursement request, attach these results.
+The budget for a flight is set depending on how its cost and travel time +compares to the flight with the lowest available fare. Travel time is +measured from the scheduled departure time of the first flight in the +itinerary to the scheduled landing time of the final flight. We use the +following table to determine the budget:
+| If the travel time for a flight is… | +the budget for that flight is… | +
|---|---|
| the same or longer than the flight with the lowest available fare | +the lowest available fare + US$100 | +
| less than three hours shorter | +the lowest available fare + US$100 | +
| between three and six hours shorter | +the lowest available fare + US$200 | +
| between six and ten hours shorter | +the lowest available fare + US$350 | +
| at least ten hours shorter | +the lowest available fare + US$600 | +
Any flight with a total cost that is within its corresponding budget is +within Policy. Any flight with a cost over its budget requires written +pre-authorization by Conservancy.
+Travelers may book their tickets on different dates or a different site as +long as they used a qualifying fare search site to determine that the +booked flights are within Policy.
+Conservancy asks that Travelers allow for flexibility with respect +to departure times during a desired day of travel, as well as longer +trips in order to reduce cost. However, Conservancy does consider +flights with two or more connections as unreasonable and does not +expect Travelers to consider those flight options to be reasonable.
+Should a team member travel on an airline that charges for a single piece of +checked baggage, such a baggage expense is eligible for reimbursement with a +receipt. Team members are responsible for charges on any baggage beyond a +single piece, unless that additional baggage is materials specifically +related to the Project’s and Conservancy’s mission (i.e., bringing t-shirts +and other promotional materials to an event).
+All air travel not adhering to the above Policies are considered Out-of-Policy +and require written pre-authorization by an officer of Conservancy.
+Cancellation fees and other penalties incurred result of a change +of plans are reimbursable at Conservancy’s discretion. In general, +Conservancy shall reimburse such fees if the Traveler can submit a +valid reason for the change of plans. Acceptable reasons include Conservancy +and/or the PLC canceling or altering the trip or unexpected delays +in flight connections. In instances where these fees are incurred +without adequate explanation, Conservancy reserves the right to refuse +to reimburse the cost of the fees.
+Ground transportation necessary as part of authorized Project trips +is considered to be a reasonable expense. Public ground transportation, +such as taxis, shuttles, buses and municipal transit, are generally +the most cost-effective options and are the standard for eligible +ground transportation reimbursements. All car rentals require pre-authorization +by the PLC or by an officer of Conservancy. When car rentals +have been pre-approved, the rental of compact cars is encouraged; +mid-size vehicles are authorized when necessary (e.g., when compact-sized +vehicles are not available or the number of passengers or volume of +baggage makes a compact vehicle impractical).
+Rail transportation as a means of travel for an authorized Project +trip is considered to be a reasonable expense. All rail transportation +must be in economy and/or coach class.
+When circumstances require Travelers to utilize their personal vehicles for +Project purposes, they can be reimbursed at the current +USA IRS Standard Mileage Rate, +plus any related parking expenses and toll fees. Drivers are encouraged to +find the lowest cost parking area reasonably near their destination.
+Travelers often seek to add extra days before or after an approved trip +(e.g., the weekend before a conference). A Traveler may seek approval for +the expenses associated with an extended stay prior to booking the trip, +provided that the additional days are solely to enable a Traveler to +conduct work within the PLC’s objectives and Conservancy’s charitable +mission, or to get a particular airfare that reduces the overall cost of +the trip. Travelers may seek approval to book travel itineraries that +include extra days for personal reasons, so long as the cost of the flight +meets the other requirements of this Policy. Other expenses incurred +during extra personal days beyond transportation costs are not reimbursable.
+Travelers are expected to be cost-conscious and prudent when booking lodging +for approved trips, and to verify that rates are within the maximum lodging +rates for the hotel’s location. See the “Rates” section above for details.
+If the lodging chosen by the Traveler and/or the PLC exceeds the maximum +lodging rate for the given location (per Traveler), the Traveler and/or the +PLC must obtain written pre-approval from Conservancy and the PLC before +booking the hotel. If written pre-approval is not sought or is not granted, +Conservancy will only reimburse up to the maximum lodging rate.
+Lodging documentation submitted as part of a reimbursement request must +include a copy of the hotel invoice detailing all charges (credit card +receipts alone are unacceptable). In particular, since Conservancy only +reimburses for room charges (plus relevant taxes and fees) for the necessary +travel dates, the receipt from the hotel must clearly show the dates of stay, +and separately list room charges and any food or service charges. +Conservancy will not reimburse Travelers for any costs associated with an +upgrade of room accommodations.
+In some cases, Conservancy, upon consultation with the PLC, may decide to +book lodging on behalf of Travelers. In this case, Conservancy-booked +lodging is always considered In-Policy.
+Conservancy will reimburse persons for Project-related expenses that +are incurred while traveling on approved Project business and/or approved +Conservancy business. Only necessary, ordinary and reasonable expenses +are eligible for reimbursement, and only those categories of expenses +listed in this document qualify.
+Travelers can submit for a per diem for meals and incidental expenses for +every day of a trip devoted to Project- and/or Conservancy-related mission +work, including the day(s) of travel itself, up to the maximum rate for the +destination of the trip. See the “Rates” section above for details.
+These per diem rates are the maximum daily rate Travelers can claim. If a +conference has provided food, or food is provided in some other form, or +the costs the Traveler incurs are lower than this rate, then the Traveler +should reasonably reduce their per diem claim.
+PLCs and/or Conservancy have the authority to set lower per diem rates +than those generated by the calculators above. In those instances, +Travelers will only be able to submit for the lower per diem rates.
+For groups of Travelers on an In-Policy trip, each Traveler should +pay for his/her own meals, seeing as all participants will have an +opportunity to submit for separate per diem reimbursements after the trip.
+For clarification purposes, this Policy does not relate to planned +group events that include meals and/or refreshments (e.g., a PLC-organized +conference that includes lunch for all attendees). Further, PLCs and/or +Conservancy retain the right to allocate a separate budget for anticipated +large group meals beyond the individual per diem limits of each Traveler, +provided that they are within the PLC’s technical objectives and/or +Conservancy’s mission. Travelers anticipating a need to cover such +a large group meal should seek pre-approval from his/her PLC and/or Conservancy +for such expenses before the trip.
+For any such group meal, Conservancy will require a written paragraph +summary of the meeting, indicating what was accomplished for the Project’s +and Conservancy’s mission.
+Travelers may occasionally have the need to invite third parties +(e.g., prospective donors, contributors, community members, etc.) to +meals in order to further a PLC’s technical direction and/or Conservancy’s +mission. Conservancy recommends that Travelers seek pre-approval from +their PLC and/or Conservancy for such meals.
+For any such organizational development meal, Conservancy will require a +written paragraph summary of the meeting, indicating what was accomplished +for the Project’s and Conservancy’s mission.
+Charges for personal phone calls (e.g., made from a hotel, or via +a mobile phone in international travel) are not reimbursable as an +expense separate from the allocated per diem.
+Any fees associated with currency conversion are not reimbursable as an +expense separate from the allocated per diem.
+Conservancy will reimburse conference registration fees up to $100 per day +for Travelers on approved Project business and/or approved Conservancy +business. For example, a $250 registration fee for a 3-day conference is +In-Policy; however, a $225 registration fee for a 2-day conference is not.
+Travelers seeking reimbursement for registration fees that exceed $100 per +day must obtain prior approval from an officer of Conservancy.
+Internet access/wi-fi fees charged by a hotel are reimbursable, provided +that they are listed on the hotel/lodging invoice submitted for +reimbursement. Other internet access fees (e.g., airport internet +services, personal wi-fi hotspots, internet cafes) are not reimbursable +except as incidental expenses to be covered by a Traveler’s per diem.
+Non-reimbursable expenses are identified throughout this policy. The +following items are typically non-reimbursable expenses:
+Partner, spouse, and/or companion travel
+First class travel (unless medically necessary)
+Upgrades to air travel, car rentals, or hotel rooms
+Purchase of clothing, luggage, toiletries and other miscellaneous +personal items
+Supplemental travel or car rental insurance
+Fines, penalties, or legal fees
+Personal entertainment or recreational expenses beyond the allotted per +diem
+Travelers are permitted to pay for their own upgrades, or use bonus +programs to upgrade Conservancy-reimbursed expenses. However, Travelers +must ensure that Conservancy does not receive nor reimburse any charges +for any such transaction.
+Reimbursed travel expenses are subject to examination by the USA Internal +Revenue Service (IRS). Travelers are responsible for retaining documentary +evidence that all expenses are strictly for Project- and/or +Conservancy-related purposes, not personal in nature, and therefore not +includable as taxable income to the Traveler. Receipts are required for +all expenses, no matter the amount.
+Travelers traveling on behalf of a Project must seek approvals and +submit expense reports to their PLC. PLCs are to review those expense +reports and pass them along to Conservancy’s accounting office for +final approval and reimbursement.
+Travelers traveling on behalf of Conservancy must seek approvals from +Conservancy’s Executive Director, and submit expense reports to +Conservancy’s accounting office for reimbursement.
+Travelers seeking reimbursement must submit an expense report to the +appropriate channel with the following information:
+Name of Traveler
+Brief description of trip and trip’s purpose (e.g., “August 2011 +trip to XYZ conference for ABC project, served as planning committee +member”; “Feb. ‘12 FOO hackfest in Portland, OR; contributed code”)
+A brief paragraph explaining what was accomplished for the project +during your travel.
+Number of days traveled (with documentary evidence, e.g., conference +itinerary, etc.) and associated per diem
+List of expenses not covered by per diem (e.g., transportation, lodging) +with substantiating receipts (or scans of receipts).
+In the event that it is impractical to obtain a required receipt and/or if + such receipt has been inadvertently destroyed or lost, the Traveler should + furnish a written statement to that effect, as well as an explanation of + the expenditure involved. When possible, secondary documentation (such as + a redacted credit card bill) should be provided instead of the + lost/destroyed receipt.
+Any expense without a substantiated receipt and/or a supporting written +statement will not be reimbursed.
+Conservancy requests that all expense reports be submitted within two weeks +of travel. Expense reports filed more than 90 days after the last day of +travel (or for other reimbursable expenses, the day expenses are incurred) +will not be reimbursed.
+Reimbursements are paid by Conservancy on a NET-30 basis, from the +date of receipt by Conservancy of the fully complete report and supporting +documentation for the travel.
+Failure to comply with this policy may result in the denial of, or delay +in payment for, reimbursement requests.
+The Conservancy reserves the right to change any terms of this Policy +from time to time. The Policy of record shall be the Policy most recently +distributed by the Conservancy.
+ +{% endblock %} diff --git a/www/conservancy/static/projects/policies/conservancy-travel-policy.html b/www/conservancy/static/projects/policies/conservancy-travel-policy.html new file mode 120000 index 0000000000000000000000000000000000000000..c23ef0af39882908a2e81bf807a43a0536a3f9fb --- /dev/null +++ b/www/conservancy/static/projects/policies/conservancy-travel-policy.html @@ -0,0 +1 @@ +conservancy-travel-policy.783dcdd92fc61f3f150e1c65782c0fe527c8ff52.html \ No newline at end of file diff --git a/www/conservancy/static/projects/policies/index.html b/www/conservancy/static/projects/policies/index.html new file mode 100644 index 0000000000000000000000000000000000000000..ba2a627455a1d0651545bf8bbe3371f777a93e92 --- /dev/null +++ b/www/conservancy/static/projects/policies/index.html @@ -0,0 +1,16 @@ +{% extends "base_projects.html" %} +{% block subtitle %}Member Project Policies - {% endblock %} +{% block submenuselection %}Policies{% endblock %} +{% block content %} + +These are the policies that member projects follow in working with Conservancy. These pages are provided as a reference to all projects participants.
+ + + +For more background about the policies, including licensing and change requests, please refer to their source code in Git.
+ +{% endblock %} diff --git a/www/conservancy/static/projects/policies/publish-travel-policy.py b/www/conservancy/static/projects/policies/publish-travel-policy.py new file mode 100755 index 0000000000000000000000000000000000000000..0aabd11c684c71b0618baf5d39ee5787f6916a80 --- /dev/null +++ b/www/conservancy/static/projects/policies/publish-travel-policy.py @@ -0,0 +1,346 @@ +#!/usr/bin/env python3 + +import argparse +import contextlib +import functools +import locale +import os +import pathlib +import shutil +import subprocess +import sys +import tempfile + +try: + import markdown + from markdown.extensions import tables as mdx_tables + from markdown.extensions import sane_lists as mdx_sane_lists + from markdown.extensions import smarty as mdx_smarty + from markdown.extensions import toc as mdx_toc + markdown_import_success = True +except ImportError: + if __name__ != '__main__': + raise + markdown_import_success = False + +TEMPLATE_HEADER = """{% extends "base_projects.html" %} +{% block subtitle %}Travel and Reimburseable Expense Policy - {% endblock %} +{% block submenuselection %}Policies{% endblock %} +{% block content %} + +""" + +TEMPLATE_FOOTER = """ + +{% endblock %} +""" + +@contextlib.contextmanager +def run(cmd, encoding=None, ok_exitcodes=frozenset([0]), **kwargs): + kwargs.setdefault('stdout', subprocess.PIPE) + if encoding is None: + mode = 'rb' + no_data = b'' + else: + mode = 'r' + no_data = '' + with contextlib.ExitStack() as exit_stack: + proc = exit_stack.enter_context(subprocess.Popen(cmd, **kwargs)) + pipes = [exit_stack.enter_context(open( + getattr(proc, name).fileno(), mode, encoding=encoding, closefd=False)) + for name in ['stdout', 'stderr'] + if kwargs.get(name) is subprocess.PIPE] + if pipes: + yield (proc, *pipes) + else: + yield proc + for pipe in pipes: + for _ in iter(lambda: pipe.read(4096), no_data): + pass + if proc.returncode not in ok_exitcodes: + raise subprocess.CalledProcessError(proc.returncode, cmd) + +class GitPath: + GIT_BIN = shutil.which('git') + CLEAN_ENV = {k: v for k, v in os.environ.items() if not k.startswith('GIT_')} + ANY_EXITCODE = range(-256, 257) + IGNORE_ERRORS = { + 'ok_exitcodes': ANY_EXITCODE, + 'stderr': subprocess.DEVNULL, + } + STATUS_CLEAN_OR_UNMANAGED = frozenset(' ?') + + def __init__(self, path, encoding, env=None): + self.path = path + self.dir_path = path if path.is_dir() else path.parent + self.encoding = encoding + self.run_defaults = { + 'cwd': str(self.dir_path), + 'env': env, + } + + @classmethod + def can_run(cls): + return cls.GIT_BIN is not None + + def _run(self, cmd, encoding=None, ok_exitcodes=frozenset([0]), **kwargs): + return run(cmd, encoding, ok_exitcodes, **self.run_defaults, **kwargs) + + def _cache(orig_func): + attr_name = '_cached_' + orig_func.__name__ + @functools.wraps(orig_func) + def cache_wrapper(self): + try: + return getattr(self, attr_name) + except AttributeError: + setattr(self, attr_name, orig_func(self)) + return getattr(self, attr_name) + return cache_wrapper + + @_cache + def is_work_tree(self): + with self._run([self.GIT_BIN, 'rev-parse', '--is-inside-work-tree'], + self.encoding, **self.IGNORE_ERRORS) as (_, stdout): + return stdout.readline() == 'true\n' + + @_cache + def status_lines(self): + with self._run([self.GIT_BIN, 'status', '-z'], + self.encoding) as (_, stdout): + return stdout.read().split('\0') + + @_cache + def has_managed_modifications(self): + return any(line and line[1] not in self.STATUS_CLEAN_OR_UNMANAGED + for line in self.status_lines()) + + @_cache + def has_staged_changes(self): + return any(line and line[0] not in self.STATUS_CLEAN_OR_UNMANAGED + for line in self.status_lines()) + + def commit_at(self, revision): + with self._run([self.GIT_BIN, 'rev-parse', revision], + self.encoding) as (_, stdout): + return stdout.readline().rstrip('\n') or None + + @_cache + def upstream_commit(self): + return self.commit_at('@{upstream}') + + @_cache + def head_commit(self): + return self.commit_at('HEAD') + + def in_sync_with_upstream(self): + return self.upstream_commit() == self.head_commit() + + @_cache + def last_commit(self): + with self._run([self.GIT_BIN, 'log', '-n1', '--format=format:%H', self.path.name], + self.encoding, **self.IGNORE_ERRORS) as (_, stdout): + return stdout.readline().rstrip('\n') or None + + def operate(self, subcmd, ok_exitcodes=frozenset([0])): + with self._run([self.GIT_BIN, *subcmd], None, ok_exitcodes, stdout=None): + pass + + +def add_parser_flag(argparser, dest, **kwargs): + kwargs.update(dest=dest, default=None) + switch_root = dest.replace('_', '-') + switch = '--' + switch_root + argparser.add_argument(switch, **kwargs, action='store_true') + kwargs['help'] = "Do not do {}".format(switch) + argparser.add_argument('--no-' + switch_root, **kwargs, action='store_false') + +def parse_arguments(arglist): + parser = argparse.ArgumentParser( + epilog="""By default, the program will pull from Git if the output path +is a Git checkout with a tracking branch, and will commit and push if +that checkout is in sync with the tracking branch without any staged changes. +Setting any flag will always override the default behavior. +""", + ) + + parser.add_argument( + '--encoding', '-E', + default=locale.getpreferredencoding(), + help="Encoding to use for all I/O. " + "Default is your locale's encoding.", + ) + parser.add_argument( + '--revision', '-r', + help="Revision string to version the published page. " + "Default determined from the revision of the source file.", + ) + add_parser_flag( + parser, 'pull', + help="Try to pull the remote tracking branch to make the checkout " + "up-to-date before making changes" + ) + add_parser_flag( + parser, 'commit', + help="Commit changes to the travel policy", + ) + parser.add_argument( + '-m', dest='commit_message', + default="Publish {filename} revision {revision}.", + help="Message for any commit", + ) + add_parser_flag( + parser, 'push', + help="Push to the remote tracking branch after committing changes", + ) + parser.add_argument( + 'input_path', type=pathlib.Path, + help="Path to the Conservancy travel policy Markdown source", + ) + parser.add_argument( + 'output_path', type=pathlib.Path, + nargs='?', default=pathlib.Path(__file__).parent, + help="Path to the directory to write output files", + ) + + if not markdown_import_success: + parser.error("""markdown module is not installed. +Try `apt install python3-markdown` or `python3 -m pip install --user Markdown`.""") + + args = parser.parse_args(arglist) + args.git_output = GitPath(args.output_path, args.encoding) + if args.pull or args.commit or args.push: + if not args.git_output.can_run(): + parser.error("Git operation requested but `git` not found in PATH") + elif not args.git_output.is_work_tree(): + parser.error("Git operation requested but {} is not a working path".format( + args.output_path.as_posix())) + if args.revision is None: + try: + args.revision = GitPath(args.input_path, args.encoding, GitPath.CLEAN_ENV).last_commit() + except subprocess.CalledProcessError: + pass + if args.revision is None: + parser.error("no --revision specified and not found from input path") + args.output_link_path = args.git_output.dir_path / 'conservancy-travel-policy.html' + args.output_file_path = args.output_link_path.with_suffix('.{}.html'.format(args.revision)) + return args + +class GitOperation: + def __init__(self, args): + self.args = args + self.git_path = args.git_output + self.exitcode = None + self.on_work_tree = self.git_path.can_run() and self.git_path.is_work_tree() + + def run(self): + arg_state = getattr(self.args, self.NAME) + if arg_state is None: + arg_state = self.should_run() + if not arg_state: + return + try: + self.exitcode = self.run_git() or 0 + except subprocess.CalledProcessError as error: + self.exitcode = error.returncode + + +class GitPull(GitOperation): + NAME = 'pull' + + def should_run(self): + return self.on_work_tree and not self.git_path.has_staged_changes() + + def run_git(self): + self.git_path.operate(['fetch', '--no-tags']) + self.git_path.operate(['merge', '--ff-only']) + + +class GitCommit(GitOperation): + NAME = 'commit' + VERB = 'committed' + + def __init__(self, args): + super().__init__(args) + try: + self._should_run = ((not self.git_path.has_staged_changes()) + and self.git_path.in_sync_with_upstream()) + except subprocess.CalledProcessError: + self._should_run = False + + def should_run(self): + return self.on_work_tree and self._should_run + + def run_git(self): + self.git_path.operate([ + 'add', str(self.args.output_file_path), str(self.args.output_link_path), + ]) + commit_message = self.args.commit_message.format( + filename=self.args.output_link_path.name, + revision=self.args.revision, + ) + self.git_path.operate(['commit', '-m', commit_message]) + + +class GitPush(GitCommit): + NAME = 'push' + VERB = 'pushed' + + def run_git(self): + self.git_path.operate(['push']) + + +def write_output(args): + converter = markdown.Markdown( + extensions=[ + mdx_tables.TableExtension(), + mdx_sane_lists.SaneListExtension(), + mdx_smarty.SmartyExtension(), + mdx_toc.TocExtension(), + ], + output_format='html5', + ) + with args.input_path.open(encoding=args.encoding) as src_file: + body = converter.convert(src_file.read()) + with tempfile.NamedTemporaryFile( + 'w', + encoding=args.encoding, + dir=args.git_output.dir_path.as_posix(), + suffix='.html', + delete=False, + ) as tmp_out: + try: + tmp_out.write(TEMPLATE_HEADER) + tmp_out.write(body) + tmp_out.write(TEMPLATE_FOOTER) + tmp_out.flush() + os.rename(tmp_out.name, str(args.output_file_path)) + except BaseException: + os.unlink(tmp_out.name) + raise + if args.output_link_path.is_symlink(): + args.output_link_path.unlink() + args.output_link_path.symlink_to(args.output_file_path.name) + +def main(arglist=None, stdout=sys.stdout, stderr=sys.stderr): + args = parse_arguments(arglist) + pull = GitPull(args) + pull.run() + if pull.exitcode: + return pull.exitcode + write_output(args) + ops = [GitCommit(args), GitPush(args)] + for op in ops: + op.run() + if op.exitcode != 0: + exitcode = op.exitcode or 0 + break + else: + exitcode = 0 + print(args.input_path.name, "converted,", + ", ".join(op.VERB if op.exitcode == 0 else "not " + op.VERB for op in ops), + file=stdout) + return exitcode + +if __name__ == '__main__': + exit(main()) + diff --git a/www/conservancy/templates/base_projects.html b/www/conservancy/templates/base_projects.html index 9b39e6143b3f664b0000608866ab6308562a1cde..2d2cd7f31e9ff380000e96d40c33cbd7be8c50a1 100644 --- a/www/conservancy/templates/base_projects.html +++ b/www/conservancy/templates/base_projects.html @@ -5,8 +5,9 @@