I have a few payment processing related projects on the go but having just gotten the first one to alpha stage I thought it would be a good time to share some thoughts (& try to get them straight in my mind). The project I am referring to is developing a Cybersource Secure Acceptance POST payment processor. For this payment processor I had a big challenge and also a big opportunity.

The challenge

The Cybersource Secure Acceptance POST gateway is one of an increasingly common breed known as transactional redirect or sometimes direct post method. The idea is that you gather most data on your site and then present the user with a credit card form on your site which POSTs to the Cybersource site. This way the credit card form is presented on your site and the customer never knowingly (depending on internet speed) leaves your site but none of the credit card information ever hits your server in any way. This gives you PCI compliance without the ugly redirect (& risk of the offsite form being fugly). The only problem is that CiviCRM isn't built for it.

The opportunity

In the not so distant past I discovered an exciting library of payment processor gateways https://github.com/omnipay/omnipay - omnipay written by

Adrian Macneil

. Omnipay normalises your interactions with payment processors so that you need less code to integrate them and you can leverage contributions from an open source community that extends far beyond CiviCRM. In this case there was not an existing Cybersource gateway but someone called Rafael Diaz-Tushman had a shell of a repo which I submitted code to and now he is actively working on too. So, already we can see this has enable collaboration outside our community.

What works?

The 2 processors I tested are Paypal and Cybersource - although I tried a few others like BitPay, Authorize.net and Payment Express. Bitpay worked locally but not on a second site and I didn't figure out the problem with that or the others. Generally I didn't have the credentials / time to explore non-relevant processors too much. However, Paypal & Cybersource both have very different flows and use the same code. If you check the processors list there are about 35 processors there. In some cases they have endpoints for functionality like recurring and refunds that we haven't fully developed in CiviCRM yet.

https://github.com/eileenmcnaughton/nz.co.fuzion.omnipaymultiprocessor

Learnings

The first thing I discovered working with Omnipay is that the maintainer has decided that thinking about processors in terms of off-site processors and on-site processors is not useful.

• Off-site gateways such as PayPal Express, where the customer is redirected to a third party site to enter payment details
• On-site (merchant-hosted) gateways such as PayPal Pro, where the customer enters their credit card details on your site

This is quite a different approach to that adopted by CiviCRM which has very different flows for on-site and off-site processors. However, I quickly came to the conclusion that the different flows are not actually that helpful and only capture a small subset of payment processing possibilities. In fact the doTransferCheckout in my implementation of the OmniPay library looks like this:

functiondoTransferCheckout(&$params,$component='contribute'){  $this->doDirectPayment($params,$component);  thrownewCRM_Core_Exception('Payment redirect failed');

}

The 2 key differences in how Core processes Onsite and Offsite payments are

1) Form Building differences

2) Pre-handover processing differences

Form Building differences

CiviCRM says 'if you are an onsite processor you need to have address fields + credit or debit card fields and I will create a billing address for you & for your contribution otherwise you don't need these fields'. As it turns out the form fields requirements of payment processors vary quite a lot and in the case of Cybersource - which in the end I had to get CiviCRM to 'think of' as an offsite processor you still need specific address fields (which are then signed). Conversely we get fairly frequent requests from people wanting us to remove billing fields as their processor doesn't require them and they prefer less fields. Basically core needs to 'ask' the processor which fields to display rather than decide based on the processor's 'type' combined with it's payment type. I achieved some of this within the extension (in particular replacing BillingBlock.tpl with a block that has no hard-coded fields in it & which displays fields based on variables assigned to the template rather than template logic. However, I feel like I'm only starting to think through the metadata involved in 'instructing the form layer' and this is something I'm going to write more about - probably in a second blog.

Pre hand-over processing differences

If you love a contribution set it free - if it comes back, change it to completed. If it doesn't - hey who knows maybe it's gone off-site & there will be an IPN later.

The above is my revised version of how CiviCRM should interact with the payment processor. In other (rather less opaque) words CiviCRM should build the pending contribution & related assets prior to handing over to the payment processor regardless of what 'type' it is and then update to completed based on 'what happens next'. How does this compare to current action? Well - at the moment CiviCRM creates a pending contribution if the processor is 'of type notifiy' but if it is of type 'form' it doesn't create one unless it is confirmed by the doDirectPayment function. This has a couple of effects. Firstly it makes the code more complex because the form layer is iffing & thenning all over the place. Secondly it makes it hard to create payment processors that don't fit standard flows.

What are the downsides of this flow change? Well, firstly this change shouldn't hurt existing processors (which is our main block to making some other changes I'm mulling) but there are 2 potential gotcha areas

1) Probably this would leave more failed transactions in the database and arguably this is a bad thing. At the moment failed credit card transactions are either not stored or rolled back out of the database - I've only been on the 'why do we lose our audit trail' side of this but there are others who prefer it this way. My suggestion would be to have a cron that cleans up failed transactions rather than not record them or introduce another setting as to whether they are desirable

2) Double transactions. Some processors create the option of a separate transaction for memberships vs contributions. I can't see this being able to be decoupled from payment processor type 'category' in the short term. In the longer term I think we could think about metadata & capabilities. I'm not going to go further into metadata & capabilities this blog but I do want to publish a blog on it

What does Omnipay Do

Basically it normalises your data & the interactions with the gateways. It doesn't matter what your gateway calls the First Name field - you set the same fieldname everytime and Omnipay takes care of that for you. So, here is my doDirectPayment function - that handles onsite processor Paypal, offsite processor Bitpay & transactional Redirect Processor Cybersource.

function doDirectPayment(&$params, $component = 'contribute') {$this->_component = strtolower($component);$this->gateway = Omnipay::create(str_replace('omnipay_', '', $this->_paymentProcessor['payment_processor_type']));$this->setProcessorFields();$this->setTransactionID(CRM_Utils_Array::value('contributionID', $params));$this->storeReturnUrls($params['qfKey']);$this->saveBillingAddressIfRequired($params);try {$response = $this->gateway->purchase($this->getCreditCardOptions($params, $component))->send();if ($response->isSuccessful()) {// mark order as complete$params['trxn_id'] = $response->getTransactionReference();//gross_amount ? fee_amount?return $params;    }elseif ($response->isRedirect()) {if ($response->isTransparentRedirect() || !empty($this->gateway->transparentRedirect)) {        CRM_Utils_System::redirect(CRM_Utils_System::url('civicrm/payment/details', $response->getRedirectData() + array('payment_processor_id'=> $this->_paymentProcessor['id'],'post_submit_url'=> $response->getRedirectURL(),          )));      }$response->redirect();    }else {//@todo - is $response->getCode supported by some / many processors?return $this->handleError('alert','failed processor transaction '. $this->_paymentProcessor['payment_processor_type'], (array) $response, 9001, $response->getMessage());    }  } catch (\Exception $e) {// internal error, log exception and display a generic message to the customer    //@todo - looks like invalid credit card numbers are winding up here too - we could handle separately by capturing that exception type - what is good fraud practice?return $this->handleError('error', 'unknown processor error '. $this->_paymentProcessor['payment_processor_type'], array($e->getCode() => $e->getMessage()), $e->getCode(), 'Sorry, there was an error processing your payment. Please try again later.');  }}

Omnipay features

Omnipay gateways support different methods and it is possible to query the gateways to find out what methods they support. The most relevant methods are Purchase, Create Token, Refund and Complete Purchase. Purchase is obvious. Complete Purchase actually means 'handle IPN'. Once again I was able to use a very generic function - although there is some argy bargy with the $_REQUEST which was because it's possible to run this from the api. (I should check if that's required)

public function processPaymentNotification($params) {$this->gateway = Omnipay::create(str_replace('omnipay_', '', $this->_paymentProcessor['name']));$this->setProcessorFields();$originalRequest = $_REQUEST;$_REQUEST = $params;$response = $this->gateway->completePurchase($params)->send();if ($response->getTransactionReference()) {$this->setTransactionID($response->getTransactionReference());  }if ($response->isSuccessful()) {try {      civicrm_api3('contribution', 'completetransaction', array('id'=> $this->transaction_id));    }catch (CiviCRM_API3_Exception $e) {if (!stristr($e->getMessage(), 'Contribution already completed')) {$this->handleError('error', $this->transaction_id  . $e->getMessage(), 'ipn_completion', 9000, 'An error may have occurred. Please check your receipt is correct');      }    }  }elseif ($this->transaction_id) {    civicrm_api3('contribution', 'create', array('id'=> $this->transaction_id, 'contribution_status_id'=> 'Failed'));  }$_REQUEST = $originalRequest;  CRM_Utils_System::redirect($this->getStoredUrl('success'));}

Refund and Create Token

These are the 2 areas I think offer most potential for development focus at the moment. At the moment we can't call an api to refund a payment & have that passed to the payment processor. With Omnipay the most difficult part of creating that api and having it work on multiple processors will be agreeing what to call it (anyone who has been party to API discussions will know that discussion is a forte).

Create token is the only option Omnipay offers for recurring payments. From the author

My feeling is that we are seeing most growth in the area of tokens at the moment and a great focus would be to build an Omnipay cron for processing recurring contributions (based on tokens stored during checkout)

Omnipay forever?

The Omnipay library is well-written and well tested. It uses modern coding standards and continuous integration.  It's being built into Drupal 8 and there is Wordpress integration along with a fleet of others such as Silverstripe.The only other library I have found which seems compelling is Payum. Payum has a few more gateways, and some more features around logging and they have integrated Omnipay into their own code. It also looks well written. I didn't perceive a big feature difference between the two - not features that seem needed. So, to my mind it comes down largely to maintenance. Omnipay has greater penetration but seems to have a single maintainer with relatively simple needs "My main use case is a shopping cart, so most of the existing gateways have been built at a lowest common denominator level of interoperability" (https://groups.google.com/forum/#!topic/omnipay/lcgkiKg7rt4) whereas Payum has less penetration but they seem to be building a business around developing Payum (it IS open source) so probably have more resources committed to it. So, the risk is that the Omnipay maintainer could get overwhelmed by what seems to be a pretty steady trickle of support requests. 

I'm trying to be even handed here but I've now spent a lot of time getting used to Omnipay and I like it and feel like it's a case of 'convince me why not Omnipay'.

The areas that I still need to work through is the aforementioned metadata because we are dealing with a situation where we are integrating multiple gateways and need to know things like 'which fields to I show for this processor', 'will this processor support 2 payments in one transaction'? and in the case of transparent redirect fields it needs to know things like whether to show a single field for expiry date or 2 fields. Some metadata options are features of the processor and others are features of your account (currency for example could be either). Another variant is Stripe which requires a javascript inclusion (and probably specific classes or similar).

Some other libraries that I didn't think were as promising

How do I add a processor (that has a Omnipay gateway) to Omnipay Multiprocessor extension

Basically add your processor to the .mgd file & see if it works ....

Note that I am using payment_type = 3 to denote that billing address fields should be displayed (but not credit card fields). The url fields are not used and the label fields need to reflect the gateway implementation - ie. the password label is 'Access Key' - which will translate to $gateway->setAccessKey('blah'); The name field is also important as it is omnipay_GatewayName

array('name'=> 'OmniPay - Cybersource','entity'=> 'payment_processor_type','params'=>array('version'=> 3,'title'=> 'OmniPay - Cybersource','name'=> 'omnipay_Cybersource','description'=> 'Omnipay Cybersource Payment Processor','user_name_label'=> 'Profile ID','password_label'=> 'Access Key','signature_label'=> 'Secret Key','class_name'=> 'Payment_OmnipayMultiProcessor','url_site_default'=> 'https://testsecureacceptance.cybersource.com/silent/pay','url_api_default'=> 'https://testsecureacceptance.cybersource.com/silent/pay','billing_mode'=> 4,'payment_type'=> 3,    ),),