How Braintree Interviews Exceptional Developers

Or, Why All Interview Practice Advice Should Be Taken with a Grain of Salt

There have been quite a few posts lately offering advice on popular interview practices. All of them offer excellent insight, however what has thus far been left unspoken is the importance of hiring the right people for your company. I'd like to share with you how Braintree looks at the hiring process, and how we leverage a variety of techniques to help us find people that were right for us.

Who are you? Who do you want to be?

Every company has an identity, a set of values that ultimately affect how the company is perceived. Is your company reliable? Innovative? Are you the premium solution in your space, or do you aim to provide the lowest cost for your customers? Are you focused on financial growth or industry admiration? What matters to you? Collectively, your people form the identity of your company, and each new person changes the company's identity, even if only slightly. It's important to be mindful of this change when you're growing your team.

There are two aspects that should be considered when evaluating a candidate in regards to your company. The first is how their values align with your values. Do your developers favor speed of delivery over code quality, or vice versa? What's your approach to testing? How important is communication and collaboration. If the candidate doesn't feel as strongly about the things that your team values, and if you hire them, over time those values will diminish. You have to decide if that's okay.

The second aspect is the team dynamic. How would the new person interact with the team? Will they gel with your developers and lead to a large productivity increase? Or will they add tension and possible conflict, reducing morale and productivity? Maybe you need a bit of fresh conflict, to challenge the existing values of the team.

While it's unlikely that any interview process, no matter how long, will provide the kind of confidence in a candidate that you'd like, there are a variety of ways to assess interviewees towards your goals. It's very important to note that every company is a snowflake, and while there are many qualities that would make a developer a strong fit for almost any company, it's your unique combination of values and work styles that determine what interview practices will be most effective.

Braintree's Identity

As a company, we set our bar very high when it comes to providing service and supporting our customers. When we have a chance to work with them, we want them to come away feeling like it was the best customer service experience ever. Sometimes this interaction is via phone or email, and other times it's through our documentation, gateway, and API. We have a low tolerance for poor experience, and we want every new person we hire to help maintain the high expectations we've led our customers to expect from us. So to that end, we seek people who are never satisfied that "good enough" is good enough, and who will work diligently to continually improve our offering. All of our other values are important because they support this one.

Communication is naturally very important to us, even for our developers. We make it easy for our customers to get help directly from our devs, and we expect every one of them to be able to handle support tickets efficiently, courteously, and with enthusiasm. Effective internal communication is more frequently needed, talking to other developers as well as non-development team members.

Collaboration is crucial to the success of larger efforts. The more effectively people are able to work together, the more effectively they're able to get things done. We value collaboration very highly, and continually work towards removing barriers. Nearly the entire company works from a large, open room. We hate cubicles and we share desks. We love to pair program. We have folks who work remotely; sometimes that's the only option and we make the best of it, but we know we're much more effective face to face.

We also believe that morale is extremely important. Happy developers write great code. It's important that our people have what they need to be successful, and that includes peers that they can count on to get things done with the same level of quality they'd expect from themselves. Our devs want to work with other exceptional devs. That's not surprising of course, but the impact of having even one or two developers unable to carry their fair share of the load can cause a remarkable impact to morale.

Of course talent is very important and can't be overlooked. At Braintree, we believe building great teams is more important than individual rock stars. I guess you could say we're essentially trying to build a rock orchestra here. While we do have very high expectations for developers from a talent stand point, talent alone will never exempt a candidate from the other values.

Our Interview Process

Pretty much all interview processes involve some form of initial screening, and we're no different. Once an applicant contacts us, we schedule a phone call to share a bit more about ourselves, ask a few additional questions, and generally gauge if the candidate is a culture fit. Would they be happy here? Are they looking for the type of work that we'd ask them to do? Do their values align with ours?

Code Review

Developers that pass the initial screen are provided a coding problem. Sometimes candidates self-select out of the process at this point, which we feel generally works in our favor. Developers who believe in the values we express in our job postings won't allow a few hours of development time to be a barrier to an opportunity to work here. Our coding problem is straightforward and we accept solutions in any language. While we always look at code candidates have published on sites like Github, solutions to our own coding problem help us to compare candidates who are solving the same problem.

Even though the problem is simple, every solution is slightly different, and gives us a fair bit of early insight into their approach to software development. Do they write tests? Test first? How did they tackle the problem? With patterns? Strong OO? Something original and clever? Did they leverage the features of the language to produce something elegant? Does their code communicate its intent to the reader? While the evaluation can be somewhat subjective, it remains a valuable tool in learning more about the candidate, and helping us decide if it's worth pursuing them. Ultimately we want to know if they value quality the way we do, are excellent at communicating via code, and have the kind of talent we like.

Phone Interview

Following the code review, we interview candidates over the phone. We try to keep the call to between a half hour to an hour, and cover as wide of a variety of topics as we can. To help keep us moving through the call, and to provide some consistency, we work through a list of questions we've prepared in advanced and reuse for each candidate. The questions are technical and we try to keep the conversation informal. This is our first real opportunity to evaluate the candidate's communication style. We probe much beyond what a resume can say and try to determine what the person's technical strengths and weaknesses are. If we feel they communicate effectively and have the skill levels that we're looking for, we invite them to a full-day, on-site interview.

Technical Interview

The first stage of our on site interview is basically an extension of the phone interview. While the phone interview focuses on a breadth of subjects, the discussion during the technical interview looks for more depth. We'll ask questions based on responses from the phone interview. We probe a candidates strengths and weaknesses further. Communication is always a factor, but we're primarily assessing technical competency. The candidates we like tend to have some decent depth in a few areas, and have strong opinions and ideas about software practices and technologies.

Lunch

Applicants join us for lunch when they interview, and it's a great opportunity for casual discussion. We love to hear what sort of side projects they have, what they do for fun. We ask a few friendly technical questions too. We're still evaluating communication, but more how they gel with the team. We want to know how we'll enjoy having them working with us everyday. It's important that new hires fit with us culturally, as it affects morale.

Logic

After lunch, we like to spend some time working on logic problems. The logic problems are non-trivial, but there are no tricks. They're chosen because solutions can be arrived at through reasoning and deduction. While they're artificial in nature, they remain a good representation of the complex problems that we're challenged with in daily work. Real businesses have complex problems that need to be solved, and we have to be good at tackling them collaboratively.

While we want to know that the candidate understands the solution when we get there, finding the solution is not nearly important as how they attack the problem itself. In fact, every candidate always gets to the solution. We coach them along when they seem to get stuck, half-collaboratively solving the problem with them. We ask them to talk through the problem solving and offer them a whiteboard. We're assessing ability for abstract reasoning along with how they communicate and collaborate.

Pairing

The final portion of the on site interview is a pairing session, where we work with them to enhance the code they submitted at the beginning of the process. This is our best opportunity to evaluate their collaboration style, and get a sense for how they'd be to work with on a day to day basis. We try to keep it as informal as possible. We've had plenty of opportunity prior to the pairing to hear them talk about software development, and this gives us a chance to see them in action.

And...

As you can see, we've constructed our interview process around our values. Our process has evolved over time, and continues to be refined and adjusted when things don't seem to be working as well as we'd like. Our approach to designing our interview process has worked very well for us. Your results may vary.

If you're interested in experiencing our interview process first-hand, we're always looking for exceptional developers of all experience levels to join our team. Check us out at joinbraintree.com. We'd love to hear from you!

Fundamental Online Security

Working at a payment processing company, I've come to appreciate the importance of online security much more than before. While many aspects of security are somewhat niche, I often think about how a few of the practices I now take for granted are actually fundamental enough that absolutely everyone should be using them. I'm talking about the kind of practices that we should be ashamed if we don't help ensure that our family and friends understand and are applying rigorously. When I boil it down to that, there's really just one thing that I hope absolutely everyone will do: use a unique password for your email.

Secure your email! (do it now; we'll wait)

Often folks who are less internet savvy overlook the importance of protecting their email account, and they tend to reuse the same passwords for every account they create on the internet. It's much more obvious that we should use a unique and complex passwords for logging into our bank accounts, for example, and we tend to think it's not as crucial that we do the same for our email. What we overlook is that anyone with access to our email can often trivially gain access to most of our other accounts by simply requesting a password reset.

If you use your email password for any other account, you're almost certainly putting too much trust in someone else's security. While well-known sites are likely to have better security in place, it's not always the case. Millions of accounts were compromised only a few months ago via Sony's PlayStation Network, and the email addresses and associated passwords have already gotten into way too many hands. It's safest to trust no one.

It's worth noting that some sites, particularly banks, have extra steps in place to make it somewhat more challenging for a stranger with access to your email account to reset your password. However, there are far more sites we've given personal information, including saved payment information. Imagine someone with access to your email account gaining access to an online merchant via password reset and using your saved payment information to make purchases.

So long as web sites use the ability to access to your email account as proof of your identity, protecting access to your email account is fundamentally important. Please tell your friends.

Anything else?

Beyond using a unique and complex password for your email account, further security suggestions require a bit more effort. While I think they're all good advice, it's really going to be up to you how much you're going to be willing to do to improve your personal online security. There are two things I recommend.

Switch to Gmail. I hate this one on principal, as I think people should be able to use whatever email service they like best. However, Gmail offers a couple of really nice security features.

In particular:

• You can set up two-factor authentication, such that you have to enter a six digit code from an authenticator app on your phone. Meaning, in general, no one can access your email without both your password and your phone/authenticator.
• Gmail also allows you to set up application specific passwords, so for example, if you use a special IM client, you can provide it a one time use password, with the ability to revoke access from that application at any time without having to change your primary email password.
• As a bonus feature, Gmail allows you to add custom strings to your email address that can help track and filter messages based on where they came from. For example, when providing my email address to a new website, I can use my_username+customstring@gmail.com. I usually make the custom string the domain name of the website so I can easily tell who I gave the address to. Messages sent to this address will ignore the + and everything after up to the @. If a website shares my email address with spammers, I will easily know who leaked my address, and Gmail makes it easy to then filter any incoming mail with my custom address straight to my spam folder.

Second, use a password generator and create unique passwords for every account. This sounds like much more of a pain than it actually is. There are applications that provide support for this, some with browser plugins, and quite frankly, they're life changing. I recommend both 1Password and Keepassx. Basically whenever you're creating a new account, you allow 1Password/Keepassx to generate the password and save your account information in its internal database. You can comfortably generate very complex passwords and not have to worry about memorizing them.

Customer Service Pachinko

or The Importance of Humane Software Development

They look like ants from up here

A few years ago I had a customer service experience that really changed the way I think about the work I do.

I'd purchased a Dance Dance Revolution dance pad off Amazon for Hammer Night. The item page showed the item as a brand new copy of the game/dance pad bundle, and was priced appropriately for a first-party sealed item. What I received was a used, third-party dance pad, with a used copy of the game stuffed inside. I'm sure when I clicked to purchase the item, Amazon made it clear that I was buying from a third party, though I'm equally sure I thought nothing of it at the time.

I contacted the merchant and politely informed them of the mistake. It quickly became clear that the merchant was actually an individual operating out of their apartment. The first response I received apologized for the mistake and suggested that two purchases had been mixed up and were sent to the wrong people. Hmm. Not quite sure I believe that, but doesn't really matter. The seller says that once I mail the item back, he'll process a refund through Amazon. Cool. I reply asking about the shipping costs. It was only about $10 each way, but obviously I shouldn't be responsible for any of that since the "mistake" was entirely on the seller's end. The seller is adamant that he shouldn't be responsible for the shipping costs. We're at an impasse.

I start investigating resolution through Amazon. They don't seem to have any policies in place for dealing with merchants sending the wrong item. In fact, most of the policies seem to be in place to protect merchants and not consumers. The processes they do have in place require a full month to have passed before you can even begin, I assume to put as much of the resolution work on the seller and buyer as possible. I continue to exchange emails and make it clear that I intend to submit the purchase to Amazon for resolution, getting nowhere.

At 30 days, I file the purchase for resolution with Amazon. They advise me, via what appeared to be an email form letter, that I should ship the item back to the seller and they would process the refund. I ship the item back as requested, and reply to the email asking about shipping costs. They tell me it's my burden since I decided I didn't want the item. I reiterate that I was shipped the wrong item. I didn't simply not want the item I'd purchased. Again I receive what seems like a form letter, reiterating their policy.

I start to get angry. What's becoming increasingly clear to me is that Amazon's policies actually make it really easy for scammers. They can sell used crap as new, and the burden of shipping costs may often be enough cause the buyers to give up and not pursue getting their proper reimbursement. I have a low tolerance for this sort of thing, and I'm convinced that Amazon cares too much about its reputation to actually let this sort of thing happen. My next email to Amazon lays out my perspective on this, acknowledges that the people I've been in contact with so far are clearly not empowered to properly resolve the matter, and I request that the issue be forwarded up the chain. Again, a seemingly canned response.

By now I'm livid. I'm ready to punch the seller for running a scam, and I'm thoroughly pissed at Amazon for protecting this guy. I google for Amazon exec emails, and I copy one on my next email. This was Friday evening. Saturday morning I get a call from a VP at Amazon. She asks for the details, assures me that Amazon does care about their customers and wants to do the right thing. She asks for the total of the shipping costs, and a credit is issued to my card to cover the shipping. We had a brief conversation where I explained my frustrations with the way the process worked, and that I was much more frustrated that I was scammed, that the policies protected the scammer, and that the only people I could reach through published support channels were unequipped to carry out what I was certain was the philosophy of Amazon's leadership. In the end I felt that she was pleased that she was able to resolve my situation, but ultimately I think the overall matter was too overwhelming for even someone in her position to affect the appropriate change.

As software developers, the products we implement are often, in fact almost always, intended to replace people and allow companies to scale way beyond what they could with purely human labor. That is, what used to take ten, a hundred, or more people, now takes far fewer, and rarely no people, to accomplish the same work, since computers are able to do much of the monotonous and repetitive work. We are powerful and valuable in this way, as we're able to dramatically increase the value of non-software development staff by increasing what they can accomplish in the same amount of time. But at the same time, we've shifted from a world where phones could only be answered by real people, to one where entire call centers are replaced by Voice Response Systems and customers only interact with computers. Our software allows companies to grow so large that the folks on the ground are so far separated from the folks at the top that the vision can no longer be carried out.

Technology generally makes our lives better, but we have to do a really good job designing our systems to be able to provide nearly the same quality of service to humans as other humans can. We're often focused on speed, scalability, quality, and the abstract elegance of the code itself. For many of us, the user experience is also key, and we invest time in making sure the user experience is easy. All of those are really important. What's also important, and often overlooked, is how we handle the atypical problems that arise. Real humans are generally quite good at assessing situations and adjusting to resolve them. We often do a poor job around the boundary between software and human support, and the broader user experience.

A girl and her bank

A friend of mine had an unpleasant experience with her bank. She had gone to a restaurant and begun to pay for her meal with her debit card. When the slip came to add the tip and signature, she changed her mind and decided to pay with cash. The authorization had already been applied to her account, and as you would expect, the restaurant didn't reverse the authorization. In fact, I'd guess they don't even know what a reversal is, much less that they should perform one in unusual situations like this.

Later that week, she made another purchase. Combined with the authorization still pending on her card, her account was now below the minimum balance for her account. The bank issued a charge for going below the minimum balance, which they deducted from her account. That charge took her balance below zero and they charged her an overdraft fee. Ultimately, none of this should have happened, and things snowballed out of control.

She went to her local bank branch and spoke with the manager. It was his strong opinion that she shouldn't have been charged an overdraft fee due to a charge from the bank. He was also generally of the opinion that she shouldn't have had the minimum balance charge due to the slowly expiring authorization. Unfortunately, all of the charges and fees are managed by the bank's central system, and even as a bank manager, there was nothing he could do about the fees. However, he did have the ability to convert the account into one with no minimum balance. This would at least prevent the same scenario from happening again, but it's disappointing to know that software was tying the hands of people who were knowledgable and humane.

I'm sure that if the bank manager knew who to talk to upwards in the company, someone would be empowered to resolve the problem properly, perhaps by requesting an adjustment to the software that determines when fees should be issued. In my opinion, an adjustment that empowers the bank manager to fix the problem is ultimately a better solution. Why do we now trust computers more than our people? Almost certainly the bank manager knows more about bank operations and customer service than anyone in the bank's IT department. Why doesn't our software better empower our people, rather than try to replace them?

CableCard pachinko

I hate automated voice systems. I dread calling support lines, knowing that I have a relatively simple question that a knowledgeable human could answer trivially, and having to navigate menus with my fingers crossed that I'll eventually find the right person. I feel like I'm a ball in a game of pachinko, with equal chance that each pin I bounce against, each menu selection I make, has equal chance to send me farther or closer to where I want to end up. It's easy to feel hopeless and become demoralized. If I'm a business owner, I never want my customers to feel this way, and I'd trade a great deal of efficiency for customer satisfaction.

Sometimes even being able to get to a human isn't enough. My TiVo recently stopped booting, and decided that replacing it would be less expensive than a repair. When I moved the cable card from the old device to the new, I discovered that the card would need to be reactivated. All of my channels were coming through clear, including HD, except the premium channels, which displayed a screen indicating I'd need to contact the cable company.

My cable provider is Comcast, renowned for terrible customer service. I rarely have to contact them, and really enjoy the high speed internet they offer, but when I have spoken with them, it's hit or miss. I've had to wait more than an hour on hold to speak to a person. On one occasion when my cable was not working, I called and the person I spoke with was unable to locate my account in their system, and suggested that I may be with a different cable provider. Based on my past phone support service, I opted for the online chat, a decision I would soon regret.

I imagine that online chat support is heavily concurrent, that each support agent is handling several conversations at one time. As long as each individual gets prompt enough responses, in theory the service could be better. This doesn't seem to be reality though. I'm already an outlier by using a cable card, and I accepted that trade off when I chose to use a TiVo instead of a standard issue cable box. But I do expect that the cable company can support the card they provided me. The individual I'm chatting with via their online support doesn't seem to have had sufficient training on cable card support. To make matters worse, I think he knows it, and is unwilling to admit it or ask for help. Our conversation lasted, not exaggerating, three hours. There seemed to be a significant amount of multitasking, where I wouldn't hear from my agent for up to fifteen minutes. I was quite clear about the events leading up to the problem, and even suggested an activation signal may need to be sent (as I'd observed when the card was initially installed). I was met with questions about whether the cable was securely attached to the sockets on both ends. I was asked to reboot my device (and I complied). About two hours in, the agent suggested that an on site service call would be the only option. And from that point, with all of the multitasking, it took another hour to schedule the service appointment. I've never been so close to leaving Comcast.

A few days later, the repair person arrives and seems equally unprepared to resolve the problem. I explain again precisely what precipitated the issue, and eventually provided the same suggested solution. It was like I was speaking Greek. The repair agent is on the phone with their support staff, trying to get advice on how to resolve the problem, checking signal strength, and basically everything unrelated to the symptoms I've described. The people they're speaking with on the phone don't seem to be helpful either. In an incredible stroke of luck, the repairman lost cell signal and was disconnected. He called back in and got a new person, who indicated that he should have called a different number. The repairman asked if she could transfer him, and she was about to, but decided that she might be able to help. He provides what information he can, though not everything I'd told him, and she looks at something on her side and says, I kid you not, "Did the customer recently change TiVos?" A ray of light. Finally someone who knows what they're doing. We're on speaker phone and I respond with a resounding yes. She sends the signal and not one minute later, everything is fixed.

I can't imagine the tangled skein that is the network of support staff for a company like Comcast, but it's surely inefficient. Like any company I've worked with, there are always at least a few people who know what they're doing. I can't even count the number of times when I've wished I could just get an engineer on the phone, someone who actually understands how something works because they built it. There must be better ways of empowering support staff to find solutions to problems. I'm sure when my online support agent was stumped, he went to a knowledge-base, as he was trained, and couldn't find what he needed. But somewhere there was another support person who knew exactly how to solve the problem, and he wasn't empowered to find that person, to find the solution and make the customer happy. In the same way, my online support person wasn't able to get an on site tech who knew how to solve the problem assigned to my appointment. And the on site tech was just plain lucky to find someone who could solve the problem.

We have the technology

As software developers, I believe we can help solve these problems. In fact, I think it's our responsibility, that we should take more accountability for developing these systems. And I don't just mean software. The nature of our discipline leads us to think more about how systems work, not just software systems, but real-world systems too. We need to do more to cross the boundary between the software we write and the people who use it. We need to be prepared to say we need more people and less software sometimes. We need to constantly remind ourselves of the hell that our software can create.

Software is an incredibly powerful tool, properly applied. The problems that come with software aren't always obvious at first. I'm sure Amazon was far more concerned with scaling their servers than they were their customer service capability. It's very hard to provide great service to that last 1% of their customers, the ones who have unusual issues. But sometimes even one customer not getting proper support is too many. Their rapid growth left gaps in their support process that need addressing. It needs to be easier for their customer support staff to escalate issues, and be encouraged to go above and beyond to resolve problems.

My friend's bank obviously cares a great deal about uniformly managing customer accounts, and letting bank managers focus on the less monotonous aspects of banking. In the process of building their systems, they transferred power from their legion of bank staff to their software developers. But my friend isn't a record in a database table. Her life isn't a series of log entries. She's a person and deserves respect. By abstracting the banking process, she was abstracted in the eyes of people like us. Her money may seem very little, and of little consequence to such a large bank, but what she has is very important to her. She deserves the benefit of the doubt, and needs bank managers who are empowered to do what's right.

As the complexity and ubiquitousness of technology increases, companies like Comcast will struggle to find enough support staff who can absorb the knowledge required to help their massive number of customers. They must continually improve the systems they employ to share knowledge internally and externally. They'll have to find better ways to enable their people to work as one team to support their users. And there's almost no aspect of their support that can't be improved by thoughtful software developers.

Shameless...

I love working for Braintree. One of the things that makes me so proud to work here is our relentless devotion to great customer service. When customers call us, one of our exceptional support staff answers the phone. There's no voice response system. If you're a developer who needs help integrating, you get help from one of our engineers. The people who help you are the people who designed and built our systems. And we're always looking for ways to support our customers better.