About Upstart

Upstart is the leading AI lending marketplace. We connect millions of consumers to almost 100 banks and credit unions who leverage Upstart’s artificial intelligence models and cloud applications to deliver superior credit products. Our platform includes personal loans, automotive retail and refinance loans, small dollar “relief” loans, and home loans.

Upstart is a digital-first company with a distributed team of over 1,800 employees across the US.

Tech Stack

DevOps: Github, Git, Datadog, Terraform, Kubernetes, Bugsnag
Business Tools: Google Suite, Slack, Jira, Confluence, Salesforce

‍

Why Incident Management?

Before Rootly, we had some very light automation set up through a tool called Demisto. It’s actually intended as a security incident platform, and it does a great job of that. But we found that when it came to automating processes for production incidents, it was just way too complex. We had set it up to do a few tasks related to starting an incident—basically just creating a Slack channel and starting a Zoom room. Those automations were really difficult to create and test with the setup we had, so we didn’t really go beyond that even though there was a need for more automation in our process so important actions weren’t being forgotten. That’s when we started looking for a tool that could do what we needed in a lower effort way.

Why Rootly?

We weren’t looking to massively disrupt the processes we had in place already. We mainly wanted a tool to help us streamline processes and automate manual tasks where possible so we could respond more efficiently and consistently. We had some minimum requirements, like the ability to fully manage an incident within Slack, and to integrate with Jira. Most tools we looked at met those minimum requirements. But when it came to implementation, a lot of the tools were very opinionated about how incident response should be done. It was kind of a one-size-fits-all approach that didn’t feel right for us. Rootly was the only tool that offered the flexibility we were looking for in order to gradually implement automation around our existing processes that were tailor made with our business in mind. All the integrations made it easy to start using Rootly alongside the tools we already have, instead of needing to do a big overhaul and replace tooling that was working for us.

Beyond the flexibility and integrations that Rootly offered out of the box, they’re also really receptive to our feedback and needs around any features we’ve wanted that didn’t exist. When we were shopping around, we asked a few companies how hard it would be to get a new feature and if we needed it, and they were really hesitant to discuss that. With Rootly, they were completely open to working with us on the solutions we needed. There have been days where we reach out to their team and say “Hey, we have a request, this is what we want.”, and Rootly has come back within an hour and let us know it's done. It's in production already. I really can't think of a feature that we've requested and hasn't been done in one way or another.

Using Rootly has definitely made being on call at Upstart a better experience. Our response and coordination is faster. We don’t miss action items anymore. The ability to do so much of our incident management exclusively within Slack means there’s less pressure to be glued to your laptop while you’re on call. Our SREs rotate through week-long on-call shifts, so we want them to be able to live their lives even when they’re on call. Knowing that you can effectively manage an incident from Slack, even from your phone, allows for a lot more flexibility while you’re on call.

Overall, the reliability of Rootly is great. The people are great. The feedback is great. They definitely take customer feedback seriously. And there's been a lot of changes since we started using Rootly, where they've driven Rootly to create a better user experience and just a better product in general. It was great to start with, and it's just even better now. It gets better every day.