Improving Business KPIs with SFTP Scalability and Automation

SFTP, or Secure File Transfer Protocol, is one of the most common ways organizations send and receive data in bulk; but SFTP doesn’t always scale easily. 

Both building new SFTP integrations and preparing the data once it’s received are time- and resource-intensive tasks. For companies that regularly receive data from customers, like 401(k) recordkeepers and benefits administrators, this translates to complex, lengthy customer onboarding and the need to constantly grow internal teams to keep up with manual processes.

To grow effectively, these companies need a faster, more scalable way to onboard new clients and more internal automation to keep the team lean and efficient.

What’s at stake when SFTP integrations don’t scale

Traditional SFTP integrations aren’t scalable because they’re difficult to implement and come with a host of manual processes. Relying on this method can limit your company’s ability to grow and hinder your profitability. 

• Limitations on business growth — Growing means bringing on new customers; but when the onboarding process takes months, you can quickly develop a backlog. Difficult onboarding processes take significant resources from your team, make for a poor first impression with the customer, and elongate your time to revenue. 
• Additional strain on internal teams — SFTP connections require significant manual work even after implementation as your Operations team works to normalize and process incoming data. As your customer base grows, so too will this workload, often forcing companies to increase headcount just to keep up.
• Harder to attract new customers — Today’s employers expect their applications to integrate seamlessly with one another. Many companies have invested heavily in API integrations as a competitive differentiator. A weeks- or months-long onboarding process can be a significant turnoff for modern buyers when other companies provide a simpler alternative. 

In short, without a scalable SFTP strategy, you risk falling behind in growth, competition, and customer satisfaction.

Challenges of scaling SFTP integrations

SFTP has been the industry standard for accessing payroll data for years, but it’s difficult to scale because each connection is unique to one client—the work is not amortized over time. And because the format of the data you receive varies from payroll system to payroll system, it’s difficult to build automation on top of the files you receive. Instead, someone has to manually normalize and process the data. For many companies, that means hiring to keep up with the work, which also doesn’t scale.

The most common challenges companies face when trying to scale SFTP integrations include:

• Lengthy onboarding processes — Onboarding a new customer with an SFTP connection is a time- and resource-intensive process that can take up to 18 weeks of back-and-forth between you, the customer, and their provider. This creates a frustrating experience for the customer, takes significant effort from your internal team, and delays time to revenue. ‍
• No data standardization — The data you receive from customers via SFTP comes in a multitude of formats. Each payroll and HRIS provider has a different set of data points and field names, which means it’s up to you to normalize that data into a usable format—a time-consuming process that is largely unavoidable. ‍
• Manual processes and a lack of automation — When you receive data via flat files over an SFTP server, manual operations are always involved—whether it’s your company’s standard practice or a backup to automated workflows. While plenty of recurring tasks could be automated with technology, building automation on top of SFTP connections is risky: non-matching data formats and changes to the data source’s underlying system, which are out of your control, can break any automated workflows you’ve built, leaving your team scrambling to identify and correct the error. Such a reliance on manual work can also make it difficult to maintain SLAs, especially in heavily regulated industries. ‍
• Growing strain on internal teams — Because of the degree of manual processing involved, companies that receive data via standard SFTP connections often have to hire new employees as they bring on new clients, just to keep up with the day-to-day workload. This isn’t a scalable solution, especially when you consider the cost of hiring and training new employees. 

Unified APIs: The secret to scalable SFTP

SFTP is still a secure, reliable way of delivering data in bulk. The challenges lie in creating the initial connection and processing the data once you receive it. Unified APIs are tools that can take SFTP to the next level—they can reduce onboarding time to minutes and standardize data from multiple sources, making it easier to automate routine processes.

Unified APIs can be implemented via a direct API integration to the provider; but they can also be used to power SFTP connections for companies that don’t have the development resources to build an integration or that prefer receiving data through files instead of directly ingesting the data into their core system. 

It works like this: the unified API provider has 1:1 API integrations with each HRIS and payroll system your customers might use. The data is pulled from each system, standardized into a uniform format, then delivered to your SFTP server as a file. Each file contains identical formatting, so your team doesn’t need to normalize the data, and you can run automations that will take much of the manual work off your team’s plate so you can work more efficiently. 

Benefits of using unified APIs to scale SFTP 

Below are a few of the benefits of using unified APIs to scale your SFTP connections. 

Onboard new customers in minutes

In a typical SFTP setup, you need to work with the customer and their data provider to configure a custom file for routine transfers. Each provider has different setup requirements, implementation timelines, and costs, meaning each employer must go through the implementation individually—a process that can take as long as 18 weeks.

To make SFTP integrations truly scalable, you need a way to apply a single setup process to multiple employers, so the same work doesn’t need to be replicated with each new customer. In this scenario, there’s a single connection between your company and the data source that can be used for every client that uses that provider. 

Unified API providers enable this through their direct integrations with the providers. Rather than creating custom file formats for every new customer, you can use pre-built connections with their HRIS or payroll provider to access the specific data you need, all without assuming the liability that comes with accessing the system yourself. No more months-long back-and-forth with the customer and provider—onboarding takes just a few minutes, regardless of the employer’s system.

You can not only save operational and technical resources for higher-priority work but also create a winning customer experience right from the get-go.

Receive standard data formats, regardless of the provider

Each HRIS and payroll system has its own data format. Processing the files you receive via SFTP requires someone on your team to manually translate that data into the format your company uses—a cumbersome process that introduces the opportunity for human error and that must be repeated with each new batch of data. 

Scaling processes requires eliminating tedious busywork like this. Unified APIs standardize data across providers automatically, meaning each file you receive through the SFTP server matches an identical format. This way, you can cut down on hundreds of hours of manual labor.

Automate previously manual processes

The key to scaling operations is automating as much as possible, so your employees can focus on complex or strategic tasks, rather than simple, rote labor. 

It’s possible to build automated processes on top of traditional SFTP workflows, but there are risks involved since the connection is fairly rigid. Non-standardized data can create Not In Good Order (NIGO) errors, and if the HRIS or payroll provider makes a change to their underlying system, it could completely break the automated workflow as new data arrives under a different field name. As a result, this “automation” still requires a ton of manual effort, both to ensure everything has been read correctly and to troubleshoot errors as they arise. 

SFTP connections powered by unified APIs eliminate much of this risk by delivering standardized data. They also act as an abstraction layer, so your organization is shielded from the ongoing maintenance of keeping up with unplanned or unexpected changes made by the HRIS or payroll systems. 

For instance, if you're a 401(k) administrator, having the right automation powered by accurate, consistent, and timely data will help you auto-enroll employees into specific plans and quickly determine plan contributions. The benefits of this automation will trickle down to other departments, from sales to product to customer success—resulting in improved products, faster processing, and fewer errors—boosting your organization's overall efficiency.

In highly regulated industries like retirement and employer-sponsored benefits, you’ll likely still need some checks and balances to ensure everything runs smoothly. The difference is that it’s more of a safeguard, as opposed to constantly monitoring automated systems for failure.

Scale the business without having to grow headcount

Without reliable automation, processing employer data is a manual task. That means that as your business grows and you bring on more customers, you’ll need to hire just to keep up with the corresponding workload. 

This solution isn’t cost-effective. It incurs both overt costs like salaries and benefits as well as hidden costs like time to onboard, employee attrition, and so on. 

Automating your routine processes provides a twofold benefit: it allows you to work more efficiently, and in turn, that allows you to keep your Operations team lean and focused on the tasked that can’t be left to technology. Inaccuracies in payroll data—whether from the employee or employer—are inevitable, but automating routine work allows your team to focus on identifying and remedying those outliers to provide the best possible sponsor experience.

Scale SFTP integrations with Finch Flatfile

Scaling SFTP integrations can be much easier with modern solutions like Finch Flatfile. Flatfile, now in beta, delivers all the benefits of a unified API, including automated delivery and standardized data formatting, without any upfront engineering investment or ongoing maintenance. 

Flatfile offers several benefits over traditional SFTP integrations:

• Onboard customers in minutes — With Flatfile, you can skip the complexities of traditional SFTP setups. Simply send the employer a link to authorize access to their HRIS or payroll provider and start receiving data—no need to build custom files.‍
• Standardized data — Unlike a direct SFTP integration with a provider, Finch delivers data to your server from each HRIS and payroll in a standardized format. This reduces manual work for your operations team and eliminates the need to manage unique data schemas and naming conventions. ‍
• Easy automation — As the data provided by Finch Flatfile is standardized and vetted for formatting errors and consistency, it’s easier to build and sustain automation with. 

Get started with Finch Flatfile today

If you’re looking to scale your SFTP connections, contact our sales team to learn more about our supported providers. You can also learn more about Finch Flatfile here.