Anna Burgess Yang

Many companies rely on a secure file transfer protocol (SFTP) integration to move data from one platform to another, yet the process is notoriously finicky. Setting up the initial connection can be tricky and time-consuming, but the headaches don’t end there — SFTP errors are common and confusing.

The integration between your customer’s HRIS and your application relies on timely data, so it’s important to identify and resolve SFTP errors quickly before the systems become out of sync. This might mean working directly with your customers to troubleshoot — even if the root cause is within the data from your customer’s HRIS and not your application. But from your customer’s perspective, your application “doesn’t work,” and they will often look to you for a resolution. 

Whether it’s a data issue or connection error, you’re left trying to troubleshoot — often without an obvious way to pinpoint the problems. You might have access to error logs, but they’re difficult, if not impossible, to understand. Meanwhile, your customer is frustrated, and you have to meet compliance and SLA requirements.

If you’re not sure where to start, we’ve listed some common SFTP errors and how to identify them. We’ve also provided some information about an alternative that eliminates the need for SFTPs with a more stable, streamlined approach. 

Related: SFTP vs. API: Which integration method is best for employment data?

SFTP file errors

With SFTP, you’re relying on the extraction of data from your customer’s HRIS into a file. That file is securely moved to a specific directory, which is then “picked up” and imported by your application.

First and foremost, the file has to exist in the specified directory at the time the other application looks for it — whether that’s with each payroll run or on a different cadence. If your customer’s data doesn’t import as expected, that’s the first thing to check. 

Assuming that the file was in the correct location at the right time, you should next look to make sure that the file is in the right format. While it was probably configured correctly when you first set it up, changes in your customer’s HRIS might unexpectedly modify the file, causing the data import to fail. 

File type and format

Files are usually in a delimited format. Delimiters are special characters that indicate the separation of different pieces of data, like commas or tabs. You should check to make sure that the file has the right delimiter and that the delimiter is the same as it was in previous formats. Applications are typically coded with a delimiter in mind, so a file can’t switch between comma- or tab-delimited.

You should also check the file type to ensure it’s as expected. A comma-delimited file, for example, could be a .csv file or a .txt file. If the extension changes, your SFTP integration could fail.

Fields included 

SFTP imports rely on an exact match of columns in your customer’s file to fields in your application. While some applications may be coded to ignore extra fields, others may not. You’d either need to have your customer remove the extra fields in your file, map those fields in your application, or tell your application to ignore those fields. 

If you’d previously mapped a field and the field is removed from your file, that can also cause SFTP errors since the application is now receiving fewer fields than expected.

If your customer has created a custom field in their HRIS and wants to send it through your SFTP integration, make sure you or the customer create a matching custom field in your application before you start sending the data in your file. SFTP integrations will not create new fields in your application. 

File name and field names

Your SFTP integration might be looking for a specific file name (such as “dailyintegration.txt”). If the file name has changed, the import might not work. Sometimes, extracts from your customer’s HRIS might include a date and timestamp at the end of the file (such as “dailyintegration20240831.text). If so, you’d need to determine if your application can handle the changing date and timestamp, or if you or your customer will need to remove it before the import can occur. 

Within the file, some SFTP integrations are looking for exact field name matches between your file and the destination system. For example, if the name in the application is “First Name,” the name in your file must also be “First Name.” In some cases, the field is even case-sensitive, so if the field name is “first name,” the transfer will fail. 

File corruption

If something interrupts file creation (such as an unstable network), your customer might end up with a corrupted file. This can also happen if there are problems during the transfer process itself, such as memory issues.

You can usually detect file corruption if you open the file and it is blank or contains odd characters. 

UTF-8 encoding protocol

Unicode transformation format - 8 bits (UTF-8) is a common encoding protocol used by SFTP servers. If files contain special characters, UTF-8 ensures that data is accurately transmitted and received without corruption. If a file doesn’t follow a UTF-8 format and your application requires that format, the transfer will fail. The reverse can also be true: your customers are sending the file in UTF-8 format and your application can’t handle it, so it corrupts the file. 

SFTP data errors 

While file errors are tied to how the files are created, data errors are contained within the file. Sometimes, issues within individual records are enough to cause your SFTP integration to fail. 

Unless you’re getting clear error messages, the only way to find data errors is to open the file and search for the data that could be causing the issue. Given the size of these files, this can require a massive amount of effort.

Field mapping

You have to establish field mapping before you can complete a successful transfer. For example, in your customer’s HRIS, you might have a field “Start Date” and you need to map it to “Hire Date” in your application. 

If not mapped correctly, the SFTP integration might not work. If your customer incorrectly maps a field “First Name” to a field “Hire Date,” the data types are a mismatch (one is text, and one is a date field). Some SFTP integrations will reject this type of mapping and cause error messages or an SFTP failure. 

Duplicate data 

Your file integration will rely on a unique identifier for each record from your customer’s HRIS (such as the employee’s social security number or email). This is how the application can tell one employee record from another employee record. 

Unique identifiers cannot be duplicated in files. Some SFTP integrations may skip the duplicate records, but in other cases, it may cause the sync to fail. 

Data format

Mapping fields often requires a very specific format. Dates might need to be in either YYYY/MM/DD or MM/DD/YYYY. If your customer is sending any fields with numbers, a field may accept or reject decimals. 

If the SFTP integration fails, check the formatting of each field to make sure it matches the application’s requirements. 

Special characters

Some applications may be unable to handle special characters (like & or $). If your fields contain these characters, they will cause SFTP errors. Special characters may not cause complete SFTP errors, but may cause the specific fields not to be populated within your application, so your sync would be incomplete. 

Your customer would need to remove these characters so your data can sync properly. 

Terminated or inactive employee data 

Many applications that sync from an HRIS want only active employee data. You’d need to ensure that your customer filters the HRIS files to remove terminated or inactive employee records before they’re sent via SFTP. 

If your customer sends a file containing terminated or inactive employees, it can cause a lot of issues. Your application may not be able to handle the records or it can create “junk” records in your application that you’d need to clean up. 

SFTP connection errors 

SFTP failure can also happen when there’s a connection issue between the source (your customer’s HRIS) and the destination (your application). You may need to involve someone from your customer’s technical support team to troubleshoot these issues. 

Destination folder

When your customer sends a file from their HRIS, it has to be placed in a destination folder for the application to ingest. If the destination folder doesn’t have the correct permissions (such as being read-only versus read-write), the file won’t transfer.

Your customer may also have issues within the folder itself. In some cases, the destination folder has to be left blank to ensure that data is ingested. Additionally, if your customer sends too many files at once — especially if they have the same file name — the sync may fail. 

Permissions

The file transfer itself may be set up as a scheduled job, which your customer needs to run with administrator access. If it’s run with a specific employee’s account, and that employee leaves without granting access, it can be difficult to transfer permission for the scheduled job to a new employee. 

Additionally, your customer’s firewall may prevent sensitive data from leaving any internal systems. Your IT department may need to disable the firewall to allow the SFTP sync to occur. 

SSH key pairing

SSH key pairing is an authentication method for SFTP that’s more secure than using a password. On your customer’s SFTP server, your customer will have a public key that acts like a “lock.” During the connection, a private key is used to authenticate and “unlock” the server. The combination of keys — the public key and private key — is the key pair. 

SSH keys can be tricky to manage, especially if your customer is doing this manually. Keys are stored in .ppk files, which may require downloading additional software to generate the files. Keys can be “orphaned” if they’re issued to employees who leave the organization and IT staff forgets to terminate the keys. Keys can also be blocked by firewalls, permissions, or server configuration. 

Erroneous actions

SFTP has many operations, such as “put” (upload), “get” (download), “rename” and more. Your customer’s SFTP needs to use the “put” action only. If you include something else (like “rename”), it may cause the sync to fail. 

Unsupported SFTP protocol

Unsupported SFTP protocol errors happen when there’s a mismatch or incompatibility between the source and the destination. Your customer’s server may only accept certain configurations. 

SFTP has multiple versions, and they’re not all compatible with each other. Additionally, SFTP runs over SSH which also has multiple versions (SSH1 versus SSH2), which can cause incompatibilities. Your customer’s security settings may also cause protocol issues. 

Related: Improving Business KPIs with SFTP Scalability and Automation

HRIS integrations: An alternative to SFTP

Even after you’ve set up your SFTP (which can be difficult by itself) and it’s running as expected, you can experience issues down the road. If there are changes in the underlying technology, such as your customer’s HRIS or in your own application, it can cause SFTP errors. You have control over your own app, but changes to your customers’ various HRIS systems are unpredictable, meaning maintaining a healthy SFTP connection is an ongoing, intensive effort.

To make matters more difficult, each HRIS has its own quirks and data schemas. Each customer will require the configuration of their own SFTP connection, but these connections can’t follow a routine protocol because each HRIS will have different data formats and connection requirements. 

By contrast, HRIS integrations powered by APIs offer an automated, standardized alternative to complex SFTP connections and a way to avoid common SFTP errors altogether. With Finch’s Unified Employment API, you can build a single API connection to access your customers’ data across hundreds of HRIS and payroll systems. 

Finch: The HRIS and Payroll Integration Solution

With Finch’s API integrations, your application can seamlessly connect to the various HRIS and payroll systems your customers use, without having to build individual SFTP connections for each customer. Better yet, the data is automatically standardized, meaning it is delivered to your application in the appropriate format, regardless of how it’s configured in the HRIS or payroll system. The data is also sent daily or weekly, as opposed to just once per pay cycle.

Finch’s integrations can be accessed in one of two ways — either through an API integration with Finch, or through Finch Flatfile, now in beta. Flatfile provides all the benefits of our API, but still delivers files via SFTP.

Finch has access to the largest network of HRIS and payroll systems, providing deep integrations without the headaches of SFTP. Companies using Finch can take advantage of Finch’s Unified Employment API and allow their customers to integrate their HRIS and payroll platforms seamlessly. 

Learn more about Finch by signing up for free or scheduling a call.

First impressions matter—which makes your user onboarding experience critical. A smooth onboarding process gets your customers up and running quickly and helps them realize value faster. Automated user onboarding is the fastest, most efficient way to create an unparalleled user experience.

With automation, you can free up resources both on the customer’s end and internally, onboarding customers with almost no manual effort. It also enhances the overall user experience since your customers can self-serve during onboarding. 

However, onboarding can be challenging in the employment sector, since your product needs to import the data held in the employer’s HRIS or payroll platform. Employee information like job titles, employment status, pay statements, or demographic data needs to be securely transferred before your customers can begin using your product. Without direct access to the employer’s system of record, moving this data to your platform requires costly SFTP setups or cumbersome manual work.

Rather than relying on a lot of back-and-forth with your customer, HRIS and payroll integrations enable you to automate onboarding by securely transferring information from their source of truth — without a lot of manual intervention. 

What is user onboarding automation?

Broadly, user onboarding automation streamlines the process of bringing customers onto a new platform with minimal manual effort. In the employment space, that means importing required employee data automatically from the employer’s source of truth.

Manual onboarding requires a lot of effort of both your team and your customers. All of the relevant employee data—whether that’s census data or individual pay statements—has to be manually imported to your platform. At best, that requires configuring a custom SFTP connection or downloading files from the HRIS or payroll system and uploading them to your platform; at worst, it requires your customer to manually add each of their employees one by one.

By contrast, automated user onboarding is much faster and easier. The use cases are broad, but the core functionality connects an application to the source of truth for employee data, whether it’s an HRIS or payroll database. Once connected, data seamlessly transfers between the two systems almost instantly. Plus, the systems are able to continually communicate, meaning data is updated in your system as it’s updated in the HRIS or payroll system.

5 Benefits of automated user onboarding

Whether your customers are SMBs or large companies, automated onboarding provides a better customer experience. 

1) Improves data quality 

Manual onboarding opens the door to human error. Your customers provide files that must be mapped and imported into your system. Customers may struggle to get files with the correct fields in the correct format. Additionally, manually exported and imported files are always point-in-time and may not reflect the current workforce in the system of record. 

Automated onboarding connects two systems using predefined rules and mapping. Customers can be assured that their data syncs are accurate. In regulated industries, this also ensures all steps for compliance are handled correctly.

Automated onboarding also sets the stage for ongoing integration between systems — which is what your customers need and expect. In fact, 84% of HR professionals say integration is “extremely important” for any tool added to their tech stack. 

2) Saves customers time

With automated onboarding, customers won’t spend time creating files or otherwise importing their employee data from one system to another. They won’t be forced to manually enter employee data if they don’t have the resources or technical expertise to create files.

This reduces the time spent onboarding and also saves time as customers continue using the product. The integration used during onboarding keeps HRIS or payroll data synced between the two systems, automatically adding new employees, de-provisioning employees as they leave, and updating information based on changes to salary, role, and other employee data. 

3) More secure

If your customers have to transfer files for manual onboarding, they risk exposing sensitive employee data. Someone internally may email the files rather than share them securely or download them to a directory that doesn’t meet internal protocols. To transfer files on an ongoing basis, they would need to set up an SFTP connection, which needs to be monitored and managed.

Automated user onboarding securely collects your customers’ credentials to connect their systems. Customer-permissioned access lets users stay in control of what data is shared through the integration. 

4) Delivers value faster

Whether manual onboarding takes days or weeks, it delays your customers’ use of your product. Some initial excitement can wear off, and it takes longer for customers to realize value — both as a company and for their employees. 

Human Interest sees integrations as a competitive differentiator. “We’re always looking for an opportunity to improve our value offering,” says Drew Obston, Manager of Product Operations. “With integrations, we’re able to go above and beyond and say, ‘You don’t have to upload payroll data. You don’t have to be provisioning someone on our team to log in to your payroll.’”

Automated onboarding takes minutes and allows your customers to focus their efforts on configuring and integrating your product into their processes. 

5) Scalable for your organization

If you’re relying on manually onboarding your customers, you need employees to work with them. Your employees may import files on behalf of customers, assist with data mapping, or make ongoing updates to keep data current. 

Manual processes will always be limited by employee bandwidth. Automated user onboarding, on the other hand, has no limits. You can onboard more customers without compromising the onboarding experience. You can focus your resources on relationship management or customer support rather than customer onboarding. 

Examples of automated user onboarding

HRIS and payroll integrations can serve a wide variety of industries, including 401(k) and retirement administrators, employee benefits solutions, insurance, HR technology, fintech, and more. 

Here are a few examples of how automation can streamline onboarding.

401(k) and retirement administrators 

With HRIS and payroll integrations, 401(k) administrators can pull employee information like employment status, start date, and date of birth directly from the employers’ system to automatically verify employees’ eligibility. The eligible employees can then be automatically enrolled in the plan, pursuant to SECURE Act 2.0, with the choice to opt out. Even better, these integrations provide evergreen access to the employer’s payroll system, so new employees can automatically be enrolled once they’ve met the sponsor’s eligibility requirements. 

Use Case: 401(k) provider Saveday uses automated user onboarding to let its customers set up plans in 15 minutes or less. Customers grant access to their payroll systems within minutes, eliminating the need for the Saveday team to manually download files and validate data. 

Employer-sponsored benefits providers

Integrations allow employers to import data for all existing employees so they can enroll in employer-sponsored benefits like ICHRAs, Earned Wage Access, and Health Payment Accounts. The provider can automatically import data for each employee, like individual pay statements, company contributions, pre- and post-tax deductions, and employment status to immediately identify eligible employees, enroll them in benefits, and manage payroll deductions on a recurring basis. 

Use Case: Exhale provides a suite of financial wellness benefits to hourly employees. By integrating with employers’ payroll systems, Exhale is able to enroll employees in benefits up to 2 weeks faster because the company doesn’t need to wait until the next pay cycle to view employees’ pay statement data.

Employee engagement and learning management systems

To effectively train employees and reward them for their efforts, engagement and learning management tools need to have a record of each employee and relevant census data like their hire date, job title, tenure, and more. Integrating with the employer’s HRIS allows these platforms to automatically create accounts for each employee and enroll them in rewards or training programs that are relevant to their specific role. As employees join or leave the company, they can automatically be enrolled or deprovisioned.

Use Case: Trainual uses HRIS and payroll integrations to instantly onboard and offboard employees as necessary, without asking customers to update their user data. That automation is critical for a platform that prides itself on scaling with its customers.

Fintech solutions

B2B fintech solutions rely on current employee data for headcount cost analysis, compensation management, or equity management. With HRIS and payroll integrations, you can securely pull in company, department, job title, and payroll data to enable pay and operational insights. Integrations keep employee data updated so your customers can see analysis and pull reports in real time.

Use Case: Mosaic has built a next-gen financial reporting and forecasting system that relies on seamless data integration from a company’s HR and payroll data. With this information, Mosaic provides detailed insights into employee costs so organizations can strategically plan their headcount. 

How to automate user onboarding

Some companies, recognizing the need for automated user onboarding and ongoing syncs between HRIS and payroll platforms, build their own integrations. For customers, this certainly improves the experience by removing the manual work on their end and keeping data up-to-date. 

However, building integrations comes with its own challenges. With so many HRIS and payroll platforms on the market, companies are faced with two paths: expend resources building a significant number of integrations, or limit your potential customers by only building a few integrations.

No matter the number of integrations you build, you will need developers to work with the APIs of different platforms. Even though integrations improve your end-user experience, building and testing them takes resources away from your core product.

Unified APIs allow companies to launch hundreds of integrations at once. As a single, standardized way to connect systems, unified APIs enable developers to work through a single interface. Your developers don’t have to work through the complexities and variations between each HRIS or payroll platform’s APIs. 

Finch provides secure and comprehensive two-way integrations with over 200 HRIS and payroll systems in the US, covering 88% of employers. 

With Finch, your end user’s experience is seamless and relies on customer-permissioned access to connect their systems to your product. 

With Finch’s unified APIs, you can automate user onboarding and rely on verified, real-time data to keep the systems in sync. 

Empower your customers from the beginning

The best user onboarding is forgettable: it is so easy that it takes almost no time or effort. Manual onboarding and ongoing updates cannot meet customer expectations, so companies need to rely on integrations. Efficient and automated user onboarding builds trust and serves as a prelude to your ongoing partnership. 

Finch’s powerful unified APIs allow companies in HR tech, benefits, insurance, and related industries to access the largest network of HRIS and payroll systems. 

Learn how Finch can benefit your organization by scheduling a call. You can also sign up to try Finch for free.